Machine learning apparatus, method, and program

Abstract

To provide a machine learning device, method, and program capable of improving the reliability of labels used to train deep learning models that process classification problems. [Solution] The machine learning device according to the embodiment includes a learning unit, a determination unit, and an update unit. The learning unit obtains a first model output including inference probabilities and / or feature vectors output from a first deep learning model by applying a first sample to a first deep learning model that processes a classification problem. The determination unit determines whether or not it is necessary to update the labels used as teachers for training the first deep learning model, based on the first model output and / or labels. If it is determined that a label update is necessary, the update unit updates the labels based on the first model output and the current update count of the labels; if it is determined that a label update is not necessary, it terminates the label update.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to machine learning apparatus, methods, and programs. [Background technology]

[0002] Machine learning, particularly deep learning models, has been deployed in various fields over the past decade. While unsupervised learning, which eliminates the need for manually assigned labels, has developed for classification problems, supervised learning remains dominant when high reliability is required, as using assigned labels generally yields higher performance. However, in cases where labeling costs are extremely high, such as in the medical field where highly specialized expertise is required for reliable labeling, or in manufacturing lines processing massive amounts of data, mislabeling is unavoidable. In other words, the very purpose of using assigned labels to increase reliability can backfire, potentially significantly degrading model performance. To prevent this, it's necessary to address the issue of deep learning models being robust to mislabeling, based on the premise that some errors will inevitably remain even after efforts to reduce mislabeling. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda, “Early-Learning Regularization Prevents Memorization of Noisy Labels,” Advances in Neural Information Proceeding Systems 33 (NeurIPS 2020). [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The problem that this invention aims to solve is to provide a machine learning device, method, and program that can improve the reliability of labels used to train deep learning models that process classification problems. [Means for solving the problem]

[0005] The machine learning apparatus according to the embodiment includes a learning unit, a determination unit, and an update unit. The learning unit obtains a first model output, which includes inference probabilities and / or feature vectors output from a first deep learning model, by applying a first sample to a first deep learning model that processes a classification problem. The determination unit determines, based on the first model output and / or the labels, whether or not it is necessary to update the labels used as teachers for training the first deep learning model. If it is determined that the labels need to be updated, the update unit updates the labels based on the first model output and the current update count of the labels; if it is determined that the labels do not need to be updated, it terminates the label update. [Brief explanation of the drawing]

[0006] [Figure 1] This figure shows an example configuration of the machine learning device according to this embodiment. [Figure 2] Diagram showing the functional configuration of the label correction unit. [Figure 3] This diagram shows an example of the label correction process performed by the label correction unit. [Figure 4] A diagram illustrating the relationship between the trend of inference probability and the number of times the analysis was terminated or updated. [Figure 5] Diagram showing the functional configuration of the learning rate estimation unit. [Figure 6] This diagram shows an example of the learning rate determination process performed by the learning rate estimation unit. [Figure 7] A diagram illustrating the behavior of inference probabilities between multiple learning rates. [Figure 8] Diagram illustrating the divergence-learning rate graph. [Figure 9] A diagram illustrating an example of the overall processing flow by a machine learning device. [Figure 10] Figure showing the transition of the accuracy rate of the pseudo label according to the comparative example [Figure 11] Figure showing the transition of the accuracy rate of the pseudo label according to the present embodiment

Mode for Carrying Out the Invention

[0007] Hereinafter, a machine learning device, method, and program according to the present embodiment will be described with reference to the drawings.

[0008] FIG. 1 is a diagram showing a configuration example of a machine learning device 1 according to the present embodiment. As shown in FIG. 1, the machine learning device 1 includes a processor 11, a storage device 12, an input device 13, a display device ️15, and a communication device 14. Transmission and reception of data and various signals among the processor 11, the storage device 12, the input device 13, the display device 15, and the communication device 14 are performed via a bus.

[0009] The processor 11 is an integrated circuit that controls the overall operation of the machine learning device 1. For example, the processor 11 has a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), and / or an FPU (Floating-Point Unit). The processor 11 may include an internal memory and an I / O interface. The processor 11 executes various processes by interpreting and calculating programs stored in advance in the storage device 12 and the like. Note that part or all of the processor 11 may be realized by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0010] The storage device 12 is a volatile memory and / or a non-volatile memory that stores various data. For example, the storage device 12 stores data and setting values used when the processor 11 executes various processes, data generated by various processes in the processor 11, and the like. The storage device 12 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), an SSD (Solid State Drive), an integrated circuit storage device, or the like. Note that the storage device 12 may have a non-temporary computer-readable storage medium that stores a program executed by the processor 11.

[0011] The input device 13 receives inputs of various operations from an operator. As the input device 13, a keyboard, a mouse, various switches, a touch pad, a touch panel display, or the like can be used. An electrical signal (hereinafter, an operation signal) corresponding to the received input of the operation is supplied to the processor 11.

[0012] The display device 15 displays various data in accordance with the control by the processor 11. As the display device 15, a CRT (Cathode-Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, an LED (Light-Emitting Diode) display, a plasma display, or any other display can be appropriately used. The display device 15 may be a projector.

[0013] The communication device 14 includes a communication interface such as a network interface card (NIC) for data communication with various devices connected to the machine learning device 1 via a network. Operation signals may be supplied from a computer connected via the communication device 14 or from an input device on that computer, and various data may be displayed on a display device on a computer connected via the communication device 14. However, for the sake of simplicity in the following explanation, unless otherwise specified, the source of the operation signals will be the input device 13, and the destination for the display of various data will be the display device 15. The input device 13 can be replaced by a computer connected via the communication device 14 or from an input device on that computer, and the display device 15 can be replaced by a display device on a computer connected via the communication device 14.

[0014] The machine learning device 1 does not need to include all of the processor 11, storage device 12, input device 13, display device 15, and communication device 14. Some of the storage device 12, input device 13, display device 15, and communication device 14 may be omitted as needed. The machine learning device 1 may also be provided with any additional hardware devices useful for executing the processing according to this embodiment. The machine learning device 1 does not need to consist of a single physical computer; it may consist of a computer system having multiple computers connected via wired or network lines for communication. The assignment of the series of processes according to this embodiment to the multiple processors 11 implemented in each of the multiple computers can be arbitrarily configured. All processors 11 may execute all processes in parallel, or specific processes may be assigned to one or some of the processors 11, and the series of processes according to this embodiment may be executed by the entire computer system.

[0015] Processor 11 performs supervised learning on a deep learning model that processes a classification problem. In supervised learning, teaching labels are used as teachers to compare with the inference probabilities output by the deep learning model in response to sample inputs. As mentioned above, teaching labels are assigned manually by humans who understand the content of the samples, or automatically by a computer after the computer analyzes the content of the samples. Therefore, teaching labels inevitably contain errors (noise). Hereafter, labels containing errors will be referred to as false labels. Deep learning that is robust to false labels is required.

[0016] Countermeasures for mislabeling are being actively researched as learning methods when the teaching labels contain noise. Currently, the mainstream approaches involve selecting reliable labeled data, relabeling, and introducing loss functions that can incorporate these effects. In particular, Non-Patent Literature 1 proposes a method to improve the reliability of labels by modifying the training label y using the inferred label p in an exponential moving average manner, such as y←mp+(1-m)y, i.e., relabeling. Here, m is a hyperparameter, approximately 0.01 to 0.1. This method uses soft labels as pseudo-labels, which are the teaching labels after relabeling, and yields more stable and accurate labels compared to using hard labels. However, verification by the inventors has revealed that this method is very sensitive to the number of label updates, and the accuracy of the pseudo-labels deteriorates significantly if the number of updates is too few or too many.

[0017] The processor 11 generates highly reliable teaching labels to ensure the reliability of the teaching labels used as teachers for training a deep learning model (hereinafter referred to as the classification model) that processes classification problems. The processor 11 then uses these highly reliable teaching labels to perform deep learning that is robust against mislabeling. As shown in Figure 1, the processor 11 has a functional configuration for this purpose, including a label correction unit 20, a learning rate estimation unit 30, a label noise-resistant learning unit 40, and a display control unit 50.

[0018] The label modification unit 20 generates highly reliable teaching labels to be used as training data for the classification model by modifying the initial labels. The initial labels are the teaching labels before modification (before update), and may be accurate labels (correct labels) in relation to the sample, or inaccurate labels (wrong labels) in relation to the sample. Label modification is performed by iteratively updating the initial labels based on the model output from the deep learning model. Label updating is also called relabeling. The label modification unit 20 generates highly reliable teaching labels by appropriately controlling the number of updates. The generated highly reliable teaching labels are called optimal teaching labels.

[0019] The learning rate estimation unit 30 estimates the optimal learning rate for training the classification model. The optimal learning rate means a learning rate that strikes a balance between suppressing overfitting and improving accuracy. The estimated optimal learning rate is called the optimal learning rate. The optimal learning rate is used for label updating (relabeling) by the label correction unit 20. The optimal learning rate may also be used for machine learning by other computers connected via the communication device 14.

[0020] The label noise-tolerant learning unit 40 performs supervised learning on the classification model. In this case, the label noise-tolerant learning unit 40 may perform supervised learning using the optimal teaching labels obtained by the label correction unit 20 as teaching labels. This enables supervised learning that is robust to label noise.

[0021] The display control unit 50 displays various information on the display device 15. For example, the display control unit 50 displays a sample, an optimal teaching label, information used to generate the optimal teaching label, the optimal learning rate, information used to estimate the optimal learning rate, and so on.

[0022] Figure 2 shows an example of the functional configuration of the label correction unit 20. As shown in Figure 2, the label correction unit 20 includes an acquisition unit 21, a learning unit 22, an update determination unit 23, and an update unit 24.

[0023] The acquisition unit 21 acquires a first sample. The first sample refers to data to be input to the first classification model and includes any form of data applicable to the classification model, such as images, videos, audio, text, sensor output, etc.

[0024] The learning unit 22 obtains a first model output from a first deep learning model (first classification model) by applying a first sample to the first deep learning model (first classification model) that processes the classification problem. The first model output includes inference probabilities and / or feature vectors output from the first classification model. The inference probability is the final output calculated by the first classification model and is a vector with a number of components (dimensions) corresponding to the number of classes. Each component has a value corresponding to the probability of belonging to that class. The inference probability can also be expressed as a label calculated by the first classification model. The inference probability is calculated as a soft label. A soft label means a label in which each component value is a continuous value. A hard label means a label in which each component value is a one-hot representation of 1 or 0. The feature vector is an intermediate output of the first classification model and is a vector with an arbitrary number of components.

[0025] The update determination unit 23 determines whether or not to update the teaching labels used as teachers for training the first classification model, based on the first model output and / or teaching labels obtained by the learning unit 22. More specifically, the update determination unit 23 calculates a first update count (hereinafter referred to as the termination update count) for stopping the updating of teaching labels, based on the first model output and / or teaching labels. The update determination unit 23 determines whether or not to update the teaching labels based on a comparison between the current update count and the termination update count. The update count means the number of times the teaching labels are updated and is synonymous with the number of epochs.

[0026] If the update determination unit 23 determines that the teaching label needs to be updated, the update unit 24 updates the teaching label based on the first model output and the teaching label for the current update count. If it determines that the teaching label does not need to be updated, the update unit 24 terminates the teaching label update. If the update unit 24 determines that the teaching label does not need to be updated, it terminates the teaching label update and modifies the teaching label for the current update count into a hard-labeled one-hot label. The modified one-hot label is then used as the optimal teaching label.

[0027] Figure 3 shows an example of the flow of the label correction process performed by the label correction unit 20. As shown in Figure 3, first, the acquisition unit 21 acquires a first sample (step SA1). The first sample is acquired from a first training dataset containing multiple first samples. In step SA1, one mini-batch of first samples is acquired. A teaching label is assigned to the first sample. Here, the initial value of the teaching label is also called the initial label. The initial label is assumed to be an incorrect label that does not match the actual first sample.

[0028] When step SA1 is performed, the learning unit 22 applies the first classification model to the first sample obtained in step SA1 and calculates the inference probability (step SA2). The first classification model is assumed to be a deep neural network (DNN) having arbitrary network layers such as fully connected layers, convolutional layers, self-attention layers and / or pooling layers. The first classification model is untrained. The learning unit 22 inputs the first sample into the first classification model and calculates the inference probability corresponding to the input first sample by applying a forward propagation process to the input first sample according to the network structure of the first classification model.

[0029] When step SA2 is performed, the update determination unit 23 determines whether or not to update the teaching label (step SA3). In step SA3, the update determination unit 23 determines whether or not to update the teaching label based on a comparison between a pre-calculated number of updates before termination and the current number of updates. More specifically, the update determination unit 23 determines that an update of the teaching label is required if the current number of updates has not reached the number of updates before termination, and determines that an update of the teaching label is not required if the current number of updates has reached the number of updates before termination. The number of updates refers to the number of times the teaching label was updated in step SA5. The number of updates before termination refers to the number of updates at which the updating of the teaching label is terminated. The calculation of the number of updates before termination will be described later.

[0030] If it is determined in step SA3 that an update is required (step SA4: required), the update unit 24 updates the teaching label for the first sample (step SA5). For example, the update unit 24 calculates the teaching label y' for the next update number based on the inference probability p of the current update number and the teaching label y of the current update number, according to equation (1) below. m is a hyperparameter that can be set to any value. For example, m is preferably set between 0.01 and 0.1. The teaching label obtained by the label update is also called a pseudo-label. In this way, the update unit 24 calculates the moving average of the teaching label y as the teaching label y'. A simple moving average, weighted moving average, exponential moving average, or any other arbitrary moving average can be used as the moving average.

[0031] y'←mp+(1-m)y (1)

[0032] When step SA5 is performed, the learning unit 22 updates the parameters of the first classification model (step SA6). In step SA6, the learning unit 22 calculates a loss according to a predetermined loss function to evaluate the error between the inference probability calculated in step SA2 and the teaching label obtained in step SA5, and updates the parameters of the first classification model so that the calculated loss is small. The parameters are assumed to be the weights or biases between the network layers of the first classification model. Any method such as stochastic gradient descent (SGD) or Adam can be used as the parameter optimization algorithm. The learning rate should preferably be the optimal learning rate estimated by the learning rate estimation unit 30.

[0033] The termination update count is set for each first sample. Therefore, the update determination unit 23 determines whether or not to update the teaching label for each first sample based on a comparison between the termination update count and the current update count, and the update unit 24 updates the teaching label for each first sample.

[0034] When step SA6 is performed, steps SA1 to SA6 are repeated for all first samples until it is determined in step SA3 that no update is needed. In step SA1, an unprocessed minibatch is retrieved from the dataset. When there are no more unprocessed minibatches, steps SA1 to SA6 are repeated again, one minibatch at a time. One cycle of minibatches is called an epoch. The number of updates is synonymous with the number of epochs.

[0035] If it is determined in step SA3 that no update is needed (step SA4: No), the update unit 24 corrects the teaching label for the current update count to a one-hot vector (step SA7). Specifically, for each first sample, the update unit 24 corrects the maximum value among the multiple component values ​​that make up the teaching label to 1 and corrects the other component values ​​to zero. By correcting the pseudo-label for the current update count to a one-hot vector in this way, the optimal teaching label is generated.

[0036] Once step SA7 is performed, the label correction process by the label correction unit 20 is completed.

[0037] Here, the method for calculating the number of updates to terminate by the update determination unit 23 will be explained in detail. The update determination unit 23 utilizes the property that when optimizing a deep learning model using stochastic gradient descent, overfitting to mislabels does not occur if the learning rate is sufficiently high compared to the batch size during training. In this embodiment, this property will be called generalizability. If a learning rate high enough to acquire generalizable features is used, it is expected that the inference probability will take only a few values ​​in the vicinity of a certain number of updates. That is, under the assumption of generalizability, if the one-hot representation of the pseudo-label changes from the one-hot representation of the initial label, it can be considered that overfitting has occurred.

[0038] Figure 4 illustrates the relationship between the progression of inference probability and the number of censorship updates E1. In Figure 4, the vertical axis represents inference probability, and the horizontal axis represents the number of epochs. The correct class probability represents the inference probability of the class that fits the sample (correct class), and the incorrect class probability represents the inference probability of the class that does not fit the sample (incorrect class). The initial label is assumed to be labeled as an incorrect class. At the start of learning (epoch number = 0), the one-hot representation of the initial label is 1 for the incorrect class and 0 for the correct class. As learning progresses under the assumption of generalizability, the fit to the correct class improves, so the correct class probability increases, and the incorrect class probability decreases accordingly. Eventually, the correct class probability exceeds the incorrect class probability. At this point, because the correct class probability is higher than the incorrect class probability, the one-hot representation of the pseudo-label is 0 for the incorrect class and 1 for the correct class. The number of epochs at which the correct class probability and the incorrect class probability reverse is the number of censorship updates E1. In other words, at the termination update count E1, the one-hot representation of the pseudo-label changes from the one-hot representation of the initial label.

[0039] Therefore, the update determination unit 23 calculates the specific number of update times when the class corresponding to the maximum value among the plurality of component values (hereinafter referred to as teaching label component values) constituting the teaching label vector before update changes to the class corresponding to the maximum value among the plurality of component values (hereinafter referred to as inference probability component values) constituting the inference probability vector, as the cut-off update number. Specifically, the cut-off update number τ can be expressed by the following formula (2) under the assumption of stationarity that the inference probability vector does not change.

[0040] [Number]

[0041] c p is the class that takes the maximum value among the plurality of inference probability component values p c constituting the inference probability vector p. In other words, c p is the inference probability component value of the class where the one-hot representation of the inference probability vector p becomes 1. c y is the class that takes the maximum value among the plurality of teaching label component values y c constituting the teaching label vector y before update. In other words, c y is the inference probability component value of the class where the one-hot representation of the teaching label vector y before update becomes 1. δp is the value obtained by subtracting p cp from p cy , that is, the inference probability component value p[[ID=二十九]] c of class c p among the plurality of inference probability component values p cp constituting the inference probability vector p, minus the inference probability component value p c of class c y among the plurality of inference probability component values p cy constituting the inference probability vector p. Similarly, δy is the value obtained by subtracting y cy from y cp , that is, the teaching label component value y c of class c y among the plurality of teaching label component values y cy constituting the teaching label vector y before update, minus the teaching label component value yc class c p Instructional label component value y cp This is the value obtained by subtracting [a certain value].

[0042] The update determination unit 23 calculates the culling update count at any number of updates. For example, the culling update count may be calculated at an update count specified in advance by the user. The update count specified in advance by the user should be empirically determined to be less than the culling update count. Alternatively, the update determination unit 23 may calculate the culling update count for each first sample. In this case, the update determination unit 23 compares the culling update count with the current update count for each first sample. If the current update count has not reached the culling update count, it updates the teaching label for that first sample. If the current update count has reached the culling update count, it corrects the teaching label for that first sample to a one-hot vector.

[0043] According to this embodiment, firstly, by utilizing an optimal learning rate with a high learning rate, it becomes possible to update labels using only generalizable features that do not overfit to noise. By updating labels with this learning behavior in mind, it becomes possible to obtain more accurate teaching labels (optimal teaching labels). Secondly, by terminating the updating of teaching labels at an appropriate number of updates that is neither too few nor too many, the degradation of the accuracy of teaching labels due to repeated updates is suppressed, and the performance of exponential moving average-like label updates is maximized. Thirdly, it is possible to mechanically determine the number of updates to terminate based only on the inference probability and pseudo-labels in a given epoch, and it is also possible to set different numbers of updates to terminate depending on the individual sample. In other words, it is possible to automatically perform label updates tailored to each individual sample, and the number of updates is also automatically determined according to each individual sample. This makes it possible to greatly simplify the complex operation of adjusting the number of updates to terminate according to each individual sample.

[0044] Due to the advantages described above, this embodiment makes it possible to correct mislabels appropriately and at a lower cost compared to Non-Patent Document 1. As a result, robust learning is possible even for datasets containing mislabels, and it is also possible to provide feedback to the teaching process by displaying samples with mislabels.

[0045] The processing performed by the label correction unit 20 described above is basically assumed to be carried out at a high learning rate (optimal learning rate) that can obtain generalizable features. For this reason, a learning rate estimation unit 30 is provided independently of the label correction unit 20.

[0046] The optimal learning rate, which is high enough to acquire generalizable features, can be determined semi-automatically. Determining the optimal learning rate utilizes the property that the optimal learning rate does not depend heavily on the intensity of the noise. For example, consider a noisy dataset in which very strong noise is artificially added to the labels. When the artificial label noise is significantly stronger than the original label noise, the accuracy measured with the labels before adding the artificial noise can be considered as the accuracy measured with labels without noise. By comparing the accuracy of the two, it becomes possible to observe overfitting to the noise, i.e., the discrepancy between the accuracy of the two. Therefore, by performing supervised learning while varying the learning rate, it is possible to estimate the learning rate at which overfitting occurs. The optimal learning rate is determined based on this learning rate. The learning rate estimation unit 30 will be described in detail below.

[0047] Figure 5 shows an example of the functional configuration of the learning rate estimation unit 30. As shown in Figure 5, the learning rate estimation unit 30 includes an acquisition unit 31, a label changing unit 32, a learning unit 33, and an optimal learning rate determination unit 34.

[0048] As shown in Figure 5, the acquisition unit 31 acquires a second sample. The second sample represents data to be input to the second classification model and includes any form of data applicable to the classification model, such as images, videos, audio, text, and sensor outputs.

[0049] The label changing unit 32 changes the initial label of the second sample acquired by the acquisition unit 31 to an artificial label, which is a label that has been artificially modified from the initial label.

[0050] The learning unit 33 uses the artificial labels obtained by the label changing unit 32 as training data to perform supervised learning on a second deep learning model (second classification model) that processes the classification problem, using multiple different learning rates.

[0051] The optimal learning rate determination unit 34 determines a specific learning rate (hereinafter referred to as the optimal learning rate) that avoids overfitting based on changes in the behavior of the output evaluation metric between multiple learning rates. The output evaluation metric is the second model output output from the second classification model when the second sample is applied to the second classification model, or the error of the second model output. Specifically, the optimal learning rate determination unit 34 determines the optimal learning rate based on the degree of deviation between a curve representing the change in the first output evaluation metric with respect to the change in the second output evaluation metric and a curve representing the change in the second output evaluation metric. The first output evaluation metric is the error between the correct label, which is the initial label that is accurate in light of the actual second sample, and the inference probability, which is the output of the second model. The second output evaluation metric is the error between the incorrect label, which is the initial label that is inaccurate in light of the actual second sample, and the inference probability, which is the output of the second model. As an index for evaluating the error, a loss based on a loss function is used. The loss functions similarly to the accuracy rate. For example, the optimal learning rate determination unit 34 determines the optimal learning rate as the learning rate at which the deviation falls below a reference value.

[0052] Figure 6 shows an example of the flow of the learning rate determination process by the learning rate estimation unit 30. As shown in Figure 6, first, the acquisition unit 31 acquires a second sample (step SB1). The second sample is acquired from a second training dataset containing multiple second samples. In step SB1, one mini-batch of second samples is acquired. The second sample is assigned a teaching label. Assume that the second training dataset is a noisy dataset containing mislabels. However, assume that the intensity of the label noise is not strong. As an example, assume that 80% of the second training dataset has true labels and the remaining 20% ​​has mislabels. The second sample may be the same as or different from the first sample.

[0053] When step SB1 is performed, the label changing unit 32 artificially changes the initial labels of the second samples obtained in step SB1 (step SB2). In step SB2, the label changing unit 32 artificially randomizes the initial labels to create a second training dataset with very strong label noise. Specifically, the label changing unit 32 artificially changes true labels to false labels and false labels to true labels for a randomly selected portion of the multiple second samples. The artificially changed initial labels are called artificial labels. That is, some second samples are assigned both initial labels and artificial labels, while others are assigned only the initial labels.

[0054] When step SB2 is performed, the learning unit 33 uses the artificial labels obtained in step SB2 as training data and performs supervised learning on the second classification model at multiple learning rates (step SB3). The second classification model is assumed to be a deep neural network having arbitrary network layers such as fully connected layers, convolutional layers, self-attention layers, and / or pooling layers. The second classification model is untrained. The network structures of the second classification model and the first classification model may be the same or different. The learning rate can be arbitrarily specified by the user.

[0055] As an example, the learning unit 33 performs supervised learning on the second classification model using initial labels and artificial labels as teachers for each of several pre-set learning rates. The learning unit 33 inputs a second sample into the second classification model and calculates the inference probability corresponding to the second sample by applying a feedforward process to the input second sample according to the network structure of the second classification model. The inference probability calculated during the supervised learning process is stored in relation to the number of epochs. As another example, the learning unit 33 may perform supervised learning using cosine annealing, which changes the learning rate in a cosine curve manner according to the number of epochs.

[0056] When step SB3 is performed, the optimal learning rate determination unit 34 determines the optimal learning rate that avoids overfitting based on the change in the behavior of inference probabilities between multiple learning rates (step SB4). The optimal learning rate determination unit 34 determines the optimal learning rate based on the degree of deviation between the first learning curve, which represents the change in the first output evaluation metric with respect to the change in the number of updates, and the second learning curve, which represents the change in the second output evaluation metric. The first output evaluation metric is the loss based on the correct label and the inference probability, and the second output evaluation metric is the loss based on the incorrect label and the inference probability.

[0057] Figure 7 illustrates the behavior of inference probability across multiple learning rates. Each graph in Figure 7, from left to right, shows the change in loss over time for learning rates η = 0.05, 0.02, 0.01, and 0.005. The vertical axis of each graph represents loss, and the horizontal axis represents the number of epochs. The thin line represents the loss for mislabeled labels measured with the correct labels, i.e., the first learning curve for the first output evaluation metric. The first learning curve (thin line) represents the loss measured with the initial labels. The thick line represents the loss for mislabeled labels measured with the incorrect labels, i.e., the second learning curve for the second output evaluation metric. The second learning curve (thick line) represents the loss measured with artificial labels. The dotted line represents the loss for correct labels, i.e., the third learning curve.

[0058] As shown in Figure 7, the second learning curve (thick line) behaves similarly to the third learning curve (dotted line), regardless of the learning rate η. When the learning rate is relatively large, such as η = 0.05 or 0.02, overfitting to label noise does not occur, so the first learning curve (thin line) and the second learning curve (thick line) behave similarly. However, when the learning rate is relatively small, such as η = 0.01 or 0.005, overfitting to label noise occurs, and as learning progresses (as the number of epochs increases), the first learning curve (thin line) and the second learning curve (thick line) diverge.

[0059] The optimal learning rate determination unit 34 determines the optimal learning rate based on the degree of deviation between the first learning curve and the second learning curve. As an example, for each of the multiple learning rates, the optimal learning rate determination unit 34 measures the deviation between the first learning curve and the second learning curve, i.e., the difference in loss, at a predetermined number of epochs (hereinafter referred to as the measurement epoch number). The measurement epoch number refers to the number of epochs at which a significant deviation has been empirically recognized when overfitting occurs. The measurement epoch number can be arbitrarily set according to user instructions, etc. Next, the optimal learning rate determination unit 34 generates a graph (deviation-learning rate graph) plotting the measured deviation in a two-dimensional coordinate space defined by the learning rate and the deviation. The deviation-learning rate graph refers to a graph relating to curves that represent the change in deviation for each of the multiple learning rates.

[0060] Figure 8 illustrates a deviation-learning rate graph. As shown in Figure 8, the deviation-learning rate graph has the learning rate on the horizontal axis and the deviation on the vertical axis. In the deviation-learning rate graph shown in Figure 8, five measurement points P11 to P15 are plotted as an example. The optimal learning rate determination unit 34 calculates a fitting curve C1 based on the five measurement points P11 to P15, and calculates the point where the fitting curve C1 intersects the horizontal axis as the optimal learning rate LR1. That is, the learning rate at which the deviation becomes zero (reference value) is calculated as the optimal learning rate LR1. Alternatively, the optimal learning rate determination unit 34 may calculate a learning rate that is higher by a margin value than the intersection point as the optimal learning rate. The margin value can be set to any value. The fitting curve C1 may be a linear function or a higher-order function of degree two or higher. Furthermore, the optimal learning rate LR1 is not limited to the learning rate at which the deviation becomes zero; it should be set to a learning rate that is less than or equal to any arbitrary threshold value, such as the lowest deviation obtained with multiple learning rates. The threshold value should be set to the upper limit of the deviation required to obtain a learning rate that avoids overfitting.

[0061] The optimal learning rate determination unit 34 may also determine the optimal learning rate according to user instructions. For example, the display control unit 50 displays the divergence-learning rate graph generated by the optimal learning rate determination unit 34 on the display device 15. It is preferable that measurement points be drawn on the divergence-learning rate graph, but the fitting curve C1 may or may not be drawn. The input device 13 specifies the optimal learning rate for the displayed divergence-learning rate graph according to user instructions. For example, the user analyzes the displayed divergence-learning rate graph and specifies a point on the divergence-learning rate graph corresponding to a desired learning rate via the input device 13. The optimal learning rate determination unit 34 determines the learning rate corresponding to the specified point as the optimal learning rate. For example, a desired point on the horizontal axis can be specified. The learning rate corresponding to the specified point is determined as the optimal learning rate. In this case, the locations that can be specified are not limited to those corresponding to the learning rates for which measurement points P11 to P15 were obtained; locations corresponding to learning rates for which measurement points P11 to P15 were not obtained may also be specified.

[0062] As another example, the display control unit 50 may display multiple graphs corresponding to multiple learning rates, as shown in Figure 7. In this case, the user specifies the graph corresponding to the desired learning rate via the input device 13. The optimal learning rate determination unit 34 determines the learning rate corresponding to the specified graph as the optimal learning rate.

[0063] Other methods for determining the optimal learning rate are described below. The optimal learning rate determination unit 34 may compare the degree of agreement between the teaching label and the class with the highest inference probability for samples where the teaching label matches before and after the change, with the degree of agreement between the class with the highest inference probability and the teaching label before the change for samples where the labels do not match before and after the change, and determine the optimal learning rate as a learning rate that causes a divergence between the two, or a learning rate that is higher by a margin value than that learning rate. Alternatively, the optimal learning rate determination unit 34 may measure the degree of agreement between the class with the highest inference probability and each teaching label for samples where the teaching labels do not match before and after the change, and focus on which is higher to find a learning rate that reverses the result of a higher learning rate. The degree of agreement may be calculated by counting and averaging binary judgment results, such as accuracy, or a loss function, typically the value of cross-entropy, may be used. A learning rate that exhibits such behavior is estimated to be a learning rate that causes overfitting.

[0064] Once step SB4 is performed, the learning rate determination process by the learning rate estimation unit 30 is completed.

[0065] Figure 9 shows an example of the overall processing flow by the machine learning device 1. The overall processing shown in Figure 9 starts when the processor 11 reads and executes the program. As shown in Figure 9, first the learning rate estimation unit 30 determines the optimal learning rate (step SA). After step SA is performed, the label correction unit 20 determines the optimal teaching label based on the optimal learning rate determined in step SA (step SB). As described above, the label correction unit 20 outputs a one-hot label, which is a hard label of the teaching label at the number of culling updates, as the optimal teaching label.

[0066] When step SB is performed, the label-tolerant noise learning unit 40 performs supervised learning on the third classification model under the optimal teaching labels determined in step SB (step SC). The third classification model is assumed to be a deep neural network having arbitrary network layers such as fully connected layers, convolutional layers, self-attention layers and / or pooling layers. The third classification model is untrained. The network structure of the third classification model and the first and second classification models may be the same or different.

[0067] The label-tolerant noise learning unit 40 performs supervised learning using the optimally taught labels as training data, enabling high-quality supervised learning. In this case, the label-tolerant noise learning unit 40 can perform supervised learning on the third classification model at any learning rate. For example, the label-tolerant noise learning unit 40 may perform supervised learning at a learning rate smaller than the optimal learning rate. In supervised learning with little label noise, overfitting to the original taught labels at a small learning rate contributes to improved generalization performance, so there is value in learning down to a low learning rate. Upon completion of supervised learning, the trained classification model is output. As another example, the learning rate may be changed according to the number of epochs, similar to cosine annealing.

[0068] Once step SC is performed, the entire process by machine learning device 1 is completed.

[0069] The above embodiment is merely an example, and various processes can be added, deleted, and / or modified. Modifications of this embodiment will be described below.

[0070] (Variation 1) In the embodiments described above, the network structure and sample types of each classification model are not particularly limited. The classification model may be a convolutional neural network or a transformer-type structure. However, it is assumed that each sample can be assigned a teaching label. This is most typically the case when teaching labels are assigned in advance, but it may also be a semi-supervised dataset where some samples have teaching labels. Similar processing can be achieved by performing a certain amount of training only on the samples with teaching labels. Furthermore, even in an unsupervised situation where not all samples have teaching labels, a standard unsupervised learning method such as SimCLR can be used to assign pseudo-labels. Therefore, it is not limited to whether or not each sample has a teaching label.

[0071] This embodiment may be combined with unsupervised learning. Specifically, unsupervised learning such as SimCLR or MoCo is first performed on the dataset to be used. Next, classification, i.e., pseudo-label generation, is performed using clustering algorithms such as k-means or DBSCAN. This makes it possible to determine the optimal learning rate and update the teaching labels using the obtained pseudo-labels.

[0072] Furthermore, the same can be done with more specific label settings. For example, this embodiment can be applied to positive-unlabeled (PU) learning in positive-negative two-class classification where some samples are known to be positive, but the rest are unlabeled. For example, by substituting unlabeled samples with randomly assigned positive or negative training labels, it is possible to determine the optimal learning rate and update the training labels. However, since randomness is introduced twice in determining the optimal learning rate, it is advisable to perform statistical processing after assigning training labels several times.

[0073] (Modification 2) If it is known that some noise-free samples are included among the noisy samples, then determining the optimal learning rate can be performed with greater accuracy by using only the noise-free samples. This situation corresponds to a case where there is a large number of samples labeled by unskilled individuals, and some of them are then labeled by skilled individuals.

[0074] If the noise intensity is known to some extent, that is, if the proportion of mislabeled samples in the dataset is known, the label correction unit 20 may assign priorities to the samples whose labels should be updated and update only the teaching labels of samples within the specified proportion of the top priority. The specified proportion of the top priority may be inferred from the results of labeling by experts and / or inexperienced users as described above, or from the recall rate during actual operation.

[0075] (Variation 3) The update unit 24 according to the above embodiment updates the teaching label based on the inference probability. The update unit 24 according to Modification 3 may calculate a combined label of a first label for the first sample to be corrected from the first sample and a first label for another first sample different from the first sample to be corrected, and update the label to be corrected based on the calculated combined label and the first model output. The update unit 24 according to Modification 3 will be described below. The first model output is assumed to be the inference probability.

[0076] Specifically, the update unit 24 identifies multiple samples whose model output approximates the sample to be corrected as other first samples, and calculates statistical values ​​of multiple teaching labels corresponding to each of the multiple first samples as teaching labels for those other first samples. As an example, the update unit 24 calculates the average value of the first teaching labels assigned to k samples that show inference probabilities close to the inference probability of the first sample to be corrected, and updates the teaching labels using the calculated average value. This method is called neighborhood label averaging. The average value may be a weighted average according to distance rather than a simple average, or it may be the average of inference probabilities instead of teaching labels. Furthermore, the distance may be measured not only by Euclidean distance, but also by Jensen-Shannon divergence or symmetric Kullback-Leibler divergence. The distance may also be calculated based on feature vectors instead of inference probabilities. Alternatively, the update unit 24 may calculate the teaching label for each sample based on a Gaussian mixture model.

[0077] (Modification 4) In the above embodiment, the update unit 24 updates the teaching label based on a moving average. However, this embodiment is not limited to this. For example, if the one-hot vector of the teaching label before the update and the one-hot vector of the inferred probability are different, the update unit 24 calculates the moving average of the teaching label before the update and the inferred probability as the teaching label after the update. If they match, the update unit 24 calculates the moving average of the teaching label before the update and the one-hot vector of the inferred probability as the teaching label after the update. This makes it possible to update labels stably even if there is some error in the estimate of the number of truncation updates. Note that a simple moving average, weighted moving average, exponential moving average, or any other moving average can be used as the moving average.

[0078] (Variation 5) The update determination unit 23 according to the above embodiment calculates the number of updates to be terminated based on the inferred probability in a certain epoch. However, this embodiment is not limited thereto. The update determination unit 23 according to Modification 5 calculates the number of updates to be terminated for the current update count as a moving average of the number of updates to be terminated provisionally calculated for the current update count and the number of updates to be terminated calculated for updates prior to the current update count. Note that a simple moving average, weighted moving average, exponential moving average, or any other moving average can be used as the moving average.

[0079] For example, the update determination unit 23 may start with a termination update count calculated at a certain update count, and for subsequent update counts, it may calculate the termination update count for the current update count as a moving average of the termination update count calculated at the current update count and the termination update count calculated up to the previous update count. This eliminates instability that arises from referring only to a specific update count. In this case, a constant may be assigned to the termination update count at the update count when the calculation of the termination update count begins, so that the number of label updates is above a certain level.

[0080] (Experimental variation 6) The update determination unit 23 according to the above embodiment calculates the number of updates to be terminated based on a moving average and generalizability. However, this embodiment is not limited thereto. In the modified example 6, the update determination unit 23 may, for example, calculate the difference between the teaching label for the current update count and the teaching label for the next update count, and calculate the number of updates to be terminated based on the calculated difference. As described above, the number of updates to be terminated means when the hard label of the teaching label changes from the hard label of the initial label, so for example, if the difference is large compared to a threshold, the update determination unit 23 may set the current update count to the number of updates to be terminated. This method can be used in situations other than label updates based on a moving average.

[0081] In the above explanation, the number of updates to terminate is stated to be when the hard label changes, but it may also be performed when the label has come close enough to be a hard label. For example, the update determination unit 23 may terminate the label update when the amount of Kullback-Libra information from the hard label falls below a threshold.

[0082] (Example 7) In the above embodiment, the update determination unit 23 does not resume label updates once the number of updates reaches the termination update count. However, this embodiment is not limited to this. In the modified example 7, the update determination unit 23 may resume updates at regular intervals. The number of epochs until the update is resumed should be set to a value greater than 1 / m, which is a typical number of updates calculated from the moving average parameter m. By resuming label updates, even if the classification model changes significantly during the learning process, it becomes possible to obtain an appropriate teaching model that follows the classification model.

[0083] (Variation 8) In the above embodiment, the update determination unit 23 does not specify the percentage of data with different labels between the initial label and the optimal teaching label, i.e., the relabeling rate. However, this embodiment is not limited to this. In the modified example 8, the update determination unit 23 may terminate the label update when the specified relabeling rate is reached. For example, the update determination unit 23 calculates the degree of agreement between the initial label including label noise and the current teaching label (pseudo-label after label update), and if the change in the degree of agreement is not small compared to a threshold, it determines not to terminate the label update and continues the label update. On the other hand, the update determination unit 23 determines to terminate the label update when the change in the degree of agreement becomes small compared to a threshold. That is, the number of updates at which the change in the degree of agreement becomes small compared to a threshold corresponds to the number of updates to terminate. If the percentage of mislabeled data is known, the determination may also be made based on how close it has come to the relabeling rate corresponding to that percentage. According to the modified example 8, by adding other termination criteria, it becomes possible to terminate the label update at a more appropriate time.

[0084] (Extreme variation 9) In the above embodiment, the optimal learning rate determination unit 34 uses the second model output, which is the output of the second classification model, as the output evaluation index used to determine the optimal learning rate. However, this embodiment is not limited to this. In the modified example 9, the optimal learning rate determination unit 34 uses the error of the second model output as the output evaluation index. Specifically, the optimal learning rate determination unit 34 applies the second sample to the second classification model to calculate the inference probability, which is the second model output, and calculates the error between the calculated inference probability and the teaching label. This may be a loss obtained by applying the inference probability and the teaching label to a loss function, or it may be cross-entropy based on the loss. Since only one cross-entropy is calculated for the inference probability vector and the teaching label vector, it is possible to easily determine the optimal learning rate compared to the inference probability obtained for each class.

[0085] (effect) Figure 10 shows the trend of the accuracy of pseudo-labels for the comparative example. For the comparative example, the accuracy was verified when the pseudo-labels were updated at predetermined intervals using the exponential moving average of equation (1) above, under a constant learning rate. In the left and right figures of Figure 10, the vertical axis represents the accuracy of the pseudo-labels, and the horizontal axis represents the number of epochs. The left figure of Figure 10 shows the trend of the accuracy when hard labels (one-hot labels) were used as pseudo-labels, with m=1 and the pseudo-labels were updated every 50 epochs. The right figure shows the trend of the accuracy when soft labels were used as pseudo-labels, with m=0.03 and the pseudo-labels were updated every epoch. Curve TT represents the accuracy of pseudo-labels with correct labels, curve FT represents the accuracy of pseudo-labels with incorrect labels, and curve T represents the average accuracy of pseudo-labels with all labels. The dataset used was CIFAR-10, and the classification model used was ResNet-18. 20% of the sample was misclassified equally into other classes.

[0086] Comparing the left and right figures in Figure 10, it can be seen that updating soft labels generally improves accuracy compared to updating hard labels. However, as the number of epochs increases, the accuracy of pseudo-labels deteriorates, and the accuracy decreases.

[0087] Figure 11 shows the trend of the accuracy of pseudo-labels according to this embodiment. The first column from the left in Figure 11 is the accuracy when pseudo-labels are updated at predetermined intervals using only the moving average, i.e., the exponential moving average of equation (1) above, which is the same as the comparative example in Figure 10. The second column is the first column (moving average) with added conditioning. The conditioning is the method according to Modification 4 without censorship. The third column is the second column (moving average + conditioning) with added censorship. Censorship is an embodiment in which label updates are stopped according to the number of censorship updates by the label correction unit 20. The fourth column is the third column (moving average + conditioning + censorship (Modification 4)) with added neighbor label average. The neighbor label average is the neighbor label average according to Modification 3. The dataset used was CIFAR-10, and the classification model used was ResNet-18. The learning rate was the optimal learning rate. The upper part of Figure 11 shows that the percentage of mislabels in the dataset is 20%, indicating relatively low noise. The lower panel shows that the mislabeling rate in the dataset is 45%, indicating relatively high noise levels.

[0088] Comparing the first column of Figure 11 with columns 2-4, this embodiment (columns 2-4) generally suppresses the decline in accuracy with the passage of epochs compared to the comparative example (column 1). In particular, the method in column 3 is effective with weak noise, the method in column 2 is effective with strong noise, and the method in column 4 is effective with both weak and strong noise.

[0089] Thus, according to the above embodiment, it becomes possible to improve the reliability of labels used to train a deep learning model that processes classification problems.

[0090] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0091] 1...Machine learning device, 11...Processor, 12...Storage device, 13...Input device, 14...Communication device, 15...Display device, 20...Label correction unit, 21...Acquisition unit, 22...Learning unit, 23...Update determination unit, 24...Update unit, 30...Learning rate estimation unit, 31...Acquisition unit, 32...Label change unit, 33...Learning unit, 34...Optimal learning rate determination unit, 40...Label noise tolerance learning unit, 50...Display control unit.

Claims

1. A first learning unit obtains a first model output including inference probabilities and / or feature vectors output from a first deep learning model by applying a first sample to a first deep learning model that processes a classification problem. A determination unit that determines whether or not it is necessary to update the labels used as training labels for the first deep learning model, based on the output of the first model and / or the labels, If it is determined that the label needs to be updated, the update unit updates the label based on the first model output and the current update count label, and if it is determined that the label does not need to be updated, the update unit terminates the label update. A machine learning device equipped with the following features.

2. The determination unit, Based on the first model output and / or the label, calculate the first update count for stopping the updating of the label. Based on a comparison of the current update count and the first update count, it is determined whether or not the label needs to be updated. The machine learning apparatus according to claim 1.

3. The machine learning apparatus according to claim 2, wherein the determination unit calculates a specific number of updates at which the class corresponding to the maximum value among the plurality of component values ​​constituting the vector of labels before the update changes to the class corresponding to the maximum value among the plurality of component values ​​constituting the inference probability vector which is the first model output, as the first update count.

4. The machine learning apparatus according to claim 3, wherein the determination unit calculates the first update count based on the specific update count calculated in the current update count and the specific update count calculated in an update count prior to the current update count.

5. The machine learning apparatus according to claim 2, wherein the determination unit calculates the first update count based on the difference between the label at a predetermined update count and the label at an update count prior to the predetermined update count.

6. The machine learning apparatus according to claim 1, wherein if the update unit determines that it is not necessary to update the label, it terminates the update of the label and modifies the label of the current update count into a hard-labeled one-hot label.

7. The machine learning apparatus according to claim 1, wherein, when the update unit determines that the label needs to be updated, it calculates the label for the next update count based on the first model output for the current update count and the label for the current update count.

8. The determination unit determines whether or not the label needs to be updated for each of the first samples. The update unit updates the label for each of the first samples. The machine learning apparatus according to claim 1.

9. The machine learning apparatus according to claim 7, wherein the update unit calculates an integrated label of the label for the first sample to be modified from the first samples and the label for another first sample different from the first sample to be modified, and updates the label to be modified based on the integrated label and the first model output.

10. The machine learning apparatus according to claim 9, wherein the update unit identifies a plurality of samples whose model output approximates the sample to be modified as the other first samples, and calculates statistical values ​​of a plurality of labels corresponding to each of the plurality of first samples as labels for the other first samples.

11. A second acquisition unit for acquiring a second sample, A modification unit that changes the initial label of the second sample to an artificial label, which is a label that has been artificially altered from the initial label, A second learning unit performs supervised learning on a second deep learning model that processes a classification problem using the aforementioned artificial labels as training data, with multiple different learning rates. A unit for determining a specific learning rate that can avoid overfitting based on changes in the behavior of an output evaluation metric between the plurality of learning rates, wherein the output evaluation metric is a second model output output from the second deep learning model by applying the second sample to the second deep learning model, or the error of the second model output, further comprising: The machine learning apparatus according to claim 1.

12. The determination unit determines the specific learning rate based on the degree of deviation between the curve representing the change in the first output evaluation index accompanying the change in the number of updates and the curve representing the change in the second output evaluation index. The first output evaluation index is the error between the correct initial label, which is accurate in light of the actual second sample, and the inference probability, which is the second model output. The second output evaluation index is the error between the mislabel, which is the initial label that is inaccurate in light of the actual second sample, and the inference probability, which is the second model output. The machine learning apparatus according to claim 11.

13. The machine learning apparatus according to claim 12, wherein the first learning unit acquires the first model output under the specific learning rate.

14. The machine learning apparatus according to claim 12, wherein the determination unit determines a learning rate in which the deviation falls below a reference value as the specific learning rate.

15. A display control unit that displays a graph on a display device relating to a curve representing the change in the deviation for each of the plurality of learning rates, The system further includes a designation unit that specifies the particular learning rate for the displayed graph according to the user's instructions, The machine learning apparatus according to claim 12.

16. Furthermore, a third learning department will be added. If the update unit determines that the label does not need to be updated, it terminates the label update, hard labels the current update count label to generate a one-hot label, The third learning unit applies supervised learning to a third deep learning model that processes a classification problem, using the one-hot label as the teacher. The machine learning apparatus according to claim 1.

17. The processor, A learning step of obtaining a first model output including inference probabilities and / or feature vectors output from a first deep learning model by applying a first sample to a first deep learning model that processes a classification problem, A determination step of determining whether it is necessary to update the labels used as training for the first deep learning model, based on the output of the first model and / or the labels, If it is determined that the label needs to be updated, the update step updates the label based on the first model output and the current update count label; if it is determined that the label does not need to be updated, the update step terminates. A machine learning method that possesses the following characteristics.

18. In the processor, A learning function that obtains a first model output including inference probabilities and / or feature vectors output from a first deep learning model by applying a first sample to a first deep learning model that processes a classification problem, A determination function that determines whether or not the labels used as training material for the first deep learning model need to be updated, based on the output of the first model and / or the labels, If it is determined that the label needs to be updated, the update function updates the label based on the first model output and the current update count label; if it is determined that the label does not need to be updated, the update function terminates the label update. A machine learning program that makes this possible.

Images