Learning device, learning method, and program

Abstract

The present invention provides a learning device, etc. for estimating rules that adversely affect a machine learning system in weakly supervised learning and presenting the rules to a subject. A learning device according to the present invention is provided with: a set of labeling functions for assigning labels to an input data set; and a discriminator that performs learning on the basis of the labeling functions. The learning device presents labeling functions to be modified or removed from the set of labeling functions on the basis of outputs from the set of labeling functions and outputs from the discriminator.

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, and a program for learning a classifier through machine learning. [Background technology]

[0002] Machine learning, particularly so-called supervised learning, is becoming widespread in a wide range of fields. In supervised learning, a training dataset is prepared in advance and a classifier is trained based on the training dataset. A training dataset is an input dataset to be analyzed, to which each judgment result (correct answer) such as discrimination, classification, regression, or identity is attached. Here, the cost required to create an input dataset (training dataset) with annotated correct answers becomes an issue in building machine learning systems.

[0003] To solve this problem, active learning and weakly supervised learning have been proposed, which involve adding training datasets through computational processing.

[0004] In active learning, an existing training dataset and classifier are used to present an input dataset from a group of input datasets without a correct answer, which will improve the performance of the classifier if the correct answer is known. The correct answer is annotated to the presented input dataset and added to the training dataset.

[0005] In weakly supervised learning, a function is implemented in the system that uses the knowledge of the entity that assigns the correct answer as a rule, and the system assigns the correct answer to the input dataset according to the function. The input dataset with the correct answer is added to the training dataset.

[0006] Non-Patent Document 1 describes a technique for sequentially adding rules through active learning in weakly supervised learning, which allows for efficient rule addition. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Osamu Saisho et al., "Human Knowledge Based Efficient Interactive Data Annotation via Active Weakly Supervised Learning", 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), 2021. Summary of the Invention [Problem to be solved by the invention]

[0008] However, in the real world, rules added by subjects are likely to contain errors. In fact, it has been shown that a certain percentage of human error occurs even in training datasets constructed using normal labeling, which is much simpler than adding rules. In weakly supervised learning, when rules containing errors are added, the subject cannot be prompted to discover the errors or correct or delete them, and they accumulate as noise, which can significantly degrade the performance of the machine learning system.

[0009] Existing technology only provides a framework for efficiently adding new rules, but does not provide a method for identifying rules that have already been added and are adversely affecting the machine learning system. This means that an environment for trial and error, which is essential when building a machine learning system, cannot be realized.

[0010] The present invention aims to provide a learning device, a learning method, and a program that estimate rules that are adversely affecting a machine learning system in weakly supervised learning and present them to a subject. [Means for solving the problem]

[0011] In order to solve the above problem, according to one aspect of the present invention, a learning device includes a group of labeling functions that assign labels to an input data set, and a classifier that performs learning based on the labeling functions, and presents labeling functions that should be modified or deleted from the group of labeling functions based on the output of the group of labeling functions and the output of the classifier.

[0012] In order to solve the above problem, according to another aspect of the present invention, a learning device includes a labeling function processing unit that uses a labeling function group including one or more labeling functions to obtain a voting result for each labeling function for input data included in an input dataset; a second probability distribution calculation unit that uses the input data as an input for a trained classifier and obtains an output probability distribution based on the trained classifier; a probability distribution calculation unit that obtains an output probability distribution of the labeling function using the voting result; and a score s indicating the need for correction or deletion of the input data using the output probability distribution based on the trained classifier and the output probability distribution of the labeling function. k (p) is calculated and the score is s k The system includes an evaluation unit that calculates a score s(i) from (p) that indicates the need for modification or deletion in the labeling function, a presentation information generation unit that generates information for presenting to the subject the labeling function whose score s(i) indicates the highest need for modification or deletion, and a labeling function modification unit that accepts input from the subject to whom the labeling function has been presented and deletes or modifies the labeling function according to the input content. [Effects of the Invention]

[0013] According to the present invention, it is possible to estimate rules that are adversely affecting a machine learning system in weakly supervised learning and present the rules to a subject. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a functional block diagram of a learning device according to a first embodiment. [Figure 2] FIG. 4 is a diagram showing an example of a processing flow of the learning device according to the first embodiment. [Figure 3]FIG. 4 is a diagram showing an example of a processing flow of the learning device according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of the configuration of a computer to which the present technique is applied. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described. In the drawings used in the following description, components having the same functions and steps performing the same processing are denoted by the same reference numerals, and duplicated explanations will be omitted. In the following description, processing performed on each element of a vector or matrix is ​​assumed to apply to all elements of that vector or matrix, unless otherwise specified.

[0016] <Features of the first embodiment> This embodiment has the following features.

[0017] Feature (1): In a real-world environment where accurate supervised information is not available, a machine learning system using weakly supervised learning can automatically identify and extract from among the implemented rules those that are likely to contain errors.

[0018] Feature (2): Feature (1) above enables active learning in weakly supervised learning to be bidirectional, with addition, modification, and deletion, making it possible for the subject and the machine learning model to work together to improve the performance of the machine learning model through trial and error. The subject can be, for example, a human, AI, or machine, and is the entity that performs the learning of the machine learning model.

[0019] <Active Learning in Weakly Supervised Learning of the First Embodiment> In active learning under weak supervision, the following three types of output probability distributions are obtained during learning:

[0020] Probability distribution based on the raw output of the labeling function before training Output probability distribution after weighting the labeling function after learning Output probability distribution based on the trained classifier In this embodiment, since the modification or deletion of the labeling function and the learning of the machine learning model are repeated, the machine learning model learned in the previous iteration of a certain iteration is referred to as "post-learning." If there is no previous iteration, that is, in the first iteration, a machine learning model trained using another learning method (for example, the learning method of Non-Patent Document 1) may be used, or a machine learning model with random parameters may be used. A model that has completed the final learning is referred to as "trained." The "probability distribution based on the raw output of the labeling function before learning" and the "output probability distribution after weighting the labeling function after learning" are collectively referred to as the "output probability distribution of the labeling function." A labeling function is a function that uses rules based on the subject's knowledge and assigns labels to input data. In an ideal environment where the implemented labeling function is free of errors, inconsistencies, and overlaps, the three probability distributions mentioned above will be roughly consistent during model training. However, it is not realistic to continually create such rules, and the probability of errors differing will vary depending on the subject and the difficulty of the task. Therefore, of the three output probability distributions mentioned above, two are selected: "a probability distribution based on the raw output of the labeling function before training" or "an output probability distribution after weighting the labeling function after training," and "an output probability distribution based on the learning device after training." Evaluation is performed using a measure of the difference between these two probability distributions.

[0021] When adding data using Bayesian active learning in conventional supervised learning, the BALD (Bayesian Active Learning by Disagreement) score for each data is calculated based on the mutual information between the parameters of the discriminative model and the output probability distribution, and data is labeled in descending order of the score.

[0022] In this embodiment, when extracting labeling functions to be modified or deleted, a score indicating the necessity is first calculated for each input data. Specifically, the scale representing the difference is used as the score for each input data. Furthermore, the score for each input data is used to calculate the score for each labeling function to be evaluated.

[0023] Based on the score calculated for each labeling function, the labeling function that should be most modified or deleted is estimated, extracted, and presented to the subject.

[0024] After the subject makes a decision (delete, modify, or do nothing) and completes the work, the machine learning model is trained again using the updated labeling function set, just as after adding the labeling function through active learning (for example, the learning method in Non-Patent Document 1).

[0025] <Learning device 100 according to the first embodiment> FIG. 1 is a functional block diagram of a learning device 100 according to the first embodiment, and FIGS. 2 and 3 show the processing flow thereof.

[0026] The learning device 100 includes a labeling function processing unit 110, a first probability distribution calculation unit 120, a classification unit 140, an evaluation unit 150, a presentation information generation unit 160, a labeling function correction unit 170, and a parameter update unit 180.

[0027] The learning device 100 receives an input data set D = (d(1), d(2), ..., d(M)), performs active learning in weakly supervised learning, presents a labeling function to be modified or deleted to a subject, accepts a request from the subject to modify or delete the labeling function, and modifies the labeling function. The input data set D includes M pieces of input data d(m), where m = 1, 2, ..., M.

[0028] The learning device 100 receives an input data set D q =(d q (1),d q (2),…,d q (Q)) is used as input, and the machine learning model is trained again using the labeling function set after modifying or deleting the labeling function. Input dataset D q contains Q input data d(q), where q=1, 2, …, Q. The input data sets D and D qThe input datasets may be the same or different datasets, and neither dataset has a correct answer.

[0029] The labeling function is modified or deleted and the machine learning model is trained repeatedly, and the classifier that satisfies predetermined conditions is output as a trained classifier.

[0030] The learning device 100 is a special device configured by loading a special program into a publicly known or dedicated computer having, for example, a central processing unit (CPU) and a main memory (RAM). The learning device 100 executes each process under the control of the central processing unit. Data input to the learning device 100 and data obtained from each process are stored in, for example, the main memory. The data stored in the main memory is read by the central processing unit as needed and used for other processes. At least a portion of each processing unit of the learning device 100 may be configured with hardware such as an integrated circuit. Each memory unit included in the learning device 100 can be configured with, for example, a main memory such as a random access memory (RAM), or middleware such as a relational database or key-value store. However, each memory unit does not necessarily need to be included inside the learning device 100. It may be configured with an auxiliary memory device including a hard disk, optical disk, or semiconductor memory element such as flash memory, and be configured external to the learning device 100.

[0031] Each part will be explained below.

[0032] <Labeling function processing unit 110> The labeling function processing unit 110 includes a labeling function group. The labeling function group includes L labeling functions, where L is any integer equal to or greater than 1. A labeling function is a function that assigns a label to input data according to a rule; assigning a label is called a vote, and not being able to assign a label is called an abstention. The output of a labeling function is also called a voting result, and the voting result also includes a value for the case of abstention.

[0033] The labeling function processing unit 110 receives an input data set D = (d(1), d(2), ..., d(M)), calculates a voting result v(m) = (v(m,1), v(m,2), ..., v(m,L)) for each labeling function for each input data d(m) (S110), and outputs v(1), v(2), ..., v(M). The voting result of the ith labeling function LF(i) for the input data d(m) is defined as v(m,i), where i = 1, 2, ..., L.

[0034] <First Probability Distribution Calculation Unit 120> The first probability distribution calculation unit 120 receives M voting results v(m) as input and calculates a probability distribution p(y b ) is calculated (S120) and output. The probability distribution corresponding to the voting result v(m) is calculated as p(m,y b ) and p(y b )=(p(1,y b ),p(2,y b ),…,p(M,y b When N types of labels are assigned, the probability distribution p(m,y b )=(p(m,y b ,1),p(m,y b ,2),…,p(m,y b ,N)). The probability distribution p(m,y b) is obtained by obtaining the number of votes for each label from the voting result v(m) for input data d(m), and dividing the number of votes for each label by the total number of votes. For example, if input data d(m) is assigned a label of "0", "1", or "2", and abstention is set to "-1", L=5, and the voting result v(m) is (0,0,1,-1,2), then from the four voting results excluding abstentions, the number of votes for label "0", 2, is divided by the total number of votes, 4, to determine the probability of label "0", and similarly, the probability of "1" is set to 0.25 and the probability of "2" is set to 0.25, and the probability distribution p(m,y b )=(0.5,0.25,0.25).

[0035] <Identification unit 140> The classifier 140 includes a trained classifier, and receives M voting results v(m) and an input data set D=(d(1), d(2), . . . , d(M)).

[0036] The identification unit 140 includes a second probability distribution calculation unit 145 and a third probability distribution calculation unit 147. The identification unit 140 corresponds to a neural network model including a soft-attention structure, with the classifier of this neural network model corresponding to the second probability distribution calculation unit 145 and the attention part corresponding to the third probability distribution calculation unit 147. In this example, the neural network model including a soft-attention structure is a model equipped with two linear transformation layers in the classifier and two linear transformation layers in the attention part as well. The neural network model including a soft-attention structure performs learning based on the labeling functions of the labeling function processing unit 110, and only input data that has been voted for by any of the labeling functions is used for learning as weakly labeled data. An input data set that has been voted for by any of the labeling functions is referred to as D k =(d k (1),d k (2),…,d k (P)), where P is the number of input data voted for by any labeling function.

[0037] The classification unit 140 uses the M voting results v(m) to extract input data that has been voted for by any of the labeling functions, and classifies the extracted input data set D k =(d k (1),d k (2),…,d k (P)) is used as weakly labeled data for training. In other words, from the input dataset D = (d(1), d(2), ..., d(M)), we filter out input data whose voting results v(m) are all abstentions.

[0038] The second probability distribution calculation unit 145 calculates the second probability distribution of the input data set D k =(d k (1),d k (2),…,d k (P)) is the input of the trained classifier, and the output probability distribution p(y c )(The output probability distribution p(y c ) is calculated (S145) and output. The third probability distribution calculation unit 147 receives M voting results v(m) as input, and calculates an output probability distribution p(y a ) is calculated (S147) and output.

[0039] <Evaluation Unit 150> The evaluation unit 150 calculates the output probability distribution p(y a ) and probability distribution p(y b ) and output probability distribution p(y c ) and M voting results v(m), and each input data d k Score s indicating the need for modification or deletion in (p) k (p) is calculated (S150-1). The score s indicates the need for correction or deletion. k As (p), a measure of the difference between the two probability distributions is used. For example, KL divergence, JS divergence, mutual information, conditional entropy, etc. can be used as a measure of the difference. One of the two probability distributions is the output probability distribution (p(y a) or p(y b )), and the other is the output probability distribution p(y c ) For example, as a score, the output probability distribution p(y c ) and the probability distribution p(y b ) using the KL divergence.

[0040] Next, the evaluation unit 150 evaluates each input data d k From the score indicating the necessity of correction or deletion in (p), a score s(i) indicating the necessity of correction or deletion in each labeling function to be evaluated is calculated (S150-2) and output. Here, p=1, 2, ..., P. This score s(i) is the score s for the input data to which the labeling function has assigned a label without rejection. k The evaluation unit 150 uses the statistical value of (p). Based on the M voting results v(m), the evaluation unit 150 determines whether the labeling function has assigned any label to the input data without abstention. For example, the average, median, maximum, minimum, sum, variance, standard deviation, coefficient of variation, etc. can be used as the statistical quantity. For example, the sum is used as a score indicating the need for correction or deletion in each labeling function.

[0041] In this embodiment, the greater the difference between the two probability distributions, the more it is determined that the labeling function should be modified or deleted. Therefore, the necessity of modifying or deleting the labeling function is determined based on the magnitude of the difference between the two probability distributions for the input data. Specifically, for input data to which the labeling function has assigned a label without rejection, in other words, for input data to which the labeling function contributes to the difference between the two probability distributions, the input data d k Score s indicating the need for modification or deletion in (p) k The statistical value of (p) is the score s(i) that indicates the necessity of correction or deletion in the labeling function. Note that a high necessity of correction or deletion in the labeling function indicates that the labeling function is likely to contain errors.

[0042] <Presentation Information Generator 160, Presentation Unit 165, and Input Unit 166> The presentation information generation unit 160 receives as input a score s(i) indicating the necessity of modification or deletion for each labeling function, generates information (presentation information) for presenting to the subject the labeling function whose score s(i) indicates the highest necessity of modification or deletion (S160), and outputs the information to the presentation unit 165. The labeling function whose score s(i) indicates the highest necessity of modification or deletion can be said to be the labeling function that should be modified or deleted from the group of labeling functions. For example, if a measure (e.g., KL divergence) is used as a measure of difference, the greater the difference between two probability distributions, the larger the value of the measure. Therefore, the greater the difference between the two probability distributions, the larger the statistical value. Therefore, the labeling function with the largest score s(i) is presented to the subject. On the other hand, if a measure (e.g., KL divergence) is used as a measure of difference, the greater the difference between two probability distributions, the smaller the statistical value. Therefore, the labeling function with the smallest score s(i) is presented to the subject.

[0043] The presentation unit 165 is a device for presenting presentation information to the subject, such as a display or a touch panel. Various presentation methods are possible. Examples of presentation methods will be described below. The presentation information is information for realizing each presentation method via the presentation unit 165.

[0044] (Presentation method 1) For example, an editing screen based on the work screen when the subject created the labeling function that most likely needs to be modified or deleted is presented to the subject. With this configuration, the subject is asked whether or not the labeling function needs to be modified or deleted. The subject may modify, delete, or do nothing with the presented labeling function. The subject modifies or deletes the labeling function from the editing screen presented on the presentation unit 165 via the input unit 166. With this configuration, modifications can be made using the same programming language and method as when the labeling function was created. For example, a button for saving the modifications and a button for deleting or disabling the labeling function itself may be added to the editing screen.

[0045] The input unit 166 is a device for receiving input by operating a main body such as a keyboard, a mouse, or a touch panel.

[0046] (Presentation method 2) Along with the labeling function editing screen in Presentation Method 1, it displays one or more pieces of input data with the highest scores for each input data among the input data voted for by the target labeling function. This input data can be said to be the input data that serves as the basis for extracting labeling functions that should be modified or deleted from the group of labeling functions. By displaying data with a high probability of incorrect label voting together with the labeling function, it is expected that the intellectual work cost for modifying and deleting labeling functions will be reduced.

[0047] Furthermore, when displaying multiple input data, it is possible to simply select the data with the highest scores, or to select data by semantic unity by combining techniques such as clustering. The latter method ensures semantic unity and efficiently encourages the subject to modify the labeling function. This method has been shown to be effective in reducing the cost of human intellectual work in batch-based active learning in weakly supervised learning.

[0048] (Presentation method 3) Regarding the display of input data in Presentation Method 2, data is displayed that visualizes which parts of the input data should be focused on. This is expected to further reduce the cost of intellectual work. For example, by combining explainable AI technology such as attention, it is possible to show which parts of each input data have been determined to be incorrect, allowing the subject to make corrections and deletions more efficiently. For example, it is possible to show which parts have been determined to be incorrect by highlighting them, etc.

[0049] <Labeling function correction unit 170> The labeling function correction unit 170 receives input from the subject via the input unit 166, and deletes or corrects the labeling function according to the input content (S170).

[0050] <Parameter update unit 180> After the subject has completed the decision-making and work, the machine learning model is trained again using the updated set of labeling functions, just as after adding the labeling functions through active learning (see Figure 3).

[0051] The learning device 100 receives an input data set D q =(d q (1),d q (2),…,d q (Q)) is input, and steps S110, S145, and S147 are performed to obtain the input data set D q The output probability distribution p(y q,c ) and the output probability distribution p(y q,a ) is found.

[0052] The parameter update unit 180 updates the output probability distribution p(y q,c ) and p(y q,a The parameters of the neural network model including the soft-attention structure of the classifier 140 are updated so that the difference between the two is small (S180).

[0053] The learning device 100 repeats S145, S147, and S180 until a predetermined condition is met (until the conditional branch in S181 becomes YES), and when the predetermined condition is met (when the conditional branch in S181 becomes YES), the neural network model including the soft-attention structure with parameters is set as the machine learning model after learning. The predetermined condition is a condition for determining whether the learning of the neural network model has converged, and may be, for example, that the number of repetitions exceeds a predetermined number, or that the difference between the parameters before and after the update is equal to or less than a predetermined value.

[0054] Furthermore, modification or deletion of the labeling functions and learning of the machine learning model are repeated until a predetermined condition is met (until the conditional branch in S182 becomes YES), and the neural network model including the soft-attention structure having parameters when the predetermined condition is met (when the conditional branch in S182 becomes YES) is defined as the trained machine learning model. The predetermined condition is a condition for determining whether the labeling function group includes a labeling function that is adversely affecting the machine learning system, and could be, for example, whether the number of repetitions exceeds a predetermined number, or whether the score s(i) corresponding to the labeling function that indicates the greatest need for modification or deletion is equal to or less than a predetermined value. Note that the conditional branch in S182 may not be provided and the subject may determine whether the labeling function group is appropriate and end the processing of the learning device 100.

[0055] <Effects> With the above configuration, it is possible to estimate rules that are adversely affecting a machine learning system in weakly supervised learning and present them to the subject. By combining this with conventional technology, it becomes possible to freely and efficiently add, modify, and delete rules in weakly supervised learning, enabling trial and error when building a machine learning model. In a user study involving multiple document classification tasks, it was confirmed that the method of this embodiment could identify the rule that needed to be modified most with a probability of 60% or more, and the top three with a probability of 80% or more.

[0056] <Other variations> The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices that execute the processes or as needed. Other modifications are possible within the scope of the present invention.

[0057] <Programs and recording media> The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 4, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0058] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0059] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0060] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with each program transferred from the server computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0061] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. The system is provided with a group of labeling functions that assign labels to input data sets and a classifier that performs learning based on the labeling functions, and presents labeling functions to be corrected or deleted from the group of labeling functions based on the output of the group of labeling functions and the output of the classifier; The labeling function to be presented is determined by a measure that measures the difference between an output probability distribution based on a group of labeling functions before learning and an output probability distribution based on a classifier after learning. Learning device.

2. 2. The learning device of claim 1, The labeling function to be modified or deleted is presented together with the input data that supports it. Learning device.

3. a labeling function processing unit that uses a labeling function group including one or more labeling functions to obtain a voting result for each labeling function for input data included in the input dataset; a second probability distribution calculation unit that uses the input data as an input to a trained classifier and calculates an output probability distribution based on the trained classifier; a probability distribution calculation unit that calculates an output probability distribution of a labeling function using the voting result; Using the output probability distribution based on the learned classifier and the output probability distribution of the labeling function, a score s indicating the need for correction or deletion in the input data is calculated. k (p) is calculated, and the score s k an evaluation unit that calculates a score s(i) from (p) that indicates the need for modification or deletion in the labeling function; a presentation information generation unit that generates information for presenting to a subject a labeling function whose score s(i) indicates the highest need for modification or deletion; a labeling function correction unit that receives input from the subject to whom the labeling function is presented and deletes or corrects the labeling function according to the input content, Learning device.

4. Using a learning device including a group of labeling functions that assign labels to input data sets and a classifier that performs learning based on the labeling functions, a labeling function to be corrected or deleted from the group of labeling functions is presented based on the output of the group of labeling functions and the output of the classifier; The labeling function to be presented is determined by a measure that measures the difference between an output probability distribution based on a group of labeling functions before learning and an output probability distribution based on a classifier after learning. How to learn.

5. A program for causing a computer to function as the learning device according to any one of claims 1 to 3.

Images