Data processing device, data processing method, and data processing program

Abstract

This data processing device uses a large language model (LLM) to assign a label to each piece of data in a candidate data set for training data for a machine learning model. The data processing device then calculates the confidence level of the label assigned to each piece of data output from the LLM, on the basis of a predicted probability distribution of the label of the piece of data. The data processing device then extracts data for which the confidence level of the label thereof is equal to or greater than a prescribed value, from the data set labeled by the LLM. The data processing device then adds the extracted data for which the confidence level of the label thereof is equal to or greater than the prescribed value to the training data for the machine learning model.

Description

Data processing device, data processing method, and data processing program

[0001] The present invention relates to a data processing device, a data processing method, and a data processing program.

[0002] Well-being (WB) has been attracting attention in recent years. Because WB is psychological, it is difficult to measure. Currently, the most widely used and reliable method for measuring WB is to conduct a survey. However, conducting a survey every time places a heavy burden on both those who create the survey and those who respond. Therefore, machine learning can be used to predict survey results. When using supervised learning to predict survey results, it is necessary to label the training data with the survey results.

[0003] Daniel Preotiuc-Pietro et al., "Modeling Valence and Arousal in Facebook Posts," Proceedings of NAACL-HLT 2016, pages 9-15, [online], [Retrieved May 21, 2024], Internet <URL: https: / / aclanthology.org / W16-0404 / > Embeddings-OpenAI API, [online], [Retrieved May 21, 2024], Internet <URL: https: / / platform.openai.com / docs / guides / embeddings> Cheng-Han Chiang et al., "Can Large Language Models Be an Alternative to Human Evaluation?," 2023, [online], [Retrieved May 21, 2024], Internet <URL: https: / / arxiv.org / abs / 2305.01937>

[0004] However, because WB depends on the situation of the person being measured, the person being measured must label the data themselves. This increases the cost of labeling training data, which in turn increases the cost of building a machine learning model for predicting WB. Therefore, an objective of the present invention is to solve the above-mentioned problem and reduce the cost of labeling training data used to train a machine learning model for predicting WB, etc.

[0005] In order to solve the above-mentioned problems, the present invention is characterized by comprising a label assignment unit that assigns a label to each piece of data in a data group of candidate training data for a machine learning model using LLMs (Large Language Models); a confidence calculation unit that calculates a confidence level of the label assigned to the data based on the distribution of predicted probabilities of each label for each piece of data output from the LLM; a filtering unit that filters data whose confidence level of the label is equal to or greater than a predetermined value; and a data expansion unit that adds the filtered labeled data whose confidence level is equal to or greater than the predetermined value to the training data of the machine learning model.

[0006] According to the present invention, it is possible to reduce the cost of labeling training data used in training a machine learning model.

[0007] FIG. 1 is a histogram of valence labels in a dataset. FIG. 2 is a diagram illustrating the pipeline of a prediction model. FIG. 3 is a graph showing measurement results of model performance when the number of training samples is reduced. FIG. 4 is a histogram of valence labels predicted by LLM and a histogram of correct labels. FIG. 5 is a diagram showing an example of the distribution of predicted probabilities for each label. FIG. 6 is a diagram illustrating an example configuration of a data processing device. FIG. 7 is a flowchart showing an example of processing procedures executed by the data processing device. FIG. 8 is a diagram showing the results of an evaluation experiment. FIG. 9 is a diagram showing the results of confirming the correlation between the difference in predicted probabilities of each label by LLM and the error between the predicted value of the label by LLM and the correct value. FIG. 10 is a diagram illustrating an example of a computer executing a data processing program.

[0008] Hereinafter, a mode (embodiment) for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment. A data processing device of this embodiment uses LLM to create labeled data for training a machine learning model.

[0009] [Problem Setting] A machine learning model is a model that predicts a user's WB (for example, survey results) from input text data. Here, let the text data be x∈R^d and the survey results be y∈R. p(y|x;θ) is learned using supervised learning. Here, we consider the case where the true data p(y|x) is a probability distribution containing noise. This corresponds to data that is prone to noise, such as human emotions.

[0010] [Experimental Settings] The experimental settings are explained below. Dataset: The dataset used was data in which experts labeled texts on Facebook (registered trademark) with arousal and valence, which are indicators of psychology, on a 10-point scale (see Non-Patent Document 1). Since two people labeled each text, the values ​​of the two labels were averaged and rounded to the nearest integer. Here, the problem of inferring valence from input text was solved (although arousal could also be used, valence was used in this study). The ratio of train (teaching data) to test (test data) was set at 3:1, and the data was divided to prevent bias in the labels. Figure 1 shows a histogram of the labels.

[0011] Model: Text is embedded into a vector using the Open AI API (see Non-Patent Document 2), and valence is predicted using a linear model. The loss used in this case is multi-class cross entropy. The overall pipeline is shown in Figure 2.

[0012] Evaluation metrics: The metrics used to evaluate the model's performance were test accuracy, Spearman rank correlation, root mean squared error (RMSE), and max error. Max error is the difference between the most incorrect prediction among all test data and the correct answer. For example, if the most incorrect prediction is "3" and the correct answer is "8," the max error is "5."

[0013] [Preliminary Experiment 1] Because it is difficult to prepare a large amount of labeled training data, we measured the deterioration of model performance when the training data for the model was reduced. The total number of train samples was 2,168, and the number of test samples was 723. The experiment was conducted while reducing the number of train samples. The number of test samples used for evaluation was fixed.

[0014] As shown in Figure 3, when the number of train samples is small, the values ​​of all evaluation indicators deteriorate. When all data is used, the results are RMSE=0.88, Spearman=0.72, test acc=0.55, and max error=3. Therefore, when all data is used to train the model, it can be seen that the model performs sufficiently well. On the other hand, when the number of train samples is reduced, the model's performance drops dramatically.

[0015] If the model's performance can be maintained even with a small number of train samples, it is believed that the model will be able to predict survey results from text using a small number of survey results.

[0016] [Preliminary Experiment 2] Next, we explain the results of predicting labels for each data set using LLM. Figure 4 shows a histogram of valence labels predicted by LLM and a histogram of correct labels. As shown in Figure 4, the distribution of labels predicted by LLM differs from the distribution of correct labels. This shows that labeling by LLM is not necessarily reliable. Furthermore, when data labeled by LLM was mixed with the training data for the model, the model's performance decreased.

[0017] [Overview] Next, an overview of the data processing device 10 of this embodiment will be described. The LLM used in this embodiment assigns pseudo-labels to input data (e.g., text data). The labels are, for example, the valence (emotional valence) of the data, and are values ​​on a 10-point scale from 1 to 10. The LLM also outputs a predicted probability for each label of the input data. For example, if the label is any one of 1 to 10, the LLM outputs a predicted probability for each label (1 to 10) of the data.

[0018] The data processing device 10 inputs a group of data (e.g., a group of text data) that are candidates for training data into the LLM. The data processing device 10 then acquires labeled data and the distribution of predicted probabilities for each label in the data from the LLM. The data processing device 10 then calculates the confidence of the labels assigned by the LLM based on the predicted distribution of each label in the data acquired from the LLM. The data processing device 10 then filters data that have been assigned labels with confidence levels equal to or greater than a predetermined value from the group of data labeled by the LLM.

[0019] For example, as shown in FIG. 5 , if the distribution of predicted probabilities for each label output by the LLM is peaked (i.e., if the predicted probability of a certain label is exceptionally high), the confidence level of the label prediction by the LLM is considered to be high. Therefore, the data processing device 10 filters data with peaked predicted probabilities for each label as described above as data with a high confidence level of the label prediction by the LLM. For example, the data processing device 10 determines the difference between the highest and second highest predicted probability values ​​for each label as the confidence level of the label prediction, and extracts data with labels assigned with a high confidence level. The data processing device 10 then adds the extracted data to the training data of the machine learning model. Subsequently, by training the machine learning model using the training data, a high-performance machine learning model can be constructed.

[0020] [Configuration Example] Next, a configuration example of the data processing device 10 will be described with reference to Fig. 6. The data processing device 10 includes, for example, an input / output unit 11, a storage unit 12, and a control unit 13.

[0021] The input / output unit 11 is an interface that controls the input and output of various data. For example, the input / output unit 11 receives input of a data group that is to be candidate training data for a machine learning model.

[0022] The storage unit 12 stores data, programs, etc. that are referenced when the control unit 13 executes various processes. The storage unit 12 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 12 stores a group of data received by the input / output unit 11.

[0023] The control unit 13 is responsible for overall control of the data processing device 10. The functions of the control unit 13 are realized, for example, by a CPU (Central Processing Unit) executing a program stored in the storage unit 12.

[0024] The control unit 13 includes, for example, a label assignment unit 130, a confidence factor calculation unit 131, a filtering unit 132, and a data extension unit 133. The learning unit 134 may or may not be provided, and the cases where it is provided will be described later.

[0025] The labeling unit 130 uses LLM to assign a label (pseudo label) to each piece of data in a data group of candidates for training data for a machine learning model.

[0026] The certainty calculation unit 131 calculates the certainty of the label assigned by the LLM. For example, the certainty calculation unit 131 acquires the predicted probability of each label for each data item from the LLM. Then, the certainty calculation unit 131 calculates the certainty of the label assigned to the data item based on the distribution of the predicted probability of each label for each data item.

[0027] For example, the certainty calculation unit 131 calculates the certainty of the label assigned to the data based on the difference in predicted probability of each label of the data. As an example, the certainty calculation unit 131 calculates the difference between the highest value and the next highest value among the predicted probability values ​​of each label as the certainty of the label assigned to the data.

[0028] Furthermore, the certainty calculation unit 131 may calculate the entropy of the distribution of the predicted probability of each label of the data in the data group output from the LLM, and calculate the certainty of the label assigned to the data based on the entropy.

[0029] The filtering unit 132 filters the data group to which labels have been assigned by the label assignment unit 130. For example, the filtering unit 132 filters data from the data group, the data having a label certainty calculated by the certainty calculation unit 131 that is equal to or greater than a predetermined value.

[0030] The data extension unit 133 adds the data (labeled data) filtered by the filtering unit 132 to the training data of the machine learning model.

[0031] According to the data processing device 10, data with high confidence in the labels used for training a machine learning model for predicting WB, etc. can be added to the training data, thereby reducing the cost of labeling the training data.

[0032] [Example of Processing Procedure] Next, an example of processing procedure executed by the data processing device will be described with reference to Fig. 7. The labeling unit 130 of the data processing device 10 uses LLM to assign a label to each piece of data in a data group of candidate training data for a machine learning model (S10: Assign label to data).

[0033] After S10, the certainty calculation unit 131 calculates the certainty of the labels assigned to the data by the LLM. For example, the certainty calculation unit 131 obtains the predicted probability of each label of each data from the LLM and calculates the difference in predicted probability between the labels (S11).

[0034] After S11, the filtering unit 132 filters data whose label certainty is equal to or greater than a predetermined value. For example, the filtering unit 132 filters data whose difference in predicted probability between the labels calculated in S11 is equal to or greater than a predetermined threshold (S12).

[0035] After S12, the data extension unit 133 adds the data filtered by the filtering unit 132 (labeled data whose label certainty is equal to or greater than a predetermined value) to the training data of the machine learning model (S13).

[0036] Thereafter, the machine learning model is trained using the training data to which the data has been added. The training of the machine learning model may be performed by the data processing device 10 or by a device other than the data processing device 10. When the data processing device 10 trains the machine learning model, the data processing device 10 further includes a training unit 134 (see FIG. 6 ). The training unit 134 trains the machine learning model using the training data to which the data has been added by the data extension unit 133.

[0037] According to the data processing device 10 described above, it is possible to reduce the cost of assigning labels to training data for a machine learning model. As a result, it is possible to reduce the cost required to build a machine learning model that predicts WB, etc. For example, if the results of questionnaire responses from a small number of users are available as training data for a machine learning model that automatically predicts a user's WB, etc. from text data, the machine learning model can be trained.

[0038] [Results of Evaluation Experiment] The results of an evaluation experiment of a machine learning model trained using training data expanded by the above-described data processing device 10 are shown below. In this experiment, data in which the difference between the highest value and the next highest value among the predicted probability values ​​of each label of the data was 0.8 or more was extracted, and the training data of the machine learning model was expanded. The results of the evaluation experiment are shown in Figure 8.

[0039] Here, we also evaluated the test accuracy, Spearman rank correlation, root mean squared error (RMSE), and max error of the machine learning model. The vertical axis of each graph in Figure 8 represents the evaluation value, and the horizontal axis represents the number of train samples of the machine learning model. As with Figure 3, the total number of train samples was 2168 and the number of test samples was 723, and experiments were conducted while decreasing the number of train samples. The number of test samples used for evaluation was fixed. The comparative example is a train sample extracted from the dataset described in Non-Patent Document 1.

[0040] As shown in FIG. 8 , when training data was expanded using the data processing device 10 of this embodiment, it was confirmed that the performance of the machine learning model was higher than that of the comparative example. Furthermore, it was confirmed that the machine learning model maintained higher performance than the comparative example even when the number of training data (number of training samples) was reduced. As a result, it was confirmed that even when, for example, approximately 22 items (0.5% of the training samples) were labeled by a human and the remaining data labeled by the data processing device 10 were used as training samples, the output of the machine learning model deviated by only about one level out of 10. Furthermore, the Spearman rank correlation was improved from approximately 0.22 in the comparative example to approximately 0.65 in this embodiment.

[0041] We also checked whether there was a correlation between the difference in the predicted probability of each label by LLM and the error between the predicted value of the label by LLM and the correct value. The results are shown in Figure 9. The horizontal axis of the graph in Figure 9 shows the difference between the highest and second highest predicted probability values ​​of each label, and the vertical axis shows the number of data. |llm pred-true| indicates the error between the predicted value of the label by LLM and the correct value.

[0042] As shown in Figure 9, it was confirmed that when the difference in the predicted probability of each label by LLM is large, the predicted value of the label by LLM is likely to match the correct value. This also confirms that expanding the training data of a machine learning model using labeled data where the difference in the predicted probability of each label is greater than or equal to a predetermined value contributes to improving the performance of the machine learning model.

[0043] [Application Example of the Present Embodiment] The above-described data processing device 10 can be applied, for example, as follows.

[0044] For example, consider the case where a machine learning model is used to evaluate how improvements in a company's policies (such as the introduction of a new internal system) have affected employees' WB (Well-being).

[0045] In this case, comments related to the internal system are obtained from chat services used by employees, and the data is input into a machine learning model, which is then used to predict employees' WB after the new internal system is introduced. This makes it possible to continuously determine whether internal measures have improved employees' WB without the need to survey employees.

[0046] In this case, first, a small number of employee comments and survey results regarding the employee's WB in the situations indicated by the comments are collected, and the training data is expanded using the data processing device 10 and trained into a machine learning model. After that, comments from the chat service used by the employee are input into the trained machine learning model to estimate the employee's WB.

[0047] [System Configuration, etc.] The components of each unit shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0048] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0049] [Program] The data processing device 10 can be implemented by installing a program (data processing program) as package software or online software on a desired computer. For example, by executing the program on an information processing device, the information processing device can function as the data processing device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

[0050] 10 is a diagram showing an example of a computer that executes a data processing program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0051] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0052] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes executed by the data processing device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the data processing device 10 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0053] Data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0054] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0055] REFERENCE SIGNS LIST 10 Data processing device 11 Input / output unit 12 Storage unit 13 Control unit 130 Label assignment unit 131 Confidence factor calculation unit 132 Filtering unit 133 Data expansion unit 134 Learning unit

Claims

1. A data processing device comprising: a label assignment unit that assigns a label to each piece of data in a data group of candidate training data for a machine learning model using LLM (Large Language Models); a confidence calculation unit that calculates the confidence of the label assigned to the data based on the distribution of predicted probabilities of each label for each piece of data output from the LLM; a filtering unit that filters data whose confidence of the label is equal to or greater than a predetermined value; and a data expansion unit that adds the filtered labeled data whose confidence is equal to or greater than the predetermined value to the training data for the machine learning model.

2. The data processing device according to claim 1, characterized in that the certainty calculation unit calculates the certainty of the label assigned to the data based on the magnitude of the difference in predicted probability of each label of the data.

3. The data processing device according to claim 2, characterized in that the certainty calculation unit determines the difference between the highest value and the second lowest value among the predicted probability values ​​of each label of the data as the certainty.

4. The data processing device according to claim 1, further comprising a learning unit that uses training data to which the labeled data has been added by the data extension unit to train the machine learning model.

5. The data processing device according to claim 1, wherein the label assigned to the data is a label indicating a predicted result of human emotion toward the data.

6. A data processing method executed by a data processing device, comprising: a step of assigning a label to each piece of data in a data group of candidate training data for a machine learning model using LLM (Large Language Models); a step of calculating the certainty of the label assigned to the data based on the distribution of predicted probabilities of each label for each piece of data output from the LLM; a step of filtering data whose certainty of the label is equal to or greater than a predetermined value; and a step of adding the filtered labeled data whose certainty is equal to or greater than the predetermined value to the training data for the machine learning model.

7. A data processing program that causes a computer to perform the following steps: assigning a label to each piece of data in a data group of candidate training data for a machine learning model using LLM (Large Language Models); calculating the confidence of the label assigned to the data based on the distribution of predicted probabilities of each label for each piece of data output from the LLM; filtering data whose confidence of the label is equal to or greater than a predetermined value; and adding the filtered labeled data whose confidence is equal to or greater than the predetermined value to the training data for the machine learning model.

Images