Generation device

Abstract

A generation device (10) has an information acquisition unit (111) that acquires information to be labeled, a feature value acquisition unit (112) that acquires a feature value relating to the information to be labeled, the feature value being acquired at the same time as the information to be labeled, a label estimation unit (12) that estimates a label from the feature value, a coupling unit (13) that couples the information to be labeled with the label estimated by the label estimation unit (12), and an output unit (14) that outputs a pair made up of the information to be labeled and the label coupled together by the coupling unit (13).

Description

generator

[0001] The present invention relates to a generating device.

[0002] Conventionally, in supervised learning, creating a machine learning model requires training data that contains a set of information and a correct label.

[0003] Burr Settles, "Active Learning Literature Survey," Computer Sciences Technical Report 1648, University of Wisconsin-Madison, Updated on: January 26, 2010. Atsuo Uo, Hiroyoshi Ito, Masaki Matsubara, Atsuyuki Morishima, and Yukino Baba, "Active Learning Considering Annotation Cost in Semantic Segmentation," The 35th Annual Conference of the Japanese Society for Artificial Intelligence, 2021. Yuma Sato, Tomokazu Matsui, Yugo Nakamura, Hirohiko Suwa, and Keiichi Yasumoto, "A Study on Nudge-Based Annotation Guidance Methods for Optimizing In-Home Activity Recognition Models," "Multimedia, Distributed, Cooperative, and Mobile (DICOMO2021) Symposium," June-July 2021.

[0004] In the past, when collecting data for training machine learning models, labeling of certain information was sometimes done manually. For example, when obtaining correct labels in the form of a questionnaire, users manually assigned correct labels to data based on the responses to the questionnaire. This manual labeling process was problematic in that it placed a burden on users and increased costs.

[0005] The present invention has been made in consideration of the above, and aims to provide a generation device that realizes automation of labeling in data collection for training a machine learning model.

[0006] In order to solve the above-mentioned problems and achieve the object, the generation device of the present invention is characterized by having: a first acquisition unit that acquires information of a target to be labeled; a second acquisition unit that acquires features related to the information to be labeled, which are acquired simultaneously with the information to be labeled; an estimation unit that estimates a label from the features; a combination unit that combines the information to be labeled and the label estimated by the estimation unit; and an output unit that outputs a pair of the information to be labeled and the label combined by the combination unit.

[0007] According to the present invention, automation of labeling is achieved in data collection for training a machine learning model.

[0008] FIG. 1 is a diagram illustrating a conventional method for generating training data. FIG. 2 is a diagram illustrating a method for generating training data according to an embodiment. FIG. 3 is a diagram illustrating an example of the configuration of a generation device according to an embodiment. FIG. 4 is a diagram illustrating an example of the configuration of a label estimation unit shown in FIG. 3. FIG. 5 is a flowchart illustrating the processing steps of a generation process according to an embodiment. FIG. 6 is a flowchart illustrating the processing steps of the label estimation process shown in FIG. 5. FIG. 7 is a flowchart illustrating the processing steps of an estimation model generation process according to an embodiment. FIG. 8 is a diagram illustrating the generation process according to an embodiment. FIG. 9 is a diagram illustrating an example of a computer on which a generation device is realized by executing a program.

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0010] [Embodiment] In the embodiment, a method for acquiring data in which information and labels are paired as learning data in collecting learning data for machine learning will be described.

[0011] Fig. 1 is a diagram illustrating a conventional method for generating training data. As shown in Fig. 1, in the conventional method, when generating training data, a user manually generates labels based on questionnaire responses, etc. ((1) in Fig. 1), and manually assigns the labels to the information to be labeled ((2) in Fig. 1).

[0012] 2 is a diagram illustrating a method for generating training data according to an embodiment. As shown in FIG. 2, in this embodiment, a trained machine learning model is used to estimate labels to be assigned to information to be labeled from features that can be acquired simultaneously with the information to be labeled ((1) in FIG. 2), and the labels are automatically assigned ((2) in FIG. 2). In this way, in this embodiment, labels are estimated using a machine learning model, and labels are efficiently generated and assigned, thereby automating labeling.

[0013] [Generation Device] A generation device according to an embodiment will be described. Fig. 3 is a diagram illustrating an example of the configuration of a generation device according to an embodiment. The generation device 10 illustrated in Fig. 3 generates training data for a machine learning model (not illustrated) to be installed in another device, for example.

[0014] The generating device 10 is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The generating device 10 is also capable of communicating with other devices via a communication interface that transmits and receives various information to and from other devices connected via a network, etc.

[0015] The generating device 10 includes an input information acquiring unit 11, a label estimating unit 12 (estimating unit), a combining unit 13, and an output unit 14.

[0016] The input information acquisition unit 11 accepts input of information to be used in the generation device 10. The input information acquisition unit 11 includes an information acquisition unit 111 (first acquisition unit) and a feature amount acquisition unit 112 (second acquisition unit).

[0017] The information acquisition unit 111 acquires information to be labeled. The information acquisition unit 111 acquires information using a sensor or the like. The information to be labeled is, for example, various types of information such as video (still images, moving images) captured by a user.

[0018] The feature acquisition unit 112 acquires feature amounts. The feature acquisition unit 112 acquires feature amounts using a sensor or the like. The feature amounts are acquired simultaneously with the information to be labeled, and are feature amounts related to the information to be labeled. For example, if the information to be labeled is a video of a user, the feature amounts are biometric information of the user appearing in the video. Examples of the biometric information include the user's heart rate, body temperature, blood flow, respiratory rate, skin potential, brain waves, etc. The biometric information is acquired, for example, by a wearable device worn by the user.

[0019] The label estimation unit 12 estimates labels from the features acquired by the feature acquisition unit 112. The label estimation unit 12 estimates labels from the features using a machine learning model that has been trained in advance using correct labels and training features. The labels are, for example, fatigue levels expressed in multiple stages.

[0020] The combining unit 13 combines the information of the labeling target acquired by the information acquiring unit 111 with the label estimated by the label estimating unit 12. The combining unit 13 combines the information of the labeling target having matching time data with the label estimated by the label estimating unit 12.

[0021] The output unit 14 outputs the pair of information to be labeled and the label combined by the combining unit 13. The output unit 14 outputs the pair of information to be labeled and the label, for example, as training data for a machine learning model (not shown) installed in another device.

[0022] [Label Estimation Unit] Fig. 4 is a diagram showing an example of the configuration of the label estimation unit 12 shown in Fig. 3. As shown in Fig. 4, the label estimation unit 12 includes a label estimation basic unit 121, an input search unit 122, an estimation unit 123 having a machine learning model 1231 that estimates a label from a feature amount, and an output unit 124.

[0023] The label estimation basic unit 121 performs learning of the machine learning model 1231 of the estimation unit 123 .

[0024] The label estimation base unit 121 includes a correct label acquisition unit 1211 , a model creation feature acquisition unit 1212 , and a model creation unit 1213 .

[0025] The correct label acquisition unit 1211 acquires a correct label. The correct label is a correct label of a feature that is training data for creating the machine learning model 1231.

[0026] The model creation feature acquisition unit 1212 acquires features. The features are learning data for creating the machine learning model 1231. The label estimation base unit 121 acquires, as learning data, pairs of biometric information of a certain user and a fatigue level at the time of acquisition of the biometric information obtained from the user's response to a questionnaire, for example.

[0027] The model creation unit 1213 uses feature quantities, which are training data, and correct labels corresponding to these feature quantities to train a machine learning model that estimates labels from the feature quantities. The model creation unit 1213 creates a machine learning model corresponding to each type of feature quantity. Alternatively, the model creation unit 1213 creates a machine learning model that can estimate a correct label from multiple feature quantities regardless of which feature quantity is used. The model creation unit 1213 sets the machine learning model 1231, whose parameters have been optimized, in the estimation unit 123.

[0028] The input search unit 122 determines what kind of feature is input from which sensor when the feature is input from the feature acquisition unit 112. The input search unit 122 searches for the type of feature to be used for label estimation by determining the sensor that is the output source of the feature. The input search unit 122 outputs the type of feature to be used for label estimation to the estimation unit 123.

[0029] The estimation unit 123 estimates a label from the feature using the machine learning model 1231. As described above, the machine learning model 1231 is a model that has been trained in advance by the model creation unit 1213 using a correct label and feature values ​​for learning. When a machine learning model 1231 is set for each type of feature value, the estimation unit 123 selects a machine learning model 1231 corresponding to the type of feature value determined by the input search unit 122 and uses the selected machine learning model 1231 for label estimation. Furthermore, the estimation unit 123 selects a machine learning model 1231 corresponding to the type of feature value determined by the input search unit 122 and uses the selected machine learning model 1231 for label estimation.

[0030] The output unit 124 outputs the label estimated by the estimation unit 123. This label is combined with the information to be labeled, which is acquired by the information acquisition unit 111.

[0031] [Generation Process] Next, a processing procedure for generating training data will be described. Fig. 5 is a flowchart showing the processing procedure for generating training data according to an embodiment.

[0032] The generating device 10 acquires information to be labeled and feature amounts acquired simultaneously with the information to be labeled (steps S1 and S2).

[0033] The generating device 10 causes the label estimating unit 12 to perform a label estimation process of estimating a label from the feature amount acquired in step S2 (step S3).

[0034] The generating device 10 combines the information of the labeling target acquired in step S1 with the label estimated in step S3 (step S4), and outputs the combined pair of the information of the labeling target and the label (step S5).

[0035] [Label Estimation Processing] FIG. 6 is a flowchart showing the processing procedure of the label estimation processing shown in FIG.

[0036] The label estimation unit 12 searches for the type of the feature input in step S2 (step S11).

[0037] The label estimation unit 12 estimates labels from the features acquired in step S2 (step S12). Based on the search result of step S11, the label estimation unit 12 may select a machine learning model 1231 according to the type of feature used for label estimation. Furthermore, before inputting the feature to the machine learning model 1231, the label estimation unit 12 may perform predetermined preprocessing so that the feature has a data format that can be input to the machine learning model 1231.

[0038] The label estimation unit 12 outputs the estimated label (step S13), and ends the label estimation process.

[0039] [Estimation Model Generation Process] FIG. 7 is a flowchart showing the processing procedure of the estimation model generation process according to the embodiment.

[0040] As shown in FIG. 7, in the generation device 10, the label estimation basic unit 121 acquires correct labels of feature quantities that are training data (step S21).

[0041] The model creation feature acquisition unit 1212 acquires features that are training data (step S22).

[0042] The model creation unit 1213 uses a machine learning model to estimate labels of feature quantities, which are training data (step S23), and updates the parameters of the machine learning model so that the estimated labels approach the correct labels (step S24). The model creation unit 1213 repeats the processes of steps S22 to S24 for these feature quantities until a predetermined termination condition is met. The termination condition may be that the processes have been repeated a predetermined number of times, or that the parameter update width has converged.

[0043] Then, the model creation unit 1213 executes the processes of steps S21 to S24 using all the features of the training target and each correct label, thereby creating a machine learning model corresponding to the features. The model creation unit 1213 sets the created machine learning model 1231 in the estimation unit 123. Note that the model creation unit 1213 may create a machine learning model 1231 for each type of feature. Furthermore, the model creation unit 1213 may create a machine learning model that can estimate labels, regardless of the type of feature.

[0044] [Evaluation Experiment] In collecting learning data for fatigue level estimation using actual video, a labeling experiment was conducted in which fatigue levels were assigned as labels to the video.

[0045] In the past, the user's fatigue level was obtained in the form of a questionnaire and the fatigue level was assigned to the corresponding information (for example, an image of the user). In contrast, in this embodiment, the user's biological information is used to automatically estimate the fatigue level and assign it to the corresponding information (for example, an image of the user).

[0046] First, we constructed a machine learning model that predicts fatigue levels into three levels (low, medium, and high) based on heart rate data. Then, in the experiment, we simultaneously acquired the user's video and heart rate data as biometric information.

[0047] The subject was one person. Heart rate data was collected over three days using a wearable device worn by the user, and was collected at 10-second intervals. The correct answer label was the fatigue level measured based on the user's questionnaire responses. The fatigue level was considered correct if it matched the user's own feelings.

[0048] When the fatigue level estimated from the heart rate data acquired simultaneously with the video using the generation device 10 was compared with the fatigue level measured based on the questionnaire responses, the accuracy rate of the labels estimated by the generation device 10 from the heart rate data was a high value of 0.794.

[0049] [Effects of the embodiment] From this evaluation experiment, it was found that the generation device 10 according to the embodiment can estimate labels from feature quantities.

[0050] FIG. 8 is a diagram illustrating a generation process according to an embodiment. As shown in FIG. 8, the generation device 10 acquires a user's biometric information (e.g., heart rate data) simultaneously with a video of the user ((1) in FIG. 8). The generation device 10 estimates a fatigue level from the biometric information using a machine learning model 1231 with sufficient estimation accuracy ((2) in FIG. 8). Because the biometric information is acquired simultaneously with the video and the video and time data match, the generation device 10 can automatically assign the estimated fatigue level to the video as a correct label ((3) in FIG. 8).

[0051] In the past, it was not possible to collect fatigue levels directly from biometric information with a high degree of accuracy, so correct labels had to be obtained manually based on questionnaire responses and then assigned to the information, which resulted in high burden and costs for users.

[0052] In contrast, the generation device 10, as described above, enables automatic assignment of correct labels, eliminating the need for a survey compared to conventional methods, thereby reducing the burden on users and costs. Therefore, the generation device 10 efficiently and accurately generates labels and automatically assigns the generated labels to information to be labeled, making it possible to collect data for training a machine learning model at low cost while ensuring sufficient accuracy.

[0053] Note that the information to be labeled is not limited to the user's video, but may also be information such as the user's voice, text by the user (transcription data of the user's voice, a document file created by the user, text entered by the user in a chat, etc.), and voice collected in the area where the user is located.

[0054] Furthermore, the feature is not limited to the user's biometric information. The feature may be a voice uttered by the user. The feature may also be text by the user (transcription data of the user's voice, a document file created by the user, text entered by the user in a chat, etc.), a video of the user, etc. The feature may also be information obtained by performing predetermined processing on information about the user. Furthermore, the generating device 10 may input, as the feature, information such as video, audio, or text about the user, as well as a determination result of emotions such as negative or positive determined based on the audio or text, into the machine learning model 1231. Furthermore, the generating device 10 may input, as the feature, information such as video, audio, or text about the user, as well as clarity analyzed based on the audio of the user, into the machine learning model 1231.

[0055] Furthermore, the label is not limited to fatigue level, but may be relaxation level, concentration level, excitement level, dissatisfaction level, or the like.

[0056] For example, the generation device 10 can be applied in a variety of situations, such as when estimating a user's level of fatigue, relaxation, concentration, excitement, and / or dissatisfaction from biometric information, voice, text, and / or video, and labeling the work results processed by the user (such as document files created by the user).

[0057] The generating device 10 may also estimate multiple types of labels from one feature (e.g., biometric information) and label the information to be labeled. The generating device 10 may also estimate one or multiple types of labels from multiple types of feature (e.g., generated information and audio information) and label the information to be labeled. The information to be labeled may include multiple types of information.

[0058] Furthermore, the generating device 10 may output only the label, in addition to outputting a pair of a label and information. For example, the generating device 10 may output a fatigue level estimated from a video of a user along with the user's identification information, and another device may assign the label output by the generating device 10 to the video to be labeled. Furthermore, the fatigue level output by the generating device 10 along with the user's identification information may be output to the user's workplace, the user's workplace's industrial physician, and / or the user himself / herself, which may lead to management of the user's health condition, etc.

[0059] [System Configuration of the Embodiment] The generation device 10 is a functional concept and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the generation device 10 is not limited to that shown in the figure, and all or part of the functions can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0060] Furthermore, all or any part of the processes performed by the generating device 10 may be realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Furthermore, each process performed by the generating device 10 may be realized as hardware using wired logic.

[0061] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0062] 9 is a diagram showing an example of a computer in which the generating device 10 is realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0063] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0064] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the generation device 10 is implemented as a program module 1093 in which code that can be executed by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the generation device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0065] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0066] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0067] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0068] 10 Generating device 11 Input information acquisition unit 12 Label estimation unit 13 Combining unit 14, 124 Output unit 111 Information acquisition unit 112 Feature acquisition unit 121 Label estimation base unit 122 Input search unit 123 Estimation unit 1211 Correct label acquisition unit 1212 Model creation feature acquisition unit 1213 Model creation unit 1231 Machine learning model

Claims

a first acquisition unit that acquires information to be labeled; a second acquisition unit that acquires a feature related to the information to be labeled, the feature being acquired simultaneously with the information to be labeled; an estimation unit that estimates a label from the feature; a combining unit that combines the information to be labeled with the label estimated by the estimating unit; an output unit that outputs the pair of the information to be labeled and the label combined by the combining unit; A generating device comprising: 　 The generation device according to claim 1 , wherein the estimation unit estimates the label from the feature using a machine learning model that has been trained in advance using a correct label and a feature for learning. 　 The first acquisition unit acquires, as information to be labeled, a video of a user, audio information about the user, and / or text information by the user; the second acquisition unit acquires, as the feature, biometric information of the user, voice information about the user, and / or text information by the user, which are acquired simultaneously with the information to be labeled; The estimation unit estimates, as the label, a level of fatigue, a level of relaxation, a level of concentration, a level of excitement, and / or a level of dissatisfaction of the user from the feature amount. The generating device according to claim 1 . 　 The generating device according to claim 3, characterized in that the output unit outputs the user's level of fatigue, relaxation, concentration, excitement, and / or dissatisfaction estimated by the estimation unit together with the user's identification information.

Images