Data processing device and data processing method

Abstract

This data processing device measures the amount of data per label for a data group in which a label corresponding to a measurement value is assigned to each item of time-series data indicating a measurement value in a continuously changing state. Thereafter, the data processing device determines whether there is a label whose amount of data is less than a predetermined threshold. When the data processing device determines that there is a label whose amount of data is less than the predetermined threshold, the data processing device changes, for that label, a label of data in a predetermined range before or after the measurement time of the data group of that label in time series and outputs the label.

Description

Data processing device and data processing method

[0001] The present invention relates to a data processing device and a data processing method for creating training data for a machine learning model.

[0002] Recently, there have been technologies that use machine learning to analyze and estimate various conditions. For example, there is a technology that uses the machine learning library Keras to train a machine learning model using images and labels as training data.

[0003] However, if the input data to the machine learning model changes to input data under different conditions than when the machine learning model was trained, there is a risk that the estimation accuracy will drop significantly. To solve this problem, for example, one possible approach is to fine-tune the machine learning model using a small amount of data (training data) every day.

[0004] Masahiro Oda, "Deep Learning with Keras", MEDICAL IMAGING TECHNOLOGY Vol.36 No.2 March 2018

[0005] However, collecting training data for fine-tuning machine learning models can be a burden for users. For example, if the collected data is insufficient, such as if some of the data is missing, it becomes necessary to collect more data.

[0006] Therefore, an object of the present invention is to solve the above-mentioned problems and reduce the time required to collect training data for a machine learning model.

[0007] In order to solve the above-mentioned problems, the present invention is characterized by comprising a measurement unit that measures the amount of data for each label for a data group in which time-series data showing measurement values ​​of a continuously changing state is assigned a label corresponding to the measurement value; a determination unit that determines whether there is a label for which the amount of data is less than a predetermined threshold; and an adjustment unit that, if it is determined that there is a label for which the amount of data is less than the predetermined threshold, changes the label of data in a predetermined range before or after the measurement time of the data group of that label in the time series to that label and outputs the data.

[0008] According to the present invention, it is possible to reduce the time required to collect learning data for a machine learning model.

[0009] FIG. 1 is a diagram illustrating an example of changes in actual states and measurement data. FIG. 2 is a diagram illustrating an overview of a data processing device. FIG. 3 is a diagram illustrating an overview of a data processing device. FIG. 4 is a diagram illustrating an example of a configuration of a data processing device. FIG. 5 is a diagram illustrating an example of processing executed by the data processing device. FIG. 6 is a diagram illustrating an example of processing executed by the data processing device. FIG. 7 is a diagram illustrating an example of labeling data. FIG. 8 is a diagram illustrating an example of labeling after data migration. FIG. 9 is a diagram illustrating an overview of a machine learning model. FIG. 10 is a diagram illustrating changes in estimation accuracy of a machine learning model. FIG. 11 is a diagram illustrating the number of data items required for fine-tuning a machine learning model. FIG. 12 is a flowchart illustrating an example of a processing procedure executed by a data processing device. FIG. 13 is a diagram illustrating an example of a computer that executes a data processing program.

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a description will be given of an embodiment of the present invention with reference to the drawings, but the present invention is not limited to the embodiment.

[0011] [Overview] An overview of the data processing device of this embodiment will be described. The data processing device creates learning data to be used for fine-tuning a machine learning model, for example.

[0012] First, the data collected by the user for creating the learning data will be described. The data collected by the user is assumed to be data that indicates a state that changes continuously over time.

[0013] For example, consider a case where the state changes minutely and continuously over time, as shown by reference numeral 101 in Fig. 1. In this case, if the state value is measured at predetermined time intervals, the value will be rounded to some extent, as shown by reference numeral 102. The shorter the measurement time interval, the closer the value will be to the actual state.

[0014] Here, the data required as learning data for the above-mentioned fine tuning is, for example, data in which correct labels indicating the state of the data are associated with data such as video, image, and audio, as shown by reference numeral 201 in Fig. 2. For example, if there are three correct labels, 0, 1, and 2, it is necessary to prepare the amount of data required for learning for each of the labels 0, 1, and 2, as shown in Fig. 2.

[0015] For example, consider a case where the collected data (collected data) is the data shown by reference numeral 301 in Fig. 3. In this case, since the amount of data for label 1 in the collected data is small, the user needs to continue collecting data until the data for label 1 is measured again.

[0016] Here, since the collected data is data for a continuous state, as shown by the symbol 302, a portion of the data before the start time t1 of the level 1 data and a portion of the data after the end time t2 of the level 1 data can be regarded as label 1 data.

[0017] Therefore, for example, as shown by reference numeral 302, when the data processing device determines that the amount of data for label 1 is small, it regards a portion of the data for label 1 before the start time t1 (see reference numeral 303) and a portion of the data for level 1 after the end time t2 (see reference numeral 304) as data for label 1. In other words, the data processing device expands the range of the data for label 1 forward and backward.

[0018] This means that even if the data collected to create learning data has little data for one label, such as data with little fluctuation in values ​​over time and some missing data, the user does not need to continue collecting data until the data for that label is measured again, thereby reducing the time required to collect data.

[0019] [Configuration Example] Next, a configuration example of the data processing device 10 will be described with reference to Fig. 4. The data processing device 10 includes, for example, an input / output unit 11, a storage unit 12, and a control unit 14.

[0020] The input / output unit 11 is an interface that controls the input and output of various data. The storage unit 12 stores data, programs, etc. that are referenced when the control unit 14 executes various processes. The storage unit 12 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 12 stores parameters of a machine learning model, learning data of the machine learning model (e.g., data for fine tuning) output by the control unit 14, etc.

[0021] The control unit 14 is responsible for overall control of the data processing device 10. The functions of the control unit 14 are realized, for example, by a CPU (Central Processing Unit) executing a program stored in the storage unit 12.

[0022] The control unit 14 includes an information input unit 141, a label input unit 142, a combining unit (label assignment unit) 143, a specified value input unit 144, a constant input unit 145, a data amount adjustment unit (adjustment unit) 146, an adjustment data output unit 147, and a learning unit 148.

[0023] The information input unit 141, label input unit 142, and combining unit 143 will be described using FIG. 4 while referring to FIG.

[0024] The information input unit 141 accepts input of data that serves as the basis for learning data for a machine learning model. The data is, for example, data that indicates measured values ​​of a continuously changing state at predetermined intervals. The data is, for example, time-series data such as video, images, and audio acquired from a sensor or the like.

[0025] The label input unit 142 receives input of a label to be assigned to the data acquired by the information input unit 141. For example, if the data is an image of a user, the label may be input at a predetermined time interval (for example, t １ ) for each data item, an input of a label (correct label) for the data item is accepted (see reference numeral 501).

[0026] The combining unit 143 combines the data (e.g., video, image, audio, etc.) received by the information input unit 141 with the correct label received by the label input unit 142. In other words, the combining unit 143 assigns the correct label of the data received by the label input unit 142 to the data (e.g., video, image, audio, etc.) received by the information input unit 141.

[0027] As a result, for example, as shown in Figure 5, data group 1 (time t0≦t≦(t0+t1)) with label 0 is created, and data group 2 (time (t0+t1)≦t≦(t0+4t1)) with label 1 is created.

[0028] Next, the specified value input unit 144, constant input unit 145, data amount adjustment unit 146, and adjustment data output unit 147 will be described using FIG. 4 while referring to FIG.

[0029] The specified value input unit 144 inputs a specified value (for example, time: t d (Details will be described later) to the data amount adjustment unit 146. The constant input unit 145 also inputs a constant (for example, r 前 , r 後 (Details will be described later) is input to the data amount adjustment unit 146. d ), constants (e.g., r 前 , r 後 ) is, for example, a value input by the user.

[0030] The data amount adjustment unit 146 adjusts the amount of data (data volume) of the data group for each label. For example, the data amount adjustment unit 146 measures the amount of data to which each label is assigned, and when it determines that there is a label for which the amount of data is less than a predetermined threshold, it changes the label of data within a predetermined range before or after the measurement time of the data group for that label to that label.

[0031] The data amount adjustment unit 146 includes a measurement unit 1461, a determination unit 1462, and an adjustment unit 1463. The measurement unit 1461 measures the amount of data for each label for a data group to which a label has been assigned by the combination unit 143. The determination unit 1462 determines whether there is a label for which the amount of data is less than a predetermined threshold, based on the measurement result of the data amount for each label by the measurement unit 1461. If the determination unit 1462 determines that there is a label for which the amount of data is less than the predetermined threshold, the adjustment unit 1463 changes the label of data within a predetermined range before or after the measurement time of the data group for that label in chronological order to that label, and outputs the data.

[0032] For example, the measurement unit 1461 of the data amount adjustment unit 146 measures the data amount of the data group for each label (S1 in FIG. 6). Then, the determination unit 1462 of the data amount adjustment unit 146 determines whether the data amount measured in S1 is equal to or smaller than a predetermined threshold (for example, a specified value t d ) (S2: Does it meet the specified value? → No), the adjustment unit 1463 extracts part of the data (constant r 前 , r 後 ) of data is transferred (S4).

[0033] For example, the amount of data is less than a predetermined threshold (e.g., a predetermined value t d ) is “1”. In this case, the adjustment unit 1463 adjusts the time series of the data group immediately before the data group with label 1 (r 前 ) data (see reference numeral 303) and a part of the data group immediately after (r 後 ) data (see reference numeral 304) is changed to "1".

[0034] The adjustment unit 1463 adjusts the amount of data in the data group of each label after the data transfer by adjusting the amount of data in the data group of each label to a predetermined threshold (for example, a specified value t d Data migration will be carried out so that the total number of users does not fall below 100.

[0035] In S2 of FIG. 6, the determination unit 1462 determines whether the amount of data measured in S1 is equal to or greater than a predetermined threshold (for example, a specified value t d), the adjustment unit 1463 does not adjust the data amount of the data group of each label, but stores it in the memory unit 12 (S3).

[0036] The adjustment data output unit 147 outputs a data group (learning data) that satisfies a specified value for all of the label data groups. For example, the adjustment data output unit 147 stores, in the storage unit 12, the data group for each label whose data amount has been adjusted by the data amount adjustment unit 146, or the data group that is determined by the data amount adjustment unit 146 to satisfy a specified value for all of the label data groups, as learning data for the machine learning model.

[0037] 4 performs learning of the machine learning model using the learning data stored in the storage unit 12. For example, when new learning data is stored in the storage unit 12, the learning unit 148 performs fine tuning of the machine learning model using the learning data.

[0038] According to the data processing device 10 described above, it is possible to reduce the time required to collect learning data for a machine learning model.

[0039] [Application Example] An application example of the data processing device 10 will be described below. For example, assume that the actual state of time-series data used as training data for a machine learning model is as shown in the graph in Figure 7. Here, the data collection interval (measurement interval) is set to 0.3. Furthermore, the measured state (x) is divided into the following three stages, and labels 0, 1, and 2 are assigned to each state.

[0040] ・Label 0 (low group): 0≦x<3 ・Label 1 (middle group): 3≦x<6 ・Label 2 (high group): 6≦x<9

[0041] In such a case, the circled portions in the graph shown in FIG. 7 are the measured values, and the following states are measured.

[0042] ・Time 0≦t<0.3: Label 2 ・Time 0.3≦t<0.3: Label 1 ・Time 0.9≦t: Label 2

[0043] In this case, the amount of data for label 0 (low group) is small. Here, looking at the values in the state before time 0.9 (for example, 0.6 < t < 0.9) in the graph shown in FIG. 7, it can be seen that the values are those classified into the low group. Therefore, as shown in FIG. 8, the data processing device 10 can assign the label of label 0 (low group) to the time before time 0.9 (for example, 0.6 < t < 0.9) in the graph.

[0044] Thus, it can be seen that when the amount of data for any label is insufficient, the label of the data at the time of measurement can be assigned to the data at a time slightly before the time of measurement of the label.

[0045] Thus, according to the data processing device 10, even when the learning data indicating a certain state is insufficient, the deficiency of the learning data of the machine learning model can be supplemented. As a result, the time required for collecting the learning data can be reduced.

[0046] Further, the data processing device 10 may create learning data for fine-tuning a machine learning model that estimates the fatigue degree of a person using an image of the person.

[0047] For example, when the machine learning model is a model that estimates the fatigue degree of a person using the input image (video), the estimation accuracy may significantly decrease when the conditions of the input image change. Therefore, in order to prevent the decrease in the estimation accuracy as described above, fine-tuning of the machine learning model is performed using learning data.

[0048] The learning data for fine-tuning this machine learning model was created by the data processing device 10. Here, the fatigue degree is divided into three levels: low, medium, and high, and three labels of low, medium, and high are assigned to the learning data. Further, the machine learning model is a machine learning model that inputs a face image and a full-body image of a person and outputs an estimation result of the fatigue degree of the person (see FIG. 9).

[0049] The estimation accuracy (correct rate) of the machine learning model after training was, for example, the value shown in the table in Figure 10. However, when the date of the input image changed, the estimation accuracy dropped to around 30% for all input images: face image only, whole-body image only, and face image and whole-body image. Therefore, we considered fine-tuning the machine learning model using a small amount of training data every day.

[0050] First, an experiment was conducted on the amount of data required for fine-tuning the machine learning model. Tables 111, 112, and 113 in Figure 11 show the amount of data used for fine-tuning the machine learning model and the accuracy rate (estimation accuracy) after fine-tuning for face images only, full-body images only, and face images and full-body images, respectively. The number of images listed in each table in Figure 11 indicates the number of images for each label. In other words, since we are estimating three levels of fatigue in this study, the actual amount of data required is the number of images listed in each table multiplied by 3.

[0051] For example, as shown in Table 113 in Fig. 11, when both face images and whole-body images are used, an accuracy of 86% can be maintained with 500 images for each label, for a total of 1500 images. In this case, it takes about 2 minutes to collect 500 images.

[0052] To ensure a sufficiently high estimation accuracy for the machine learning model, it is desirable to collect at least 10 minutes of video data for each label (a total of at least 30 minutes).

[0053] Previous experiments have shown that if a person's fatigue level is low, once the fatigue level is measured, it will not change for about 15 minutes. However, if a person's fatigue level is high, it is likely to change if they take a break after measuring their fatigue level. For this reason, it has sometimes been difficult to collect data for 10 minutes from the time of measuring their fatigue level.

[0054] Therefore, if the data processing device 10 determines that sufficient data cannot be collected during the time from when the person's fatigue level was measured until they took a rest, the data processing device 10 will label data going back to a time before the measurement (for example, three minutes before), which makes it possible to acquire the amount of learning data necessary for fine-tuning the machine learning model.

[0055] 12, the data processing device 10 acquires a video of a person (S11) and measures the person's fatigue level in a questionnaire format (S12). The data processing device 10 then determines whether or not it has collected video of the same fatigue level for 10 minutes (S13). If the data processing device 10 determines that it has collected video of the same fatigue level for 10 minutes (Yes in S13), it assigns a label based on the fatigue level at the time of measurement (S14).

[0056] On the other hand, if the data processing device 10 determines in S13 that it has not been able to collect video of the same fatigue level for 10 minutes (No in S13), it goes back 3 minutes from the time of measurement and assigns a label to the video based on the fatigue level at the time of measurement (S15).

[0057] By having the data processing device 10 execute the above-described processing, it becomes possible to obtain the amount of training data necessary for fine-tuning the machine learning model.

[0058] [System Configuration, etc.] The components of each unit shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0059] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0060] [Program] The data processing device 10 can be implemented by installing a program (data processing program) as package software or online software on a desired computer. For example, by executing the program on an information processing device, the information processing device can function as the data processing device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

[0061] 13 is a diagram showing an example of a computer that executes a data processing program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0062] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0063] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes executed by the data processing device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the data processing device 10 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0064] Data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0065] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0066] REFERENCE SIGNS LIST 10 Data processing device 11 Input / output unit 12 Storage unit 14 Control unit 141 Information input unit 142 Label input unit 143 Combination unit (label assignment unit) 144 Specified value input unit 145 Constant input unit 146 Data amount adjustment unit 147 Adjusted data output unit 148 Learning unit 1461 Measurement unit 1462 Determination unit 1463 Adjustment unit

Claims

1. A data processing device comprising: a measurement unit that measures the amount of data for each label in a data group in which time-series data showing measurement values ​​of continuously changing states is assigned a label corresponding to the measurement value; a judgment unit that determines whether there is a label in which the amount of data is less than a predetermined threshold; and an adjustment unit that, if it is determined that there is a label in which the amount of data is less than the predetermined threshold, changes the label of data in a predetermined range before or after the measurement time of the data group in the time series with that label to that label and outputs the data.

2. The data processing device according to claim 1, characterized in that the adjustment unit changes the label of part of a data group immediately before or after a data group with a given label in chronological order to that label so that the amount of data for each label is equal to or greater than the predetermined threshold.

3. The data processing device according to claim 1, further comprising a learning unit that performs learning of a machine learning model using a data group in which the amount of data for each label has been determined by the determination unit to be equal to or greater than a predetermined threshold, or a data group in which the labels of some of the data have been changed by the adjustment unit.

4. A data processing method executed by a data processing device, comprising the steps of: measuring the amount of data for each label in a data group in which time-series data indicating measurement values ​​of continuously changing states is assigned a label corresponding to the measurement value; determining whether there is a label in which the amount of data is less than a predetermined threshold; and, if it is determined that there is a label in which the amount of data is less than the predetermined threshold, changing the label of data in a predetermined range in the time series before or after the measurement time of the data group with that label to that label and outputting the data.

Images