Learning device and learning method

Abstract

With respect to a machine learning model, this learning device updates the parameters of an extractor, a first estimator, and a second estimator so as to reduce estimation error with respect to information of a task performed by the first estimator and to increase estimation error with respect to information of a timing used by the second estimator. The machine learning model comprises: the extractor which extracts a feature amount from brain wave data; the first estimator which estimates, from the extracted feature amount, the information of a designated task; and the second estimator which estimates, from the extracted feature amount, the information of the timing at which the brain wave data is acquired. Thereafter, the learning device outputs the updated parameter of the extractor.

Description

Learning device and learning method

[0001] The present invention relates to a learning device and a learning method for training an extractor that extracts features from electroencephalogram data.

[0002] Various pieces of information related to perception and movement are embedded in EEGs, and a brain-computer interface is realized by extracting and utilizing this information. In recent years, extractors have been proposed that extract target information from EEGs with high accuracy by performing machine learning on large amounts of data (see Non-Patent Documents 1 and 2).

[0003] Robin Tibor Schirrmeister et al., 2018. “Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human EEG,” Human Brain Mapping 38 (11): 5391-5420.C. Spampinato et al., “Deep Learning Human Mind for Automated Visual Classification,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 6809-6817.Ren Li et al., 2021. “The Perils and Pitfalls of Block Design for EEG Classification Experiments,” IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (1): 316-333.

[0004] It is known that biological signals, including electroencephalograms, have temporal autocorrelation. It has been reported that if an extractor is trained without taking into account the temporal autocorrelation of electroencephalograms, it will focus on signals derived from temporal artifacts in the electroencephalograms and will fail to focus on signals derived from target information contained in the electroencephalograms (information corresponding to a task designated by a user) (see Non-Patent Document 3).

[0005] To avoid the above problem, it is conceivable to randomize the influence of autocorrelation noise by distributing the data into multiple blocks when creating machine learning training data, for example. However, with the above method, it is not possible to add training data when, for example, a machine learning task needs to be added.

[0006] Therefore, an object of the present invention is to solve the above-mentioned problems and to construct an extractor that can accurately extract features for estimating various information related to perception and movement, even when electroencephalogram data contains autocorrelation noise.

[0007] In order to solve the above-mentioned problems, the present invention provides a machine learning model including an extractor that extracts features from EEG data, a first estimator that estimates information about a specified task from the extracted features, and a second estimator that estimates information about the timing at which the EEG data was acquired from the extracted features, characterized in that the machine learning model further includes an update unit that updates parameters of the extractor, the first estimator, and the second estimator so as to reduce an estimation error of task information by the first estimator and increase an estimation error of timing information by the second estimator, and an output unit that outputs the updated parameters of the extractors.

[0008] According to the present invention, it is possible to construct an extractor that can accurately extract features for estimating various pieces of information related to perception and movement, even when electroencephalogram data contains autocorrelation noise.

[0009] Fig. 1 is a diagram illustrating an example of the configuration of a learning device according to a first embodiment. Fig. 2 is a flowchart illustrating an example of a processing procedure executed by the learning device according to the first embodiment. Fig. 3 is a diagram illustrating an example of the configuration of a learning device according to a second embodiment. Fig. 4 is a diagram illustrating an example of a computer that executes a learning program.

[0010] Hereinafter, with reference to the drawings, a description will be given of modes (embodiments) for carrying out the present invention. The present invention is not limited to each embodiment. Note that a task in this embodiment is, for example, to estimate, from electroencephalograms, various pieces of information relating to the perception and movement of a person whose electroencephalograms are being measured. It is assumed that the above task is specified in advance by a user of the learning device.

[0011] First Embodiment First, an overview of a learning device according to the first embodiment will be described. The learning device according to the first embodiment learns a machine learning model that estimates information corresponding to a task from the features of input electroencephalogram data, as well as information related to the timing at which the electroencephalogram data was acquired (e.g., date and time, location, session number, acquisition order, etc.).

[0012] Here, the learning device performs learning on the machine learning model so as to increase the estimation accuracy of information corresponding to a task, but decrease the estimation accuracy of information related to timing.

[0013] This enables the learning device to construct an extractor that accurately extracts features that focus on signals derived from task-related information contained in EEG data (e.g., signals corresponding to perception, movement, etc.), even when the EEG data contains autocorrelated noise that depends on the timing of acquisition.

[0014] As a result, the learning device can, for example, realize a small-scale feature extractor by partially using the training data, or realize a feature extractor that can handle a wider range of tasks by acquiring additional data.

[0015] [Configuration Example] Next, a configuration example of the learning device 10 will be described with reference to Fig. 1. The learning device 10 performs training on a machine learning model including a feature extraction unit (extractor) 131, a score estimation unit (first estimator) 132, and a session information estimation unit (second estimator) 133, as shown in Fig. 1. Here, the learning device 10 trains the above machine learning model so as to increase the estimation accuracy of the task score by the score estimation unit 132, but decrease the estimation accuracy of the session information by the session information estimation unit 133.

[0016] 1 , the learning device 10 includes, for example, a score input unit 111, a session information input unit 112, and an electroencephalogram input unit 113. The learning device 10 also includes a score estimation parameter input unit 121, a feature extraction parameter input unit 122, and a session information parameter input unit 123.

[0017] The learning device 10 also includes a feature extraction unit 131, a score estimation unit 132, and a session information estimation unit 133. The learning device 10 also includes a score estimation error calculation unit 141, a session information estimation error calculation unit 142, a total error calculation unit 143, a parameter update unit (update unit) 151, and a parameter output unit (output unit) 152.

[0018] The learning device 10 also includes a storage unit 16, which stores various data referenced by the learning device 10 when it executes processing, and processing results by the learning device 10. For example, the storage unit 16 stores parameters (feature extraction parameters, score estimation parameters, session information estimation parameters) of the calculation models used by the feature extraction unit 131, score estimation unit 132, and session information estimation unit 133.

[0019] Each unit will be described in detail below. The electroencephalogram input unit 113 receives input of electroencephalogram data. The electroencephalogram data is, for example, time-series data of electroencephalograms. The electroencephalogram data may be time-series data that has been preprocessed, such as by removing noise, or may be data obtained by converting acquired time-series data or preprocessed time-series data into electroencephalogram feature quantities such as power spectral density and instantaneous phase.

[0020] The score input unit 111 receives input of the task score (correct score value) for the electroencephalogram data.

[0021] For example, if the task is "estimating the type of movement," the task score is expressed by a one-hot vector or the like representing the type of movement. Also, if the task is "estimating an emotional state," the task score is expressed by a two-dimensional real vector or the like representing the valence and arousal strength that represent the emotional state. Also, if the task is "estimating what the user (the person whose EEG was being measured) was looking at when EEG data was acquired," the task score is expressed by a multidimensional one-hot vector or the like representing the category of what the user was looking at when EEG data was acquired. Also, if the task is "estimating the intensity of each emotional category," the task score is expressed by a multidimensional real vector or the like representing the intensity of each multidimensional emotional category.

[0022] The session information input unit 112 accepts input of the session information (correct value of the session information) of the electroencephalogram data.

[0023] Examples of session information: the location where the EEG data was acquired, the date and time of the EEG data, the block number if the data was acquired in multiple blocks, information indicating the order in which the EEG data was acquired, etc.

[0024] The score estimation parameter input unit 121 inputs a parameter (score estimation parameter) θ of a calculation model used by the score estimation unit 132. t The feature extraction parameter input unit 122 also receives input of the initial value of the parameter (feature extraction parameter) θ of the calculation model used by the feature extraction unit 131. f Furthermore, the session information parameter input unit 123 receives input of the initial value of the parameter (session information estimation parameter) θ of the calculation model used by the session information estimation unit 133. i Accepts input of the initial value.

[0025] The feature extraction unit 131 extracts features from the input electroencephalogram data based on a computational model (feature extraction model) in which feature extraction parameters are set. The feature extraction model is realized by, for example, a multi-layered neural network such as a deep learning model.

[0026] The score estimation unit 132 estimates information about a specified task (e.g., the score of the task) from the feature amounts of the electroencephalogram data extracted by the feature amount extraction unit 131. For example, the score estimation unit 132 estimates the score of the task from the feature amounts of the electroencephalogram data based on a calculation model (score estimation model) in which score estimation parameters are set. Note that the score estimation model is realized by a neural network such as a deep learning model or a multilayer perceptron model.

[0027] Furthermore, the score estimation unit 132 uses a calculation model suited to the format of the score to be estimated. For example, if the format of the task score is a one-hot vector or the like, the score estimation unit 132 uses a calculation model equipped with a classification layer. Furthermore, if the format of the task score is a real number vector or the like, the score estimation unit 132 uses a calculation model equipped with a regression layer.

[0028] The session information estimation section 133 estimates information (session information) about the timing at which the electroencephalogram data was acquired from the feature amounts of the electroencephalogram data extracted by the feature amount extraction section 131 .

[0029] For example, the session information estimation unit 133 estimates session information from the feature quantities of the electroencephalogram data based on a computational model (session information estimation model) in which session information estimation parameters are set. The session information model is also realized by a neural network such as a deep learning model or a multilayer perceptron model.

[0030] The session information estimation unit 133 uses a calculation model suited to the format of the session information to be estimated. For example, if the session information to be estimated is information with low temporal continuity, such as a location, date, or block number, the session information is expressed, for example, as a one-hot vector, which is a multidimensional vector. Therefore, the session information estimation unit 133 uses a calculation model equipped with a classification layer.

[0031] On the other hand, if the session information to be estimated is information with high temporal continuity, such as time or the order in which electroencephalogram data was acquired, the session information can be regarded as a continuous value. Therefore, the session information estimation unit 133 uses a computational model including a regression layer.

[0032] The score estimation error calculation unit 141 calculates the error L between the task score input by the score input unit 111 and the task score estimated by the score estimation unit 132. t Calculate.

[0033] For example, when the task score is information expressed by a one-hot vector or the like, the score estimation error calculation unit 141 calculates the error L t In addition, when the task score is information expressed as a real number vector, the score estimation error calculation unit 141 calculates the error L t Calculate.

[0034] The session information estimation error calculation unit 142 calculates the error L between the session information input by the session information input unit 112 and the session information estimated by the session information estimation unit 133. i Calculate.

[0035] For example, when the session information is expressed as a one-hot vector or the like, the session information estimation error calculation unit 142 calculates the error L i In addition, when the session information is expressed as a continuous value, the session information estimation error calculation unit 142 calculates the error L i Calculate.

[0036] That is, when the session information to be estimated is information with low temporal continuity, such as location, date, or block number, the learning device 10 performs information estimation and error estimation as a classification problem. On the other hand, when the session information to be estimated is information with high temporal continuity, such as time or the order in which EEG data was acquired, the learning device 10 performs information estimation and error estimation as a regression problem. This allows the learning device 10 to perform learning based on various session information.

[0037] The total error calculation unit 143 calculates the estimated error L t and the estimated error of the session information L i At this time, the total error calculation unit 143 calculates the total error L from the weighted sum of the estimated error L of the session information. i For example, the total error calculation unit 143 inverts the sign of L=L t -λL i Calculate.

[0038] The parameter update unit 151 updates the score estimation error L t and the session information estimation error L i For example, the parameter update unit 151 updates each parameter (feature extraction parameter, score estimation parameter, and session information estimation parameter) based on the estimated error L t and the session information estimation error L i Each parameter is updated so as to increase

[0039] For example, the parameter update unit 151 updates each parameter so as to minimize the total error L calculated by the total error calculation unit 143 .

[0040] If the computational model used by the feature extraction unit 131, the score estimation unit 132, and the session information estimation unit 133 is a neural network, the parameter update unit 151 updates the parameters based on, for example, backpropagation.

[0041] By performing the above processing, the parameter update unit 151 can update each parameter so as to improve the estimation accuracy of the task score (task information) by the score estimation unit 132 and reduce the estimation accuracy of the session information (timing information) by the session information estimation unit 133.

[0042] The parameter output unit 152 outputs the parameters updated by the parameter update unit 151. For example, the parameter output unit 152 outputs the feature extraction parameters updated by the parameter update unit 151.

[0043] [Example of Processing Procedure] Next, an example of processing procedure executed by the learning device 10 will be described with reference to Fig. 2. First, the learning device 10 accepts input of EEG data, a task score for the EEG data (correct value of the score), and session information for the EEG data (correct value of the session information) (S1: Acceptance of input of EEG data, score, and session information).

[0044] For example, the electroencephalogram input unit 113 accepts input of electroencephalogram data, the score input unit 111 accepts input of task scores for the electroencephalogram data, and the session information input unit 112 accepts input of session information for the electroencephalogram data.

[0045] The learning device 10 also receives input of each parameter (feature extraction parameter, score estimation parameter, and session information estimation parameter) (S2).

[0046] For example, the feature extraction parameter input unit 122 accepts input of initial values ​​of feature extraction parameters, the score estimation parameter input unit 121 accepts input of initial values ​​of score estimation parameters, and the session information parameter input unit 123 accepts input of initial values ​​of session information estimation parameters.

[0047] Next, the feature extraction unit 131 extracts features from the electroencephalogram data input in S1 using a feature extraction model in which the feature extraction parameters input in S2 are set (S3: Extraction of features from electroencephalogram data).

[0048] Thereafter, the score estimation unit 132 estimates a score from the feature amounts of the electroencephalogram data extracted in S3 (S4). For example, the score estimation unit 132 estimates the score of the task for the electroencephalogram data from the feature amounts of the electroencephalogram data extracted in S3 using a score estimation model in which the score estimation parameters input in S2 are set.

[0049] After S4, the score estimation error calculation unit 141 calculates the error (estimation error) between the score input in S1 and the score estimated in S4 (S5: calculate the score estimation error).

[0050] The session information estimation unit 133 also estimates session information from the feature amounts of the electroencephalogram data extracted in S3 (S6). For example, the session information estimation unit 133 estimates the session information of the electroencephalogram data from the feature amounts of the electroencephalogram data extracted in S3 using a session information estimation model in which the session information estimation parameters input in S2 are set.

[0051] After S6, the session information estimation error calculation unit 142 calculates the error (estimation error) between the session information of the electroencephalogram data input in S1 and the session information estimated in S6 (S7: Calculate the estimation error of the session information).

[0052] Thereafter, the total error calculation unit 143 calculates the estimated error (L t ) and the estimated error of the session information calculated in S7 (L i ) and calculates the total error (L). At this time, the total error calculation unit 143 inverts the sign of the estimated error of the session information (S8). For example, the total error calculation unit 143 calculates L=L t -λL i Calculate.

[0053] After S8, the parameter update unit 151 updates each parameter (feature extraction parameter, score estimation parameter, session information estimation parameter) based on the total error calculated in S8 (S9). Thereafter, the parameter output unit 152 outputs each parameter updated in S9 (S10).

[0054] According to the learning device 10 described above, even if the EEG data contains autocorrelated noise that depends on the timing of acquisition, it is possible to construct an extractor (feature extraction unit 131) that extracts features by focusing on signals derived from task-related information contained in the EEG data (e.g., signals corresponding to perception, movement, etc.).

[0055] Second Embodiment The learning device 10 may update the score estimation parameters and the session information estimation parameters based on a weighted sum of the score estimation error by the score estimation unit 132 and the session information estimation error by the session information estimation unit 133, and may update the feature extraction parameters based on a weighted sum of the score estimation error and a value obtained by inverting the sign of the session information estimation error. This embodiment will be described as the second embodiment. The same components as in the first embodiment are assigned the same reference numerals, and their description will be omitted.

[0056] The learning device 10a of the second embodiment includes a parameter update unit 151a, as shown in Fig. 3. The parameter update unit 151a updates the score estimation parameter and the session information estimation parameter based on a weighted sum of the score estimation error by the score estimation unit 132 and the session information estimation error by the session information estimation unit 133, and updates the feature extraction parameter based on a weighted sum of the above score estimation error and the above session information estimation error with their signs reversed.

[0057] According to this learning device 10a, even when the EEG data contains autocorrelation noise that depends on the timing of acquisition, it is possible to construct an extractor (feature extraction unit 131) that more accurately extracts features that focus on signals derived from task-related information contained in the EEG data.

[0058] [System Configuration, etc.] The components of each unit shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0059] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0060] [Program] The learning device 10 can be implemented by installing a program (learning program) as package software or online software on a desired computer. For example, by executing the program on an information processing device, the information processing device can function as the learning device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

[0061] 4 is a diagram showing an example of a computer that executes a learning program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0062] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0063] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes executed by the learning device 10 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, on the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the learning device 10 is stored on the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0064] Data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0065] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0066] 10, 10a Learning device 16 Memory unit 111 Score input unit 112 Session information input unit 113 Electroencephalogram input unit 121 Score estimation parameter input unit 122 Feature extraction parameter input unit 123 Session information parameter input unit 131 Feature extraction unit (extractor) 132 Score estimation unit (first estimator) 133 Session information estimation unit (second estimator) 141 Score estimation error calculation unit 142 Session information estimation error calculation unit 143 Total error calculation unit 151, 151a Parameter update unit (update unit) 152 Parameter output unit (output unit)

Claims

1. A learning device characterized by comprising, for a machine learning model comprising an extractor that extracts features from electroencephalogram data, a first estimator that estimates information about a specified task from the extracted features, and a second estimator that estimates information about the timing at which the electroencephalogram data was acquired from the extracted features, an update unit that updates parameters of the extractor, the first estimator, and the second estimator so as to reduce an estimation error of task information by the first estimator and increase an estimation error of timing information by the second estimator, and an output unit that outputs the updated parameters of the extractor.

2. The learning device described in claim 1, characterized in that the update unit updates the parameters of the extractor, the first estimator, and the second estimator based on a weighted sum of an estimation error of task information by the first estimator and a value obtained by inverting the sign of an estimation error of timing information by the second estimator.

3. The learning device described in claim 1, characterized in that the update unit updates the parameters of the first estimator and the second estimator based on a weighted sum of an estimation error of task information by the first estimator and an estimation error of timing information by the second estimator, and updates the parameters of the extractor based on a weighted sum of an estimation error of task information by the first estimator and a value obtained by inverting the sign of the estimation error of timing information by the second estimator.

4. A learning method executed by a learning device, comprising: a step of updating parameters of an extractor that extracts features from electroencephalogram data, a first estimator that estimates information about a specified task from the extracted features, and a second estimator that estimates information about the timing at which the electroencephalogram data was acquired from the extracted features, for a machine learning model comprising the extractor, the first estimator, and the second estimator so as to reduce the estimation error of task information by the first estimator and increase the estimation error of timing information by the second estimator; and a step of outputting the updated parameters of the extractor.

Images