Behavior inference system, measurement point selection device, behavior inference device, control method for information processing device, control method for behavior inference device, and program
The behavior estimation system uses brain wave data analysis at select points to objectively assess memorable scenes in videos, reducing processing load and improving evaluation efficiency.
Patent Information
- Application Number
- JP2020209743
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-17
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2040-12-17
AI Technical Summary
Conventional video evaluation methods rely on subjective assessments and struggle to objectively determine memorable scenes during or immediately after video broadcast, leading to high processing loads.
A behavior estimation system that collects brain wave data at select measurement points on a subject's head, identifies a representative point set based on functional connectivity analysis, and estimates behavior with reduced processing load by focusing on a smaller number of measurement points.
Enables objective evaluation of memorable scenes with reduced processing load, allowing for efficient estimation of video viewing behaviors and content memorability.
Smart Images

Figure 0007595858000001 
Figure 0007595858000002 
Figure 0007595858000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a behavior inference system, a measurement point selection device, a behavior inference device, a control method for an information processing device, a control method for a behavior inference device, and a program. [Background technology]
[0002] In evaluating video content, it is important to consider whether the scene being played back leaves an impression on the viewer, such as whether the viewer will remember it. For example, there is a known technology that allows a broadcasting station to obtain marketing information useful for improving the quality of program broadcast data by using data broadcasting (Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2014-217024 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the above-mentioned conventional techniques rely solely on subjective evaluation, and it is difficult to estimate whether or not a memorable scene was present during or immediately after the broadcast of a video.
[0005] The present invention has been made in consideration of the above-mentioned circumstances, and has as an object to provide a behavior inference system, a measurement point selection device, a behavior inference device, a control method for an information processing device, a control method for a behavior inference device, and a program that support a reduction in the processing load of behavior-related inference and enable behavior-related inference to be performed in a manner that involves a low processing load. [Means for solving the problem]
[0006] One aspect of the present invention that solves the problems of the conventional examples includes a measurement means for measuring electroencephalogram data of a subject performing a predetermined type of behavior at N predetermined measurement points (N is an integer where N>2) and M reference measurement points (M is a natural number) on the subject's head, and a selection means for selecting at least a pair of measurement points from the N measurement points as a representative point set based on functional connectivity analysis of the electroencephalogram data at a pair of measurement points selected from the N measurement points and the result of the behavior presented by the subject, and information identifying the representative point set is provided for acquiring electroencephalogram data at n points (n is an integer where N>n≧2) including the representative point set identified by the information of the subject performing the predetermined type of behavior, and providing the data to a process of estimating the result of the behavior of the subject. Effect of the Invention
[0007] According to the present invention, it is possible to estimate the outcome of an action based on electroencephalogram data at a relatively small number of n points. For example, even while watching a video, an objective evaluation based on electroencephalogram data can be performed with low processing load, thereby reducing the processing load of estimation regarding actions such as watching a video. [Brief description of the drawings]
[0008] [Figure 1] 1 is a configuration block diagram illustrating an example of a behavior inference system according to an embodiment of the present invention. [Diagram 2] 1 is a functional block diagram illustrating an example of a measurement point selection device according to an embodiment of the present invention. [Diagram 3] FIG. 4 is a flowchart illustrating an example of the operation of the measurement point selecting device according to the embodiment of the present invention. [Figure 4] 3 is an explanatory diagram illustrating an example of information held by a measurement point selecting device according to an embodiment of the present invention. FIG. [Diagram 5] FIG. 4 is a flowchart illustrating an example of an analysis process of the measurement point selection device according to the embodiment of the present invention. [Figure 6] 1 is a functional block diagram illustrating an example of a behavior inference device according to an embodiment of the present invention. [Figure 7] FIG. 4 is a flowchart illustrating an example of the operation of the behavior inference device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] An embodiment of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, a behavior estimation system 1 according to the embodiment of the present invention includes a measurement point selection device 10 and a behavior estimation device 20. Here, both the measurement point selection device 10 and the behavior estimation device 20 are information processing devices such as a computer. In the following description, the measurement point selection device 10 and the behavior estimation device 20 are separate devices, but one computer device may function as each device.
[0010] The behavior inference system 1 of this embodiment performs different processes at a measurement point selection stage in which a measurement point selection device 10 is used to select measurement points for an electroencephalogram, and at an inference processing stage in which a behavior result is inferred using a behavior inference device 20.
[0011] In the measurement point selection stage, a previously selected measurement subject (a person who cooperated with the experiment, hereafter referred to as the experimental subject) is asked to perform an action such as watching a video, while their brain wave data is measured at multiple measurement points. Separately, the experimental subject is provided with information on the results of their actions (e.g., by having them declare scenes in the video that made an impression on them), and some of the multiple measurement points mentioned above are selected as representative measurement points.
[0012] In the estimation processing stage, electroencephalogram data is measured at the selected representative measurement points for a measurement subject (hereinafter referred to as an estimated subject) who is engaged in the same type of behavior as the experimental subject. Then, the results of the behavior (information specifying an impressive scene in the video, etc.) are estimated based on the measured electroencephalogram data. In other words, in the estimation processing stage, processing is performed only on electroencephalogram data from fewer measurement points than the electroencephalogram data measured in the measurement point selection stage, so that the processing load is reduced, and as a result, estimation can be performed on a larger number of estimated subjects.
[0013] [Device configuration] As illustrated in FIG. 1, the measurement point selection device 10 used in the measurement point selection stage includes a control unit 11, a memory unit 12, an operation unit 13, an output unit 14, and an input / output unit 15, and the behavior estimation device 20 includes a control unit 21, a memory unit 22, an operation unit 23, an output unit 24, and an input / output unit 25.
[0014] In addition, in one example of this embodiment, a multi-channel electroencephalograph 30 is connected to the measurement point selection device 10, and an electroencephalograph 40 having a smaller number of channels (measurement points) than the multi-channel electroencephalograph 30 is connected to the behavior estimation device 20.
[0015] The control unit 11 of the measurement point selection device 10 is a program-controlled device such as a CPU, and operates according to a program stored in the storage unit 12. In one example of this embodiment, the control unit 11 receives, via the input / output unit 15, information on the electroencephalogram of a subject performing a predetermined action from the multi-channel electroencephalograph 30.
[0016] Here, the multi-channel electroencephalograph 30 measures and outputs electroencephalograms at a predetermined N measurement points (N is an integer greater than 2) on the subject's head and M reference measurement points (M is a natural number), and the control unit 11 accepts electroencephalogram data as the measurement results at these N measurement points and records the accepted electroencephalogram data (measurement results) in the memory unit 12.
[0017] In addition, the control unit 11 executes a process of selecting at least a pair of measurement points (a set of two measurement points) from the N measurement points as a representative point set based on a functional connectivity analysis of electroencephalogram data measured at a pair of measurement points (a set of two measurement points) selected from the N measurement points and information related to the results of the above behavior provided by the corresponding experimental subject. The detailed process of the control unit 11 will be described later.
[0018] The storage unit 12 is a memory device, a disk device, or the like, and holds a program executed by the control unit 11. In the example of the present embodiment, this program may be provided by being stored in a computer-readable and non-transitory recording medium, and may be stored in the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11, and holds electroencephalogram data, which is the measurement result, and the like.
[0019] The operation unit 13 is a mouse, a keyboard, or the like, and receives the contents of the operation performed by the user of the measurement point selection device 10, and outputs information indicating the contents of the operation to the control unit 11. The output unit 14 is a display, or the like, and displays a screen according to instructions input from the control unit 11.
[0020] The input / output unit 15 is an interface device or the like, and is connected to the multi-channel electroencephalograph 30. Further, the input / output unit 15 outputs the measurement results (electroencephalogram data) input from the multi-channel electroencephalograph 30 to the control unit 11.
[0021] In addition, in one example of this embodiment, this input / output unit 15 may be connected to a device (such as an optical scanner device) that accepts information related to the results of the behavior provided by the experimental subject, accepts the information output by the device, and outputs it to the control unit 11.
[0022] The control unit 21 of the behavior inference device 20 used in the inference processing stage is a program-controlled device such as a CPU, and operates according to a program stored in the storage unit 22. In one example of this embodiment, the control unit 21 receives electroencephalogram data of the inferred subject who is performing the same type of behavior as the behavior performed by the experimental subject from the electroencephalograph 40 via the input / output unit 25. Here, the electroencephalograph 40 is attached to the inferred subject so as to measure and output electroencephalogram data of the inferred subject at n points (n is an integer such that N>n≧2) including the representative point set selected by the measurement point selection device 10.
[0023] The control unit 21 then estimates the outcome of the behavior of the person to be estimated based on the acquired electroencephalogram data. The operation of the control unit 21 will also be described in detail later.
[0024] The storage unit 22 is a memory device, a disk device, or the like, and holds a program executed by the control unit 21. In the example of the present embodiment, this program is provided by being stored in a computer-readable and non-transitory recording medium, and may be stored in the storage unit 12. The storage unit 22 also operates as a work memory for the control unit 21, and holds information on the measurement results of the electroencephalograms, etc.
[0025] The operation unit 23 is a mouse, a keyboard, or the like, and receives the contents of an operation performed by the user of the behavior inference device 20, and outputs information indicating the contents of the operation to the control unit 21. The output unit 24 is a display, or the like, and displays information according to instructions input from the control unit 21.
[0026] The input / output unit 25 is an interface device or the like, and is connected to the electroencephalograph 40. The input / output unit 25 also outputs information on the measurement results input from the electroencephalograph 40 to the control unit 21.
[0027] [Functional configuration of the measurement point selection device] Next, the operation of the control unit 11 of the measurement point selection device 10 in the measurement point selection stage will be described. The control unit 11 executes a program stored in the storage unit 12 to realize a configuration that functionally includes a measurement result receiving unit 31, a measurement result recording unit 32, an information receiving unit 33, an analysis processing unit 34, a selection processing unit 35, and an output processing unit 36, as illustrated in FIG.
[0028] Here, the behavior of the experimental subject is assumed to be watching a video, and the video that the experimental subject is made to watch is assumed to be about 10 to 90 minutes long. The video is divided into "scenes" at predetermined time intervals (for example, 1 minute), and each scene is assigned a unique scene identifier.
[0029] A multi-channel electroencephalograph 30 is connected to the measurement point selection device 10 in the measurement point selection stage, and this multi-channel electroencephalograph 30 is connected to electrodes arranged at N measurement points (N is an integer greater than 2) on the subject's head and M reference measurement points (M is a natural number). Note that measurements are performed by injecting conductive gel between the subject's scalp and each electrode of the multi-channel electroencephalograph 30 and adjusting the contact impedance to 10 kΩ or less.
[0030] In this embodiment, 30 measurement points determined by the extended international 10 / 20 system are used, N measurement points are 28 points excluding measurement points A1 and A2 on the pinnae of both ears (i.e., N=28), and M reference measurement points are measurement points A1 and A2 on the pinnae of both ears (i.e., M=2).
[0031] This multi-channel electroencephalograph 30 includes an electroencephalogram amplifier and a PC (personal computer) for electroencephalogram measurement, and electrodes attached to each measurement point are connected to the electroencephalogram amplifier via a cable. The multi-channel electroencephalograph 30 uses at least one of the M reference measurement points as reference data, subtracts the reference data from each measurement data (raw data) measured at the N measurement points, and amplifies the result with the electroencephalogram amplifier to obtain electroencephalogram data at each measurement point (differential amplification).
[0032] The multi-channel electroencephalograph 30 also records the eye movements and blinks of the subject during the action by a separate electrooculography measurement. The multi-channel electroencephalograph 30 inputs the electroencephalogram data at each measurement point, which is differentially amplified by the electroencephalogram amplifier, to the PC for electroencephalogram measurement as the electroencephalogram data of the measurement result. The PC for electroencephalogram measurement monitors the electroencephalogram data of the measurement result, and outputs the electroencephalogram data of the measurement result to the measurement point selection device 10 of this embodiment.
[0033] This multi-channel electroencephalograph 30 measures the electroencephalograms at each of the measurement points while the subject is performing a specified action, and outputs the measurement results, that is, electroencephalogram data, to the measurement point selection device 10 in sequence.
[0034] The measurement result receiving unit 31 receives, from a user (operator) of the measurement point selecting device 10, input of subject identification information that identifies an experimental subject.
[0035] While the subject is watching the video, the measurement result receiving unit 31 sequentially receives the measurement result information (electroencephalogram data, which is the measurement result at each of the N measurement points. Hereinafter, a collection of electroencephalogram data at all measurement points will be referred to as multi-channel electroencephalogram information) output by the multi-channel electroencephalograph 30, and outputs it together with subject identification information to the measurement result recording unit 32. In the following explanation, it is assumed that the electroencephalogram data is input to the measurement point selecting device 10 in a state in which it is possible to identify which measurement point the electroencephalogram data is from.
[0036] The measurement result recording unit 32 receives subject identification information from the measurement result receiving unit 31, and sequentially receives input of multi-channel electroencephalogram information, and stores this multi-channel electroencephalogram information in the memory unit 12 in association with the received subject identification information. At this time, the measurement result recording unit 32 records information (time information) specifying the measurement time point in association with the multi-channel electroencephalogram information. In this example, the time information is set to 0 for the start time of the action (here, the start time of the video playback), and T is the elapsed time from the start time of the action to the measurement time point.
[0037] By the operation of this measurement result recording section 32, the memory section 12 stores the electroencephalogram data, which are the measurement results at the N measurement points, in chronological order for each subject.
[0038] The information receiving unit 33 receives information relating to the results of the behavior from each experimental subject. In this example, the information received is information identifying scenes that the experimental subject answered as having left an impression (e.g., "I remember"). The information receiving unit 33 may further receive information identifying scenes that the experimental subject answered as having no impression (e.g., "I don't remember"). Hereinafter, the scenes that the experimental subject answered as having left an impression are referred to as "memorable scenes," and the scene identifiers identifying the memorable scenes are referred to as "memorable scene information." Hereinafter, the scenes that the experimental subject answered as having no impression are referred to as "non-memorable scenes," and the scene identifiers identifying the non-memorable scenes are referred to as "non-memorable scene information."
[0039] Specifically, this answer is acquired as follows. That is, here, after the subject watches a video, the operator of the measurement point selection device 10 presents the subject with a key image (which may be a still image) that has been selected in advance for each scene of the video that the subject watched. The subject is then asked to answer at least one (for example, five) key images that he remembers best (that left the strongest impression).
[0040] The operator of the measurement point selection device 10 inputs into the measurement point selection device 10 the scene related to the answered key image as a memorized scene, and memorized scene information that identifies the memorized scene.
[0041] Furthermore, when the information receiving unit 33 receives information identifying a non-memorized scene, the operator of the measurement point selection device 10 has the subject answer at least one (e.g., five) key images that he or she does not remember (is not memorable) from among the previously presented key images. In this case, the operator of the measurement point selection device 10 inputs non-memorized scene information identifying the non-memorized scene into the measurement point selection device 10, regarding the scene related to the answered key image as a non-memorized scene.
[0042] The information accepting unit 33 associates the memorized scene information input based on the answer with the subject person identifying information and stores it as answer information in the storage unit 12. When non-memorized scene information is accepted, the information accepting unit 33 stores the non-memorized scene information together with the memorized scene information as answer information in the storage unit 12 in association with the subject person identifying information.
[0043] The analysis processing unit 34 reads out the multi-channel electroencephalogram information and response information stored in the memory unit 12 from the memory unit 12, and sequentially selects the multi-channel electroencephalogram information associated with each subject identification information as the processing target to perform the next processing.
[0044] The analysis processing unit 34 performs preprocessing on each piece of electroencephalogram data included in the multi-channel electroencephalogram information selected as the processing target. The preprocessing may be any widely known preprocessing of electroencephalogram data, such as a process for removing the influence of blinking from electroencephalogram data during blinking (a process using independent component analysis may be used).
[0045] Then, the analysis processing unit 34 divides the preprocessed electroencephalogram data at the N measurement points into electroencephalogram data (per unit behavior) for the time period during which each scene was viewed. Since the scenes are divided into one minute intervals from the start of playback of the video, the electroencephalogram data may be divided into scenes based on the corresponding recorded time information.
[0046] In the following, for ease of explanation, the subject of the experiment related to the electroencephalogram data to be processed is denoted as u (u=u1, u2, ...), the measurement point is denoted as k (k=1, 2, ...N), and the scene identifier for identifying the scene is denoted as i (i=1, 2, ...). In this way, among the electroencephalogram data of the experimental subject u at the measurement point k, the divided electroencephalogram data when the experimental subject u was watching the scene with the scene identifier i is written as E_u(k, i). In addition, for the sake of distinction, the electroencephalogram data E_u(k, i) obtained by dividing is called divided electroencephalogram data.
[0047] The analysis processing unit 34 performs a predetermined process such as filtering, for example, applying a bandpass filter in the theta band (4 to 7 Hz), on each of the divided electroencephalogram data E_u(k,i) of the experimental subject u. Then, the analysis processing unit 34 extracts the electroencephalogram (here, the theta band component) T_u(k,i) after the process. Hereinafter, the extracted electroencephalogram data (here, the theta band component) is referred to as the electroencephalogram data to be analyzed.
[0048] The analysis processing unit 34 obtains a set of analysis target electroencephalogram data {T_u(p,i),T_u(q,i)} (i=1,2,..., p,q=1,2,...N, where p≠q) at two different measurement points from the analysis target electroencephalogram data T_u(k,i) of the experimental subject u. Here, the analysis processing unit 34 obtains sets of analysis target electroencephalogram data {T_u(p,i),T_u(q,i)} relating to all possible pairs of measurement points.
[0049] The analysis processing unit 34 selects each of the pair of electroencephalogram data to be analyzed at the pair of measurement points in the group, performs functional connectivity analysis on the selected pair of electroencephalogram data to be analyzed Tu(p,i) and Tu(q,i), and obtains an analysis result C_u(p,q,i).
[0050] The functional connectivity analysis evaluates the phase synchrony based on the phase difference between a selected pair of analysis target electroencephalogram data Tu(p,i) and Tu(q,i), and specific examples of the phase synchrony include information such as PLV (Phase Locking Value), iPLV (imaginary PLV), ciPLV (corrected imaginary PLV) obtained by a method derived from the method of obtaining PLV, PLI (Phase Lag Index), wPLI (weighted PLI), dwPLI (debiased wPLI) obtained by a method derived from the method of obtaining PLI, or PPC (Pairwise Phase Consistency), wPPC (weighted PPC) obtained by a method derived from the method of obtaining PPC, etc. The processing of this functional connectivity analysis is not limited to the evaluation of the phase synchrony, and may be processing for analyzing amplitude-phase coupling, Granger causality, transfer entropy, PSI (Phase Slope Index), coherence, etc. In the following description, the case of evaluating the phase synchronism will be taken as an example.
[0051] The analysis processing unit 34 also extracts the memory scene information (i=Mu1, Mu2, ...) related to the experimental subject u from the response information associated with the subject identification information to be processed. The analysis processing unit 34 then extracts the analysis results C_u(p,q,Mu1), ..., C_u(p,q,Mu2), ... related to the memory scene information from the analysis results C_u(p,q,i) of the functional connectivity analysis, and obtains a statistical value (e.g., average) CM_u(p,q) of the analysis results for each set of common measurement points. Hereinafter, the analysis processing unit 34 calculates the average as the statistical value, and the statistical value obtained here is called the memory-time phase synchrony average CM_u(p,q).
[0052] The analysis processing unit 34 also extracts the results C_u(p, q, j) (where j is not M1, M2, ...) of the functional connectivity analysis for the experimental subject u relating to scenes other than the memory scene, and calculates the average non-memory phase synchrony CN_u(p, q), which is an example of its statistical value.
[0053] The analysis processing unit 34 performs the above processing for each experimental subject, and obtains the stored phase synchrony degree average and the non-stored phase synchrony degree average for each experimental subject.
[0054] For each combination (p, q) of two measurement points, the selection processing unit 35 compares the memory-time phase synchrony average CM_u1(p, q), CM_u2(p, q), ... with the non-memory-time phase synchrony average CN_u1(p, q), CN_u2(p, q), ... for each experimental subject u (u = u1, u2, ...), and selects as a representative point set the combination (p, q) of measurement points whose difference is judged to be statistically significant at a predetermined significance level.
[0055] One of the features of this embodiment is that when the selection processing unit 35 selects a representative point set because the difference between the stored phase synchrony average and the non-stored phase synchrony average is significant, it generates criterion information indicating which of the stored phase synchrony average and the non-stored phase synchrony average is larger, and outputs it together with information specifying the representative point set (information specifying a pair of measurement points). This criterion information is, for example, information such as "+" when the stored phase synchrony average is larger than the non-stored phase synchrony average, and "-" when the stored phase synchrony average is smaller than the non-stored phase synchrony average.
[0056] In this manner, in this embodiment, by performing calculations using statistics of phase synchrony for multiple experimental subjects, a smaller number of measurement points that show significant differences between memory and non-memory states are selected, eliminating the influence of individual differences.
[0057] The output processing unit 36 outputs information specifying the combination of measurement points selected by the selection processing unit 35 (information including at least one piece of information (p, q) specifying the combination of measurement points) as a representative point set. As an example, the output processing unit 36 may store the information specifying the representative point set selected by the selection processing unit 35 (information representing the combination of measurement points) in the storage unit 12.
[0058] [Other examples of analysis and selection processing] Also, in the explanation given here, the analysis processing unit 34 determines the phase synchrony of the electroencephalogram data at a pair of measurement points based on the electroencephalogram data obtained while the experimental subject was viewing a memorable scene and the electroencephalogram data obtained while the subject was viewing a scene other than the memorable scene, but this embodiment is not limited to this example.
[0059] For example, as already mentioned, when non-memorable scene information (information about scenes that the subject responded as not leaving an impression) is obtained from the subject in addition to memorable scene information, the following processing may be carried out.
[0060] In this example, the analysis processing unit 34 first processes the memory scene in the same manner as described above. That is, the analysis processing unit 34 extracts the memory scene information (i=Mu1, Mu2, ...) related to the experimental subject u from the answer information associated with the subject identification information to be processed. Then, the analysis processing unit 34 extracts the analysis results C_u(p,q,Mu1), ..., C_u(p,q,Mu2), ... related to the memory scene information from the analysis results C_u(p,q,i) of the functional connectivity analysis, and calculates the memory phase synchrony average CM_u(p,q), which is a statistical value of the analysis results for each set of common measurement points.
[0061] The analysis processing unit 34 also extracts the results C_u(p, q, Nu1), ..., C_u(p, q, Nu2), ... of the functional connectivity analysis related to the non-memory scene information (j = Nu1, Nu2 ...) for each experimental subject u, and calculates the non-memory phase synchrony average CN_u(p, q), which is a statistical value of the analysis results for each set of common measurement points.
[0062] Then, for each combination (p, q) of two measurement points, the selection processing unit 35 compares the memory-time phase synchrony average CM_u1(p, q), CM_u2(p, q), ... for each experimental subject u (u = u1, u2, ...) with the non-memory-time phase synchrony average CN_u1(p, q), CN_u2(p, q), ..., and selects as a representative point set the combination (p, q) of measurement points whose difference is judged to be statistically significant at a predetermined significance level.
[0063] [Operation of the measuring point selection device] Next, the operation of the measurement point selection device 10 according to the above configuration will be described. Electrodes with a conductive gel applied thereto and adjusted to a contact impedance of 10 kΩ or less are arranged at 30 measurement points (28 points including Fp1, Fp2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, T8, C3, Cz, C4, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, O1, Oz, and O2, in addition to the two reference measurement points A1 and A2) on the head surface of each of the multiple experimental subjects u1, u2, ..., determined by the extended international 10 / 20 system, and connected to the multi-channel electroencephalograph 30.
[0064] Here, the multi-channel electroencephalograph 30 may be, for example, LiveAmp (registered trademark) from Brain Products, a German company.
[0065] Each subject is then made to watch a 25-minute video while the multi-channel electroencephalograph 30 acquires electroencephalogram data at each of the measurement points for 25 minutes.
[0066] The multi-channel electroencephalograph 30 uses the electroencephalogram data measured for each experimental subject at reference electrodes (predetermined here as electrodes placed at measurement points A1 and A2 on the pinnae of both ears) as reference data, and obtains the electroencephalogram data of the experimental subject while watching a video at 28 measurement points other than the reference electrodes by differential amplification.
[0067] The multi-channel electroencephalograph 30 then sequentially outputs sets of electroencephalogram data for each test subject obtained by differential amplification as multi-channel electroencephalogram information for each test subject.
[0068] The operator of the measurement point selection device 10 inputs subject identification information that identifies the experimental subject, and causes the measurement point selection device 10 to accept the multi-channel electroencephalogram information of the experimental subject identified by the subject identification information, which is sequentially output by the multi-channel electroencephalograph 30.
[0069] The measurement point selection device 10 performs the process shown in Fig. 3 according to the instruction of the operator, receives input of subject identification information and sequentially receives input of multi-channel electroencephalogram information, and stores this multi-channel electroencephalogram information in the storage unit 12 in association with the received subject identification information (S11). As a result, the storage unit 12 stores the electroencephalogram data at the N measurement points in chronological order for each experimental subject, as shown in Fig. 4.
[0070] Meanwhile, the operator of the measurement point selection device 10 presents the experimental subjects, who have finished watching the 25-minute video, with key images selected for each scene in the video (in this case, the video is divided into one-minute segments, each of which is a separate scene), and asks them to answer about the five key images they remember best (that made the biggest impression) and the five key images they do not remember (that made the least impression).
[0071] In addition, when the operator of the measurement point selection device 10 obtains a response from the experimental subject u, the operator inputs information Mu1, Mu2, Mu3, Mu4, and Mu5 identifying the scene related to the key image which the experimental subject answered that he remembers best (left an impression), and information Nu1, Nu2, Nu3, Nu4, and Nu5 identifying the scene related to the key image which the experimental subject answered that he does not remember (left an impression), into the measurement point selection device 10 as remembered scene information and non-memorized scene information, respectively, together with subject identification information identifying the experimental subject who answered.
[0072] The measurement point selecting device 10 receives the answer information including the memorized scene information and the non-memorized scene information, and the subject specifying information, and stores them in the storage unit 12 in association with each other (S12).
[0073] The measurement point selection device 10 repeatedly executes steps S11 and S12 until multi-channel electroencephalogram information and response information for all experimental subjects have been input, and when information input for all experimental subjects has been completed, starts the following processing at the instruction of the operator.
[0074] That is, the measurement point selection device 10 reads out the multi-channel electroencephalogram information and response information stored in the memory unit 12 from the memory unit 12, sequentially selects each experimental subject identified by the subject identification information as a subject (S13), and executes the following processing.
[0075] The measurement point selection device 10 reads out the multi-channel EEG information associated with the subject identification information of the selected subject u, and performs pre-processing, such as processing to remove the effects of blinking, on each piece of EEG data contained in the multi-channel EEG information (S14).
[0076] The measurement point selection device 10 divides the preprocessed electroencephalogram data at the N measurement points into electroencephalogram data for each minute from the start of viewing the video, to obtain divided electroencephalogram data E_u(k,i) (S15). In this example, the video is divided into scenes every minute, so this process corresponds to dividing the electroencephalogram data into the electroencephalogram data for the time period during which each scene was viewed. Here, k (k=1,2,...N) represents the measurement points, and i corresponds to the scene identifier (i=1,2,...).
[0077] The measurement point selection device 10 applies a bandpass filter in the theta band (4 to 7 Hz) to each of the divided EEG data E_u(k,i) obtained in step S15, and extracts the EEG data to be analyzed T_u(k,i), which is the theta band component (S16).
[0078] Next, the measurement point selection device 10 executes an analysis process (S17). In this analysis process, the measurement point selection device 10 lists pairs of measurement points (p, q) that are different from each other, as shown in Fig. 5 (S31). In this example, there are 28 measurement points (excluding the reference measurement points), namely Fp1, Fp2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, T8, C3, Cz, C4, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, O1, Oz, and O2. A set of (Fp1, Fp2), The set (Fp1, F7), … The pair (Fp1, O2), The set (Fp2, F7), … (Oz,O2) pair Thus, there are 378 combinations, which is the number of combinations in which 2 points are taken out of 28 points. The measurement point selection device 10 lists these combinations and then selects a set of electroencephalogram data to be analyzed for each combination for each of the scene identifiers Mu1, Mu2, Mu3, Mu4, Mu5, Nu1, Nu2, Nu3, Nu4, and Nu5 included in the response information of the subject u. {T_u(Fp1,i),T_u(Fp2,i)}, {T_u(Fp1,i),T_u(F7,i)},… Then, for each set of EEG data to be analyzed, the dwPLI value as the analysis result of the functional connectivity analysis was: C_u(Fp1,Fp2,i), C_u(Fp1,F7,i),… where i=Mu1, Mu2, Mu3, Mu4, Mu5, Nu1, Nu2, Nu3, Nu4, Nu5 (S32). In this way, the measurement point selection device 10 obtains the analysis result of the functional connectivity analysis of the analysis target electroencephalogram data when the subject u viewed the memory scene, and the analysis result of the functional connectivity analysis of the analysis target electroencephalogram data when the subject u viewed the non-memory scene.
[0079] The measurement point selection device 10 selects, for each set of electroencephalogram data to be analyzed, an average dwPLI value (average phase synchrony at the time of memory), which is an analysis result of a functional connectivity analysis of the electroencephalogram data to be analyzed when the memory scene is viewed. CM_u(Fp1,Fp2), CM_u(Fp1,F7),… (S33).
[0080] In addition, the measurement point selection device 10 calculates, for each set of electroencephalogram data to be analyzed, the average dwPLI value (average non-memory phase synchrony) which is the analysis result of the functional connectivity analysis of the electroencephalogram data to be analyzed when viewing a non-memory scene. CN_u(Fp1,Fp2), CN_u(Fp1,F7),… (S34).
[0081] The measurement point selection device 10 returns to step S18 in FIG. 3, and then returns to step S13, where it selects a new subject u that has not been selected so far and repeats the processes from steps S14 to S17 until all the experimental subjects have been processed.
[0082] After determining the average phase synchrony at memory time and the average phase synchrony at non-memory time for all experimental subjects, the measurement point selection device 10 compares (S19) the average phase synchrony at memory time CM_u1(p,q), CM_u2(p,q), ... with the average phase synchrony at non-memory time CN_u1(p,q), CN_u2(p,q), ... for each experimental subject u (u = u1, u2, ...) for each combination of pairs of measurement points (p, q) = (Fp1, Fp2), (Fp1, F7), ... that are the same as those listed in step S31, and determines whether the difference between them is statistically significant at a predetermined significance level.
[0083] Then, the measurement point selection device 10 selects as a representative point set a combination of a pair of measurement points for which the difference between the stored phase synchrony average and the non-stored phase synchrony average is judged to be statistically significant at a predetermined significance level (S20), and outputs the selected representative point set (S21).
[0084] When the measurement point selection device 10 selects the representative point set in step S19, it generates criterion information indicating which of the stored phase synchrony averages, the difference of which is significant, is larger, and outputs this together with information identifying the representative point set (information identifying a pair of measurement points) in step S20. This criterion information is, for example, "+" when the stored phase synchrony average is larger than the non-stored phase synchrony average, and "-" when the stored phase synchrony average is smaller than the non-stored phase synchrony average.
[0085] Specifically, in one example of this embodiment, the measurement point selection device 10 tests, for each combination (p, q) of a pair of measurement points, a null hypothesis that there is no statistically significant difference between the distribution of the memory-time phase synchrony averages CM_u1(p, q), CM_u2(p, q), ... for each experimental subject u (u=u1, u2, ...) and the distribution of the non-memory-time phase synchrony averages CN_u1(p, q), CN_u2(p, q), ... by t-test, and determines whether the null hypothesis can be rejected at a predetermined significance level (e.g., 5% or 1%).The measurement point selection device 10 then selects the combination (p, q) of measurement points for which the null hypothesis can be rejected at a predetermined significance level as a representative point set.
[0086] In one example of this embodiment, when the significance level is set to 5%, the measurement point selection device 10: (FC5,Fp2)(+) (O2,Fp2)(-) (P7,T7)(+) (T7,CP5)(+) (CP5,FC6)(+) (FC2,FC6)(-) (FC6,F4)(-) (FC5,FC6)(+) (Pz,CP6)(+) (CP2,CP6)(+) (CP1,FC5)(-) (CP1,FC2)(-) (C4,Fp2)(+) These 13 pairs were selected as representative point pairs because they showed significant differences. Here, (+) or (-) is the criterion information, and (+) is used when the average phase synchrony degree when stored is greater than the average phase synchrony degree when not stored, and (-) is used when the average phase synchrony degree when not stored is greater than the average phase synchrony degree when stored.
[0087] Furthermore, when the significance level was set to 1%, the measurement point selection device 10 determined that only the combination (FC5, Fp2)(+) had a significant difference, and selected this pair as the representative point pair.
[0088] In other words, in this embodiment, it is assumed that functional connections exist between channel pairs in which a significant difference in the phase synchrony of EEG is observed between when a video scene is remembered and when it is not, as in the example here, and the channel pairs determined to have functional connections are set as representative point pairs.
[0089] [Another example of representative point set] In the explanation so far, the representative point set is selected based on the phase synchrony of the EEG data at a pair of measurement points, and is therefore selected as a pair of measurement points, but this embodiment is not limited to this.
[0090] For example, the phase synchrony between EEG data for each combination of three or more measurement points may be evaluated, and while viewing a memorable scene, a combination of measurement points whose phase synchrony is significantly different from the phase synchrony between EEG data associated with other combinations of measurement points may be found, and the measurement points associated with the found combination may be selected as a representative point set.
[0091] [Functional configuration of the behavior estimation device] Next, a description will be given of the operation of the control unit 21 of the behavior inference device 20. The control unit 21 executes a program stored in the storage unit 22 to realize a configuration that functionally includes an electroencephalogram information acquisition unit 41, an inference processing unit 42, and an output processing unit 43, as exemplified in FIG.
[0092] The electroencephalograph 40 connected to the behavior inference device 20 focuses on a network between at least a pair of measurement points included in the representative point set selected by the measurement point selection device 10, and measures electroencephalogram data at the measurement points. Therefore, the electroencephalograph 40 is simpler than the multi-channel electroencephalograph 30, and the device cost and the burden required for measurement are relatively light.
[0093] In one example of this embodiment, the electroencephalograph 40 sequentially acquires electroencephalogram data of the subject in action, and measures electroencephalogram data at a total of four measurement points, including two measurement points FC5 and Fp2 (measurement points selected by the measurement point selection device 10 at a significance level of 1%) on the head surface of the subject, and measurement points A1 and A2 on the pinnae of both ears as reference measurement points.The electroencephalograph 40 subtracts the electroencephalogram data measured at the measurement points A1 or A2 as reference data from each of the electroencephalogram data measured at the two points FC5 and Fp2 (raw data), amplifies the data, and outputs it as electroencephalogram data at each of the points FC5 and Fp2.Note that FC5 and Fp2 are both measurement points defined by the extended international 10 / 20 system.
[0094] Also, for ease of explanation, the behavior is assumed to be watching a video, and the behavior estimation device 20 is assumed to estimate whether a scene contained in the video was memorable for the person to be estimated (whether it became a memorable scene).
[0095] The electroencephalogram information acquiring unit 41 sequentially receives the input of electroencephalogram data of the estimation subject during behavior at each of the above-mentioned points FC5 and Fp2 from the electroencephalograph 40. In the following description, it is assumed that the electroencephalogram data is input to the behavior estimation device 20 in a state in which it is possible to identify which measurement point the electroencephalogram data is from.
[0096] The estimation processing unit 42 receives and stores information specifying the representative point set selected by the measurement point selection device 10 for a given behavior and judgment criteria information for the representative point set. When the brain wave information acquisition unit 41 sequentially receives each piece of brain wave data, the estimation processing unit 42 divides each piece of brain wave data at a given timing to obtain divided brain wave data E(k,i) (k represents a measurement point, i is an identifier that identifies the unit of behavior, and in this example, i is a scene identifier, where i=1, 2, ...) for each unit of behavior (for each scene being viewed in the case of video viewing). For example, the estimation processing unit 42 extracts the brain wave data at each point received in the past minute every minute as divided brain wave data at each measurement point.
[0097] The estimation processing unit 42 performs a predetermined filtering process or the like (equivalent to the process performed by the measurement point selection device 10, for example, a process of applying a bandpass filter in the theta band (4 to 7 Hz) in this case) on each of the extracted divided EEG data, and obtains the EEG data to be analyzed T(k,i) from each divided EEG data E(k,i) (here, by extracting the theta band components).
[0098] The estimation processing unit 42 then performs functional connectivity analysis on the analysis target EEG data T(p,i) and T(q,i) at a pair of measurement points (p,q) related to the analysis target, for at least a pair of measurement points including the representative point set selected by the measurement point selection device 10, and obtains the analysis result C(p,q,i).
[0099] The functional connectivity analysis performed by the estimation processing unit 42 is the same as that in the measurement point selection device 10, and is used to evaluate the phase synchrony based on the phase difference between a pair of electroencephalogram data to be analyzed. Specifically, this functional connectivity analysis is performed by employing a well-known method for obtaining any of the following information: PLV (Phase Locking Value), iPLV (imaginary PLV), ciPLV (corrected imaginary PLV), etc. derived therefrom, PLI (Phase Lag Index), wPLI (weighted PLI), dwPLI (debiased wPLI), etc. derived therefrom, or PPC (Pairwise Phase Consistency), wPPC (weighted PPC), etc. derived therefrom. The processing of this functional connectivity analysis is not limited to the evaluation of phase synchrony, and may be processing for analyzing amplitude-phase coupling, Granger causality, transfer entropy, PSI (Phase Slope Index), coherence, etc.
[0100] For each set of representative points, the estimation processing unit 42 refers to the result C(p, q, i) of the functional coupling analysis corresponding to each scene identifier (for example, the value of dwPLI) and the corresponding determination criterion information, and searches for a scene identifier among the analysis results C(p, q, i) of the set of representative points (p, q) corresponding to scene identifiers i = 1, 2, … that satisfies the determination criterion indicated by the determination criterion information corresponding to the set of representative points (p, q).
[0101] Specifically, when the determination criterion information is "+", the estimation processing unit 42 searches for a scene identifier j such that C(p, q, j) > CM(p, q) and C(p, q, j) - CM(p, q) is greater than a predetermined threshold with respect to the average CM(p, q) of the analysis results C(p, q, i) related to the set of representative points (p, q). Then, when such a scene identifier j is found, the estimation processing unit 42 determines that the scene identified by the found scene identifier is a scene that leaves an impression.
[0102] Also in this case, for the set of representative points (p, q) where the determination criterion information is "-", the estimation processing unit 42 searches for a scene identifier j such that C(p, q, j) < CM(p, q) and CM(p, q) - C(p, q, j) is greater than a predetermined threshold with respect to the average CM(p, q) of the analysis results C(p, q, i) related to the set of representative points (p, q). Then, when such a scene identifier j is found, the estimation processing unit 42 determines that the scene identified by the found scene identifier is a scene that leaves an impression.
[0103] Through such processing, the estimation processing unit 42 finds the scene identifier of the scene estimated to leave an impression. Then, the output processing unit 43 outputs the scene identifier found by the estimation processing unit 42.
[0104] [Operation of the Action Estimation Device] Next, the operation of the behavior inference device 20 will be described. In this embodiment, a relatively small number of measurement points for measuring electroencephalogram data are selected in advance by the measurement point selection device 10. As already described, the measurement point selection device 10 selects at least a pair of measurement points as a representative point set using electroencephalogram data at N measurement points of a subject performing a predetermined behavior.
[0105] A user of the behavior estimation device 20 (an operator who estimates the behavior of the person to be estimated) places electrodes of the electroencephalograph 40 at measurement points including at least one pair of the representative point set and at reference measurement points on the head of the person to be measured (person to be estimated). The person to be estimated will be performing the same behavior as the experimental subject when the representative point set was selected.
[0106] Specifically, in the following example, as in the examples already described, the behavior of watching a video is assumed to be performed, and among the scenes in the video (here, each scene is defined as one minute of the video being watched), a behavior of remembering a scene (having watched an impressive scene) is inferred, and which scene was the impressive one is detected.
[0107] Furthermore, prior to using this behavior estimation device 20, when the measurement point selection device 10 selects a set of representative points, the video viewed by the experimental subject from whom the EEG data was obtained may be different from the video viewed by the estimated subject from whom the EEG data to be input into this behavior estimation device 20 is obtained.
[0108] The behavior estimation device 20 receives the electroencephalogram data measured by the electroencephalogram 40. Here, the electroencephalogram 40 focuses on a network between at least a pair of measurement points included in the representative point set selected by the measurement point selection device 10, and measures and outputs electroencephalogram data at the measurement points of the estimated subject watching a video. In the following, the electroencephalogram 40 measures electroencephalogram data at a total of four measurement points, including two points FC5 and Fp2 as a representative point set (measurement points selected by the measurement point selection device 10 at a significance level of 1%) among the measurement points on the head surface of the estimated subject, and measurement points A1 and A2 on the pinnae of both ears as reference measurement points. In addition, the judgment criterion information corresponding to the representative point set (FC5, Fp2) is "+".
[0109] The behavior inference device 20 sequentially receives electroencephalogram data from the electroencephalograph 40 and performs the process illustrated in Fig. 7. The behavior inference device 20 receives and stores in advance inputs of information for identifying the representative point set selected by the measurement point selection device 10 for the behavior of watching videos and judgment criteria information for the representative point set.
[0110] The behavior inference device 20 performs preprocessing (which may be the same process as step S14 of the measurement point selection device 10) on each of the electroencephalogram data at each measurement point sequentially received from the electroencephalograph 40, and then divides the data into electroencephalogram data for each scene (here, for each minute).The behavior inference device 20 then obtains divided electroencephalogram data E(FC5,i), E(Fp2,i) (i is an identifier that identifies a unit of behavior, and in this example, it is a scene identifier, i=1, 2, ...) for each scene (S51).
[0111] The behavior inference device 20 applies a bandpass filter in the theta band (4 to 7 Hz) to each of the divided EEG data extracted in step S51, extracts theta band components of each of the divided EEG data E(FC5,i) and E(Fp2,i), and obtains EEG data to be analyzed T(FC5,i) and T(Fp2,i) (S52).
[0112] Then, the behavior inference device 20 takes each of the measurement points included in the representative point set selected by the measurement point selection device 10 as an analysis target, and repeats the following process for each analysis target and for each scene identifier.
[0113] In the course of this repetitive processing, the behavior inference device 20 performs a process of obtaining, for example, dwPLI as a functional connectivity analysis on the analysis subject EEG data T(FC5, i) and T(Fp2, i) at a pair of measurement points (FC5, Fp2) related to the analysis subject, and obtains the analysis result C(FC5, Fp2, i) (S53).
[0114] After obtaining the analysis results C for all scenes, the behavior inference device 20 refers to the dwPLI value C(FC5, Fp2, i) corresponding to each scene identifier and the criterion information (here, "+") corresponding to the representative point set, for each representative point set, and searches for a scene identifier that satisfies the criterion indicated by the criterion information "+" corresponding to the representative point set (FC5, Fp2) among the analysis results C(FC5, Fp2, i) of the representative point sets (FC5, Fp2) corresponding to scene identifiers i=1, 2, ... (S54).
[0115] That is, here, the behavior estimation device 20 searches for a scene identifier j where C(FC5,Fp2,j)>CM(FC5,Fp2) and C(FC5,Fp2,j)-CM(FC5,Fp2) is greater than a predetermined threshold value, for the average CM(FC5,Fp2) of the analysis result C(FC5,Fp2,i) related to the representative point set (FC5,Fp2). When such a scene identifier j is found, the behavior estimation device 20 outputs the found scene identifier j (S55).
[0116] In this way, the user of the behavior estimation device 20 obtains a scene identifier of a scene that is estimated to leave an impression on the estimation target person.
[0117] The above process is then repeated for multiple different estimated subjects, and identifiers of scenes estimated to be memorable for each estimated subject are obtained, and these are used for processing such as analyzing which parts of the video are the most memorable scenes.
[0118] According to this embodiment, it is possible to perform an objective analysis of behavior without conducting a questionnaire survey. In addition, since a pair of measurement points from which characteristic electroencephalogram data can be detected during a predetermined behavior is selected in advance without using a multi-channel electroencephalograph, it is possible to estimate behavior using a simple electroencephalograph with a small number of channels.
[0119] In addition, because the number of processing channels is small, estimations regarding scene memory can be made with relatively light processing load, making it possible to collect estimation results from a larger number of subjects than in the past.
[0120] Furthermore, the results estimated in this way can be used to provide an objective evaluation of how memorable the presented video or other content is, which can contribute to the goal of providing more memorable videos or videos that generate more anticipation.
[0121] Furthermore, by measuring an individual using an electroencephalograph and analyzing scenes that are likely to be memorable through the above-mentioned processing, the results can also be used for processing such as recommending content to each individual. [Explanation of symbols]
[0122] 1 behavior estimation system, 10 measurement point selection device, 11 control unit, 12 memory unit, 13 operation unit, 14 output unit, 15 input / output unit, 20 behavior estimation device, 21 control unit, 22 memory unit, 23 operation unit, 24 output unit, 25 input / output unit, 30 multi-channel electroencephalograph, 31 measurement result receiving unit, 32 measurement result recording unit, 33 information receiving unit, 34 analysis processing unit, 35 selection processing unit, 36 output processing unit, 40 electroencephalograph, 41 electroencephalogram information acquisition unit, 42 estimation processing unit, 43 output processing unit.
Claims
1. A behavior estimation system for estimating a result of a predetermined type of behavior performed by a measurement subject by measuring an electroencephalogram, the system including a measurement point selection device and a behavior estimation device, The measurement point selection device, a measuring means for measuring electroencephalogram data of a subject performing the predetermined type of behavior at N predetermined measurement points (N is an integer greater than 2) and M reference measurement points (M is a natural number) on the subject's head, and dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior; a selection means for selecting at least a pair of the N measurement points as a representative point set based on a functional connectivity analysis of electroencephalogram data for each unit behavior at a pair of measurement points selected from the N measurement points and a behavior result of the unit behavior presented by the measurement subject for each unit behavior; Including, The behavior estimation device, an acquisition means for acquiring electroencephalogram data at n points (n is an integer satisfying N>n>2) including the representative point set selected by the measurement point selection device of the measurement subject performing the predetermined type of behavior, and dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior; an estimation processing means for estimating a result of each unit behavior of the subject based on the acquired electroencephalogram data; A behavior estimation system comprising:
2. a measuring means for measuring electroencephalogram data of a subject performing a predetermined type of behavior at N measurement points (N is an integer greater than 2) on the subject's head and M reference measurement points (M is a natural number), and dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior; a selection means for selecting at least a pair of the N measurement points as a representative point set based on a functional connectivity analysis of electroencephalogram data for each unit behavior at a pair of measurement points selected from the N measurement points and a behavior result of the unit behavior presented by the measurement subject for each unit behavior; Including, A measurement point selection device which acquires electroencephalogram data at n points (n is an integer such that N>n>2) including the representative point set identified by information identifying the representative point set of a measurement subject performing a predetermined type of behavior, divides the data into electroencephalogram data for each predetermined unit behavior during the behavior, and uses the data to process an estimation of the results of each unit behavior of the measurement subject.
3. The measurement point selection device according to claim 2, The selection means is a measurement point selection device that performs a phase synchronization analysis based on any of information such as PLV (Phase Locking Value) or information obtained by a method derived from a method for obtaining a PLV, PLI (Phase Lag Index) or information obtained by a method derived from a method for obtaining a PLI, or PPC (Pairwise Phase Consistency) or information obtained by a method derived from a method for obtaining a PPC, as the functional connectivity analysis process for all combinations of two measurement points that can be selected from the N measurement points, and selects at least a pair of points from the N measurement points as a representative point set based on the result of the phase synchronization analysis and the result of the behavior presented by the measurement subject.
4. The measurement point selection device according to claim 2 or 3, The selection means is a measurement point selection device that, as a process of the functional connectivity analysis, selects at least a pair of points from the N measurement points as a representative point set by comparing the result of the functional connectivity analysis when the result of the behavior satisfies a predetermined condition with the result of the functional connectivity analysis when the result of the behavior does not satisfy a predetermined condition.
5. The measurement point selection device according to any one of claims 2 to 4, the behavior is a behavior of watching a video, and the unit behavior is a behavior of memorizing the content of each scene included in the video, The selection means selects two of the N measurement points as candidates for a representative point set based on functional connectivity analysis of EEG data at two measurement points selected from the N measurement points for each viewed scene as EEG data for each unit behavior, and information indicating whether or not the scene corresponding to the unit behavior presented by the measurement subject for each unit behavior is remembered, and selects two of the N measurement points as representative points based on the selection result for each scene.
6. A behavior inference device that infers a result of a predetermined type of behavior performed by a subject by measuring an electroencephalogram, comprising: an acquisition means for acquiring electroencephalogram data at n points (n is an integer greater than or equal to 2) including the representative point set identified by information on two points selected from N measurement points (N is an integer greater than or equal to 2) on the head of a measurement subject who has performed the predetermined type of behavior, the two points being selected based on measurement results for each unit behavior during the behavior, and dividing the electroencephalogram data into electroencephalogram data for each unit behavior during the behavior; an estimation processing means for estimating a result of the behavior of the subject for each unit behavior based on the acquired electroencephalogram data; A behavior estimation device comprising:
7. a measuring step of measuring electroencephalogram data of a measurement subject performing a predetermined type of behavior at N measurement points (N is an integer greater than 2) on the subject's head and M reference measurement points (M is a natural number), and dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior; a selection step of selecting at least a pair of the N measurement points as a representative point set based on a functional connectivity analysis of electroencephalogram data for each unit behavior at two measurement points selected from the N measurement points and a behavior result of the unit behavior presented by the measurement subject for each unit behavior; Including, A control method for an information processing device, in which information identifying the representative point set is used to obtain electroencephalogram data at n points (n is an integer such that N>n>2) including the representative point set identified by the information of a subject performing a behavior of a predetermined type, and divide the data into electroencephalogram data for each predetermined unit behavior during the behavior, thereby estimating the results of the behavior of the subject for each unit behavior.
8. A method for controlling a behavior inference device that causes the behavior inference device to infer a result of a predetermined type of behavior performed by a measurement subject by measuring an electroencephalogram, comprising: a step of acquiring electroencephalogram data at n points (n is an integer greater than or equal to 2) including the representative point set specified by the information among N measurement points (N is an integer greater than or equal to 2) on the head of a measurement subject that has performed the predetermined type of behavior and M reference measurement points (M is a natural number) selected based on a measurement result for each unit behavior during the behavior, and dividing the electroencephalogram data into electroencephalogram data for each unit behavior during the behavior; A step of estimating a result of the behavior of the subject for each unit behavior based on the acquired electroencephalogram data; A control method for a behavior inference device comprising:
9. Computer, a measuring means for measuring electroencephalogram data of a subject performing a predetermined type of behavior at N measurement points (N is an integer greater than 2) on the subject's head and M reference measurement points (M is a natural number), and dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior; a selection means for selecting at least a pair of the N measurement points as a representative point set based on a functional connectivity analysis of electroencephalogram data for each unit behavior at two measurement points selected from the N measurement points and a behavior result of the unit behavior presented by the measurement subject for each unit behavior; Function as a A program for acquiring electroencephalogram data at n points (n is an integer such that N>n>2) including the representative point set identified by information of a subject performing a behavior of the predetermined type, dividing the data into electroencephalogram data for each predetermined unit behavior during the behavior, and subjecting the data to a process of estimating the results of the behavior of the subject for each unit behavior.
10. A program for causing a computer to function to estimate the result of a predetermined type of behavior performed by a subject by measuring electroencephalograms, the program comprising: an acquisition means for acquiring electroencephalogram data at n points (n is an integer greater than or equal to 2) including the representative point set identified by information on two points selected from N measurement points (N is an integer greater than or equal to 2) on the head of a measurement subject who has performed the predetermined type of behavior, the two points being selected based on measurement results for each unit behavior during the behavior, and dividing the electroencephalogram data into electroencephalogram data for each unit behavior during the behavior; an estimation processing means for estimating a result of the behavior of the subject for each unit behavior based on the acquired electroencephalogram data; A program that functions as a
Citation Information
Patent Citations
Data broadcasting system
JP2014217024A
Music hearing experience presence / absence estimation method, music hearing experience presence / absence estimation device and music hearing experience presence / absence estimation program
JP2017042562A
