State estimation device, state estimation method, and program

Abstract

The present invention provides a technology in which time series data relating to the blinking of a subject when executing a task is used to estimate the mental / cognitive state of the subject when executing the task. The invention includes: a time series data acquisition unit that acquires time series data relating to the blinking of a subject during a prescribed task (hereinafter referred to as the time series data of the subject); and a state estimation unit that generates a state estimation result indicating the mental / cognitive state of the subject, from the time series data of the subject and blinking-related time series data serving as a reference (hereinafter referred to as reference time series data).

Description

[Technical field]

[0001] The present invention relates to a technique for estimating a psychological / cognitive state of a subject when performing a task. [Background technology]

[0002] It is known that the frequency of blinking (also called eye blinking) changes depending on the emotion of the person blinking and the task the person is performing (see Non-Patent Document 1). For example, the emotion of anger increases the frequency of blinking, and the task of reading decreases the frequency of blinking. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] John A. Stern, Larry C. Walrath, and Robert Goldstein, “The endogenous eyeblink,” Psychophysiology, Volume 21, Issue 1, pp.22-33, 1984. Summary of the Invention [Problem to be solved by the invention]

[0004] Consider quantitatively evaluating the athletic performance of athletes. For example, in a competition where speed is the focus, such as car racing, there are traditionally indices related to the athletic performance of athletes, such as time and speed. On the other hand, indices related to the psychological and cognitive states of athletes, such as attention state, concentration state, and excitement state, do not exist, despite the findings disclosed in Non-Patent Document 1.

[0005] Therefore, an object of the present invention is to provide a technique for estimating the psychological / cognitive state of a subject when performing a task, using time-series data regarding the subject's blinking when performing a task. [Means for solving the problem]

[0006] One aspect of the present invention includes a time series data acquisition unit that acquires time series data regarding the blinking of a subject during a specified task (hereinafter referred to as subject time series data), and a state estimation unit that generates a state estimation result indicating the psychological / cognitive state of the subject from the subject time series data and reference time series data regarding blinking (hereinafter referred to as reference time series data). Effect of the Invention

[0007] According to the present invention, it is possible to estimate the psychological / cognitive state of a subject when performing a task. [Brief description of the drawings]

[0008] [Figure 1] FIG. 13 is a diagram showing time-series data as experimental results. [Diagram 2] FIG. 1 is a diagram showing blink synchronization. [Diagram 3] FIG. 13 is a diagram showing time-series data as experimental results. [Figure 4] FIG. 13 is a diagram showing the relationship between the rotation speed and the strength of synchronization. [Diagram 5] FIG. 1 is a block diagram showing a configuration of a state estimation device 100. [Figure 6] 4 is a flowchart showing the operation of the state estimation device 100. [Figure 7] FIG. 2 is a block diagram showing a configuration of a state estimation unit 120. [Figure 8] 4 is a flowchart showing the operation of a state estimation unit 120. [Figure 9] FIG. 2 is a diagram illustrating an example of a functional configuration of a computer that realizes each device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described in detail. Note that components having the same functions are given the same reference numbers and duplicated explanations will be omitted.

[0010] Before describing each embodiment, the notation used in this specification will be described.

[0011] ^ (caret) represents a superscript. For example, x y^z Yes z is a superscript to x, and x y^z Yes z is a subscript to x. Also, _ (underscore) represents a subscript. For example, x y_z Yes z is a superscript to x, and x y_z Yes z is a subscript on x.

[0012] The superscripts "^" and "~" such as ^x and ~x for a certain letter x should actually be written directly above the "x", but due to the constraints of the description in the specification, they are written as ^x and ~x. <Technical background> An embodiment of the present invention is based on the newly discovered findings that (1) time-series data of blinking when performing a specific task (e.g., driving a race car) shows similar tendencies between the same subject and different subjects, and (2) the degree of synchronization between time-series data of blinking varies depending on differences in psychological / cognitive load, and utilizes these findings to estimate the psychological / cognitive state of a subject when performing a task.

[0013] The experiments on which the present invention is based and the results thereof will be described below. [experiment] The blinking of Formula car drivers is measured while they are going around a racing course, and time series data on the drivers' blinking is compared. [Experimental Results] Figure 1 shows the time series data of the experimental results. Figure 1 shows two types of graphs: one based on the video of the drivers' blinks, and the other based on the steering angle data. Graphs (A) and (B) show the blinking behavior of each driver in each lap, and graph (C) shows the steering behavior of the two drivers. The horizontal axis of the three graphs represents the normalized distance ratio, which indicates the distance from the starting point when the distance from the starting point to the goal point is set to 1. The vertical axis of graphs (A) and (B) represents the number of laps, and the small vertical lines in graphs (A) and (B) represent the timing of the drivers' blinks. On the other hand, the vertical axis of graph (C) represents the steering angle. Comparing graphs (A) and (B), we can see that the blinking frequency of Driver A and Driver B is significantly different, but there are places where the two drivers blink in common (shaded areas in graphs (A) and (B)). In addition, when comparing this with graph (C), we can see that the driver blinks less frequently when going through corners on the racing course, and blinks more frequently when going through the initial straight sections of the racing course.

[0014] In other words, when the position on the racing course is used as a reference point, it can be seen that similar blinking behavior occurs within one driver, and similar blinking behavior occurs between two drivers (see Figure 2). Hereinafter, this phenomenon will be referred to as the synchronization phenomenon of eye blinking behavior. This synchronization phenomenon of eye blinking behavior suggests that drivers are not always in a constant, flat psychological / cognitive state when going around the racing course, but rather dynamically control their psychological / cognitive state according to their position on the racing course.

[0015] In addition, by defining an index that represents the strength of blinking synchronization at a common place on the racing course and analyzing the time series data in Figure 1, it is found that the strength of synchronization changes depending on the speed of the lap. The explanation is as follows. In the graphs of Figure 1 (A) and (B), it can be seen that there are laps where blinking occurs sporadically (see Figure 3). These laps where blinking occurs sporadically are laps with slow times. Therefore, when dividing the laps into laps with slow times, laps with medium times, and laps with fast times and examining the relationship with the strength of synchronization, the result is shown in Figure 4. Figure 4 shows the strength of synchronization between Driver A and Driver B on laps with slow times, laps with medium times, and laps with fast times. It can be seen that the strength of synchronization is weak on laps with slow times, while strong synchronization appears on laps with medium times and fast times. Since the speed of the lap is thought to be strongly related to the difficulty for the driver, it is thought that the degree of concentration, which is a psychological / cognitive load, can be expressed by an index called the degree of synchronization.

[0016] It can be seen that the above experimental results provide the findings (1) and (2) above. First Embodiment The state estimation device 100 estimates the psychological / cognitive state of the subject when performing a task based on time series data on the blinking of the subject during a predetermined task (hereinafter referred to as subject time series data). Here, the predetermined task is a task of going around a specific course, or a task of performing a specific action at a predetermined timing. Examples of tasks of going around a specific course include car racing, bicycle racing, boat racing, and foot races. Examples of tasks of performing a specific action at a predetermined timing include hitting in baseball, swinging a bat at the timing when the pitcher throws the ball, stealing bases from first base to second base at the timing when the pitcher throws the ball, the action of pulling the bow to hit the target in archery, and the action of throwing the ball in shot put.

[0017] Hereinafter, the state estimation device 100 will be described with reference to FIG. 5 and FIG. 6. FIG. 5 is a block diagram showing the configuration of the state estimation device 100. FIG. 6 is a flowchart showing the operation of the state estimation device 100. As shown in FIG. 5, the state estimation device 100 includes a time series data acquisition unit 110, a state estimation unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the state estimation device 100. The recording unit 190 records time series data related to a blink that serves as a reference (hereinafter, referred to as reference time series data) in advance. Here, the reference time series data is data used to estimate the psychological / cognitive state of the subject, and is time series data related to a blink in the same task as the task of the subject time series data. The reference time series data may be data obtained in advance by the subject himself performing the task, or may be data obtained in advance by one or more persons other than the subject performing the task.

[0018] The operation of the state estimating device 100 will be described with reference to FIG.

[0019] In S110, the time-series data acquiring unit 110 acquires the subject's time-series data. The time-series data acquiring unit 110 can acquire the subject's time-series data from, for example, a video capturing the subject's blinking while performing a task.

[0020] In S120, the state estimation unit 120 generates a state estimation result indicating the psychological / cognitive state of the subject, from the subject time series data acquired in S110 and the reference time series data recorded in the recording unit 190. Hereinafter, the state estimation unit 120 will be described with reference to Figs. 7 and 8. Fig. 7 is a block diagram showing the configuration of the state estimation unit 120. Fig. 8 is a flowchart showing the operation of the state estimation unit 120. As shown in Fig. 7, the state estimation unit 120 includes an index value calculation unit 121 and a state estimation result generation unit 122.

[0021] The operation of the state estimation unit 120 will be described with reference to FIG.

[0022] In S121, the index calculation unit 121 calculates an index (hereinafter, referred to as a synchronization index) indicating the degree of synchronization between the subject time series data and the reference time series data from the subject time series data and the reference time series data. For example, the z value can be used as the synchronization index.

[0023] In S122, the state estimation result generating unit 122 generates a state estimation result from the synchronization index value calculated in S121. For example, if the state estimation result indicates the degree to which the subjects are concentrating, the state estimation result indicates that the higher the synchronization index value, the higher the degree to which the subjects are concentrating. of This will be indicative of the

[0024] According to an embodiment of the present invention, it is possible to estimate the psychological / cognitive state of a subject when performing a task. <Additional Notes> 9 is a diagram showing an example of a functional configuration of a computer 2000 that realizes each of the above-mentioned devices. The processing in each of the above-mentioned devices can be implemented by having a recording unit 2020 load a program for causing the computer 2000 to function as each of the above-mentioned devices, and having the control unit 2010, the input unit 2030, the output unit 2040, and the like operate.

[0025] The device of the present invention has, as a single hardware entity, an input section to which a keyboard or the like can be connected, an output section to which a liquid crystal display or the like can be connected, a communication section to which a communication device (e.g., a communication cable) capable of communicating with the outside of the hardware entity can be connected, a CPU (which may also have a central processing unit, cache memory, registers, etc.), memories such as RAM and ROM, an external storage device such as a hard disk, and buses connecting the input section, output section, communication section, CPU, RAM, ROM, and external storage device so that data can be exchanged between them. If necessary, the hardware entity may also be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity equipped with such hardware resources is, for example, a general-purpose computer.

[0026] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required in the processing of these programs (not limited to the external storage device, but for example the programs may be stored in a ROM, which is a read-only storage device). Data obtained by the processing of these programs is stored appropriately in the RAM, the external storage device, etc.

[0027] In the hardware entity, each program stored in an external storage device (or ROM, etc.) and the data required for processing each program are loaded into memory as necessary, and interpreted, executed, and processed by the CPU as appropriate, so that the CPU realizes a predetermined function (each component represented as the above, "unit," "means," etc.).

[0028] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. In addition, the processes described in the above-described embodiment are not limited to being executed in chronological order according to the order described, but may be executed in parallel or individually depending on the processing capacity of the device that executes the processes or as necessary.

[0029] As described above, when the processing functions of the hardware entities (the devices of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

[0030] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium. Specifically, for example, a hard disk drive, a flexible disk, a magnetic tape, or the like can be used as the magnetic recording device, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), or the like can be used as the optical disk, an MO (Magneto-Optical disc), or the like can be used as the magneto-optical recording medium, and an EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) or the like can be used as the semiconductor memory.

[0031] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0032] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored in its own storage device and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. The server computer may not transfer the program to the computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes the processing function only by issuing an execution instruction and obtaining the result. Note that the program in this embodiment includes information used for processing by an electronic computer that is equivalent to a program (data that is not a direct instruction to the computer but has a nature that specifies the processing of the computer, etc.).

[0033] In addition, in this embodiment, a hardware entity is configured by executing a specific program on a computer, but at least a part of the processing contents may be realized by hardware.

[0034] The foregoing description of the embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings. The embodiments have been chosen and depicted to provide a best illustration of the principles of the invention and to enable those skilled in the art to utilize the invention in various embodiments and with various modifications as may be suitable for the practical use contemplated. All such modifications and variations are within the scope of the invention as defined by the appended claims interpreted in accordance with the breadth to which they are fairly, legally, and equitably entitled.

Claims

1. A time series data acquisition unit that acquires time series data regarding the blinking of a subject during a predetermined task (hereinafter referred to as subject time series data); a state estimation unit that generates a state estimation result indicating a psychological / cognitive state of the subject from the subject time series data and reference time series data related to blinking (hereinafter referred to as reference time series data); A state estimation device comprising: The state estimation unit is an index value calculation unit that calculates an index value (hereinafter referred to as a synchronization index value) indicating a degree of synchronization between the subject time series data and the reference time series data from the subject time series data and the reference time series data; a state estimation result generating unit that generates the state estimation result from the synchronization index value State estimator.

2. The state estimation device according to claim 1 , The state estimation result indicates the degree to which the subject is concentrating, and a higher synchronization index value indicates a higher degree to which the subject is concentrating. A state estimation device comprising:

3. The state estimation device according to claim 1 , The predetermined task is a task of going around a specific course, or a task of performing a specific action at a specific timing. A state estimation device comprising:

4. a time series data acquiring step in which the state estimation device acquires time series data regarding blinking of the subject during a predetermined task (hereinafter referred to as subject time series data); a state estimation step of generating a state estimation result indicating a psychological / cognitive state of the subject from the subject time series data and reference time series data related to blinking (hereinafter referred to as reference time series data) by the state estimation device; A state estimation method comprising: The state estimation step includes: an index value calculation step of calculating an index value (hereinafter referred to as a synchronization index value) indicating a degree of synchronization between the subject time series data and the reference time series data from the subject time series data and the reference time series data; A state estimation result generating step of generating the state estimation result from the synchronization index value. State estimation methods.

5. A program for causing a computer to function as the state estimating device according to any one of claims 1 to 3.

Images