Concentration state estimation device and method

Abstract

In an aspect of the present invention, when the concentration state of an estimation target user is estimated on the basis of the pupil diameter of the estimation target user, first pupil diameter information including, as a main component, a component corresponding to a luminance change in an activity environment of the estimation target user is acquired, second pupil diameter information representing a change in the pupil diameter of the estimation target user being engaged in an activity is acquired, the acquired first pupil diameter information is compared with the acquired second pupil diameter information, and third pupil diameter information is generated by removing the component corresponding to the luminance change from the second pupil diameter information on the basis of the comparison result. Then, an estimation target unit period defined as a predetermined length of time is set for the third pupil diameter information, change trend information representing a trend of the change in the pupil diameter is generated in each estimation target unit period, and the concentration state of the estimation target user is estimated on the basis of the change trend information.

Description

Concentration state estimation device and method

[0001] One aspect of the present invention relates to a concentration state estimation device and method for estimating a person's concentration state based on pupil diameter.

[0002] It is known that pupil diameter reflects a person's state of concentration. Therefore, attempts have been made to utilize pupil diameter as an indicator that objectively and quantitatively reflects a person's state of concentration. For example, by using the size and fluctuation of pupil diameter as an indicator of the concentration state while viewing a lecture video or the like, it is possible to estimate whether the lecture content was viewed with concentration at a specific time, and to redisplay only the content that was conveyed during the time when the viewer was not viewing with concentration (see, for example, Patent Document 1).

[0003] However, because pupil diameter is affected by the luminance of the environment, even if a change in pupil diameter is observed, it is difficult to simply assume that this change reflects the state of concentration. To address this problem, a method has been proposed for distinguishing between changes in pupil diameter that reflect the state of concentration and changes in pupil diameter that reflect luminance (see, for example, Patent Document 2).

[0004] International Publication No. 2022 / 107288 International Publication No. 2022 / 185550

[0005] However, the technology described in Patent Document 2 is intended to estimate states of concentration that require high real-time performance, such as when driving a car, which requires constant attention and increases cognitive fatigue, and is designed to estimate states of concentration that change over a relatively short period of time, such as in units of a few seconds.

[0006] On the other hand, in situations where the state of concentration does not change in real time, such as when watching a lecture video, it is necessary to estimate the type of concentration state that changes over a period of several tens of seconds, and technology to do so is highly desirable.

[0007] This invention has been made in light of the above circumstances, and aims to provide a technology that enables accurate estimation of a type of concentration state that does not change in real time on a second-by-second basis.

[0008] In order to solve the above problem, one aspect of a concentration estimation device or method according to the present invention, when estimating a concentration state of a target user based on pupil diameter, acquires first pupil diameter information containing, as a main component, a component corresponding to a change in luminance in an environment in which the target user is active, and acquires second pupil diameter information representing changes in pupil diameter during the target user's activity, compares the acquired first pupil diameter information with the second pupil diameter information, and generates third pupil diameter information in which the component corresponding to the change in luminance has been removed from the second pupil diameter information based on the comparison result, sets a predetermined target estimation unit period for the third pupil diameter information, generates change trend information representing a change trend in pupil diameter for each target estimation unit period, and estimates the target user's concentration state based on the change trend information.

[0009] According to one aspect of the present invention, the concentration state of the estimation target user is estimated based on the change trend of pupil diameter over an estimation target unit interval set to a relatively long time, for example, about 5 to 15 seconds, making it possible to estimate a type of concentration state in which the concentration state does not change in real time but changes over a relatively long time interval. Moreover, because the change component of pupil diameter due to brightness changes in the activity environment is removed, the influence of brightness changes in the activity environment is eliminated, making it possible to accurately estimate the concentration state of the estimation target user.

[0010] That is, according to one aspect of the present invention, it is possible to provide a technique that enables accurate estimation of a type of concentration state that does not change in real time.

[0011] FIG. 1 is a block diagram showing an example of the hardware configuration of a concentration state estimating device according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the software configuration of a concentration state estimating device according to an embodiment of the present invention. FIG. 3 is a flowchart showing an example of the processing procedure and processing content in a pre-setting mode executed by a control unit of the concentration state estimating device shown in FIG. 2. FIG. 4 is a flowchart showing an example of the processing procedure and processing content in an estimation mode executed by a control unit of the concentration state estimating device shown in FIG. 2. FIG. 5 is a diagram showing an example of the pupil diameter to be sensed. FIG. 6 is a diagram showing an example of the relationship between time-series data showing the average pupil diameter and time-series data of the pupil diameter to be estimated.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] [One Embodiment] (Overview) The inventors conducted a psychological experiment in which participants were asked to perform a series of psychological tasks while measuring their pupil diameter. The time-series data of the obtained pupil diameter was subjected to a smoothing process, taking a moving average over a period of, for example, 5 to 10 seconds, to calculate the amount of change in pupil diameter for 5 to 10 seconds before and after the timing of a reaction (e.g., a button press) in the psychological task. As a result, they found that, across multiple psychological tasks, pupil diameter consistently decreased when the participant's state of concentration was high (e.g., when the button press response was fast) and tended to increase when the participant's state of concentration was low (e.g., when the button press response was slow).

[0014] However, in an environment where brightness changes, such as when watching a video, pupil diameter is also affected by this change in brightness. Therefore, if the obtained change in pupil diameter is simply used as is to estimate the concentration state, accurate estimation is difficult due to the influence of the change in brightness.

[0015] Therefore, in one embodiment of the present invention, average pupil diameter data containing only the change component of pupil diameter due to the above-mentioned change in luminance is prepared in advance, and the component due to the change in luminance is removed by subtracting the above-mentioned average pupil diameter data from the pupil diameter data measured for the user to be estimated, and the concentration state of the user to be estimated is estimated based on this pupil diameter data from which the change component in luminance has been removed.

[0016] In this way, by estimating the concentration state based on the average pupil diameter along the time axis, it is possible to estimate the type of concentration state that does not change in real time but changes over relatively long time periods of tens of seconds. Moreover, because the transition of the concentration state is calculated based on pupil diameter data that has been filtered out for the effects of brightness changes in the lecture video, highly accurate estimation that is not affected by brightness changes is possible.

[0017] (Configuration example) One embodiment of the present invention will be described using an example of estimating the concentration state of a target user (hereinafter referred to as a target user) when giving a lecture to participants (hereinafter referred to as a target user) at a seminar or the like using a training video.

[0018] (1) Concentration state estimating device CE The concentration state estimating device CE is configured by a personal computer used, for example, by the organizer of a seminar, etc. Note that the concentration state estimating device CE may be configured by a server computer located on the Web or the cloud, or may be configured by a personal computer used by the person to be estimated.

[0019] 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration of the concentration state estimating device CE according to an embodiment of the present invention.

[0020] The concentrated state estimation device CE includes a control unit 1 using a hardware processor such as a central processing unit (CPU), etc. A storage unit having a program storage unit 2 and a data storage unit 3, a sensor interface (hereinafter, interface will be referred to as I / F) unit 4, and an input / output I / F unit 5 are connected to the control unit 1 via a bus 6.

[0021] A camera CM is connected to the sensor I / F unit 4. The camera CM is, for example, an infrared camera, and captures an image of the face of the estimated target user, including the eye area. Note that the camera CM may capture an image of only the eye area, or an image of the upper body. The sensor I / F unit 4 receives the image data output from the camera CM.

[0022] An input device IP and a display device DP are connected to the input / output I / F unit 5. The input device IP is composed of, for example, a keyboard and a mouse, and is used to input various control commands related to the estimation process to the concentration state estimation device CE. Note that an external storage medium such as a USB memory may be connected to the input / output I / F unit 5.

[0023] The display device DP is formed of a display using, for example, a liquid crystal display or an organic electroluminescence display, and is used to display information representing the estimation result of the concentration level of the estimation target user obtained by the concentration state estimation device CE. The input / output I / F unit 5 receives the control commands and the like inputted via the input device IP, and outputs the concentration level estimation information to the display device DP.

[0024] The program storage unit 2 is, for example, a combination of a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a read only memory (ROM), and stores application programs necessary for executing various processes related to one embodiment of the present invention, in addition to middleware such as an operating system (OS).

[0025] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an HDD or SSD as a storage medium that can be written to and read from at any time, and a volatile memory such as a RAM (Random Access Memory), and its storage area includes a reference image storage unit 31, an average pupil diameter storage unit 32, an estimated target image storage unit 33, an estimated target pupil diameter storage unit 34, a differential pupil diameter storage unit 35, and a concentration estimation information storage unit 36.

[0026] The reference image storage unit 31 stores, for example, time-series image data of the faces of a plurality of reference users, which is used to calculate the average pupil diameter.

[0027] The average pupil diameter storage unit 32 is used to store time-series data of the average pupil diameter calculated based on the video data of the plurality of reference users.

[0028] The estimated target image storage unit 33 stores image data of the face of the estimated target user, including the eye area, until the estimation process of the concentration level of the estimated target user is completed.

[0029] The estimated target pupil diameter storage unit 34 stores time-series data representing the pupil diameter of the estimated target user, which is generated based on video data obtained by capturing an image of the estimated target user.

[0030] The differential pupil diameter storage unit 35 stores differential data between the time series data of the pupil diameter of the estimation target user and the time series data of the average pupil diameter.

[0031] The concentration level estimation information storage unit 36 ​​stores information indicating the estimation result of the concentration level estimated by the control unit 1 .

[0032] The control unit 1 includes, as processing units according to one embodiment of the present invention, a reference image acquisition processing unit 11, an average pupil diameter calculation processing unit 12, an estimation target image acquisition processing unit 13, an estimation target pupil diameter data generation processing unit 14, a pupil diameter comparison processing unit 15, a pupil diameter trend component calculation processing unit 16, and a concentration estimation processing unit 17.

[0033] The processing units 11 to 17 are all realized by causing a hardware processor in the control unit 1 to execute an application program stored in the program storage unit 2. Note that some or all of the processing units 11 to 17 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0034] In the preset mode, the reference video acquisition processing unit 11 acquires video data of the faces of multiple reference users captured by the camera CM while they are viewing a lecture video via the sensor I / F unit 4. The received video data is then stored as reference video data in the reference video storage unit 31. It is desirable to select users with average sensitivity characteristics to changes in brightness as the reference users.

[0035] In the preset mode, the average pupil diameter calculation processing unit 12 reads out the reference video data of the plurality of reference users from the reference video storage unit 31, recognizes pupil images from each of the read reference video data using, for example, pattern recognition technology, and generates pupil diameter data (first pupil diameter information) representing the size of the pupil diameter. The average pupil diameter calculation processing unit 12 then aligns the time axes of the pupil diameter data and averages them, and stores the time-series data obtained thereby in the average pupil diameter storage unit 32.

[0036] In addition, the concentration state estimation device CE may not have the function of acquiring the above-mentioned reference video data and calculating the average pupil diameter data, but may instead acquire average pupil diameter data generated separately on another terminal or server, etc., and store it in the average pupil diameter memory unit 32.

[0037] In the estimation mode, the estimated target video acquisition processing unit 13 acquires video data of the estimated target user's face captured by the camera CM while viewing a lecture video via the sensor I / F unit 4, and stores the acquired video data in the estimated target video storage unit 33 as estimated target video data while associating it with the identification information of the estimated target user (hereinafter referred to as user ID).

[0038] The estimation target pupil diameter data generation processing unit 14 recognizes an image of the pupil of the estimation target user from the estimation target video data using, for example, pattern recognition technology, and generates pupil diameter data (second pupil diameter information) representing the size of the pupil diameter based on the recognized pupil image.The generated pupil diameter data is then stored in the estimation target pupil diameter storage unit 34.

[0039] The pupil diameter comparison processing unit 15 compares the pupil diameter data of the estimation target stored in the estimation target pupil diameter storage unit 34 with the average pupil diameter data stored in the average pupil diameter storage unit 32 along the same time axis to generate differential pupil diameter data (third pupil diameter information).The pupil diameter comparison processing unit 15 then stores the differential pupil diameter data in the differential pupil diameter storage unit 35.

[0040] The pupil diameter trend component calculation processor 16 calculates a pupil diameter trend component that indicates the tendency of change in pupil diameter for each predetermined estimation target time period (e.g., 5 to 10 seconds) from the pupil diameter difference data. An example of the calculation process of this pupil diameter trend component will be described in the section on operation.

[0041] The concentration level estimation processing unit 17 estimates the concentration level tendency of the estimation target user based on the pupil diameter trend component for each estimated time length, and stores the estimation result in the concentration level estimation information storage unit 36. An example of the concentration level estimation process will also be described in the operation example.

[0042] (Example of Operation) Next, an example of operation of the concentration state estimating device CE configured as above will be described.

[0043] (1) Presetting Mode For example, when the presetting mode is designated by operating the input device IP, the control unit 1 of the concentration state estimating device CE executes the following process for presetting average pupil diameter data.

[0044] FIG. 3 is a flowchart showing an example of the processing procedure and processing contents of a series of processing executed by the control unit 1 of the concentration state estimating device CE in the pre-setting mode.

[0045] (1-1) Obtaining Reference Video Data In the preset mode, for example, multiple reference users are made to watch the entire same lecture video, and while they are watching, the camera CM captures an image of the face of each reference user, including the eye area.

[0046] In response to this, when the control unit 1 of the concentration state estimation device CE recognizes the operation of specifying the preset mode in step S10, first in step S11, under the control of the reference video acquisition processing unit 11, the control unit 1 acquires video data of the face of each of the reference users captured by the camera CM via the sensor I / F unit 4. Then, the acquired reference video data of the face of each of the reference users is stored in the reference video storage unit 31. In step S12, the reference video acquisition processing unit 11 determines whether or not all of the reference video data of each of the reference users has been acquired, and repeatedly executes the reference video acquisition process in step S11 until acquisition is complete.

[0047] (1-2) Calculation of Average Pupil Diameter After completing acquisition of the reference video data for each reference user, the control unit 1 of the concentration state estimation device CE then, under the control of the average pupil diameter calculation processing unit 12, first reads out the stored reference video data for the multiple reference users one by one from the reference video storage unit 31 in step S13, and recognizes a pupil image from the read reference video data using, for example, pattern matching or feature extraction technology. Pupil diameter data representing changes in pupil diameter size are then generated from the pupil image. Figure 5 shows an example of a recognized pupil image and the pupil diameter PD measured from this pupil image.

[0048] Next, in step S14, the average pupil diameter calculation processing unit 12 performs a process of aligning the time axis between the multiple pupil diameter data and averaging them, and stores the data obtained in this way in the average pupil diameter memory unit 32 as average pupil diameter data.

[0049] Here, since it is assumed that the concentration levels of the multiple reference users change randomly, the average pupil diameter data is considered to change regardless of the concentration state. Therefore, by preparing a sufficient number of reference users, the average pupil diameter data obtained by averaging the pupil diameter data of these reference users can be considered to reflect only changes in luminance.

[0050] (2) Estimation Mode When the estimation mode is designated by operating the input device IP, the control unit 1 of the concentration state estimation device CE executes the process of estimating the concentration state of the estimation target user based on the pupil diameter as follows.

[0051] FIG. 4 is a flowchart showing an example of the processing procedure and processing content of a series of processes executed by the control unit 1 of the concentration state estimating device CE in the estimation mode.

[0052] (2-1) Acquisition of Estimation Target Video Data In the estimation mode, while the estimation target user is watching a lecture video, the camera CM captures the face of the estimation target user, including the eye area, over the entire section. Note that if the section to be estimated for concentration level is limited to a part of the scene in the lecture video, the camera CM may capture only the period corresponding to that part of the scene.

[0053] In response to this, when the control unit 1 of the concentration state estimation device CE recognizes the operation of specifying the estimation mode in step S20, first in step S21, under the control of the estimation target video acquisition processing unit 13, the control unit 1 acquires video data of the face of the estimation target user captured by the camera CM via the sensor I / F unit 4. Then, the acquired video data of the face of the estimation target user is stored in the estimation target video storage unit 33 in association with the user ID of the estimation target user.

[0054] (2-2) Generation of Pupil Diameter Data Once the estimation target video data has been acquired, the control unit 1 of the concentration state estimating device CE, under the control of the estimation target pupil diameter data generation processing unit 14, reads the estimation target video data from the estimation target video storage unit 33 in step S22. The estimation target pupil diameter data generation processing unit 14 then extracts a pupil image from the read estimation target video data using, for example, pattern matching or feature extraction technology, and generates estimation target pupil diameter data representing changes in pupil diameter size from the pupil image. The generated estimation target pupil diameter data is then stored in the estimation target pupil diameter storage unit 34 in association with the user ID.

[0055] (2-3) Comparison of Pupil Diameter Data Next, in step S23, under the control of the pupil diameter comparison processing unit 15, the control unit 1 of the concentration state estimating device CE reads the estimation target pupil diameter data from the estimation target pupil diameter storage unit 34 and, in parallel with this, reads the average pupil diameter data from the average pupil diameter storage unit 32. The pupil diameter comparison processing unit 15 then compares the read estimation target pupil diameter data and the average pupil diameter data, aligning the time axes, and generates difference data therebetween. For example, the difference data is generated by subtracting the time series data of the average pupil diameter from the time series data of the estimation target pupil diameter. The pupil diameter comparison processing unit 15 then stores the generated pupil diameter difference data in the difference pupil diameter storage unit 35.

[0056] FIG. 6 shows an example of the comparison process of the pupil diameter data, where PDsub represents the time series change of the pupil diameter data to be estimated, and PDavr represents the time series change of the average pupil diameter data.

[0057] Here, the pupil diameter difference data is considered to be a combination of the pupil diameter change occurring inside the user to be estimated and the pupil diameter change caused by the user to be estimated in response to a change in luminance, minus the pupil diameter change caused by the latter change in luminance. In other words, the pupil diameter comparison process obtains pupil diameter data from which the change component caused by a change in luminance has been removed.

[0058] The means for comparing the pupil diameter data is not limited to the subtraction, but may be multiplication, division, frequency analysis, time resolution analysis, or the like.

[0059] (2-4) Extraction of Pupil Diameter Trend Component Next, in step S24, the control unit 1 of the concentration state estimating device CE, under the control of the pupil diameter trend component calculation processing unit 16, sets an estimation target unit interval of a predetermined estimation target time length (e.g., 5 to 10 seconds) for the pupil diameter difference data. Then, the pupil diameter trend component calculation processing unit 16 extracts a pupil diameter trend component that represents the tendency of change in pupil diameter for each estimation target unit interval. Note that the time length of the estimation target unit interval may be set to any time length equal to or greater than 10 seconds.

[0060] For example, the pupil diameter trend component calculation processing unit 16 performs a smoothing process to take a moving average of the pupil diameter difference data for each estimation target unit interval, and then calculates the amount of change in the pupil diameter value between the estimation target unit interval and the adjacent estimation target unit interval based on the smoothed data.

[0061] In addition, in the estimation target unit section, as a means for calculating the tendency of change in pupil diameter difference data, other than calculating a moving average, for example, frequency analysis or time resolution analysis may be used.

[0062] (2-5) Estimation of concentration level Next, in step S25, under the control of the concentration level estimation processing unit 17, the control unit 1 of the concentration state estimation device CE estimates the concentration level of the user to be estimated based on the amount of change in the pupil diameter value in each estimation target unit interval calculated by the pupil diameter trend component calculation processing unit 16.

[0063] For example, the concentration level estimation processing unit 17 estimates that the concentration level is low in a section where the amount of change is increasing, and that the concentration level is high in a section where the amount of change is decreasing.

[0064] The concentration level estimation processing unit 17 then stores information representing the concentration level estimation result obtained as described above in the concentration level estimation information storage unit 36, in association with information representing the time position of the unit section to be estimated in the lecture video.

[0065] Finally, in step S26, the concentration level estimation processing unit 17 reads information representing the estimation result of the concentration level from the concentration level estimation information storage unit 36, and outputs the read information representing the estimation result of the concentration level to the display device OD from the input / output I / F unit 5. Thus, the display device OD displays the estimation result of the concentration level of the estimation target user while watching the lecture video, in association with each estimation target unit section.

[0066] The control unit 1 of the concentration state estimation device CE executes a series of processes in steps S21 to S27 for each estimation target user until it recognizes in step S27 that an instruction to end the estimation mode has been input. Then, when the instruction to end the estimation mode is input, the estimation process is terminated.

[0067] (Effects) As described above, in one embodiment, first, in a preset mode, average pupil diameter data, the primary component of which is a luminance change component of the lecture video, is generated based on pupil diameter data of multiple reference users while they are watching a lecture video. Next, in an estimation mode, pupil diameter data of the estimation target user while watching the lecture video is acquired, and differential data is calculated between the acquired pupil diameter data of the estimation target user and the average pupil diameter data, thereby generating pupil diameter data from which the pupil diameter change component due to luminance changes in the lecture video has been removed from the pupil diameter data of the estimation target user. Then, an estimation target unit interval of approximately 5 to 10 seconds is set for the generated pupil diameter data, and a pupil diameter trend component is calculated from this interval. Based on the calculated pupil diameter trend component, the estimation target user's concentration level while watching the lecture video is estimated for each estimation target unit interval.

[0068] Therefore, since the concentration level is estimated based on the pupil diameter trend component in the estimation target unit interval, which is set to a relatively long time of about 5 to 10 seconds, it is possible to estimate the type of concentration state that does not change in real time but changes in relatively long time intervals of tens of seconds. Moreover, by removing the component of pupil diameter change due to brightness changes in the lecture video, the influence of brightness changes in the lecture video is eliminated, making it possible to accurately estimate the concentration level of the estimation target user.

[0069] Other Embodiments (1) In one embodiment, data representing the average pupil diameter is obtained by calculating the average of time-series data representing the pupil diameters of multiple reference users. However, rather than simply averaging the pupil diameters of multiple reference users, the multiple reference users may be classified into multiple groups based on their attributes (e.g., age, gender, and various nervous system characteristics that affect pupil diameter), and the average pupil diameter may be calculated for each of the multiple groups. In this case, in the estimation mode, the average pupil diameter of the corresponding group is selected and used based on the attributes of the user to be estimated. This makes it possible to use average pupil diameter data that more closely matches the characteristics of the user to be estimated in response to changes in luminance.

[0070] (2) For example, the target user may be made to view a video simulating a change in luminance multiple times, and data representing the pupil diameter obtained during each viewing may be averaged to obtain data representing the average pupil diameter. In this way, it is possible to set average pupil diameter data that reflects the unique characteristics of each target user in response to a change in luminance.

[0071] (3) In one embodiment, the description has been given taking as an example a case where the concentration level of participants in a seminar, etc. However, the application is not limited to this, and the present invention can also be applied to, for example, a case where the concentration level of a user who evaluates the viewing of an advertising video is estimated, or a case where the concentration level of a user who inspects products or parts on a production line where the products or parts appear at a regular rhythm is estimated.

[0072] (4) In one embodiment, a case has been described in which a personal computer used by a seminar organizer or the like is used as the concentration state estimation device CE. However, the present invention is not limited to this, and can also be applied to cases in which a personal computer, tablet terminal, smartphone, HMD (Head Mount Display), smart glasses, or the like used by a target user such as a participant is used as the concentration state estimation device.

[0073] In this case, the concentration state estimation device is provided with the function of downloading and storing data representing the average pupil diameter in advance from the seminar organizer's personal computer or server via a network, and transmitting concentration level estimation information to the seminar organizer's personal computer or server via the network.

[0074] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0075] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0076] DESCRIPTION OF THE SYMBOLS CE...concentration state estimation device CM...camera IP...input device DP...display device 1...control unit 2...program storage unit 3...data storage unit 4...sensor I / F unit 5...input / output I / F unit 6...bus 11...reference video acquisition processing unit 12...average pupil diameter calculation processing unit 13...estimation target video acquisition processing unit 14...estimation target pupil diameter data generation processing unit 15...pupil diameter comparison processing unit 16...pupil diameter trend component calculation processing unit 17...concentration level estimation processing unit 31...reference video storage unit 32...average pupil diameter storage unit 33...estimation target video storage unit 34...estimation target pupil diameter storage unit 35...differential pupil diameter storage unit 36...concentration level estimation information storage unit

Claims

1. A concentration state estimation device that estimates the concentration state of a target user based on the pupil diameter, comprising: a first processing unit that acquires first pupil diameter information including, as a main component, a component corresponding to a luminance change in the activity environment of the target user; a second processing unit that acquires second pupil diameter information representing a change in the pupil diameter during the activity of the target user; a third processing unit that compares the first pupil diameter information and the second pupil diameter information, and generates third pupil diameter information in which a component corresponding to the luminance change is removed from the second pupil diameter information based on the comparison result; a fourth processing unit that sets an estimation target unit period having a predetermined time length for the third pupil diameter information, and generates change tendency information representing a change tendency of the pupil diameter for each estimation target unit period; and a fifth processing unit that estimates the concentration state of the target user based on the change tendency information.

2. The concentration state estimation device according to claim 1, wherein the first processing unit acquires, as the first pupil diameter information, either first average pupil diameter information generated by averaging time series information representing changes in the pupil diameters of a plurality of reference users in the activity environment over the time axis, or second average pupil diameter information generated by averaging time series information representing changes in the pupil diameter of the target user measured in advance a plurality of times in the activity environment over the time axis.

3. The concentration state estimation device according to claim 1, wherein the first processing unit acquires time series information representing changes in the pupil diameters of a plurality of reference users in the activity environment respectively, and uses third average pupil diameter information generated by averaging the acquired plurality of time series information over the time axis for each attribute of the reference users as the first pupil diameter information; the third processing unit selects the third average pupil diameter information corresponding to the attribute of the target user as the first pupil diameter information, and compares the selected first pupil diameter information and the second pupil diameter information.

4. A method for estimating the concentration state of a target user based on the pupil diameter, which is executed by an information processing apparatus. The method includes: a process of acquiring first pupil diameter information including, as a main component, a component corresponding to a luminance change in the activity environment of the target user; a process of acquiring second pupil diameter information representing a change in the pupil diameter during the activity of the target user; a process of comparing the first pupil diameter information and the second pupil diameter information, and generating third pupil diameter information in which a component corresponding to the luminance change is removed from the second pupil diameter information based on the comparison result; a process of setting an estimation target unit period with a predetermined time length for the third pupil diameter information, and generating change tendency information representing a change tendency of the pupil diameter for each estimation target unit period; and a process of estimating the concentration state of the target user based on the change tendency information.

Images