Psychological state estimation device and psychological state estimation method

Abstract

To provide a psychological state estimation apparatus and a psychological state estimation method which can properly estimate a pupil diameter or a pupil area even when a peripheral environment changes, and can estimate the psychological state of a user on the basis of the estimated pupil diameter or pupil area.SOLUTION: A psychological state estimation apparatus comprises: a pupil information acquisition unit which acquires a pupil diameter or area in a state where a psychological state of a user is normal; a first measurement unit which measures environmental illuminance; a second measurement unit which measures illuminance of a screen or window on which an object to which a sight line is directed is reflected; a third measurement unit which measures luminance of the object and size of the object reflected on the screen or window; a pupil diameter estimation unit which estimates the pupil diameter or pupil area of the user by multiplying correction coefficients based on the measured environmental illuminance, measured illuminance of the screen or window, the illuminance of the object reflected on the screen or window and the measured size of the object; and a psychological state estimation unit which estimates the psychological state of the user by comparing the acquired pupil diameter or pupil area in the normal time with the estimated pupil diameter or pupil area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a psychological state estimation device and a psychological state estimation method. [Background technology]

[0002] Attempts have been made to measure pupil diameter and estimate internal states such as human psychology (see, for example, Patent Document 1). The technology described in Patent Document 1 involves photographing the subject's eyes and then performing image processing on the photographed image to determine pupil area and pupil diameter. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-174541 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a real environment such as a conference room, in addition to the illuminance of the surrounding environment (ambient illuminance) such as the brightness of the room and the illuminance and brightness of the entire screen that the conference participants are looking at, there is also contrast within the screen, such as the size and brightness of objects on the screen that the participants are actually gazing at. For this reason, the technology described in Patent Document 1 cannot accurately correct changes in pupil diameter due to the light reflex unless it takes into account the effect of light from objects on the screen.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a psychological state estimation device and a psychological state estimation method that can appropriately estimate pupil diameter or pupil area even when the surrounding environment changes, and can estimate a user's psychological state based on the estimated pupil diameter or pupil area. [Means for solving the problem]

[0006] (1) In order to achieve the above object, a psychological state estimation device according to one embodiment of the present invention includes a pupil information acquisition unit that acquires the pupil diameter or area of ​​a user when the user's psychology is normal; a first measurement unit that measures the environmental illuminance of the environment in which the user is located; a second measurement unit that measures the illuminance of a screen or window that displays an object toward which the user is looking; a third measurement unit that measures the luminance of the object displayed on the screen or window and the size of the object; a pupil diameter estimation unit that estimates the pupil diameter or pupil area of ​​the user by multiplying the measured environmental illuminance, the measured illuminance of the screen or window that displays the object toward which the user is looking, the illuminance of the object displayed on the screen or window, and the measured size of the object; and a psychological state estimation unit that compares the acquired pupil diameter or pupil area at normal times with the estimated pupil diameter or pupil area to estimate the user's psychological state.

[0007] (2) In the psychological state estimation device according to the aspect of the present invention, the size of the object may be a value converted into a viewing angle.

[0008] (3) In addition, in the psychological state estimation device according to one aspect of the present invention, the second measurement unit may extract the screen and its contours from an image captured by a camera, calculate an average brightness, and determine the illuminance.

[0009] (4) In addition, in a psychological state estimation device according to one aspect of the present invention, the pupil diameter estimation unit may use the measured ambient illuminance and the measured illuminance of the screen or window on which the object toward which the user is looking is displayed as logarithms.

[0010] (5) In the psychological state estimation device according to an aspect of the present invention, the pupil diameter estimation unit may use the reciprocal of the pupil area.

[0011] (6) In order to achieve the above object, a psychological state estimation method according to one aspect of the present invention includes a pupil information acquisition unit that acquires the pupil diameter or area of ​​a user when the user's psychology is normal, a first measurement unit that measures the environmental illuminance of the environment in which the user is present, a second measurement unit that measures the illuminance of a screen or window that displays an object at which the user is looking, a third measurement unit that measures the size of the object at which the user is looking and the luminance of the object reflected on the screen or window, a pupil diameter estimation unit that estimates the pupil diameter or pupil area of ​​the user by multiplying the measured environmental illuminance, the measured illuminance of the screen or window that displays the object at which the user is looking, the illuminance of the object reflected on the screen or window, and the measured size of the object, and a psychological state estimation unit that compares the acquired pupil diameter or pupil area at normal psychology with the estimated pupil diameter or pupil area to estimate the user's psychological state. [Effects of the Invention]

[0012] According to (1) to (6), the pupil diameter or pupil area can be appropriately estimated even when the surrounding environment changes, and the psychological state of the user can be estimated based on the estimated pupil diameter or pupil area. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a psychological state estimation device according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining illuminance measurement according to the embodiment. [Figure 3] FIG. 10 is a diagram for explaining luminance measurement according to the embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of a second illuminance measurement unit. [Figure 5] FIG. 10 is a diagram showing a correlation coefficient in the first correction formula. [Figure 6] FIG. 1 is a diagram for explaining an evaluation method. [Figure 7] FIG. 10 is a diagram showing an example of an evaluation result. [Figure 8] 10 is a flowchart of a processing procedure of the psychological state estimation device according to the embodiment. [Figure 9]FIG. 10 is a diagram showing the relationship between illuminance and pupil diameter. [Figure 10] FIG. 10 is a diagram showing an approximate expression of the relationship between illuminance and pupil diameter. [Figure 11] FIG. 10 is a diagram showing the relationship between the slope and intercept of an approximation equation with respect to the illuminance of the environment. [Figure 12] 10A and 10B are diagrams showing examples of the luminance of an image display device used in an experiment and the luminance of a displayed object. [Figure 13] FIG. 10 is a diagram showing the relationship between the luminance of an object and the normalized (1 / pupil area) as an experimental result. [Figure 14] FIG. 10 is a diagram illustrating parameters of an approximate formula. [Figure 15] FIG. 10 is a diagram illustrating the relationship between parameters and the size of an object. [Figure 16] FIG. 10 is a diagram illustrating parameters of an approximate formula. [Figure 17] FIG. 10 is a diagram showing the relationship between parameters and the size of an object converted into a viewing angle. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component is appropriately changed so that each component can be recognized. In all the drawings for explaining the embodiments, the same reference numerals are used for components having the same functions, and repeated explanations will be omitted. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, any information).

[0015] <Summary> First, an outline of this embodiment will be described. In this embodiment, when a user looks at a display device or at the windshield from inside a vehicle, for example, the change in pupil diameter due to the light reflex is corrected using four variables: ambient illuminance, screen (front) luminance, luminance of an object present on the screen (at the gaze point), and size of the object present on the screen (at the gaze point).Then, in this embodiment, the pupil diameter based on the measured value is compared with the estimated pupil diameter value to estimate the user's psychological state.

[0016] <Mental state estimation device> First, an example of the configuration of the mental state estimation device 1 will be described. Fig. 1 is a diagram showing an example of the configuration of a psychological state estimation device according to this embodiment. As shown in Fig. 1, the psychological state estimation device 1 includes a first illuminance measurement unit 101 (first measurement unit), a second illuminance measurement unit 102 (second measurement unit), a luminance measurement unit 103 (third measurement unit), a size measurement unit 104 (third measurement unit), a first correction unit 105 (pupil diameter estimation unit), a second correction unit 106 (pupil diameter estimation unit), a pupil diameter estimation unit 107, an actual pupil diameter measurement value acquisition unit 108 (pupil information acquisition unit), a pupil diameter change calculation unit 109, a psychological state estimation unit 110, and an output unit 111.

[0017] Next, the operation of each component of the mental state estimation device 1 will be described with reference to Figures 1 to 4. Note that, in the following example, a state in which a user is looking at the image display device 2 will be described as an example, but the present invention is not limited to this. Fig. 2 is a diagram for explaining illuminance measurement according to the present embodiment, Fig. 3 is a diagram for explaining luminance measurement according to the present embodiment, and Fig. 4 is a diagram showing an example of a second illuminance measurement unit.

[0018] The first illuminance measurement unit 101 is, for example, an illuminance meter, and measures the illuminance 3 of the environment in which the user hu is viewing, for example, the image display device 2. Note that the first illuminance measurement unit 101 may include, for example, a camera and illuminance or brightness detection software.

[0019] The second illuminance measurement unit 102 is, for example, an illuminance meter, and measures the illuminance 4 of an object in the line of sight of the user hu, for example, looking at the image display device 2. The second illuminance measurement unit 102 may be attached to a glasses-type device 201 worn by the user, as shown in FIG. 4. The glasses-type device 201 may also have the functions of the pupil diameter actual measurement value acquisition unit 108, a communication function, etc. The second illuminance measurement unit 102 may also have, for example, a camera and illuminance or brightness detection software. When the second illuminance measurement unit 102 acquires the illuminance in the line of sight using a camera, it extracts the contours of the screen and the image plane, and calculates the average brightness.

[0020] The brightness measurement unit 103 is, for example, a camera and brightness or illuminance detection software, and measures the brightness 5 of an object obj that a user hu is looking at, for example, on an image display device 2. Note that it is preferable to measure the brightness of an object using a camera and brightness or illuminance detection software rather than a luminance meter. The reason for this is that a luminance meter can only measure points, and therefore cannot accurately obtain the illuminance or brightness of an object.

[0021] The size measurement unit 104 is, for example, a camera equipped with an image processing circuit, and measures the size of an object obj on, for example, the image display device 2 that the user hu is looking at.

[0022] The first correction unit 105 acquires the first illuminance measured by the first illuminance measurement unit 101 and the second illuminance measured by the second illuminance measurement unit 102. The first correction unit 105 substitutes the acquired values ​​into a first correction formula to calculate the reciprocal of the pupil area (1 / S). The first correction formula will be described later.

[0023] The second correction unit 106 acquires the luminance and size of the object obj displayed on the image display device 2. The second correction unit 106 calculates the reciprocal of the pupil area by substituting the acquired values ​​into a second correction formula. The second correction formula will be described later.

[0024] The pupil diameter estimation unit 107 calculates the reciprocal of the pupil area using the first and second correction formulas. Because the shape of a human movement is round, the pupil diameter estimation unit 107 estimates the pupil diameter from the calculated pupil area.

[0025] The pupil diameter measurement value acquisition unit 108 acquires the pupil diameter or area measured when the user is in a normal mental state.

[0026] Pupil diameter change calculation section 109 calculates the difference between the actually measured pupil diameter value and the estimated pupil diameter.

[0027] The psychological state estimation unit 110 estimates the psychological state of the user based on the change calculated by the pupil diameter change calculation unit 109 using a known method.

[0028] The output unit 111 outputs to an external device information indicating the psychological state of the user estimated by the psychological state estimation unit 110. The external device is, for example, a personal computer, an image display device, a printing device, a tablet terminal, a smartphone, or the like.

[0029] <Measuring the brightness and size of an object> Here, an example of a method for measuring the brightness and size of an object will be described. The illuminance, brightness, and size of the object are determined by extracting the contour from the camera image. That is, the illuminance at the line of sight can also be measured by using the brightness measurement unit 103 attached to the eyeglass-type device shown in FIG. 4 as a camera, which also has the function of the second illuminance measurement unit 102.

[0030] It is preferable that each measurement unit has a camera with a wide dynamic range and no auto-gain function. Contrast adjustment by the auto-gain function and halation due to sensor sensitivity saturation can cause a decrease in the accuracy of luminance extraction. However, the auto-gain function can avoid a decrease in luminance extraction accuracy by creating a correction table for each gain.

[0031] <First correction formula> The first correction formula will be explained. The first correction formula is formed from the ambient illuminance and the line-of-sight illuminance (average screen luminance) as shown in the following formula (1).

[0032]

number

[0033] In equation (1), S is the pupil area, Ill env is the ambient illuminance (lux), Ill dis is the illuminance in the direction of the line of sight. dis is, for example, the average screen luminance at 255 gradations. Furthermore, ln represents the natural logarithm. The predicted pupil diameter value and the measured pupil diameter value obtained by this formula (1) have a coefficient of determination (R-squared) of 0.89. Furthermore, when a comparison is made when calculating with each parameter, the result is as shown in Figure 5, which shows that the coefficient of formula (1) is the combination with the highest coefficient of determination. Figure 5 is a diagram showing the correlation coefficient in the first correction formula. Note that each coefficient shown in Figure 5 is an example, and any value appropriate for the experimental environment (each illuminance, luminance of the image display device, size of the object, distance between the user and the image display device, etc.) may be used.

[0034] The relationship between the illuminance and luminance and the pupil diameter used in the first correction formula is as follows: The first correction formula is the relationship between pupil diameter and pupil area and ambient illuminance and line-of-sight illuminance (average screen luminance). Because the equation deals with the amount of light entering the pupil, pupil area fits better than pupil diameter, but pupil diameter can also be used. Also, the equation fits better when the reciprocal of pupil diameter and pupil area is used.

[0035] In addition, the pupil diameter in the dark can be used to normalize the first correction formula to obtain the pupil constriction rate. The formula fits better when the illuminance and luminance are logarithmic, but it is not necessary to use logarithms. Furthermore, the illuminance and brightness may be measured using an illuminance meter, a luminance meter, or brightness of 255 levels extracted from an image.

[0036] As described above, according to this embodiment, by using the first correction formula, the accuracy is improved by using the pupil area instead of the pupil diameter. According to this embodiment, the prediction accuracy is further improved by taking the reciprocals of the pupil diameter and pupil area. According to this embodiment, the prediction accuracy can be improved by taking the logarithms of the illuminance and luminance.

[0037] <Second correction formula> Next, the second correction formula will be described. The second correction formula is formed from the brightness and size of the object, as shown in the following formula (2).

[0038]

number

[0039] In equation (2), Bri tar is the brightness of the object, and Siz tar is the size of the object. Note that each coefficient in equation (2) is an experimentally determined value.

[0040] The second correction formula is a coefficient for correcting the pupil diameter and pupil area calculated by the first correction formula (formula (1)), as shown in the following formula (3).

[0041]

number

[0042] The pupil diameter d is calculated using the following equation (4).

[0043]

number

[0044] <Evaluation> Next, an example of the results of evaluation using such a correction formula will be described. FIG. 6 is a diagram for explaining the evaluation method. As shown in FIG. 6, the predicted data is a parameter and a coefficient of determination. The first evaluation is a case where the first correction formula is used but the second correction formula is not used, i.e., the size and brightness of the object are not corrected. The second evaluation is a case where the first and second correction formulas are used, i.e., the size and brightness of the object are corrected. In the evaluation, actual measured pupil diameter data for verification (not used to create the correction formula) was prepared and used. In addition, in the evaluation, the coefficient of determination was calculated from the predicted pupil diameter calculated from the correction formula and the prepared actual measured pupil diameter data.

[0045] FIG. 7 is a diagram showing an example of the evaluation results. In FIG. 7, the horizontal axis represents the measured pupil diameter (mm), and the vertical axis represents the predicted pupil diameter (mm). Point g11 represents the result without correction in the first evaluation, and point g12 represents the result with correction in the second evaluation. The dashed line g21 represents the approximation line for the ideal case (y=0.0566x+4.2494, R 2 =0.653), and the dashed line g22 is the approximate line with correction in the second evaluation (y=0.6795x+09814, R 2 =0.730).

[0046] As shown in Figure 7, when the second correction formula is not used and correction for the luminance and size of the object is not included (point g11), the illuminance at the line of sight (average screen luminance) is almost constant, and the change in pupil diameter cannot be reproduced. In other words, the influence of the object cannot be taken into account using only the first correction formula. In contrast, by correcting the brightness and size of the object using the second correction formula as in this embodiment, the correlation with the actual measured value is improved. That is, according to this embodiment, by using the second correction formula, correction is possible with a coefficient of determination of 0.7 or more using a model including the target size and brightness. In this way, according to this embodiment, by using the first correction formula and the second correction formula, the accuracy of the predicted pupil diameter when an object is present in the line of sight can be improved.

[0047] <Example of processing procedure> Next, an example of the processing procedure of the mental state estimation device 1 will be described. FIG. 8 is a flowchart of the processing procedure of the psychological state estimation device according to this embodiment.

[0048] (Step S1) The first illuminance measuring unit 101 measures the illuminance of the environment in which the user hu is viewing the image display device 2.

[0049] (Step S2) The second illuminance measuring unit 102 measures the illuminance of an object in front of the line of sight of the user hu, at which the image display device 2 is being viewed.

[0050] (Step S3) First correction unit 105 substitutes the first illuminance measured by first illuminance measurement unit 101 and the second illuminance measured by second illuminance measurement unit 102 into a first correction formula to calculate information about the pupil area (the reciprocal of the pupil area (1 / S), or pupil area S). Note that first correction unit 105 may also calculate the pupil diameter using the calculated pupil area.

[0051] (Step S4) The brightness measurement unit 103 measures the brightness of the object obj on the image display device 2 that the user hu is looking at.

[0052] (Step S5) The size measurement unit 104 measures the size of the object on the image display device 2 that the user hu is looking at.

[0053] (Step S6) The second correction unit 106 substitutes the brightness and size of the object displayed on the image display device 2 into the second correction formula to calculate information about the pupil area (the reciprocal of the pupil area (1 / S), or the pupil area S).

[0054] (Step S7) The pupil diameter measurement value acquisition unit 108 acquires the pupil diameter value or pupil area actually measured by the user hu when the user hu is in a normal psychological state (for example, when at rest). Note that the pupil diameter measurement value acquisition unit 108 may acquire the pupil diameter value actually measured by the user in advance.

[0055] (Step S8) Pupil diameter change calculation unit 109 calculates the difference between the actually measured pupil area and the estimated pupil area. Alternatively, pupil diameter change calculation unit 109 calculates the difference between the actually measured pupil diameter value and the estimated pupil diameter.

[0056] (Step S9) The psychological state estimation unit 110 estimates the psychological state of the user based on the change calculated by the pupil diameter change calculation unit 109 using a well-known method.

[0057] <Calculation example of the first correction formula> Next, a calculation example of the first correction formula will be described with reference to FIGS. Fig. 9 is a diagram showing the relationship between illuminance and pupil diameter. In Fig. 9, the horizontal axis represents the ambient illuminance and the illuminance (lux) of the image display device, and the vertical axis represents the pupil diameter (mm). The squares and line g31 represent the case where the ambient illuminance is 20 lux, the triangles and line g32 represent the case where the ambient illuminance is 30 lux, and the circles and line g33 represent the case where the ambient illuminance is 70 lux. The illuminance (brightness values) of the image display device are RGB = 32, 64, 128, and 192. 9, a dashed arrow g34 indicates a change in pupil diameter relative to the illuminance of the environment, and a dashed arrow g35 indicates a change in pupil diameter relative to the image display device.

[0058] Fig. 10 is a diagram showing the relationship between illuminance and pupil diameter using an approximate formula. In Fig. 10, the horizontal axis represents the illuminance of the environment and the logarithmic illuminance (lux) of the image display device, and the vertical axis represents 1 / pupil area (m -2 The square and line g41 represent the case where the environmental illuminance is 20 lux, the triangle and line g42 represents the case where the environmental illuminance is 30 lux, and the circle and line g43 represents the case where the environmental illuminance is 70 lux. As shown in Figure 10, if the vertical axis is the inverse of the pupil area and the horizontal axis is the logarithm of the illuminance, the approximation equation approaches a straight line.

[0059] FIG. 11 is a diagram showing the relationship between the slope and the intercept of the approximate formula with respect to the illuminance of the environment. In FIG. 11, the horizontal axis is the illuminance (lux) of the environment, and the vertical axis is the intercept and the slope. Line g51 is the change in the intercept with respect to the illuminance, and line g52 is the change in the slope with respect to the illuminance. Based on these examination results, the above-described first correction formula (Equation (1)) and each coefficient were determined. In Equation (1), {0.0013(Ill env ) + 0.0394} corresponds to the slope, and ln(Ill env + Ill dis ) - {(0.0071(Ill env ) + 0.0117)} corresponds to the intercept.

[0060] <Calculation Example of the Second Correction Formula> Next, a calculation example of the second correction formula will be described while referring to FIGS. 12 to 15. FIG. 12 is a diagram showing an example of the luminance of the image display device used in the experiment and the luminance of the object to be displayed. In the experiment, the average luminance (RGB gray scale) of the image display device 2 was unified to be 64. Then, objects (circles) having different sizes and luminances were displayed as shown in FIG. 12. Image g61 is the case where the luminance of the image display device and the luminance of the object are the same. Images g62 to g64 are examples where the size of the object is the same and the luminance of the object is different (g62 < g63 < g64). Images g64 to g65 are examples where the luminance of the object is the same and the size of the object is different (g64 < g65).

[0061] FIG. 13 shows the results of an experiment conducted on a user using the image in FIG. 12. FIG. 13 is a diagram showing the relationship between the luminance of an object and normalized (1 / pupil area), which is the experimental result. The horizontal axis represents the luminance of the object (grayscale (0 to 255)), and the vertical axis represents normalized (1 / pupil area). Line g71 represents the case where the object size is 100 dots, line g72 represents the case where the object size is 200 dots, line g73 represents the case where the object size is 300 dots, line g74 represents the case where the object size is 400 dots, line g75 represents the case where the object size is 600 dots, and line g76 represents the case where the object size is 800 dots. The dashed line g78 represents the luminance where the luminance of the object and the luminance of the image display device are equal. Each dashed line represents the result of a quadratic approximation.

[0062] FIG. 14 is a diagram showing the parameters of the approximation formula. The parameters of the approximation formula are the gradient (×E -3 ) and intercept, and the results shown in Figure 14 were obtained from the results in Figure 13. The relationship between these obtained parameters and the size of the object can be graphed as shown in Figure 15. Figure 15 is a diagram showing the relationship between the parameters and the size of the object. The horizontal axis is the size of the object (dot scale), the left vertical axis is the slope, and the right vertical axis is the intercept. Line g81 is the slope, and line g82 is the intercept. Each dotted line is the result of a quadratic approximation formula.

[0063] As shown in Figures 13 and 15, similar to the case of the environmental illuminance and the luminance of the image display device 2, the approximation is close to a straight line for one over the pupil area. Furthermore, there is a linear response to the luminance of the object, and the larger the size of the object, the greater the effect. Based on this experimental result, the second correction formula and each parameter described above were determined.

[0064] <Example of creating a correction formula based on the size and brightness of the object> Next, an example of creating a correction formula based on the size and luminance of an object will be described with reference to Fig. 13 and Figs. 16 and 17. The experimental conditions and methods are the same as those described with reference to Figs. 12 to 15.

[0065] FIG. 16 is a diagram showing the parameters of the approximation formula. In FIG. 14, the parameters were classified by the size of the object, but in FIG. 16, the size of the object is classified by the value converted into the viewing angle. The parameters of the approximation formula are the inclination (×E -3 ) and the intercept. Figure 17 shows the relationship between these parameters and the size of the object.

[0066] Figure 17 shows the relationship between parameters and object size converted into viewing angle. The horizontal axis is object size (converted into viewing angle, degrees), the left vertical axis is tilt, and the right vertical axis is intercept. Line g91 is tilt, and line 982 is intercept. Each dashed line is the result of a quadratic approximation formula.

[0067] As in the cases of the environmental illuminance and the luminance of the image display device, the approximation formula fits well to 1 / pupil area, as shown in Figures 16 and 17. There is also a linear response to the target luminance, and the effect increases as the target size increases. Based on these results, when the size of the object is converted into the viewing angle, B in the second correction formula is given by the following formula (5): Note that the values ​​of each parameter in formula (5) are just examples and will vary depending on the experimental conditions.

[0068]

number

[0069] In the above examples and experimental method examples, a circle (circle) was used as an example of the object, but the shape is not limited to this. Also, while the example of the object being displayed on the image display device 2 has been described, the object is not limited to this and may be a real object. Furthermore, pupil diameter estimation can also be applied to, for example, a case where the object is viewed through a windshield or window from inside a vehicle. In this case, the psychological state estimation device 1 measures the illuminance or luminance of the windshield or window.

[0070] As described above, in this embodiment, when viewing an image display device or viewing the windshield from inside a vehicle, the pupil diameter is calculated using a correction formula that corrects changes in pupil diameter due to the pupillary reflex using four variables: ambient illuminance, screen (front) luminance, and the luminance and size of an object present on the screen (at the gaze point).Then, in this embodiment, the user's psychological state is estimated based on the estimated changes in pupil diameter.

[0071] As a result, according to this embodiment, the pupil diameter or pupil area can be appropriately estimated even if the surrounding environment changes, and the psychological state of the user can be estimated based on the estimated pupil diameter or pupil area.

[0072] A program for realizing all or part of the functions of the psychological state estimation device 1 of the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the psychological state estimation device 1. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when the program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0073] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0074] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0075] 1...Mental state estimation device, 101...first illuminance measurement unit, 102...second illuminance measurement unit, 103...luminance measurement unit, 104...size measurement unit, 105...first correction unit, 106...second correction unit, 107...pupil diameter estimation unit, 108...actual pupil diameter measurement value acquisition unit, 109...pupil diameter change calculation unit, 110...mental state estimation unit, 111...output unit

Claims

1. a pupil information acquisition unit that acquires a pupil diameter or area when the user's state of mind is normal; a first measurement unit that measures an environmental illuminance of an environment where the user is present; a second measuring unit that measures the illuminance of a screen or a window on which an object toward which the user is looking is displayed; a third measuring unit that measures the luminance of the object displayed on the screen or the window and the size of the object; a pupil diameter estimation unit that estimates a pupil diameter or pupil area of ​​the user based on the measured environmental illuminance, the measured illuminance of the screen or the window on which an object toward which the user is looking is reflected, the luminance of the object reflected on the screen or the window, and the measured size of the object; a psychological state estimation unit that compares the acquired pupil diameter or pupil area under normal conditions with an estimated pupil diameter or pupil area to estimate the psychological state of the user; A psychological state estimation device comprising:

2. The size of the object is a value converted into a viewing angle. The psychological state estimation device according to claim 1 .

3. The second measurement unit extracts the screen and its contour from the image captured by the camera, calculates average luminance, and obtains illuminance. The psychological state estimation device according to claim 1 or 2.

4. the pupil diameter estimation unit uses, as logarithms, the measured environmental illuminance and the measured illuminance of the screen or the window on which the object toward which the user is looking is displayed. The psychological state estimation device according to any one of claims 1 to 3.

5. the pupil diameter estimation unit uses the inverse of the pupil area. The psychological state estimation device according to any one of claims 1 to 4.

6. The pupil information acquisition unit acquires the pupil diameter or area when the user's mind is normal, a first measurement unit measuring an environmental illuminance of an environment in which the user is present; a second measuring unit measuring the illuminance of a screen or a window on which an object toward which the user is looking is reflected; a third measuring unit measuring the size of the object toward which the user is looking and the luminance of the object reflected on the screen or the window; a pupil diameter estimation unit estimates a pupil diameter or pupil area of ​​the user based on the measured environmental illuminance, the measured illuminance of the screen or the window on which the object toward which the user is looking is reflected, the luminance of the object reflected on the screen or the window, and the measured size of the object; a psychological state estimation unit that compares the acquired pupil diameter or pupil area under normal conditions with the estimated pupil diameter or pupil area to estimate the psychological state of the user; A method for estimating psychological states.

Images