Impersonation detection system, impersonation detection method, and computer program

Abstract

The system accurately detects the eye movements of the target person to determine if they are impersonating someone else. [Solution] The impersonation detection system includes a display control means that displays the point of focus that the target person is looking at in a manner that repeats a first period in which it moves at a predetermined speed and a second period in which the point of focus is stopped for a predetermined period; an estimation means that estimates the viewpoint position of the target person from the target person's image; a deviation amount production means that calculates the deviation amount between the estimated viewpoint position and the actual viewpoint position; and an impersonation detection means that detects that the target person is being impersonated when the deviation amount is small, and does not detect that the target person is being impersonated when the deviation amount is large.

Description

Technical Field

[0001] This disclosure relates to the technical field of a spoofing determination system, a spoofing determination method, and a computer program for estimating the line of sight of a subject.

Background Art

[0002] As this type of system, one that detects the line of sight of a subject is known. For example, in Patent Document 1, a technique for estimating a line-of-sight position from visual stimulus information defined in advance on a display and the movement of the subject's eyes is disclosed. In Patent Document 2, a technique for performing a detection process for detecting a line of sight, such as the corneal reflection method, using each of the images captured at a predetermined frame rate is disclosed. In Patent Document 3, a technique for performing a line-of-sight detection process using an image of a user received from an imaging device is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] The method of having the subject gaze at a stopped fixation point not only places a large burden on the subject but may also induce fluctuations in unconscious eye movements. In each of the above-mentioned patent documents, measures against fluctuations in the line of sight are not sufficient and there is room for improvement.

[0005] An object of this disclosure is to provide a spoofing determination system, a spoofing determination method, and a computer program for solving the above-described problems. [Means for solving the problem]

[0006] One aspect of the impersonation detection system disclosed herein includes: a display control means that displays a point of focus that a subject is looking at in such a way that it moves in a sequence of first periods in which it moves at a predetermined speed and second periods in which it remains stationary for a predetermined period; an estimation means that estimates the subject's viewpoint position from an image of the subject; a deviation amount production means that calculates the deviation amount between the estimated viewpoint position and the actual viewpoint position; and an impersonation detection means that detects that the subject is being impersonated when the deviation amount is small, and does not detect that the subject is being impersonated when the deviation amount is large.

[0007] One aspect of the impersonation detection method of this disclosure involves at least one computer displaying a point of gaze that a subject is looking at, such that it moves in a sequence of a first period at a predetermined speed and a second period in which the point of gaze is stopped for a predetermined period; estimating the subject's viewpoint position from the subject's image; calculating the discrepancy between the estimated viewpoint position and the actual viewpoint position; detecting that the subject is being impersonated when the discrepancy is small; and not detecting that the subject is being impersonated when the discrepancy is large.

[0008] One aspect of the computer program of this disclosure displays a point of focus that a subject is looking at in a manner that alternates between a first period in which the point of focus moves at a predetermined speed and a second period in which the point of focus remains stationary for a predetermined period; estimates the subject's viewpoint position from the subject's image; calculates the discrepancy between the estimated viewpoint position and the actual viewpoint position; detects that the subject is being impersonated when the discrepancy is small; and does not detect that the subject is being impersonated when the discrepancy is large. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the overall configuration of the gaze estimation system according to the first embodiment. [Figure 2] This is a block diagram showing the hardware configuration of the gaze estimation system according to the first embodiment. [Figure 3] This flowchart shows the operation flow of the gaze estimation system according to the first embodiment. [Figure 4] This is a conceptual diagram (part 1) showing the display mode of the gaze point by the gaze estimation system according to the second embodiment. [Figure 5] This is a conceptual diagram (part 2) showing the display mode of the gaze point by the gaze estimation system according to the second embodiment. [Figure 6] This is a conceptual diagram (part 3) showing the display mode of the gaze point by the gaze estimation system according to the second embodiment. [Figure 7] This is a conceptual diagram (part 4) showing the display mode of the gaze point by the gaze estimation system according to the second embodiment. [Figure 8] This is a conceptual diagram (part 1) showing how the trajectory of the point of fixation is displayed by the gaze estimation system according to the second embodiment. [Figure 9] This is a conceptual diagram (part 2) showing how the trajectory of the point of fixation is displayed by the gaze estimation system according to the second embodiment. [Figure 10] This is a conceptual diagram (part 3) showing how the trajectory of the point of fixation is displayed by the gaze estimation system according to the second embodiment. [Figure 11] This is a block diagram showing the overall configuration of the gaze estimation system according to the third embodiment. [Figure 12] This flowchart shows the operation flow of the gaze estimation system according to the third embodiment. [Figure 13] This is a block diagram showing the overall configuration of the gaze estimation system according to the fourth embodiment. [Figure 14] This is a flowchart showing the operation flow of the gaze estimation system according to the fourth embodiment. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of a gaze estimation system, a gaze estimation method, and a computer program will be described while referring to the drawings.

[0011] <First Embodiment> The gaze estimation system according to the first embodiment will be described with reference to FIGS. 1 to 3.

[0012] (System Configuration) First, while referring to FIG. 1, the overall configuration of the gaze estimation system according to the first embodiment will be described. FIG. 1 is a block diagram showing the overall configuration of the gaze estimation system according to the first embodiment.

[0013] As shown in FIG. 1, the gaze estimation system 10 according to the first embodiment includes a display control unit 101, a motion detection unit 102, and a following determination unit 110 as functional blocks for realizing its functions.

[0014] The display control unit 101 is configured to be able to display a fixation point on a display device having, for example, a display. The display control unit 101 controls the display so that the fixation point moves along a predetermined movement path. More specifically, the display control unit 101 controls each parameter related to the behavior and display mode of the fixation point, such as the movement path, movement speed, size, color, etc. of the fixation point. A specific display example of the fixation point will be described in detail later.

[0015] The motion detection unit 102 estimates the movement of the eyes of the subject (i.e., the person who is gazing at the fixation point) from an image of the subject. The motion detection unit 102 may acquire an image of the subject, for example, from a camera or the like installed around the display device on which the fixation part is displayed. The motion detection unit 102 may, for example, detect the face region of the subject from the image of the subject and detect the movement of the eyes from the image of the face region. Further, the motion detection unit 102 may estimate the gaze of the subject (for example, which position of the display unit 20 the subject is gazing at) from the movement of the eyes in the image of the subject. Note that since an existing technique can be appropriately adopted for a more specific method of detecting the movement of the eyes, detailed description here is omitted.

[0016] The tracking determination unit 110 is configured to determine whether the subject's eyes are tracking the point of focus, based on the relationship between the movement of the point of focus controlled by the display control unit 101 and the eye movement detected by the motion detection unit 102. For example, the tracking determination unit 110 should determine that the subject's eyes are tracking the point of focus if the subject's eyes move in a manner that follows the movement of the point of focus. However, there is a certain amount of time delay (a discrepancy due to a delay in reaction) between the movement of the point of focus and the movement of the subject. Therefore, the tracking determination unit 110 may take such time delay into consideration when determining tracking. Also, if there is no discrepancy at all between the movement of the point of focus and the movement of the subject's eyes, the unit may determine that the subject's eyes are not tracking the point of focus (for example, that some kind of misconduct is occurring).

[0017] (Hardware configuration) Next, the hardware configuration of the viewpoint position estimation device 10 according to the first embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing the hardware configuration of the viewpoint position estimation device according to the first embodiment.

[0018] As shown in Figure 2, the gaze estimation system 10 according to the first embodiment includes a CPU (Central Processing Unit) 11, RAM (Random Access Memory) 12, ROM (Read Only Memory) 13, and a storage device 14. The gaze estimation system 10 may further include an input device 15 and an output device 16. The CPU 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected via a data bus 17. The gaze estimation system 10 may also include multiple CPUs 11, RAM 12, ROM 13, storage devices 14, input devices 15, and output devices 16.

[0019] The CPU 11 reads a computer program. For example, the CPU 11 is configured to read a computer program stored in at least one of the RAM 12, ROM 13, and storage device 14. Alternatively, the CPU 11 may read a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The CPU 11 may also obtain (i.e., read) a computer program from a device (not shown) located outside the gaze estimation system 10 via a network interface. By executing the read computer program, the CPU 11 controls the RAM 12, storage device 14, input device 15, and output device 16. In this embodiment in particular, when the CPU 11 executes the read computer program, a functional block is realized within the CPU 11 for controlling the display of the gaze point, estimating the eye movements of the subject, and determining tracking (see Figure 1).

[0020] RAM12 temporarily stores computer programs executed by CPU11. RAM12 also temporarily stores data that CPU11 uses temporarily while executing computer programs. RAM12 may be, for example, D-RAM (Dynamic RAM).

[0021] ROM13 stores the computer program executed by CPU11. ROM13 may also store other static data. ROM13 may be, for example, a P-ROM (Programmable ROM).

[0022] The storage device 14 stores data that the gaze estimation system 10 stores long-term. The storage device 14 may also operate as a temporary storage device for the CPU 11. The storage device 14 may include, for example, at least one of a hard disk drive, a magneto-optical disk drive, an SSD (Solid State Drive), and a disk array device.

[0023] The input device 15 is a device that receives input instructions from the user of the gaze estimation system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.

[0024] The output device 16 is a device that outputs information related to the gaze estimation system 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) capable of displaying information related to the gaze estimation system 10.

[0025] (Flow of operations) Next, the operation flow of the viewpoint position estimation device 10 according to the first embodiment will be described with reference to Figure 3. Figure 3 is a flowchart showing the operation flow of the viewpoint position estimation device according to the first embodiment.

[0026] As shown in Figure 3, when the viewpoint position estimation device 10 according to the first embodiment is in operation, the display control unit 101 first starts display control of the point of gaze (step S11). Note that display control of the point of gaze may be started, for example, by the subject's terminal operation, or it may be started automatically when the presence of the subject is detected in the vicinity of the display device, etc., where the point of gaze is displayed.

[0027] When control of the gaze point display is initiated, the motion detection unit 102 acquires an image of the subject (step S12). Then, the motion detection unit 102 detects the subject's eye movements from the acquired image of the subject (step S13).

[0028] Next, the tracking determination unit 110 determines whether the subject's eyes are tracking the point of focus, based on the relationship between the movement of the point of focus controlled by the display control unit 101 and the eye movements detected by the motion detection unit 102 (step S14).

[0029] (Technical effects) Next, an example of the technical effects obtained by the viewpoint position estimation device 10 according to the first embodiment will be described.

[0030] As explained in Figures 1 to 3, the viewpoint position estimation device 10 according to the first embodiment can detect the movement of the subject's eyes by having the subject fixate on a point of fixation. In particular, since the point of fixation is controlled to move, it is possible to suppress fluctuations in the subject's gaze compared to, for example, having the subject fixate on a stationary point of fixation.

[0031] In this embodiment, the detected eye movements determine whether the subject is tracking the point of fixation. Therefore, it is possible to determine whether the subject is properly fixating on the moving point of fixation. This determination result can be used not only to determine whether the eye movements can be properly detected, but also for processes such as correcting the subject's viewpoint position or detecting impersonation, as described in the second and third embodiments later.

[0032] <Second Embodiment> The gaze estimation system according to the second embodiment will be described with reference to Figures 4 to 10. The second embodiment differs from the first embodiment described above only in some operations (mainly operations related to the display of the stopping point); otherwise, it is generally the same. Therefore, the differences from the first embodiment will be described in detail below, and other overlapping parts will be omitted as appropriate.

[0033] (System Configuration) The configuration of the gaze estimation system according to the second embodiment may be the same as that of the gaze estimation system according to the first embodiment (see Figure 1), so its description will be omitted. Similarly, the hardware configuration of the gaze estimation system according to the second embodiment may be the same as that of the hardware configuration of the gaze position estimation system 10 according to the first embodiment (see Figure 2), so its description will be omitted.

[0034] (Flow of operations) The operation flow of the gaze estimation system 10 according to the second embodiment may be the same as the operation flow of the gaze estimation system 10 according to the first embodiment (see Figure 3), so its explanation will be omitted. However, in the gaze estimation system 10 according to the second embodiment, the display control unit 101 controls the display mode of the point of fixation or the display mode of the trajectory of the point of fixation as follows.

[0035] (Display method of the point of focus) The display method of the gaze point in the gaze estimation system 10 according to the second embodiment will be explained with reference to Figures 4 to 7. Figure 4 is a conceptual diagram (1) showing the display method of the gaze point by the gaze estimation system according to the second embodiment. Figure 5 is a conceptual diagram (2) showing the display method of the gaze point by the gaze estimation system according to the second embodiment. Figure 6 is a conceptual diagram (3) showing the display method of the gaze point by the gaze estimation system according to the second embodiment. Figure 7 is a conceptual diagram (4) showing the display method of the gaze point by the gaze estimation system according to the second embodiment.

[0036] As shown in Figure 4, the point of focus X moves along a predetermined trajectory on the display surface of the display unit 20, alternating between moving and stopping. More specifically, the point of focus X moves along a predetermined straight line, stops at the end of that line for a predetermined period (for example, a few seconds), then moves along another straight line, and stops at the end of that line for a predetermined period, repeating this operation. In this way, it is easy to make the subject focus on the point of focus X. It is preferable that the point of focus X moves so as to pass through all the points on the display surface of the display unit 20. Here, the shape of the point of focus X is, for example, an ellipse, and the position of the point of focus X is, for example, the center position of the point of focus X. However, the shape and position of the point of focus X are not limited to these.

[0037] The subject is instructed to visually follow the movement of the fixation point X. To get the subject to follow the fixation point X, specific instructions may be output to the subject before the display control of the fixation point X begins. For example, instructions such as text to follow the fixation point X may be displayed on the display unit 20. Alternatively, the display control may be performed in a way that allows the subject to naturally follow the fixation point X with their eyes. For example, the fixation point X may be displayed as a character or object that attracts the subject's interest.

[0038] The movement speed and size of the fixation point X are pre-set. However, the movement speed of the fixation point X may be appropriately changed in response to the subject's reaction. For example, for subjects with a relatively large delay between the movement of their gaze and the movement of the fixation point, the movement of fixation point X may be slowed down. Alternatively, the size of fixation point X may be appropriately changed in response to the subject's reaction. For example, for subjects with a relatively large variation in their viewpoint position, the size of fixation point X may be increased. Furthermore, both the movement speed and size of fixation point X may be appropriately changed in response to the subject's reaction. Control to change the movement speed of fixation point X, or control to change the size of fixation point X, should be executed using the results immediately after the display control of fixation point X begins (for example, the results measured when moving along the first straight line).

[0039] As shown in Figure 5, the point of focus X may be controlled to flash (in other words, be highlighted) at predetermined intervals while moving. This makes it easy to get the subject to fixate on the point of focus X. Also, because the point of focus X flashes at regular intervals, the subject can more easily predict the movement of the point of focus X. Therefore, it becomes easy to reduce the time delay of the parallax (i.e., the delay in the movement of the line of sight relative to the movement of the point of focus) to a constant value. In addition, in the display configuration shown in Figure 5, as explained in Figure 4, the state in which the point of focus X moves and the state in which it remains stationary may be alternately repeated.

[0040] As shown in Figure 6, the point of focus X may be controlled to alternate between being visible (X1 in the figure) and invisible (X2 in the figure) while moving. Even in this case, it is easy to get the subject to fixate on the point of focus X. Furthermore, by periodically alternating between being visible and invisible, the subject can more easily predict the movement of the point of focus X. Therefore, it becomes easier to reduce the time delay of the parallax to a constant value. Note that, even in the display configuration shown in Figure 6, the point of focus X may alternate between being in a moving state and being stationary, as explained in Figure 4.

[0041] As shown in Figure 7, the point of focus X may be controlled to alternate between being displayed larger while moving (X3 in the figure) and being displayed smaller (X4 in the figure). Even in this case, it is easy to get the subject to focus on the point of focus X. Furthermore, because the size of the point of focus X changes periodically, the subject can more easily predict the movement of the point of focus X. Therefore, it becomes easy to reduce the time delay of the parallax to a constant value. Note that even in the display configuration shown in Figure 7, the point of focus X may alternate between being in a moving state and being stationary, as explained in Figure 4.

[0042] In addition to the point of focus X, a number for counting may also be displayed. This number may, for example, count up from a preset initial value to an upper limit, or count down from a preset initial value to a lower limit. The number may be counted over time (for example, every second). It may also be counted during the period from when the point of focus X starts moving until it stops, or each time it flashes, or it may be counted when the position of the point of focus X and the position of the eye are continuously within a predetermined distance.

[0043] (Technical effects of the display method of the point of focus) Next, an example of the technical effects obtained by the viewpoint position estimation device 10 according to the display mode of the point of gaze of the second embodiment will be described.

[0044] According to the viewpoint position estimation device 10 of the second embodiment, the display mode of the point of focus is controlled as shown in Figures 4 to 7. Therefore, it becomes easier to get the subject to focus on the point of focus X. Note that the display modes shown in Figures 4 to 7 may be combined as appropriate. For example, the point of focus X, which moves by repeatedly moving and stopping along a predetermined trajectory as shown in Figure 4, may be controlled to flash at predetermined intervals during movement as shown in Figure 5.

[0045] (Display method of the trajectory of the point of focus) Next, the display modes of the trajectory of the gaze point X will be explained with reference to Figures 8 to 10. Figure 8 is a conceptual diagram (1) showing the display mode of the gaze point trajectory by the gaze estimation system according to the second embodiment. Figure 9 is a conceptual diagram (2) showing the display mode of the gaze point trajectory by the gaze estimation system according to the second embodiment. Figure 10 is a conceptual diagram (3) showing the display mode of the gaze point trajectory by the gaze estimation system according to the second embodiment.

[0046] As shown in Figure 8, the movement trajectory of the point of focus X may be displayed. In this way, the subject can see how the point of focus X has moved so far, or how it will move in the future. As a result, the subject will be able to predict the movement of the point of focus X more easily.

[0047] As shown in Figures 9 and 10, the movement trajectory of the point of focus X may be displayed differently for movement trajectories that have already been moved and movement trajectories that are yet to be moved. Specifically, as shown in Figure 9, the movement trajectory that has already been moved may be displayed as a solid line, while the movement trajectory that is yet to be moved may be displayed as a dotted line. Alternatively, as shown in Figure 10, the movement trajectory that has already been moved may be displayed as usual, while the movement trajectory that is yet to be moved may not be displayed (i.e., only the movement trajectory that has already been moved may be displayed).

[0048] (Technical effects of the display method for the trajectory of the point of gaze) Next, an example of the technical effects obtained by the viewpoint position estimation device 10 according to the display method of the viewpoint trajectory of the point of gaze of the second embodiment will be described.

[0049] According to the viewpoint position estimation device 10 of the second embodiment, the display mode of the trajectory of the point of fixation is controlled as shown in Figures 8 to 10. Therefore, it becomes easier for the subject to predict the movement of the point of fixation X, making it easier to reduce the time delay of parallax to a certain value. An example of the technical effect of reducing the time delay of parallax to a certain value will be explained in detail in the third embodiment described later.

[0050] <Third Embodiment> The gaze estimation system according to the third embodiment will be described with reference to Figures 11 and 12. In the third embodiment, an example will be given in which the gaze estimation system functions as a system for correcting (calibrating) the viewpoint position. The third embodiment differs from the first and second embodiments described above only in some configurations and operations; otherwise, it is generally the same. Therefore, the parts that differ from the first and second embodiments will be described in detail below, and other overlapping parts will be omitted as appropriate.

[0051] (System Configuration) First, the overall configuration of the gaze estimation system according to the third embodiment will be described with reference to Figure 11. Figure 11 is a block diagram showing the overall configuration of the gaze estimation system according to the third embodiment. Note that the hardware configuration of the gaze estimation system according to the third embodiment may be the same as the hardware configuration of the gaze position estimation system 10 according to the first embodiment (see Figure 2), so its explanation will be omitted.

[0052] As shown in Figure 11, the gaze estimation system 10 according to the third embodiment is connected to a display unit 20 and an imaging unit 30. The display unit 20 is a display positioned where the subject whose gaze position is to be estimated can be seen, and it displays the point of fixation for estimating the gaze position. The display control of the point of fixation on the display unit 20 is performed by the gaze estimation system 10. The imaging unit 30 is a camera installed around the display unit 20, and it is positioned to capture an image of the subject (particularly an image of the area around the face). The image of the subject captured by the imaging unit 30 is output to the gaze estimation system 10.

[0053] The gaze estimation system 10 according to the third embodiment includes, in addition to the components of the first embodiment (see Figure 1), a disparity calculation unit 103, a statistical processing unit 104, a time delay calculation unit 105, a deviation amount calculation unit 106, a correction value integration unit 107, and a gaze correction output unit 108.

[0054] The parallax calculation unit 103 calculates the parallax, which is the difference between the position of the point of focus controlled by the display control unit 101 and the viewpoint position of the subject estimated by the motion detection unit 102. The parallax calculated by the parallax calculation unit 103 is calculated as a value that includes random statistical errors and time delays. The parallax calculation unit 103 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0055] The statistical processing unit 104 is configured to perform statistical processing to remove statistical errors included in the parallax calculated by the parallax calculation unit 103. Specifically, the statistical processing unit 104 removes statistical errors by averaging the parallax calculated by the parallax calculation unit 103 over a certain period of time. Note that the statistical processing described above is merely an example, and other statistical processing methods may be used to remove statistical errors. The statistical processing unit 104 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0056] The time delay calculation unit 105 is configured to calculate the time delay portion included in the parallax calculated by the parallax calculation unit 103. The time delay calculation unit 105 calculates the time delay portion of the parallax based at least on the parallax when the point of focus is moving. The time delay calculation unit 105 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0057] The deviation calculation unit 106 calculates the deviation between the viewpoint position estimated by the motion detection unit 102 and the true value of the viewpoint position (i.e., the position actually seen by the subject) based on the parallax from which statistical errors have been removed by the statistical processing unit 104 and the time delay calculated by the time delay calculation unit 105. The deviation calculation unit 106 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0058] The correction value integration unit 107 calculates a correction value (in other words, a correction amount to reduce the deviation of the viewpoint position) for the viewpoint position estimated by the motion detection unit 102, based on the deviation amount calculated by the deviation amount calculation unit 106. The correction value integration unit 107 also performs integration processing on the correction values ​​calculated at multiple locations in the display unit 20 and generates a correction formula for correcting the viewpoint position. Using this correction formula, it becomes possible to correct the viewpoint position even for points where a correction value has not actually been calculated. The correction value integration unit 107 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0059] The viewpoint correction output unit 108 outputs the correction formula generated by the correction value integration unit 107 as information for performing viewpoint position calibration. The viewpoint correction output unit 108 may also have a function to store the generated correction formula and correct and output the viewpoint position (i.e., a function to output the corrected viewpoint position). The viewpoint correction output unit 108 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0060] (Flow of operations) Next, the operation flow of the gaze estimation system 10 according to the third embodiment will be described with reference to Figure 12. Figure 12 is a flowchart showing the operation flow of the gaze estimation system according to the third embodiment.

[0061] As shown in Figure 12, when the gaze estimation system 10 according to the third embodiment is in operation, the display control unit 101 first starts display control of the gaze point on the display unit 20 (step S101). Note that display control of the gaze point may be started, for example, by the subject's terminal operation, or it may be started automatically when the presence of the subject is detected around the display unit 20.

[0062] When control of the gaze point display is initiated, the motion detection unit 102 acquires an image of the subject from the imaging unit 30 (step S102). Then, the motion detection unit 102 estimates the subject's viewpoint position from the acquired image of the subject (step S103).

[0063] Next, the parallax calculation unit 103 calculates the parallax, which is the difference between the position of the point of fixation and the estimated viewpoint position (step S104). Once the parallax is calculated, the statistical processing unit 104 performs statistical processing to remove statistical errors included in the parallax (step S105).

[0064] Subsequently, the time delay calculation unit 105 calculates the time delay occurring in the parallax (step S106). Then, the deviation amount calculation unit 106 calculates the deviation amount between the estimated viewpoint position and the actual viewpoint position based on the parallax and the time delay of the parallax from which statistical errors have been removed (step S107).

[0065] Once the deviation amount is calculated, it is determined whether the display control of the point of focus by the display control unit 101 has finished (step S108). That is, it is determined whether the point of focus has repeatedly moved and stopped along a predetermined path and has completed its movement to the end point. If it is determined that the display control has not finished (step S108: NO), the process is repeated again from step S102. As a result, the deviation amount is calculated at multiple points along the movement path of the point of focus.

[0066] If it is determined that the display control has ended (step S108: YES), the tracking determination unit 110 determines whether the subject's eyes were tracking the point of fixation (step S109). The tracking determination unit 110 may also determine whether the subject's eyes were tracking the point of fixation before the display control ends (i.e., while the point of fixation is displayed). For example, the tracking determination unit 110 may determine whether the subject's eyes were tracking the point of fixation each time the point of fixation X stops. If it is determined that the subject's eyes were not tracking the point of fixation (step S109: NO), the subsequent processing is omitted and the series of operations ends. This is because if the subject's eyes are not tracking the point of fixation, appropriate viewpoint position correction cannot be performed.

[0067] On the other hand, if it is determined that the subject's eyes were following the point of fixation (step S109: YES), the correction value integration unit 107 calculates multiple correction values ​​from each of the deviation amounts calculated at multiple locations and performs integration processing on the multiple correction values ​​(step S109). That is, it calculates a correction formula for the viewpoint position based on the multiple correction values. Then, the viewpoint correction output unit 108 outputs the correction formula generated by the correction value integration unit 107 as information for performing viewpoint position calibration (step S110).

[0068] (Correction of viewpoint position) Next, the correction of the viewpoint position by the gaze estimation system 10 according to the second embodiment will be described in detail. In the following, it will be assumed that the point of focus X is displayed in the manner shown in Figure 6, which was described in the second embodiment.

[0069] If the viewpoint position estimated by the motion detection unit 102 of the gaze estimation system 10 according to this embodiment is "Xgaze,est", the correction value is "ΔXcalib", and the statistical error is "ε", then the true value of the subject's viewpoint position "Xgaze,true" can be expressed by the following formula (1).

[0070] JPEG2026050495000002.jpg33170

[0071] Furthermore, if the parallax does not include statistical error, the above equation (1) can also be expressed as the following equation (2). In this case, it is possible to ignore "ε" in the following explanation and proceed with the calculation.

[0072] Furthermore, if we denote the position of the point of focus X as "Xc" and the time delay as "δXdelay", it can also be expressed as shown in equation (3) below.

[0073] JPEG2026050495000004.jpg29170

[0074] Furthermore, from equations (1) and (3) above, the parallax "Xgaze,est-Xc" calculated by the parallax calculation unit 103 can be expressed by the following equation (4).

[0075] JPEG2026050495000005.jpg28170

[0076] Here, the statistical error ε can be removed by the statistical processing performed by the statistical processing unit 104. As a result, it becomes possible to calculate the correction value ΔXcalib using Xgaze,est-Xc calculated by the disparity calculation unit 103 and the time delay δXdelay calculated by the time delay calculation unit 105.

[0077] The correction value ΔXcalib is calculated using parallax, at least when the fixation point X is moving. When the fixation point X is moving, the time delay δXdelay decreases after a sufficient amount of time has elapsed since the fixation point X began to move. Specifically, because the fixation point moves at a constant speed, the subject can predict the movement of the fixation point, thus reducing the time delay and approaching a constant value. This time delay can be estimated, for example, from the viewpoint position when the fixation point X stops, or the elapsed time until the viewpoint settles into that stopped position. Therefore, using the parallax when the fixation point X is moving allows for easier and more accurate calculation of the correction value ΔXcalib. The parallax when the fixation point is moving can be calculated, for example, at the midpoint of the line along which the fixation point X is moving.

[0078] The correction value ΔXcalib is calculated at multiple locations on the display unit 20. The correction value integration unit 107 then integrates the correction values ​​ΔXcalib calculated at multiple locations to generate a correction formula for the viewpoint position. The correction formula is generated as an equation such as equation (5) below, which includes predetermined coefficients A and b.

[0079] TIFF2026050495000006.tif42170

[0080] Furthermore, the correction value ΔXcalib may be calculated by combining not only the correction value calculated when the point of focus X is moving, but also the correction value calculated when the point of focus X is stationary.

[0081] Alternatively, the position Xc of the point of focus X may be the position on the circumference of the point of focus X that is closest to the viewpoint position Xgaze,est (i.e., the intersection of the line segment connecting the viewpoint position Xgaze,est and the center position of the point of focus X, and the circumference of the point of focus X).

[0082] In the example described above, we explained the case where disparity calculation, disparity statistical processing, disparity time delay calculation, and deviation amount calculation (i.e., the processes from steps S104 to S107 in Figure 5) are executed sequentially, but it is not necessary for all of these processes to be executed.

[0083] For example, the viewpoint position could be corrected by performing parallax calculations. This would allow for correction that reduces the effects of parallax.

[0084] Alternatively, the viewpoint position may be corrected by performing parallax statistical processing. This allows for correction that reduces the impact of statistical errors in parallax.

[0085] Alternatively, the viewpoint position may be corrected by calculating the parallax time delay. This allows for correction that reduces the impact of the parallax time delay.

[0086] Alternatively, the viewpoint position could be corrected by calculating the deviation amount. This would allow for correction that reduces the impact of the deviation amount in the viewpoint position.

[0087] Furthermore, the processes of disparity calculation, disparity statistical processing, disparity time delay calculation, and deviation amount calculation may be performed in combination of at least two of them.

[0088] (Technical effects) Next, an example of the technical effects obtained by the gaze estimation system 10 according to the third embodiment will be described.

[0089] As explained in Figures 11 and 12, the gaze estimation system 10 according to the third embodiment allows for the calculation of a correction value (in other words, a calibration value) to correct the estimated gaze position by having the subject gaze at the point of fixation X. Furthermore, in this embodiment, since the correction value is calculated using parallax when the point of fixation X is moving, the effect of the time delay δXdelay is reduced, and the correction value can be obtained over a wider range compared to when the point of fixation X is stationary. As a result, it becomes possible to accurately estimate where the subject is actually looking.

[0090] <Fourth Embodiment> Next, the gaze estimation system 10 according to the fourth embodiment will be described with reference to Figures 13 and 14. In the fourth embodiment, an example will be given in which the gaze estimation system 10 functions as a system for determining whether a person is impersonating another person. The third embodiment differs from the first to third embodiments already described only in some configurations and operations, and the other parts are generally the same. Therefore, the parts that differ from the first to third embodiments will be described in detail below, and the explanation of other overlapping parts will be omitted as appropriate.

[0091] (System Configuration) First, the overall configuration of the gaze estimation system 10 according to the fourth embodiment will be described with reference to Figure 13. Figure 13 is a block diagram showing the overall configuration of the gaze estimation system according to the fourth embodiment. Note that the hardware configuration of the gaze estimation system according to the fourth embodiment may be the same as the hardware configuration of the gaze position estimation system 10 according to the first embodiment (see Figure 2), so its explanation will be omitted.

[0092] As shown in Figure 13, the gaze estimation system 10 according to the fourth embodiment includes, in addition to the components of the gaze estimation system 10 according to the third embodiment (see Figure 1), a spoofing detection unit 201 and a determination result output unit 202. More precisely, the gaze estimation system 10 according to the fourth embodiment includes a spoofing detection unit 201 and a determination result output unit 202 in place of the correction value integration unit 107 and the viewpoint correction output unit 108 according to the third embodiment.

[0093] The impersonation detection unit 201 is configured to detect whether impersonation (i.e., fraudulent activity using video, etc.) is occurring based on the determination result of the tracking determination unit 110. For example, if the subject is actually present in front of the imaging unit 30, the deviation amount is calculated to be a value of a considerable magnitude due to the time delay. On the other hand, if the subject is not actually present in front of the imaging unit 30, and the video, etc., of the subject is pointed towards the imaging unit 30, there is no time delay specific to humans, and the deviation amount is calculated to be an extremely small value. Therefore, the tracking determination unit 110 can determine whether the subject is performing normal tracking of the gaze point X by comparing the calculated deviation amount with a predetermined threshold. Accordingly, the impersonation detection unit can detect whether impersonation is occurring based on the determination result of the tracking determination unit 110. Specifically, if the calculated deviation amount is greater than the predetermined threshold, the tracking determination unit 110 determines that the subject is performing normal tracking of the gaze point X. In this case, the impersonation detection unit 201 does not detect that impersonation is occurring. On the other hand, the tracking determination unit 110 determines that the subject is not tracking the gaze point X normally if the calculated deviation amount is smaller than a predetermined threshold. In this case, the impersonation detection unit 201 detects that impersonation is occurring. The impersonation detection unit 201 can be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0094] The judgment result output unit 202 is configured to output the judgment result of the impersonation detection unit 201. The judgment result output unit 202 may output only the result of whether or not impersonation is occurring, or it may perform a predetermined action (for example, an alert action) when impersonation is detected. The judgment result output unit 202 may be implemented, for example, in the CPU 11 (see Figure 1) as a functional block having the above-described function.

[0095] (Flow of operations) Next, the operation flow of the gaze estimation system 10 according to the fourth embodiment will be described with reference to Figure 14. Figure 14 is a flowchart showing the operation flow of the gaze estimation system according to the fourth embodiment.

[0096] As shown in Figure 14, when the gaze estimation system 10 according to the fourth embodiment is in operation, the processes from steps S101 to S108 are executed first, similar to the first embodiment. That is, the subject is asked to follow the gaze point X with their eyes, and the amount of deviation between the estimated gaze position and the true value is calculated from the image captured at that time.

[0097] In the fourth embodiment, in particular, when it is determined that the display control of the gaze point has ended (step S108: YES), the tracking determination unit 110 determines whether or not the subject's eyes were tracking the gaze point (step S109). If it is determined that the subject's eyes were not tracking the gaze point (step S109: NO), the impersonation detection unit 201 detects that impersonation is occurring (step S201). On the other hand, if it is determined that the subject's eyes were tracking the gaze point (step S109: YES), the impersonation detection unit 201 does not detect that impersonation is occurring (step S202). The determination result output unit 202 then outputs the determination result from the impersonation determination unit 201 (step S202).

[0098] In the fourth embodiment, the impersonation detection unit 201 may detect impersonation before the display control of the gaze point X is completed (in other words, the tracking determination unit 110 may perform a determination before the display control of the gaze point X is completed). For example, the tracking determination unit 110 may determine that tracking is not occurring when it has determined that the calculated value of one deviation corresponds to the value of impersonation, without waiting for the movement of the gaze point X to be completed, and at that timing, the impersonation detection unit 201 may detect that impersonation is occurring.

[0099] (Technical effects) Next, an example of the technical effects obtained by the gaze estimation system 10 according to the fourth embodiment will be described.

[0100] As explained in Figures 13 and 14, the gaze estimation system 10 according to the fourth embodiment can determine and detect whether or not impersonation is occurring based on the deviation amount calculated by the deviation amount calculation unit 106. In other words, it can appropriately determine and detect impersonation by utilizing the difference between the movement of an actual human's viewpoint and the movement of a mechanical viewpoint. The gaze estimation system 10 according to the fourth embodiment is particularly useful when performing authentication processing that utilizes viewpoint position.

[0101] In the gaze estimation system 10 according to the fourth embodiment, it is preferable not to display the trajectory of the point of fixation. Alternatively, as shown in Figure 10, it is preferable to display the movement trajectory of the movement that has already been made, while not displaying the movement trajectory of the movement that will be made in the future.

[0102] <Note> The embodiments described above may also be described in the following appendix, but are not limited to these.

[0103] (Note 1) The gaze position estimation system described in Appendix 1 is a gaze estimation system characterized by comprising: display control means for displaying the point of fixation that a subject is looking at in a predetermined manner of movement; detection means for detecting the movement of the subject's eyes from the subject's image; and tracking determination means for determining whether or not the subject's eyes are following the point of fixation based on the relationship between the movement of the point of fixation and the movement of the eyes.

[0104] (Note 2) The gaze estimation system described in Appendix 2 is characterized in that the display control means displays the gaze point in at least one of the following display modes: a first mode in which the gaze point is moved such that a first period in which the gaze point moves with a predetermined acceleration and a second period in which the gaze point is stopped for a predetermined period are alternately repeated; a second mode in which the gaze point is highlighted at regular intervals; a third mode in which the gaze point is alternately displayed and hidden at regular intervals; and a fourth mode in which the size of the gaze point is changed while it is moving.

[0105] (Note 3) The gaze estimation system described in Appendix 3 is a gaze estimation system described in Appendix 1 or 2, characterized in that the display control means displays the movement trajectory of the gaze point that the gaze point has passed through and the trajectory that it has not passed through in different display modes, or displays at least one of the movement trajectory of the gaze point that the gaze point has passed through and the trajectory that it has not passed through.

[0106] (Note 4) The gaze estimation system described in Appendix 4 is a gaze estimation system according to any one of Appendix 1 to 3, characterized in that it includes a correction means for correcting the detected parameters relating to the eye movement of the subject, depending on whether or not the subject's eyes are following the point of fixation.

[0107] (Note 5) The gaze estimation system described in Appendix 5 is characterized in that the correction means calculates a parallax, which is the difference between the position of the point of fixation and the viewpoint position of the subject, from the detected eye movement of the subject; calculates the time delay of the parallax calculated while the point of fixation is moving; calculates the deviation of the estimated viewpoint position from the true value based on the parallax and the time delay of the parallax; and determines a correction value to correct the viewpoint position of the subject based on the deviation.

[0108] (Note 6) The gaze estimation system described in Appendix 6 includes a correction means which calculates the deviation amount at multiple locations in the region where the gaze point moves, and integrates multiple correction values ​​determined based on the deviation amounts calculated at the multiple locations to correct the gaze position. The gaze estimation system described in Appendix 5 is characterized by the above features.

[0109] (Note 7) The gaze estimation system described in Appendix 7 is the gaze estimation system according to any one of claims 1 to 6, further comprising a spoofing detection means for detecting impersonation of the subject when the subject's eyes are not following the point of fixation.

[0110] (Note 8) The gaze estimation system described in Appendix 8 is characterized in that the impersonation detection means calculates a parallax, which is the difference between the position of the point of fixation and the viewpoint position of the target person, from the eye movements of the detected target person; calculates the time delay of the parallax calculated while the point of fixation is moving; calculates the deviation amount from the true value of the estimated viewpoint position based on the parallax and the time delay amount of the parallax; and detects impersonation of the target person based on the deviation amount.

[0111] (Note 9) The gaze estimation device described in Appendix 8 is a gaze estimation system described in Appendix 7 or 8, characterized in that the display control means does not display the movement trajectory of the point of focus when the impersonation detection means detects impersonation.

[0112] (Note 10) The gaze estimation method described in Appendix 10 is characterized by displaying the point of fixation that the subject is looking at in a predetermined manner, detecting the movement of the subject's eyes from the image of the subject, and determining whether or not the subject's eyes are following the point of fixation based on the relationship between the movement of the point of fixation and the movement of the eyes.

[0113] (Note 11) The computer program described in Appendix 11 is characterized by displaying the point of focus that the subject is looking at in a predetermined manner, detecting the subject's eye movements from the subject's image, and operating the computer to determine whether or not the subject's eyes are following the point of focus based on the relationship between the movement of the point of focus and the eye movements.

[0114] This disclosure may be modified as appropriate, without contradicting the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and such modifications to gaze estimation systems, gaze estimation methods, and computer programs are also included in the technical idea of ​​this disclosure. [Explanation of Symbols]

[0115] 10. Eye-tracking estimation system 20 Display section 30 Imaging Unit 101 Display Control Unit 102 Motion detection unit 103 Parallax calculation unit 104 Statistical Processing Section 105 Time Delay Calculation Unit 106 Discrepancy Amount Calculation Unit 107 Correction Value Integration Unit 108 Viewpoint Correction Output Unit 110 Tracking determination unit 201 Impersonation detection unit 202 Judgment Result Output Unit

Claims

1. A display control means that displays the point of focus that the subject is looking at in such a way that it moves repeatedly in a first period of time when it moves at a predetermined speed, and in a second period when it is stopped for a predetermined time, Estimation means for estimating the viewpoint position of the subject from the image of the subject, A means for calculating the amount of discrepancy between the estimated viewpoint position and the actual viewpoint position, An impersonation detection means that detects when the discrepancy is small that the subject is being impersonated, and does not detect when the discrepancy is large, A system for detecting impersonation.

2. The system further includes tracking determination means that determines that the subject's eyes are not tracking the point of gaze if the amount of deviation is less than a predetermined threshold, and determines that the subject's eyes are tracking the point of gaze if the amount of deviation is greater than a predetermined threshold. The impersonation detection system according to claim 1.

3. The aforementioned impersonation detection means is If it is determined that the subject's eyes are not following the point of focus, it is detected that the subject is being impersonated. If it is determined that the subject's eyes are following the point of focus, the system will not detect that the subject is being impersonated. The impersonation detection system according to claim 2.

4. The parallax, which is the difference between the position of the point of fixation and the estimated viewpoint position, is calculated. From the parallax calculated while the point of focus is moving, the time delay portion of the parallax is calculated. The system further includes correction means for correcting the viewpoint position based on the parallax and the time delay of the parallax. The impersonation detection system according to claim 1 or 2.

5. The display control means displays the movement trajectory of the point of focus, specifically the trajectory through which the point of focus passed and the trajectory through which it did not pass, in different display modes. The impersonation detection system according to claim 1 or 2.

6. The display control means displays the point of focus in at least one of the following display modes: a first mode in which the point of focus moves in a first period of time at a predetermined speed and a second period in which the point of focus is stopped for a predetermined period, a second mode in which the point of focus is highlighted at regular intervals, a third mode in which the point of focus is displayed and hidden alternately at regular intervals, and a fourth mode in which the size of the point of focus is changed while it is moving. The impersonation detection system according to claim 1.

7. At least one computer, The object of focus that the subject is looking at is displayed in such a way that it moves repeatedly in a first period where it moves at a predetermined speed, and a second period where the object of focus is stopped for a predetermined period. The viewpoint position of the subject is estimated from the image of the subject. The difference between the estimated viewpoint position and the actual viewpoint position is calculated, When the discrepancy is small, it is detected that the subject is impersonating someone else, and when the discrepancy is large, it is not detected that the subject is impersonating someone else. Methods for detecting impersonation.

8. The object of focus that the subject is looking at is displayed in such a way that it moves repeatedly in a first period where it moves at a predetermined speed, and a second period where the object of focus is stopped for a predetermined period. The viewpoint position of the subject is estimated from the image of the subject. The difference between the estimated viewpoint position and the actual viewpoint position is calculated, When the discrepancy is small, it is detected that the subject is impersonating someone else, and when the discrepancy is large, it is not detected that the subject is impersonating someone else. A computer program that makes a computer work in a certain way.

Images