Viewpoint detection device, calibration method and program

Abstract

[Problem] To perform a calibration operation accurately and quickly. [Solution] This viewpoint detection device is provided with: a visual target presentation unit which presents a visual target to an eye and continuously moves the visual target according to a prescribed trajectory; a gaze point detection unit which detects the gaze point of the eye; and a numerical optimization problem solving unit which solves a numerical optimization problem in order to find a correction that transforms a first set of a plurality of detected gaze points into a second set that satisfies a specific condition with respect to the prescribed trajectory.

Description

[Technical Field]

[0001] The present invention relates to a viewpoint detection device, a calibration method, and a program. [Background technology]

[0002] Currently, head-mounted displays capable of providing 3D images are becoming increasingly popular in order to provide virtual reality (VR) in games and videos. A head-mounted display is an image providing device that provides a 3D image to a user inside a housing that covers the user's head. Some head-mounted displays are equipped with a gaze detector that detects the user's gaze direction, and are configured to change the 3D image by, for example, increasing the resolution of only the image in the gaze direction.

[0003] Such gaze detectors require a calibration operation to align the direction in which the user is actually looking with the direction detected by the gaze detector. Conventional devices present calibration targets at multiple positions and prompt the user to look at the targets. While the user is looking at the targets, the gaze detector detects the gaze direction. Calibration can be performed by calculating the difference between this detected gaze direction and the actual gaze direction (the direction of the targets) (see, for example, Patent Document 1).

[0004] However, when multiple targets are presented sequentially, the gaze detector may not obtain a detection signal due to blinking or noise, and in such cases the detection operation must be repeated, which takes a long time for the calibration operation and places a burden on the user.

[0005] [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2019-21049 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a viewpoint detection device, a calibration method, and a program that can perform a calibration operation accurately and quickly. [Means for solving the problem]

[0008] The gaze point detection device according to the present invention comprises a gaze point presenting unit that presents a gaze point to the eye and continuously moves the gaze point along a predetermined trajectory, a gaze point detection unit that detects the gaze point of the eye, and a numerical optimization problem solving unit that solves a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a viewpoint detection device, a calibration method, and a program that can perform a calibration operation accurately and quickly. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram schematically illustrating an overview of a video system 1 according to a first embodiment. [Figure 2] 2 is a perspective view schematically showing the configuration of a video display unit 140 according to the first embodiment. FIG. [Figure 3] 2 is a schematic cross-sectional view showing a configuration of an image display unit 140 according to the first embodiment. FIG. [Figure 4] 1 is a block diagram showing the configuration of a main part of a head-mounted display 100 and a video playback device 200 according to a first embodiment. [Figure 5] 3 is a conceptual diagram illustrating the operation of a target presentation unit 205 and a correction amount calculation unit 206 according to the first embodiment. FIG. [Figure 6]4 is a flowchart illustrating a procedure for performing a calibration operation in the video system 1 of the first embodiment. [Figure 7] 4 is a flowchart illustrating a procedure for performing a calibration operation in the video system 1 of the first embodiment. [Figure 8] 3 is a schematic diagram illustrating a procedure for performing a calibration operation in the video system 1 of the first embodiment. FIG. [Figure 9] FIG. 10 is a schematic diagram illustrating a comparative example. [Figure 10] FIG. 10 is a schematic diagram illustrating a procedure for performing a calibration operation in the video system 1 according to the second embodiment. [Figure 11] FIG. 10 is a schematic diagram illustrating a procedure for performing a calibration operation in the video system 1 according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be designated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

[0012] Although the present embodiment has been described in sufficient detail to enable those skilled in the art to implement the present disclosure, it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0013] The contents of the embodiment of the present invention will be listed and described below. One embodiment of the present invention has the following configuration. [Item 1] a visual target presenting unit that presents a visual target to the eyes and continuously moves the visual target along a predetermined trajectory; a gaze point detection unit for detecting a gaze point of the eye; and a numerical optimization problem solver that solves a numerical optimization problem to find a correction that transforms a first set of the detected multiple gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory. [Item 2] Item 1. The gaze point detection device according to item 1, wherein the numerical optimization problem solver finds the correction that transforms the first set of detected gaze points from the predetermined trajectory to the second set of nearest neighbors. [Item 3] 2. The gaze point detection device according to item 1, wherein the numerical optimization problem solver uses a segment of the predetermined trajectory for each gaze point in the first set of detected gaze points. [Item 4] 3. The viewpoint detection device according to item 1 or 2, wherein the numerical optimization problem solver uses latency information as a constraint in optimization. [Item 5] 4. The viewpoint detection device according to item 3, wherein the segments of the predetermined trajectory are selected based on latency information. [Item 6] presenting a visual target to the eyes and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; The computer performs the calibration method. [Item 7] presenting a visual target to the eyes and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; A program that causes a computer to execute the following.

[0013] [First embodiment] A video system 1 according to a first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram schematically illustrating an overview of the video system 1. The video system 1 according to the first embodiment includes a head-mounted display 100 and a video playback device 200. As shown in Fig. 1, the head-mounted display 100 may be a shielded type head-mounted display that is worn on the head of a user.

[0014] The video playback device 200 generates image data to be displayed on the head mounted display 100 and transmits the image data to the head mounted display 100 via wired or wireless communication. As an example, the video playback device 200 is a device that can play images from a personal computer, a stationary game console, a portable game console, a tablet terminal, a smartphone, a tablet, a phablet, a video player, a television, etc.

[0015] The wireless connection between the video playback device 200 and the head mounted display 100 can be realized using known wireless communication technologies such as Wi-Fi (registered trademark), Bluetooth (registered trademark), and infrared communication. Furthermore, image transmission between the head mounted display 100 and the video playback device 200 is performed in accordance with standards such as Miracast (trademark), WiGig (trademark), WHDI (trademark), or Ethernet (trademark). The video playback device 200 may be configured integrally with the head mounted display 100 or may be built into the head mounted display 100.

[0016] The head mounted display 100 includes a housing 110, a head fixing unit 120, headphones 130, and an image display unit 140. As will be described later, the housing 110 houses a transmission module and various sensors in addition to the image display unit 140. The head fixing unit 120 is a member that causes the head mounted display 100 to be worn on the user's head.

[0018] The headphones 130 output the sound of the image reproduced by the video playback device 200. Although not shown, the head mounted display 100 may include a gyro sensor or the like for detecting the tilt direction of the head mounted display 100. The video playback device 200 can change the display state of the image according to the detection result of this gyro sensor. The video display unit 140 is housed in the housing 110 and displays the video transmitted from the video playback device 200.

[0019] Fig. 2 is a perspective view showing a schematic configuration of the video display unit 140, and Fig. 3 is a schematic cross-sectional view thereof. Fig. 4 is a block diagram showing the configuration of the main parts of the head-mounted display 100 and the video playback device 200.

[0020] As shown in Fig. 2, the image display unit 140 presents an image to each of the user's left and right eyes E (EL, ER), thereby making it possible to present a stereoscopic image to the user. As shown in Fig. 3, the image display unit 140 includes an infrared light source 141 (141a-c), an objective lens 142, a half mirror 143, a camera 144, a hot mirror 145, and a display 146. Note that the configuration of the image display unit 140 is the same for both the left and right eyes E, and therefore only the configuration of one side will be described below.

[0021] The infrared light source 141 is a light source used to detect the positions of the center of the pupil of the eye, the center of the cornea, and further the center of the eyeball, and is used to detect the line of sight of the eye E, and is a light source capable of emitting light in the near-infrared wavelength band (approximately 780 nm to 2500 nm). At least three infrared light sources 141 are provided. Here, as an example, it is assumed that three infrared light sources 141a to 141c are provided (infrared light source 141c is not shown in FIG. 3). Note that, hereinafter, the infrared light sources 141a to 141c may be collectively referred to as "infrared light source 141."

[0022] The objective lens 142 is disposed so as to be positioned in front of the user's eye E when the head mounted display 100 is in use. The infrared light sources 141a to 141c can be disposed around the objective lens 142. Here, the three infrared light sources 141a to 141c are disposed at positions that are not at least line-symmetrical with respect to the optical axis of the objective lens 142, and are preferably disposed at approximately equal angles around the optical axis.

[0023] A half mirror 143 and a hot mirror 145 are provided in the optical path between the objective lens 142 and the display 146. The half mirror 143 transmits a portion of the light from the eye E and reflects the remainder. A camera 144 is provided in the optical path on the reflection side of the half mirror 143, and the hot mirror 145 is provided in the optical path on the transmission side.

[0024] Camera 144 is an imaging device that captures an image of eye E on which light from infrared light source 141 is projected and a bright spot is formed based on the light. The image of the eye including this bright spot is used to detect the positions of the center of the eye's pupil and the center of the cornea, and ultimately the direction of gaze. Display 146 is a video display device for displaying images transmitted from video playback device 200, and can be realized using, for example, a known liquid crystal display or organic EL display. Hot mirror 145 has the property of transmitting visible light emitted by display 146 while reflecting near-infrared light.

[0025] Infrared light emitted from infrared light sources 141a to 141c and reaching user's eye E is reflected by eye E and travels toward objective lens 142. A portion of the light that passes through objective lens 142 is reflected by half mirror 143 and enters camera 144. Camera 144 may be equipped with a filter (not shown) that blocks visible light.

[0026] 4, the head mounted display 100 includes a control unit 101 and a communication unit 102 in addition to an infrared light source 141, a camera 144, and a display 146. The control unit 101 controls the entire head mounted display 100 including the infrared light source 141, the camera 144, and the display 146. The communication unit 102 also controls data communication with the video playback device 200.

[0027] The video playback device 200 also includes a control unit 201 (processor) for controlling the detection unit 203 and the video generation unit 204, and a communication unit 202 for managing data communication with the head mounted display 100. The communication unit 202 communicates with the communication unit 102 in the head mounted display 100, receives various data obtained by the head mounted display 100 and supplies it to the control unit 201, and conversely, transmits data obtained by the video playback device 200 to the head mounted display 100. The video playback device 200 also includes the detection unit 203 and the video generation unit 204.

[0028] The detection unit 203 detects the positions of the pupil center, the cornea center, and the eyeball center of the user's eye based on an image of the eye E captured by the camera 144 with infrared light from the infrared light sources 141a to 141c projected onto the eye E. The detection unit 203 then detects the gaze direction of the eye E based on data on the positions of the pupil center, the cornea center, and / or the eyeball center. The image generation unit 204 generates various images to be displayed on the display 146 and transmits them to the head mounted display 100 via the communication unit 202.

[0029] The video playback device 200 also includes a visual target presentation unit 205 and a correction amount calculation unit 206 as components for executing the calibration operation in the detection unit 203. The visual target presentation unit 205 displays a visual target M on the display 146 for the user to gaze upon during the calibration operation. As shown in FIG. 5, the visual target presentation unit 205 is configured to continuously move the visual target M along a predetermined trajectory TR. The movement may be continuous, and the speed of the movement may be constant or variable. The shape of the trajectory TR is also not important as long as the calibration operation can be performed. In the example of FIG. 5, the trajectory TR is an open curve having a start point Ps and an end point Pe, but it may also be a closed curve. As an example, the curve may include a parabola, a conic section, a spiral, a Lissajous curve, a cycloid curve, a clothoid curve, a straight line, or a combination thereof.

[0030] In the calibration operation, the visual target M is continuously moved along a trajectory TR on the display 146 by the visual target presenting unit 205, while the camera 144 captures an image of the eye E illuminated by the infrared light sources 141a-c. The detection unit 203 then analyzes the positions of the bright spots formed on the cornea based on the infrared light sources 141a-c using the image captured by the camera 144, and identifies the position of the corneal center according to the analysis results. The detection unit 203 also identifies the center position of the pupil according to the image captured by the camera 144, and further identifies the center position of the eyeball. Based on the data thus identified, the detection unit 20 performs gaze detection.

[0031] When a calibration operation is performed by having a user gaze at a target M that moves along a trajectory TR, the gaze direction SL2 detected by the detection unit 203 ideally coincides with the trajectory TR. However, due to individual differences between users, misalignment of the housing 110 relative to the user's head, assembly errors, and other reasons, the gaze direction SL2 detected by the detection unit 203 may not coincide with the direction SL1 (ground truth) toward the trajectory TR. The point P where the screen of the display 146 and the gaze direction SL2 intersect i The detection unit 203 determines that the person is looking at point P iIt may happen that does not match the position of the target M. Note that the index t represents the timestamp below, so all Pi above is replaced with Pt.

[0032] Therefore, in the video system 1 of the first embodiment, a calibration operation is performed, and the difference between the line of sight direction SL2 detected by the detection unit 203 and the direction SL1 (the direction of the target M) that the user is actually looking in is calculated in the correction amount calculation unit 206. Then, the correction amount calculation unit 206 calculates the amount of correction according to the calculation result. In this example, the point P i The correction amount is calculated by calculating the positional relationship, that is, the distance ΔD, between the position of the target M and the corresponding position of the target M.

[0033] Detected point P i The position of the target M corresponding to point P i The point on the trajectory TR that is closest to the point P can be detected. i The distance between the target M and the point P is calculated as the distance ΔD. The amount of correction can be calculated by acquiring data on this distance ΔD multiple times while continuously moving the target M. i The nearest point of can be found by solving the following equation: Note that an optimization algorithm can be used to minimize the total distance between the calibrated point Pt and the trajectory TR. The optimization algorithm can be local or global, and can be, but is not limited to, steepest descent or least squares.

[0034] [Formula 1] TIFF0007770031000001.tif1357

[0035] However, →T(→e(t), →c) is the point P iis a calibration function for converting the position vector →e(t) of the trajectory TR (Curve) into the position vector of a point on the trajectory TR (Curve). Also, →c is a calibration parameter →c used in the calibration function. Furthermore, Dist is a function that indicates the distance between the point determined by the position vector →e and the converted point. The distance is expressed as L 2 The calculation can be performed using the norm or other similar methods. E indicates the moment at which a sufficient number of bright points can be obtained to perform gaze detection, thereby providing reliable data for gaze detection. According to the above formula, the point P i The nearest position on the trajectory TR can be identified from [Equation 2]. The cost function for the optimization with respect to the parameter c is defined. The result of the minimization process is fin This can be expressed as [Equation 2] in the case of the above cost function. [Formula 2] TIFF0007770031000002.tif1383 Once calibration is complete, fin can be used for gaze correction in the following way: [Formula 3] TIFF0007770031000003.tif833[Equation 3] is the c calculated by [Equation 2] fin Transform the uncalibrated gaze output x to x* by applying the calibration transformation T using , which can increase the accuracy of the eye-tracking system.

[0036] The procedure for performing the calibration operation in the video system 1 of the first embodiment will be described with reference to the flowchart in Fig. 6. When the calibration operation starts, first, the target presentation unit 205 displays the target M on the display 146 and starts an operation of continuously moving the target M along the trajectory TR (step S11).

[0037] The detection unit 203 detects the line of sight SL2 of the eye E (point P) while the user is gazing at the target M moving along the trajectory TR. i) is detected (step S12). When the set of points Pi is detected, the points P i The position of the target M on the trajectory TR corresponding to c is determined by the initial value c ini t (S13). Usually, the initial value satisfies the following equation for any x: [Formula 4] Next, we begin the iterative optimization. As shown in S14, each estimated gaze Pt is corrected by a calibration transformation T(Pt, c), which can be expressed as [Equation 5]. [Formula 5] TIFF0007770031000005.tif731 Next, in S15, for each Pt*, the closest point on the trajectory TR can be found. In the next step S16, a cost function needs to be calculated. In this embodiment, the cost function is the sum of the distances Dist(Pt*, [TR]). Then, in S17, the optimizer updates the parameters based on the resulting cost value. In S18, a stopping condition is checked, and S14 to S18 are repeated as necessary. The stopping condition in this case may be, but is not limited to, a running time, a target cost value to be achieved, etc. When the optimization process is complete, the final value of c is c fin In general, depending on the choice of optimization algorithm, it may be necessary to perform the cost function calculation (S14-S16) several times with different parameters c to perform the iterations of optimization S17. As an example, for point P i By identifying the closest point from the target to the trajectory TR, the position of the corresponding target M can be identified. Once the position of the corresponding target M is identified, the position of the target M and the point P i As shown in FIG. 8, the procedure of steps S11 to S14 is executed multiple times while the target M moves along the trajectory TR, and multiple points P i (P1, P2, . . ., P m ), and multiple distances ΔD (ΔD1, ΔD2, . . . , ΔD m ) data is acquired. m) the correction amount can be calculated.

[0038] The calibration operation in the comparative example will be described with reference to Fig. 9. In this comparative example, visual targets Mi (i = 1, 2, ...) are sequentially and discretely (discretely) presented at a plurality of positions spaced apart on the display 146. The plurality of visual targets M i Each time one of the images is projected, infrared light is emitted from the infrared light sources 141a to 141c, and the gaze direction is detected by the detection unit 203 according to information on the positions of the bright spots of the infrared light sources 141a to 141c, the positions of the pupils, and / or the positions of the centers of the eyeballs captured by the camera 144.

[0039] In this comparative example, all the targets M i However, in reality, due to blinking, noise, etc., the calibration may not be accurate. i If the number of failures increases, the amount of data required to perform proper calibration will be insufficient, and the calibration operation will need to be repeated, which will increase the burden on the user.

[0040] In contrast to this, according to the method of the first embodiment shown in Fig. 5, the visual target M is not presented discretely but is presented continuously along the trajectory TR. If the visual target M moves continuously along the trajectory TR and the eye E follows the movement of the visual target M, even if blinking or noise occurs discontinuously along the way, the detection unit 203 can detect multiple points P i The information of multiple points P i In each of these, the position of the corresponding target M can also be identified. Therefore, according to the method shown in Fig. 5 and Fig. 6, the calibration operation can be performed accurately and quickly.

[0041] [Second embodiment] Next, a video system 1 according to a second embodiment will be described with reference to Fig. 10. The overall configuration of the video system 1 is similar to that of the first embodiment (Figs. 1 to 4), so a duplicated description will be omitted. The calibration operation is basically similar to the procedures shown in Figs. 5 and 6.

[0042] However, the calibration operation of the second embodiment is performed at point P i The method of identifying the position of the target M corresponding to the target (details of step S13 in FIGS. 6 and 7) differs from that of the first embodiment. Specifically, in the first embodiment, the function Dist(x, Tr) was the distance from point x to the point on the locus TR that is closest to point x. In the second embodiment, the fragment FragTr of the locus TR is t is specified for each individual gaze output Pt. The overall process is the same as in the first embodiment except for block S15. In the second embodiment of S15, Mt is calculated by dividing the trajectory FragTr t As shown in Figure 10, the trajectory TR is divided into multiple fragments CF, and the detected point P i In the fragment CF nearest to i The position of the target M corresponding to is identified.

[0043] The procedure for performing the calibration operation in the second embodiment, particularly the point P i A method for specifying the position of the target M corresponding to the target M will be described with reference to FIGS. 7 and 10. First, as in the first embodiment, an operation of continuously moving the target M along the trajectory TR is started (step S11). Then, the detection unit 203 detects the line of sight SL2 of the eye E (point P) while the user is gazing at the target M moving along the trajectory TR. i ) is detected (step S12).

[0044] point P i Once the position vector e(t) is detected, this point P i The nearest fragment CF (→g(t-f1):→g(t+f2)) is identified, and the point Pi The nearest points from are identified based on the following formula: All points Pt are measured and the curve FragTr t Once each fragment of is associated with each Pt(S14b), an optimization problem is solved. For example, the optimization problem can be formulated as the following equation:

[0045] [Formula 6] TIFF0007770031000006.tif1394

[0046] Note that instead of [Equation 6], the nearest neighbor point may be calculated by further multiplying by a weighting factor w(t). t Furthermore, to make the algorithm robust to outliers, an additional weighting factor wt can be applied to each distance at each iteration. Thus, wt may be updated at each iteration.

[0048] According to the video system 1 of the second embodiment, in the calibration operation, the point P i The correspondence between the trajectory fragments FragTr and the corresponding points on the trajectory TR can be calculated based on the fragments obtained by dividing the trajectory TR. Therefore, the calculation load can be reduced compared to the first embodiment, and the calibration operation can be performed more quickly and accurately. Furthermore, the second embodiment does not use the trajectory fragment FragTr in the cost function of optimization. t This differs from the first embodiment by introducing the following: This increases the chance that the local optimization algorithm will find the correct global minimum. In addition, it reduces the computational load, allowing the calibration operation to be performed more quickly and accurately.

[0049] [Third embodiment] Next, a video system 1 according to a third embodiment will be described. The overall configuration of the video system 1 is the same as that of the first embodiment (FIGS. 1 to 4), so duplicated explanations will be omitted. The calibration operation is basically the same as the procedures shown in FIGS. 5, 6, and 7.

[0050] However, the calibration operation of the third embodiment is performed at point P i The method of specifying the position of the target M corresponding to the target (details of step S13 in FIGS. 6 and 7) differs from that of the first embodiment. t Specifically, in the third embodiment, after the visual target M is presented, the point P i Delay time information T until L (latency) is grasped in advance, and according to this delay time information ΔT, the point PiFragTr t Identify the position of the target M corresponding to

[0051] The procedure for performing the calibration operation in the third embodiment, particularly the point P i A method for identifying the position of the target M corresponding to the above will be described with reference to FIGS. First, as in the first embodiment, an operation of continuously moving the target M along the trajectory TR is started (step S11). After that, the detection unit 203 detects the gaze direction SL2 (point Pt) of the eye E while the user is gazing at the target M moving along the trajectory TR (step S12). At this point, a timestamp t can be defined as the moment when the detection unit 203 detects the gaze Pt. Therefore, the trajectory TR can be expressed as a trajectory parameterized by time. The movement of the target M is controlled according to a predetermined time table (FIG. 11). For example, at the start of movement (elapsed time Telp=0), the target M moves from a starting point Ps (coordinates (X s , Y s ) and at the elapsed time Tel = Te, the target M reaches the end point Pe (coordinates (X e , Y e)). The position (coordinate data) of the target M at each elapsed time Telp is also stored in advance in the time table. In addition, information on the delay time ΔT from when the target M is presented at a certain position on the trajectory TR until the point Pi is identified is also stored in advance.

[0052] The detection unit 203 detects the line of sight SL2 of the eye E (point P) while the user is gazing at the target M moving along the trajectory TR. i ) is detected (step S12). At a certain elapsed time, for example, at elapsed time Telp=Tj, the point P i (coordinates (Xp j , Yp j Once the point P i The position of the target M corresponding to point P is identified according to the time table and the delay time information ΔT (step S13). i The positional relationship between the fragment FragTr and the fragment FragTr, that is, the distance ΔD, is calculated in the same manner as in the first embodiment (step S14). t can be specified relative to time t (when the measurement of Pt is completed) and depends on the estimated latency of the system TL. [Formula 7] TIFF0007770031000007.tif7113 In this embodiment, fragment FragTr t is shifted by a time TL from the time Pt ​​was measured. The length of the fragment is 2ΔTL.

[0053] In this way, in the third embodiment, the point P identified by the detection unit 203 i The position of the target M corresponding to the target is specified according to the delay time information ΔT. This method can also achieve the same effect as the previous embodiment. Since the delay time information ΔT is calculated and stored in advance, the target M can be specified by simply referring to a time table, so the calculation load for calculating the correction amount is smaller than in the previous embodiment. As described above, in the third embodiment, the FragTr tWe propose that is related to the latency of the system TL. The system latency may be measured automatically or configured externally. The system latency may include technical latency (e.g., caused by data transmission, processing time, etc.) and physiological latency (the time required for a person to catch up with the position of the target M).

[0054] [others] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. For example, in the above-described embodiment, an example in which the gaze point detection device is installed in a video system having a head-mounted display 100 has been mainly described, but the scope of application of the gaze point detection device of the present invention is not limited thereto and can be applied to various devices and systems that can use a gaze detection function.

[0055] In the above-described embodiment, the processor in the gaze detection device executes a gaze detection program or the like to identify the user's gaze direction, but the gaze detection device may be realized by hardware such as a logic circuit or dedicated circuit formed on an integrated circuit. These circuits may be realized by one or more integrated circuits, and the multiple functional blocks shown in the above-described embodiment may be realized by a single integrated circuit.

[0056] In addition, the gaze detection program may be recorded on a processor-readable recording medium, and the recording medium may be a "non-transitory tangible medium," such as a tape, disk, card, semiconductor memory, or programmable logic circuit. [Explanation of symbols]

[0057] 1...Video system, 100...Head-mounted display 101...Control unit 102…Communications Department 110…Housing 120…Head fixation part 130...Headphones 140...Video display section 141, 141a~c...Infrared light source 142...Objective lens 143...Half mirror 144...Camera 145...Hot mirror 146...Display 200...Video playback device 201...Control unit (processor) 202…Communications Department 203...Detection unit 204...Image generation unit 205…Optotype presentation unit 206...Correction amount calculation section.

Claims

1. a visual target presenting unit that presents a visual target to the eyes and continuously moves the visual target along a predetermined trajectory; a gaze point detection unit for detecting a gaze point of the eye; a numerical optimization problem solver that solves a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; Equipped with The numerical optimization problem solver identifies, for each of the detected gaze points in the first set of gaze points, a nearest target position in the predetermined trajectory segment. A viewpoint detection device comprising:

2. A visual target presenting unit that presents a visual target to the eyes and continuously moves the visual target along a predetermined trajectory; a gaze point detection unit for detecting a gaze point of the eye; a numerical optimization problem solver that solves a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; Equipped with The viewpoint detection device, wherein the numerical optimization problem solver uses latency information as a constraint in the optimization.

3. A visual target presenting unit that presents a visual target to the eyes and continuously moves the visual target along a predetermined trajectory; a gaze point detection unit for detecting a gaze point of the eye; a numerical optimization problem solver that solves a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; Equipped with the numerical optimization problem solver uses the predetermined trajectory fragment for each of the gaze points in the first set of detected gaze points; The viewpoint detection device of claim 1 , wherein the segments of the predetermined trajectory are selected based on latency information.

4. presenting a visual target to the eyes and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; The computer executes In the step of solving the numerical optimization problem, the computer identifies, for each of the detected gaze points in the first set of gaze points, a position of a visual target in the predetermined trajectory segment that is closest to the gaze point. A calibration method characterized by:

5. A method of presenting a visual target to the eyes and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; The computer executes In solving the numerical optimization problem, the computer uses latency information as a constraint in the optimization. A calibration method characterized by:

6. presenting a visual target to the eyes and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; A program for causing a computer to execute the above, and in solving the numerical optimization problem, causing the computer to identify, for each of the detected gaze points in the first set of gaze points, a position of a visual target in the predetermined trajectory segment that is closest to the gaze point. A program characterized by.

7. A method of presenting a visual target to an eye and continuously moving the visual target along a predetermined trajectory; detecting a gaze point of the eye; solving a numerical optimization problem to find a correction that transforms a first set of the detected gaze points into a second set that satisfies certain conditions with respect to the predetermined trajectory; A program for causing a computer to execute the above, causing the computer to use latency information as a constraint in the optimization in solving the numerical optimization problem; A program characterized by.

Images