Display device, control method for display device, program

Abstract

To provide a display device that enables high-precision automatic IPD adjustment. [Solution] A display device that can be attached to the user's head and has a display unit for displaying images comprises: an image acquisition means for acquiring an eyeball image taken of the user's eye; a detection means for detecting at least one feature point on the outer circumference of the eyeball based on the eyeball image; and a position adjustment means for moving the display unit based on the detected at least one feature point.

Description

Technical Field

[0001] The present invention relates to a display device wearable on a user's head, a control method for the display device, and a program.

Background Art

[0002] Some head-mounted displays (HMDs) have a mechanism that automatically adjusts the position of the optical axes of the left and right display displays and the viewing optical system for observing the display displays according to the user's interpupillary distance. These adjustments are called so-called automatic IPD (Interpupillary Distance) adjustments.

[0003] Patent Document 1 describes a method of recording information on adjustment records related to the characteristics of a user and the distance between the pupils of the user, and driving a display display to a position corresponding to the corresponding adjustment record when the user's characteristic information is detected.

Prior Art Documents

[0007] One aspect of the present invention is, A display device that can be worn on the user's head and has a display unit for displaying images, Image acquisition means for acquiring an eyeball image of the user's eye, A detection means for detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment means for moving the display unit based on the at least one feature point detected, This is a device characterized by having [a certain feature].

[0008] One aspect of the present invention is, A control method for a display device that can be worn on a user's head and has a display unit for displaying images, Image acquisition step: Obtain an eyeball image by capturing the user's eye, A detection step of detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment step of moving the display unit based on the at least one feature point detected, This is a control method for a display device characterized by having [a certain feature]. [Effects of the Invention]

[0009] According to the present invention, a display device capable of achieving automatic IPD adjustment with high precision can be provided. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram illustrating the HMD according to Embodiment 1. [Figure 2] This is an external view of the HMD body according to Embodiment 1. [Figure 3] This is a diagram illustrating the eyepiece according to Embodiment 1. [Figure 4] This diagram illustrates the position of the display unit and the eyeball according to Embodiment 1. [Figure 5]It is a diagram for explaining the position of the display unit and the eyeball according to Embodiment 1. [Figure 6] It is a diagram for explaining the position of the display unit and the eyeball according to Embodiment 1. [Figure 7] It is a flowchart of the processing of the HMD according to Embodiment 1. [Figure 8] It is a diagram showing the structure of the eyeball of the human body according to Embodiment 1. [Figure 9] It is a diagram for explaining the method of calculating the driving amount according to Embodiment 1. [Figure 10] It is a flowchart of the processing of the HMD according to Embodiment 2. [Figure 11] It is a diagram showing the positional relationship between the center line of the image and the pupil center according to Embodiment 2.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described in detail based on the accompanying drawings.

[0012] <Embodiment 1> Hereinafter, the HMD1 according to Embodiment 1 will be described. FIG. 1A is an external view of the HMD1 which is a display device wearable on the user's head. The HMD1 has an HMD main body 101 (main body part), a connecting part 102, and a wearing part 103. The HMD main body 101 is connected via the connecting part 102 to the wearing part 103 which is worn around the user's head in a headband shape.

[0013] FIGS. 2A and 2B are external views of the HMD main body 101 as seen from the user side wearing the HMD1. The HMD main body 101 has a right display unit 104 which is a display unit for displaying an image for the right eye, and a left display unit 105 which is a display unit for displaying an image for the left eye. The horizontal position of the right display unit 104 is changeable. On the other hand, when the center line of the HMD main body 101 is defined by the broken line in FIGS. 2A and 2B, the left display unit 105 is configured symmetrically with the right display unit 104 based on this center line. The horizontal position of the left display unit 105 is changeable in the same manner as the right display unit 104.

[0014] FIG. 2A is a diagram showing the HMD main body 101 in a state where the right display unit 104 and the left display unit 105 are moved (driven) to the outermost position. FIG. 2B is a diagram showing the HMD main body 101 in a state where the right display unit 104 and the left display unit 105 are moved (driven) to the innermost position.

[0015] FIG. 2C shows a configuration diagram of the right display unit 104 and the left display unit 105. The right display unit 104 has an eyepiece optical system and a display panel 108. The right display unit 104 can be moved (driven) left and right (in the horizontal direction) by a stepping motor 109. The left display unit 105 has an eyepiece optical system and a display panel 110. The left display unit 105 can be moved (driven) left and right by a stepping motor 111. By controlling the stepping motors 109 and 111 (position adjustment units) respectively, the positions of the right display unit 104 and the left display unit 105 can be controlled independently. Also, the right display unit 104 and the left display unit 105 can be adjusted in position not only by driving by the stepping motors 109 and 110 but also manually by the user himself / herself.

[0016] Here, let the distance from the center line of the HMD main body 101 to the optical axis center of the right display unit 104 be denoted as Xr, and the distance from the center line of the HMD main body 101 to the optical axis center of the left display unit 105 be denoted as Xl. Then, the ranges of the distance Xr and the distance Xl can be defined as the ranges between the minimum value X_min and the maximum value X_max as follows, respectively. X_min≦Xr≦X_max X_min≦Xl≦X_max

[0017] Figure 3 is a detailed view of the eyepieces of the right-side display unit 104 and the left-side display unit 105. As shown in Figure 3, in the right-side display unit 104 and the left-side display unit 105, a camera 107 for observing the eyeball is positioned directly below the optical axis of the display optical system. The camera 107 is an image acquisition unit that acquires eyeball images of the user's eyeball. In automatic IPD adjustment, the drive target position of each display unit is determined based on the eyeball images acquired by the camera 107.

[0018] Using Figures 4 to 6B, we will explain in detail how to determine the drive target position of the display unit based on the eyeball image captured by the camera 107.

[0019] Figure 4 shows an example of the position of the display units (right display unit 104 and left display unit 105) and the eyeballs. Figure 4 is a top view of the HMD body 101 and shows the positional relationship between the display units (right display unit 104 and left display unit 105) and the eyeballs. The left eyeball is located on the optical axis of the left display unit 105. On the other hand, the right eyeball is shifted inward by a distance L from the optical axis of the right display unit 104.

[0020] When the user is wearing the HMD1, the distance from the display unit to the eyeball is normally the standard distance Z0. Camera 107 can capture an image within a predetermined imaging angle that is wider than the size of the eyeball. Furthermore, camera 107 is positioned directly below the optical axis of the display optical system, and when viewed from above, the imaging optical axis coincides with the optical axis of the display unit. In automatic IPD adjustment, the distance L is determined based on the eyeball image, and the stepping motor 109 drives the right display unit 104 by distance L. This makes it possible to achieve a state where the eyeball is positioned on the optical axis of each display unit. The method for determining distance L will be described later.

[0021] The standard distance Z0 is determined by the design values ​​of the components of the HMD1, including the mounting part 103 that is worn around the user's head like a headband. However, the distance between the display unit and the eyeball (hereinafter referred to as the "imaging distance") may differ from the standard distance Z0 depending on the shape of the user's head.

[0022] Figure 5 shows an example of the positional relationship between the display unit and the eyeball when the imaging distance differs from the standard distance Z0. In Figure 5, the imaging distance is longer than the standard distance Z0 by ΔZ. As in Figure 4, the position of the left eyeball is on the optical axis of the left display unit 105. On the other hand, the position of the right eyeball is shifted inward by a distance L from the optical axis of the right display unit 104.

[0023] Figure 6A shows the right eyeball in an eyeball image. In Figure 6A, eyeball region 601 represents the right eyeball captured from a position at a standard distance Z0, and is shifted to the left by a distance of X pixels from the center line of the eyeball image. On the other hand, eyeball region 602 represents the right eyeball captured from a position at a standard distance Z0+ΔZ, and is shifted from the center line by a distance of X1 pixels, which is shorter than the distance X. As the imaging distance increases, the imaging magnification decreases, so eyeball region 602 is smaller than eyeball region 601.

[0024] Figure 6B shows the position of the right-side display unit 104 and the eyeballs as viewed from the side of the HMD body 101. Yes. The camera 107 is tilted so that the imaging optical axis and the display optical axis intersect at a position a standard distance Z0 away from the right-side display unit 104. Therefore, in Figure 6A, when the eyeball position is shifted by ΔZ, the eyeball region 602 is shifted downward compared to the eyeball region 601.

[0025] If K is the imaging magnification at the standard distance Z0, K1 is the imaging magnification at the distance Z0+ΔZ, and P is the pixel pitch, then the following equations 1 and 2 hold true. L=X×P×K=...(Formula 1) L=X1×P×K1=...(Formula 2)

[0026] Therefore, if ΔZ is large compared to the standard distance Z0, the imaging magnification must be calculated according to the distance from the camera 107 to the eyeball. However, in Embodiment 1, it is assumed that ΔZ is sufficiently small, and the imaging magnification is K even if the distance between the display unit and the eyeball is distance Z0 + ΔZ.

[0027] To determine the distance Z+ΔZ to the eyeball, for example, multiple LEDs are placed around the left and right display optical systems in Figure 3, and these LEDs emit light onto the eyeball. When the multiple LEDs are lit, camera 107 observes multiple Purkinje images formed on the eyeball. The distance Z+ΔZ can then be calculated based on the distances between these multiple Purkinje images. Since this technique is a known technique, a detailed explanation is omitted here.

[0028] The processing of the HMD1 according to Embodiment 1 will be explained with reference to the flowchart in Figure 7. In the following, as shown in Figure 1B, the control unit 120 (processor, etc.) included in the HMD body 101 executes the processing of each step according to the program stored in the memory unit 121. Note that the processing in this flowchart starts when the power of the HMD1 is turned ON.

[0029] In step S701, the control unit 120 moves the right display unit 104 and the left display unit 105 to symmetrical positions (initial positions) with respect to the center line of the HMD1. For example, the control unit 120 moves the right display unit 104 and the left display unit 105 to a position where Xr=Xl=31mm. This allows the right display unit 104 and the left display unit 105 to be positioned appropriately for the average user, as the average IPD of an adult is approximately 62mm. Furthermore, the right display unit 104 and the left display unit 105 are positioned symmetrically with respect to the center line of the HMD1 (arranged symmetrically). This encourages the user to align their midline with the center line of the HMD1 when wearing the HMD1.

[0030] In step S702, the control unit 120 controls the camera 107 to acquire an eyeball image by capturing images of the user's right and left eyeballs and surrounding areas.

[0031] In step S703, the control unit 120 performs image processing on the eyeball image to detect feature points on the outer periphery of the eyeball in the eyeball image. Figure 8 shows the structure of the human eyeball. When the user is wearing the HMD1, it is expected that at least one of the feature points of the lower eyelid, upper eyelid, lacrimal caruncle, and outer canthus will be captured in the eyeball image. Therefore, the control unit 120 identifies the positions of these feature points by image processing.

[0032] In step S704, the control unit 120 determines whether the lower eyelid, upper eyelid, lacrimal caruncle, and outer canthus have been detected. If it is determined that all of these feature points have been detected, the process proceeds to step S706. If it is determined that none of these feature points have been detected, the process proceeds to step S705.

[0033] In step S705, the control unit 120 individually drives the right display unit 104 and the left display unit 105 to perform automatic IPD adjustment. For example, if the lacrimal caruncle is not detected in the eyeball image of the right eye, the control unit 120 moves the right display unit 104 nasally by a predetermined amount. Also, if the outer canthus is not detected in the eyeball image of the right eye, the control unit 120 moves the right display unit 104 nasally by a predetermined amount in the opposite direction to the nose. The process in steps S702 to S705 is repeated until the lower eyelid, eyelid apex, lacrimal caruncle, and outer canthus can all be detected from both the eyeball image of the right eye and the eyeball image of the left eye.

[0034] In step S706, the control unit 120 calculates the drive amount for the right display unit 104 and the left display unit 105 based on the detected feature points.

[0035] Figure 9 is a diagram illustrating a method for calculating the drive amount based on feature points. Figure 9 shows an eyeball image of the right eye captured by the camera 107 of the right-side display unit 104. Here, the position of the lacrimal caruncle in the eyeball image is indicated by point A, the position of the outer canthus is indicated by point B, the position of the upper eyelid is indicated by point C, and the position of the lower eyelid is indicated by point D. In this case, the control unit 120 sets the intersection point of the line segment connecting point A and point B and the line segment connecting point C and point D as point E, and calculates the distance X from this point E to the center line of the eyeball image. Note that point E may be the point with the average coordinates of points A, B, C, and D.

[0036] For example, if the pixel pitch P is 5um and the imaging magnification K is 20x, and assuming the distance X is 20 pixels, the control unit 120 calculates the drive amount (distance L) of the right-side display unit 104 as 2mm according to Equation 1. Here, we have described an eyeball image captured from the right eye, but the drive amount can be calculated for an eyeball image captured from the left eye using a similar calculation.

[0037] In step S707, the control unit 120 individually drives the right display unit 104 and the left display unit 105 based on the drive amount determined in step S706. This allows the control unit 120 to perform automatic IPD adjustment.

[0038] In Embodiment 1, as described above, feature points around the eyeball are detected from the eyeball image captured by the camera 107. Then, automatic IPD adjustment is performed based on the positions of the detected feature points. Therefore, highly accurate automatic IPD adjustment is possible without being affected by the rotation of the eyeball.

[0039] In Embodiment 1, the lower eyelid, upper eyelid, lacrimal caruncle, and outer canthus are detected as characteristic points around the eyeball, but this is not limited to these, and other characteristic points in the eye (such as the left eyelid and right eyelid) may also be detected.

[0040] Furthermore, in the flowchart of Figure 4, the display unit is driven (moved) and image is captured repeatedly until these four feature points are detected. However, if one or more of these four feature points are detected, the process may proceed to step S706. For example, the control unit 120 may determine the drive amount by setting the midpoint of the line segment connecting points A and B as point E (= reference point for calculating distance X), or by setting the midpoint of the line segment connecting points C and D as point E. Alternatively, the control unit 120 may determine the drive amount by setting either point C or point D as point E.

[0041] Furthermore, sunlight or illumination light may appear as ghosts in the eyeball image, making it impossible to detect feature points such as the lower eyelid, upper eyelid, lacrimal caruncle, and outer canthus. For this reason, the control unit 120 may switch point E (method of calculating point E) in step S706 depending on whether all feature points are detected or whether any feature points are not detected.

[0042] <Embodiment 2> In Embodiment 1, the HMD1 performs automatic IPD adjustment based on the detection of feature points around the eyeball. In Embodiment 2, the HMD1 performs automatic IPD adjustment by referring not only to feature points around the eyeball but also to the pupil position. The appearance and configuration of the HMD1 in Embodiment 2 are the same as those of the HMD1 in Embodiment 1, so a further explanation is omitted below.

[0043] The processing of the HMD1 according to Embodiment 2 will be described with reference to the flowchart in Figure 10. Below, only the differences between the flowchart in Figure 10 and the flowchart in Figure 7 will be explained.

[0044] The process in step S1001 begins when the processes in steps S701 to S707 shown in Figure 7 are executed. In step S1001, the control unit 120 displays indicators (indicators showing the position that the user should focus on) on the display optical axes of the right display unit 104 and the left display unit 105. The control unit 120 also displays display items on the right display unit 104 and the left display unit 105 that prompt the user to focus on the indicators.

[0045] In step S1002, the control unit 120 controls the camera 107, similar to step S702, to image the user's eyeball and the area around it. Through this, the control unit 120 acquires an image of the eyeball.

[0046] In step S1003, the control unit 120 detects the pupil center by performing image processing on the eyeball image. The method for detecting the pupil center is a known technique. For example, in Patent Document 2, the edge coordinates of the pupil are detected based on the difference in brightness of the image captured by the camera. The pupil center can be determined by approximating the pupil to a perfect circle from the edge coordinates. Artificial Intelligence (AI) technology may also be used to determine the pupil and the pupil center.

[0047] In step S1004, the control unit 120 determines whether the pupil center is located near the center line of the eyeball image. If it is determined that the pupil center is located near the center line of the eyeball image, it is determined that the user is fixating on the indicator, and the process proceeds to step S1005. If it is determined that the pupil center is not located near the center line of the eyeball image, it is determined that the user is not fixating on the indicator, and the process in this flowchart ends. In other words, in this case, the automatic IPD adjustment based on the pupil center from step S1005 onward is skipped, and the series of operations ends.

[0048] Figure 11 shows an example of the positional relationship between the centerline of the eyeball image and the pupil center. Figure 11 shows the state after automatic IPD adjustment has been performed so that point E, which was determined based on the feature points around the eyeball, coincides with the centerline of the eyeball image. The pupil center detected in step S1003 is indicated by point F.

[0049] In Figure 11, dotted lines LA and LB are drawn to indicate positions at a distance ΔX from the center line. The control unit 120 then determines whether or not point F exists between dotted lines LA and LB. If point F exists between dotted lines LA and LB, it is determined that the pupil center is near the center line of the eyeball image (= the distance between the center line and the pupil center is less than or equal to distance ΔX). On the other hand, if point F does not exist between dotted lines LA and LB, it is determined that the pupil center is not near the center line of the eyeball image. In other words, it is determined that the user is looking at something other than the target, and that automatic IPD adjustment cannot be performed accurately due to the effect of eyeball rotation.

[0050] Note that the appropriate value for distance ΔX may vary depending on the individual. Therefore, distance ΔX may be user-configurable, or a predetermined value may be used.

[0051] In step S1005, the control unit 120 calculates the drive amount for the right display unit 104 and the left display unit 105 based on the pupil centers (point F) of the left and right pupils. It calculates the distance X from the coordinates of point F to the center line of the eyeball image. The drive amount (distance L) can be calculated based on the distance X according to Equation 1.

[0052] In step S1006, the control unit 120 performs automatic IPD adjustment by individually driving the right display unit 104 and the left display unit 105 based on the drive amount determined in step S1005. This completes the series of automatic IPD adjustment operations.

[0053] Immediately after putting on the HMD1, the position of the eyes and the position of the display unit may be significantly misaligned. In this state, even if indicators are displayed on the display unit, the eyeball is likely to be far from the display optical axis, and therefore, due to the influence of pupil rotation, automatic IPD adjustment may not be possible. For this reason, in Embodiment 2, the HMD1 first adjusts the eyeball to be positioned near the display optical axis based on characteristic points around the eyeball, and then performs automatic IPD adjustment based on the pupil center. This makes it possible to perform automatic IPD adjustment more appropriately.

[0054] If all feature points are detected in step S704, automatic IPD adjustment based on the pupil center may be initiated from step S1001 onwards.

[0055] <Example 1> Furthermore, the HMD1 may pre-display indicators on its display unit and switch the automatic IPD adjustment method depending on whether the user is fixating on the indicators (result of gaze detection). The process in Modification 1 is best performed when it can be assumed that there is little discrepancy between the position of the display unit and the position of the eyeball (for example, when the user using the HMD1 this time is the same as the user who used it last time, or after the user has manually adjusted their IPD).

[0056] Specifically, first, the control unit 120 displays an index in the same manner as in step S1001 and acquires an eyeball image in the same manner as in step S1002. Then, the control unit 120 detects feature points in the same manner as in step S703 and also detects the pupil center in the same manner as in step S1003.

[0057] Subsequently, the control unit 120 determines whether the user is gazing at the indicator based on the feature point and the pupil center. For example, the control unit 120 determines that the user is gazing at the indicator if the horizontal difference (lateral direction of the HMD1) between the coordinates of point E and the pupil center is within a predetermined range. The control unit 120 determines that the user is not gazing at the indicator if the horizontal difference between the coordinates of point E and the pupil center is not within a predetermined range.

[0058] If the control unit 120 determines that the user is fixating on the indicator, it performs automatic IPD adjustment based on the pupil center, similar to steps S1005 and S1006. When it is determined that the user is fixating on the indicator, the position where the eyeball is fixating can be accurately determined, so appropriate automatic IPD adjustment based on the pupil center can be achieved. On the other hand, if the control unit 120 determines that the user is not fixating on the indicator, it performs automatic IPD adjustment based on feature points, similar to steps S706 and S707. When it is determined that the user is not fixating on the indicator, the position where the eyeball is fixating cannot be accurately determined, so automatic IPD adjustment based on feature points can be achieved more appropriately than based on the pupil center.

[0059] Furthermore, in the above, "If A is greater than or equal to B, proceed to step S1, and if A is less than (lower than) B, proceed to step S2" should be changed to "If A is greater than (higher than) B, proceed to step S1 This can be rephrased as, "Proceed to step S1, and if A is less than or equal to B, proceed to step S2." Conversely, "Proceed to step S1 if A is greater than (higher than) B, and proceed to step S2 if A is less than or equal to B" can be rephrased as, "Proceed to step S1 if A is greater than or equal to B, and proceed to step S2 if A is less than (lower than) B." Therefore, as long as no contradiction arises, "greater than or equal to A" can be rephrased as "greater than (higher; longer; more) than A," and "less than or equal to A" can be rephrased as "less than (lower; shorter; fewer) than A." Furthermore, "greater than (higher; longer; more) than A" can be rephrased as "greater than or equal to A," and "less than (lower; shorter; fewer) than A" can be rephrased as "less than or equal to A."

[0060] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). Multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) may share the processing to control the entire device.

[0061] Furthermore, the above-mentioned processors are processors in a broad sense, including general-purpose processors and specialized processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Specialized processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0062] Furthermore, although embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0063] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit that implements one or more functions.

[0064] The above-disclosed embodiments include the following configurations, methods, and programs. (Composition 1) A display device that can be worn on the user's head and has a display unit for displaying images, Image acquisition means for acquiring an eyeball image of the user's eye, A detection means for detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment means for moving the display unit based on the at least one feature point detected, A display device characterized by having the following features. (Configuration 2) The aforementioned at least one feature point includes at least one of the lower eyelid, upper eyelid, left eyelid, right eyelid, lacrimal caruncle, and outer canthus. The display device according to configuration 1, characterized in that... (Composition 3) The position adjustment means moves the display unit based on a point determined from the coordinates of the detected at least one feature point. A display device according to configuration 1 or 2, characterized by the above. (Composition 4) The position adjustment means moves the display unit based on the average coordinates of the at least one detected feature point. The display device according to configuration 3, characterized by the above. (Composition 5) If the at least one feature point is not detectable by the detection means, the position adjustment means moves the display unit to a position where the at least one feature point can be detected by the detection means. The detection means detects the at least one feature point after the display unit has moved to a position where the detection means can detect the at least one feature point. A display device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The position adjustment means moves the display unit based on the detected at least one feature point, and then moves the display unit based on the center of the user's pupil. A display device according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The display unit displays an indicator to cause the user to focus on a specific location. A display device according to any one of configurations 1 to 6, characterized by the above. (Composition 8) 1) Detects the pupil center of the user from the eyeball image, and 2) has a gaze determination means that determines whether the user is gazing at the indicator based on the detected at least one feature point and the pupil center. The display device according to configuration 7, characterized by the features described above. (Composition 9) The aforementioned gaze determination means is If the difference in a specific direction between the coordinates of the point determined by the detected at least one feature point and the coordinates of the pupil center is within a predetermined range, it is determined that the user is gazing at the indicator. The display device according to configuration 8, characterized by the above. (Composition 10) The position adjustment means is If it is determined that the user is not paying attention to the indicator, the display unit is moved based on the detected at least one feature point. If it is determined that the user is fixating on the indicator, the display unit is moved based on the pupil center. The display device according to configuration 8 or 9, characterized by the above. (method) A control method for a display device that can be worn on a user's head and has a display unit for displaying images, Image acquisition step: Obtain an eyeball image by capturing the user's eye, A detection step of detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment step of moving the display unit based on the at least one feature point detected, A method for controlling a display device, characterized by having the following features. (program) A program for causing a computer to function as one of the display devices described in any of configurations 1 to 10. [Explanation of symbols]

[0065] 1:HMD (display device), 104: Right-side display unit, 105: Left-side display unit, 107: Camera, 109, 111: Stepping motor, 120: Control Unit

Claims

1. A display device that can be worn on the user's head and has a display unit for displaying images, Image acquisition means for acquiring an eyeball image of the user's eye, A detection means for detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment means for moving the display unit based on the at least one feature point detected, A display device characterized by having the following features.

2. The aforementioned at least one feature point includes at least one of the lower eyelid, upper eyelid, left eyelid, right eyelid, lacrimal caruncle, and outer canthus. The display device according to claim 1, characterized in that...

3. The position adjustment means moves the display unit based on a point determined from the coordinates of the detected at least one feature point. The display device according to feature 1.

4. The position adjustment means moves the display unit based on the average coordinates of the at least one detected feature point. The display device according to feature 3.

5. If the at least one feature point is not detectable by the detection means, the position adjustment means moves the display unit to a position where the at least one feature point can be detected by the detection means. The detection means detects the at least one feature point after the display unit has moved to a position where the detection means can detect the at least one feature point. The display device according to feature 1.

6. The position adjustment means moves the display unit based on the detected at least one feature point, and then moves the display unit based on the center of the user's pupil. The display device according to feature 1.

7. The display unit displays an indicator to cause the user to focus on a specific location. The display device according to feature 1.

8. 1) Detects the pupil center of the user from the eyeball image, and 2) has a gaze determination means that determines whether the user is gazing at the indicator based on the detected at least one feature point and the pupil center. The display device according to feature 7.

9. The aforementioned gaze determination means is If the difference in a specific direction between the coordinates of the point determined by the detected at least one feature point and the coordinates of the pupil center is within a predetermined range, it is determined that the user is gazing at the indicator. The display device according to feature 8.

10. The position adjustment means is If it is determined that the user is not paying attention to the indicator, the detected small At the very least, based on one feature point, the display unit is moved, If it is determined that the user is fixating on the indicator, the display unit is moved based on the pupil center. The display device according to feature 8.

11. A control method for a display device that can be worn on a user's head and has a display unit for displaying images, Image acquisition step: Obtain an eyeball image by capturing the user's eye, A detection step of detecting at least one feature point on the outer periphery of the eyeball based on the aforementioned eyeball image, A position adjustment step of moving the display unit based on the at least one feature point detected, A method for controlling a display device, characterized by having the following features.

12. A program for causing a computer to function as one of the means of a display device according to any one of claims 1 to 10.

Images