Electronic device, control method thereof, computer program product, and storage medium

Abstract

The invention discloses an electronic device, a control method thereof, a computer program product, and a storage medium. The electronic apparatus includes a first detection unit for detecting a line of sight of a left eye of a user viewing a first display unit, a second detection unit for detecting a line of sight of a right eye of a user viewing a second display unit, and a control unit for performing control in a case where a predetermined condition is satisfied, the first detection unit or the second detection unit is configured to change any one of the first detection unit or the second detection unit from a first state to a second state in which power consumption is less than that of the first state.

Description

Technical Field

[0001] This disclosure relates to an electronic device and its control method, a computer program product, and a storage medium. Background Technology

[0002] Recently, electronic devices that use information about the user's gaze as a user interface have been used in various fields; examples of such devices include head-mounted displays (HMDs). Some HMDs are capable of using virtual reality (VR) or augmented reality (AR).

[0003] Specifically, when a user wears an HMD and uses VR content, they can have an experience as if they have entered a virtual space (VR space) where virtual objects are displayed. Furthermore, when a user wears an HMD and uses AR content, they can add digital content that does not exist in reality (e.g., subjects such as characters) to real-world space (AR space), thus having an experience as if they perceive the existence of things that do not exist in the real world.

[0004] Japanese Patent Application Publication No. 2024-109785 describes a head-mounted display in which two eye-tracking devices are arranged to monitor the corresponding gaze directions of the user's left and right eyes.

[0005] Users can use the head-mounted display described in Japanese Patent Application Publication No. 2024-109785 to select virtual objects in VR space or subjects in AR space based on the corresponding gaze directions of the user's left and right eyes. In this case, the head-mounted display consumes a large amount of power because it uses two eye-tracking devices to monitor the corresponding gaze directions of the user's left and right eyes. Summary of the Invention

[0006] The purpose of this disclosure is to reduce the power consumption of electronic devices when performing gaze detection on a user.

[0007] According to one aspect of this disclosure, an electronic device includes a first detection unit for detecting the gaze of a user viewing a first display unit's left eye, a second detection unit for detecting the gaze of a user viewing a second display unit's right eye, and a control unit for performing control under predetermined conditions to change either the first detection unit or the second detection unit from a first state to a second state where power consumption is less than that of the first state.

[0008] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of embodiments is by way of example. Attached Figure Description

[0009] Figure 1A and Figure 1B This is a perspective view showing an example of the appearance of a head-mounted display (HMD) according to the first embodiment.

[0010] Figure 2 This is a block diagram illustrating a configuration example of an HMD according to the first embodiment.

[0011] Figure 3 This is a diagram illustrating the principle of gaze detection.

[0012] Figure 4A This is a schematic diagram of the eyeball image formed by a light-receiving lens. Figure 4B yes Figure 4A A schematic diagram of the brightness distribution in region α.

[0013] Figure 5 This is a flowchart regarding the gaze detection process according to the first embodiment.

[0014] Figure 6A and Figure 6B This is a diagram illustrating a method for calculating the gaze position obtained when a user looks at a subject with both eyes, according to a first embodiment.

[0015] Figure 7 This is a flowchart regarding the mode switching process according to the first embodiment.

[0016] Figure 8A and Figure 8B This is a diagram illustrating the timing of performing a predetermined processing operation based on line of sight in a corresponding mode according to the first embodiment.

[0017] Figure 9A and Figure 9B This is a flowchart regarding the mode switching process according to the second embodiment. Detailed Implementation

[0018] Various embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. In the following embodiments, a head-mounted display (HMD) is described as an example of an electronic device performing gaze detection. The video image viewed by a user through the HMD is referred to as an “external video image (three-dimensional (3D) space)”. As an example, a case is described where the HMD selects a subject based on the user’s gaze when capturing an external video image including a subject using a camera device included in the HMD. However, this disclosure can also be applied, for example, to cases where the HMD selects a virtual object in a virtual reality (VR) space based on the user’s gaze when worn on a user’s head, or to cases where the HMD selects a subject in an augmented reality (AR) space based on the user’s gaze when worn on a user’s head.

[0019] <First Embodiment>

[0020] Figure 1A and Figure 1B This is a perspective view showing an example of the appearance of the HMD 100 according to the first embodiment, wherein, Figure 1A This is a perspective view seen from the front of the HMD 100, and Figure 1B This is a perspective view of the HMD 100 from the rear. The HMD 100 is equipped with a headband 200. The user applies the HMD 100 to the user's eye area and secures the HMD 100 to the user's head with the headband 200.

[0021] The HMD 100 includes camera devices 105 (left camera 105a and right camera 105b). The left camera 105a is used to capture images of the left display screen, which is located at a position corresponding to the user's left eye. Figure 1A and Figure 1B A camera that captures external video images on a right display (not shown). The right camera device 105b is used to capture images to be displayed on the right display (located at a position corresponding to the user's right eye). Figure 1A and Figure 1B (Not shown in the image) External video image of the camera.

[0022] External video images captured by camera devices 105 (105a and 105b) are displayed on a monitor (display unit) and are visible through eyepiece units 102 (left eyepiece unit 102a and right eyepiece unit 102b), respectively. The external video image captured by the left camera device 105a is displayed on the left display, visible through the left eyepiece unit 102a located at a position corresponding to the user's left eye. The external video image captured by the right camera device 105b is displayed on the right display, visible through the right eyepiece unit 102b located at a position corresponding to the user's right eye.

[0023] By observing the left eyepiece unit 102a and the right eyepiece unit 102b with the user's left and right eyes respectively, the user can view the captured external video images displayed on the monitors located at the corresponding positions corresponding to the user's left and right eyes.

[0024] [Block diagram description of HMD 100]

[0025] Figure 2 This is a block diagram illustrating a configuration example of the HMD 100 according to the first embodiment.

[0026] The control unit 104 includes a central processing unit (CPU) that serves as a computing unit, as well as memories such as read-only memory (ROM) for storing programs that can be executed by the CPU and random access memory (RAM) for storing and retrieving various types of data, and controls the HMD 100. Furthermore, the control unit 104 is capable of performing controls to overlay virtual objects or graphical user interfaces (GUIs) such as pointers or menus onto the VR space, and to perform controls to add subjects such as characters to the AR space.

[0027] The display 103 (left display 103a and right display 103b), the camera device 105 (left camera device 105a and right camera device 105b), the gaze detection unit 106 (left gaze detection unit 106a and right gaze detection unit 106b), the motion detection unit 107, the operation unit 108, and the power supply 109 are connected to the control unit 104 via corresponding control lines.

[0028] Displays 103 (103a and 103b) display external video images captured by camera devices 105 (105a and 105b) on a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display, based on signals received from control unit 104. Specifically, the left display 103a displays the external video image captured by the left camera device 105a based on signals received from control unit 104, and the right display 103b displays the external video image captured by the right camera device 105b based on signals received from control unit 104.

[0029] Each camera device 105 (105a and 105b) is a camera that captures external video images and sends the captured external video images to the control unit 104. Specifically, the camera devices 105 send external video images captured by the left camera device 105a and the right camera device 105b to the control unit 104, respectively.

[0030] The gaze detection units 106 (106a and 106b) are units that detect the gaze of a user viewing the display 103. When a user is wearing the HMD 100 and viewing an external video image, the user views the external video image through the display 103 located inside the HMD 100. The gaze detected by the gaze detection units 106 is only the gaze of the user viewing the display 103, not the gaze of a user viewing a subject in the external video image from a position away from the display 103 as seen by the user. Specifically, the left gaze detection unit 106a detects the gaze of the user's left eye 101a as it views the left display 103a located inside the HMD 100 through the left eyepiece unit 102a. The right gaze detection unit 106b detects the gaze of the user's right eye 101b as it views the right display 103b located inside the HMD 100 through the right eyepiece unit 102b.

[0031] Thus, the HMD 100, including left gaze detection unit 106a and right gaze detection unit 106b corresponding to the user's left and right eyes respectively, enables accurate calculation of the user's gaze position when the user has viewed the subject in the external video image with both eyes, based on the detection results obtained from the respective gaze detection units 106. Furthermore, the HMD 100's ability to accurately calculate the user's gaze position when the user has viewed the subject in the external video image with both eyes also improves the accuracy of performing predetermined processing operations based on gaze. Specifically, even when the camera device 105 selects a subject image in the external video image based on the gaze and then captures an image, the camera device 105 can capture an image including the subject desired by the user. Furthermore, even when a pointer indicating the gaze position is displayed based on the gaze, the pointer can be displayed at the position the user is aiming at.

[0032] The following reference Figures 3 to 5 The internal configuration of each gaze detection unit 106 and the gaze detection processing operations performed by the gaze detection unit 106 are described. Information about the user's gaze (e.g., gaze position or gaze direction) detected by each gaze detection unit 106 is sent to the control unit 104.

[0033] The motion detection unit 107 detects, for example, the amount of rotation, direction of rotation, and posture of the HMD 100.

[0034] Motion information, such as the amount of rotation, direction of rotation, and posture detected by the motion detection unit 107, is sent to the control unit 104. The motion detection unit 107 is configured to detect the aforementioned motion information and is equipped with, for example, a gyroscope sensor for detecting the rotation of the HMD 100 or a geomagnetic sensor for detecting the posture of the HMD 100.

[0035] The operation unit 108 is a collective term for multiple user-operable input devices (e.g., buttons, switches, and dials). When an operation is detected on an input device, the control unit 104 performs a processing operation corresponding to the detected operation. Although the first embodiment employs a configuration where the HMD 100 includes the operation unit 108, external devices such as controllers wirelessly connected to the HMD 100 may include operation units capable of operating the HMD 100. In this case, when an operation is detected on an operation unit included in the external device, the control unit 104 performs a processing operation corresponding to the detected operation.

[0036] Under the control of the control unit 104, the power supply 109 provides the necessary power to control each of the blocks, including the display 103, camera device 105, gaze detection unit 106, motion detection unit 107, and operation unit 108. The power supply 109 is configured with a rechargeable and rechargeable battery. This battery may be replaceable. Although the HMD 100 is configured to include the power supply 109 in the first embodiment, the HMD 100 can be configured to receive power from an external device.

[0037] [Description of gaze detection processing]

[0038] Reference Figures 3 to 5 Describe the gaze detection process.

[0039] Figure 3 This diagram illustrates the principle of gaze detection. Illumination sources 13a and 13b are arranged approximately symmetrically with respect to the optical axis of the light-receiving lens 16, and illuminate infrared light towards the eyeball 14 of the user viewing the HMD 100. The light-receiving lens 16 forms an eye image on the imaging plane of the eye image sensor 17, generated from the infrared light reflected from the eyeball 14.

[0040] Figure 4A This is a schematic diagram of the eye image formed by the light-receiving lens 16. Figure 4B yes Figure 4A A schematic diagram of the brightness distribution in region α.

[0041] Each of the left gaze detection unit 106a and the right gaze detection unit 106b includes Figure 3 Each of the illumination sources 13a and 13b, the light receiving lens 16, and the eye image sensor 17 is shown, and the gaze detection process described below is performed for the user.

[0042] Figure 5This is a flowchart of the gaze detection process according to the first embodiment. The gaze detection process is performed by each of the left gaze detection unit 106a and the right gaze detection unit 106b, respectively corresponding to the user's left and right eyes. For example, the gaze detection process can be performed when an object (eye) is detected approaching the eyepiece unit 102. Known alternative methods (e.g., using a proximity sensor located near the eyepiece unit 102) can be used to detect whether an object (eye) is approaching the eyepiece unit 102. The gaze detection process can be initiated in response to a command issued by the user via the operation unit 108. Figure 5 The process shown in the flowchart is executed by the control unit 104 that controls each unit. Furthermore, in response to instructions received from the control unit 104, the process is repeated. Figure 5 The process is shown in the flowchart.

[0043] In step S501, the control unit 104 causes the illumination source drive circuit (not shown) to turn on the illumination sources 13a and 13b to emit light. This causes infrared light to be emitted from the illumination sources 13a and 13b toward the outside of the HMD 100. The infrared light is reflected from the user's eyeball in the viewing eyepiece unit 102 and then enters the light receiving lens 16.

[0044] In step S502, the control unit 104 causes the eye image sensor 17 to perform image capture. The eye image sensor 17 converts the eye image formed by the light receiving lens 16 into an image signal. The image signal is converted from analog to digital (A / D) by a gaze detection circuit (not shown) and then input to the control unit 104 as eye image data.

[0045] In step S503, the control unit 104 obtains the coordinates of the corneal reflection images Pd' and Pe' of the illumination sources 13a and 13b, and the coordinates of the image c' of the pupil center c, from the eye image data acquired in step S502. The eye image obtained by the eye image sensor 17 includes the reflection images Pd' and Pe' corresponding to the images Pd and Pe of the illumination sources 13a and 13b appearing on the cornea 142, such as... Figure 4A As shown.

[0046] like Figure 4A As shown, the horizontal direction is set as the X-axis, and the vertical direction is set as the Y-axis. In this case, the X-axis coordinates of the centers of the reflected images Pd' and Pe' of the illumination sources 13a and 13b included in the eye image are represented as Xd and Xe, respectively. Furthermore, the X-axis coordinates of the images a' and b' of the pupil ends a and b, which are the ends of the pupil 141, are represented as Xa and Xb, respectively.

[0047] like Figure 4BAs shown, the brightness values ​​at coordinates Xd and Xe corresponding to the reflected images Pd' and Pe' of illumination sources 13a and 13b become much higher than the brightness values ​​at other locations. On the other hand, the brightness values ​​in the range between coordinates Xa and Xb corresponding to the pupil 141 region become very low, except for the brightness values ​​at coordinates Xd and Xe. Furthermore, the brightness values ​​in the coordinate range of the iris 143 outside the pupil 141, which are smaller than coordinate Xa and larger than coordinate Xb, are brightness values ​​between the brightness values ​​of the reflected images Pd' and Pe' of illumination sources 13a and 13b and the remaining brightness value of the pupil 141.

[0048] Based on this characteristic of brightness level in the X-axis direction, the control unit 104 can detect the X-axis coordinates Xd and Xe of the reflected images Pd' and Pe' of the illumination sources 13a and 13b, and the X-axis coordinates Xa and Xb of the images a' and b' at the pupil ends a and b, respectively, from the eye image. Furthermore, in applications such as those in the first embodiment, the rotation angle θx of the optical axis of the eyeball 14 relative to the optical axis of the light-receiving lens 16 is relatively small. In this case, the X-axis coordinate Xc of the image c' at the pupil center c in the eyeball can be expressed as "Xc≈(Xa+Xb) / 2". Thus, the control unit 104 can obtain the coordinates of the reflected images Pd' and Pe' of the illumination sources 13a and 13b, and the X-axis coordinates of the image c' at the pupil center c, from the eye image. Although, in Figure 3 and Figure 4A , Figure 4B The image shows an example of a control unit 104 that obtains the X-axis coordinate; however, the control unit 104 is also able to obtain the Y-axis coordinate in a similar manner.

[0049] In step S504, the control unit 104 calculates the imaging magnification β of the eye image. The imaging magnification β is a magnification determined by the position of the eyeball 14 relative to the light receiving lens 16, and can be obtained as a function of the distance (Xd-Xe) between the reflected images Pd' and Pe' of the illumination sources 13a and 13b.

[0050] In step S505, the control unit 104 calculates the rotation angle of the eyeball. The X-axis coordinate of the midpoint between the images Pd and Pe of the illumination sources 13a and 13b appearing on the cornea 142 is almost identical to the X-axis coordinate of the center of curvature O of the cornea 142. Therefore, when the standard distance between the center of curvature O of the cornea 142 and the center c of the pupil 141 is represented as Oc, the rotation angle θx of the optical axis of the eyeball 14 in the ZX plane can be obtained from the relational expression “β×Oc×sinθx≈{(Xd+Xe) / 2}-Xc”.

[0051] Although Figure 3 and Figure 4A and Figure 4B The diagram illustrates an example of calculating the rotation angle θx in a plane perpendicular to the Y-axis, but the rotation angle θy in a plane perpendicular to the X-axis can also be calculated in a similar manner. Thus, the control unit 104 obtains the rotation angles θx and θy of the eyeball. The control unit 104 is able to calculate the gaze position based on the rotation angles of the eyeball.

[0052] In step S506, the control unit 104 retrieves correction coefficients from RAM. Correction coefficients are coefficients used to correct for individual differences in user gaze. These correction coefficients are generated during a calibration operation and then stored in RAM before the gaze detection process begins. In the case where correction coefficients for multiple users are stored in RAM, the control unit 104 uses the correction coefficient associated with the current user, for example, by querying the user at an optional time interval.

[0053] In step S507, the control unit 104 uses the eye rotation angles θx and θy calculated in step S505 to calculate the user's gaze coordinates (gaze position) on the display 103. Furthermore, the control unit 104 determines that the user's gaze position is the coordinates (Hx, Hy) corresponding to the center c of the pupil 141 on the display 103, and therefore can calculate the user's gaze position according to the expressions "Hx = m × (Ax × θx + Bx)" and "Hy = m × (Ay × θy + By)".

[0054] Here, coefficient m is a transformation coefficient used to convert rotation angles θx and θy into coordinates corresponding to the center c of the pupil 141 on the display 103, and is determined by the characteristics of the eyepiece unit 102. Coefficient m can be pre-stored in RAM. Furthermore, coefficients Ax, Bx, Ay, and By are correction coefficients obtained in step S506.

[0055] In step S508, the control unit 104 records the gaze position and the time (gaze detection time) of the image signal converted in step S502 in RAM, and then ends the gaze detection process.

[0056] After detecting the gaze position in the manner described above and storing it in RAM, the control unit 104 performs predetermined processing based on the gaze position information stored in RAM. Specifically, the predetermined processing is either processing for displaying a pointer indicating the gaze position or processing for selecting a subject located at the gaze position.

[0057] [A method for calculating the gaze positions of a user when viewing a video image with both eyes, based on the gaze positions of the left and right eyes]

[0058] The HMD 100 includes a left gaze detection unit 106a and a right gaze detection unit 106b corresponding to the user's left and right eyes, respectively, so that the user's gaze position can be accurately calculated based on the detection results obtained from the corresponding gaze detection units 106 when the user views the subject in the external video image with both eyes.

[0059] Specifically, refer to Figure 6A Describes a method for calculating the gaze position obtained when a user views an image of a subject with both eyes.

[0060] Figure 6A This diagram illustrates a user's left eye 101a and right eye 101b viewing a subject Obj (dog) in AR space through left display 103a and right display 103b. At this time, left gaze detection unit 106a detects the gaze of the left eye 101a, and right gaze detection unit 106b detects the gaze of the right eye 101b.

[0061] pass Figure 5 The gaze detection process shown can calculate the coordinates (Hax, Hay) of the gaze position Ha of the user's left eye 101a when viewing the left display 103a. The center of the left eye's optical axis in the left display 103a is denoted as Ca. Furthermore, through... Figure 5 The gaze detection process shown can calculate the coordinates (Hbx, Hby) of the gaze position Hb of the user viewing the right display 101b. The center of the optical axis of the right eye in the right display 103b is denoted as Cb.

[0062] Here, if the distance between the subject Obj in the AR space and the display 103 of the HMD 100 worn by the user is greater than or equal to a predetermined value, it is necessary to take into account the distance between the HMD 100 and the subject Obj and calculate the gaze position Io obtained when the user views the subject Obj in the AR space with both eyes.

[0063] The distances Da between the left display 103a and the subject Obj, and Db between the right display 103b and the subject Obj, are calculated based on the focal lengths of the lenses included in the left camera device 105a and the right camera device 105b, respectively. The method for calculating the distance between the display 103a and the subject Obj can be based on output information received from a distance measurement sensor included in the HMD 100, or it can be based on estimation using external video images.

[0064] Furthermore, when the distance to the subject Obj in the AR space is less than a predetermined value, the coordinates of the intersection of the left eye gaze direction based on the gaze position Ha of the user's left eye 101a and the right eye gaze direction based on the gaze position Hb of the user's right eye 101b can be calculated as the gaze position Io obtained when the user views the subject with both eyes.

[0065] The coordinates (Iax, Iay, Iaz) of the left gaze position Ia, obtained when the user views the subject Obj with both eyes using the center Co of the HMD 100 as the origin of the coordinate system, are calculated using the following formulas: Iax = Hax × Da × αax + Hax × βax × γax (αax, βax, and γax are conversion coefficients); Iay = Hay × Da × αay + Hay × βay × γay (αay, βay, and γay are conversion coefficients); and

[0066] Iaz = Da × Ka (Ka is the conversion coefficient).

[0067] In addition, the coordinates (Ibx, Iby, Ibz) of the right gaze position Ib obtained when the user views the subject Obj with both eyes, using the center Co of the HMD 100 as the origin of the coordinate reference:

[0068] Ibx = Hbx × Db × αbx + Hbx × βbx × γbx (αbx, βbx, and γbx are conversion coefficients); Iby = Hby × Db × αby + Hby × βby × γby (αby, βby, and γby are conversion coefficients); and

[0069] Ibz = Db × Kb (Kb is the conversion factor).

[0070] Here, the coefficients αax, αay, βax, βay, γax, γay, αbx, αby, βbx, βby, γbx and γby, which are used as conversion coefficients, are specifically the following correction coefficients.

[0071] These conversion coefficients are used to calculate the X and Y coordinates of the left and right viewing positions Ia and Ib obtained when the user views the subject Obj with both eyes, based on the viewing position Ha of the left display 103a and the viewing position Hb of the right display 103b. Furthermore, coefficients Ka and Kb, which are also conversion coefficients, are used to calculate the Z coordinates of the left and right viewing positions Ia and Ib obtained when the user views the subject Obj with both eyes, based on the subject distances Da and Db. These correction coefficients are predetermined based on, for example, the characteristics of the optical system of the eyepiece unit 102, the characteristics of the optical system of the imaging device 105 (e.g., focal length), the center Co of the HMD 100, the center of the left eye optical axis Ca, and the center of the right eye optical axis Cb, and are then stored in RAM.

[0072] The control unit 104 sets the average of the calculated coordinates (Iax, Iay, Iaz) of the left gaze position Ia and the calculated coordinates (Ibx, Iby, Ibz) of the right gaze position Ib as the gaze position Io obtained when the user views the subject Obj with both eyes. Then, the control unit 104 selects the subject based on the calculated gaze position Io.

[0073] Furthermore, for example, if either the left gaze position Ia or the right gaze position Ib cannot be calculated due to failures such as gaze detection or subject distance calculation, or if the calculation is unreliable, the control unit 104 can set the gaze position that has been successfully calculated or has high reliability as the gaze position Io obtained when the user views the subject with both eyes. Additionally, if the subject distance is infinitely large, the control unit 104 can set the coordinates of the intersection of the left and right gaze directions as the gaze position Io obtained when the user views the subject with both eyes.

[0074] [A method for calculating the gaze position when a user views a video image with both eyes, based on the gaze position of either the left or right eye]

[0075] However, as described above, there are cases where the control unit 104 does not need to calculate the gaze position based on the left eye gaze and the right eye gaze calculated by the left gaze detection unit 106a and the right gaze detection unit 106b, respectively. Therefore, there are cases where the control unit 104 can calculate the gaze position based on the gaze calculated by any one of the gaze detection units 106, and then calculate the gaze position Io obtained when the user views the subject Obj with both eyes based on the calculated gaze position.

[0076] Assuming that the distance between the left gaze position Ia and the right gaze position Ib obtained when the user views the subject with both eyes is less than or equal to a predetermined value (i.e., a predetermined condition is met), this is a case where the user is stably looking at the same subject Obj with both eyes. Therefore, assuming that the left gaze position Ia and the right gaze position Ib obtained when the user views the subject with both eyes are almost identical, this is a case where the user is stably looking at the same subject Obj with both eyes. If the user is stably looking at the same subject Obj with both eyes, the gaze position Io obtained when the user views the subject Obj with both eyes can be calculated based on the gaze position calculated from the gaze detected by any of the gaze detection units 106. Thus, the control unit 104 determines whether the predetermined condition is met based on the distance between the left and right gaze positions, and if the predetermined condition is met, the gaze position Io can be detected even if the gaze detection process in one of the gaze detection units 106 is stopped. Stopping the gaze detection process in one of the gaze detection units 106 can reduce the power consumption of the HMD 100 while calculating the gaze position Io obtained when the user views the subject Obj with both eyes. One of the gaze detection units 106 that does not need to perform gaze detection processing may include stopping the operation of one of the gaze detection units 106 or slowing down the operation cycle (gaze detection cycle) as described above.

[0077] Furthermore, the distance between the left gaze position Ia and the right gaze position Ib obtained when the user views the subject with both eyes can be set as the average of the distances between the left gaze position Ia and the right gaze position Ib for any number of frames. Additionally, a predetermined value is determined based on the amount of noise measured during gaze calibration, thereby reducing the impact of individual differences.

[0078] Reference Figure 6B This describes a method for calculating the gaze position Io when a user views the subject Obj with both eyes, based on the gaze position calculated from any gaze detection unit 106. Figure 6B In this context, it is assumed that the user has pre-set the primary gaze detection unit as the left gaze detection unit 106a and the secondary gaze detection unit as the right gaze detection unit 106b.

[0079] The control unit 104 calculates the gaze position Io obtained when the user views the subject with both eyes, based on the gaze position Ha of the user viewing the left display 103a, which is detected from the main left gaze detection unit 106a. Therefore, the control unit 104 makes the gaze position Io obtained when the user views the subject with both eyes equal to the left gaze position Ia obtained when the user views the subject with both eyes. The coordinates (Iax, Iay, Iaz) of the left gaze position Ia are calculated using the following formulas: Iax = Hax × Da × αax + Hax × βax × γax (αax, βax, and γax are conversion coefficients); Iay = Hay × Da × αay + Hay × βay × γay (αay, βay, and γay are conversion coefficients); and

[0080] Iaz = Da × Ka (Ka is the conversion coefficient).

[0081] Here, the conversion coefficients αax, αay, βax, βay, γax, and γay are used to calculate the X and Y coordinates of the left viewing position Ia obtained when the user views the subject Obj with both eyes, based on the viewing position Ha of the left display 103a. Furthermore, the conversion coefficient Ka is used to calculate the Z coordinate of the left viewing position Ia obtained when the user views the subject Obj with both eyes, based on the subject distance Da. These conversion coefficients are predetermined based on, for example, the characteristics of the eyepiece unit 102, the center Co of the HMD 100, the center of the left eye optical axis Ca, and the center of the right eye optical axis Cb, and are then stored in RAM. The control unit 104 sets the coordinates (Iax, Iay) of the left viewing position Ia thus calculated as the coordinates (Iox, Ioy) of the viewing position Io obtained when the user views the subject with both eyes.

[0082] Furthermore, if the user steadily gazes at the same subject Obj with both eyes, the coordinates (Iox, Ioy) of the gaze position Io can be used as the coordinates of the right gaze position. Therefore, the control unit 104 can estimate the gaze position Hc of the right display 103b based on the coordinates of the right gaze position. The control unit 104 uses the estimated gaze position Hc of the right display 103b to determine whether to continue the power-saving mode as described below or to execute the pointer display. The coordinates of the estimated gaze position Hc of the user viewing the right display 103b are represented as (Hcx, Hcy). The coordinates (Hcx, Hcy) are calculated using the following formula:

[0083] Iox = Hcx × Da × αbx + Hcx × βbx + γbx; and

[0084] Ioy=Hcy×Da×αby+Hcy×βby+γby.

[0085] Here, the conversion coefficients αbx, αby, βbx, βby, γbx, and γby are used to calculate the X and Y coordinates of the right viewing position Ib obtained when the user views the subject Obj with both eyes, based on the viewing position Hb of the right display 103b. These conversion coefficients are predetermined based on, for example, the characteristics of the eyepiece unit 102, the center Co of the HMD 100, the center Ca of the left eye optical axis, and the center Cb of the right eye optical axis, and are then stored in RAM.

[0086] The method for calculating the estimated gaze position Hc of the user viewing the right display 103b is not limited to the methods described above. For example, the control unit 104 cuts an image around the gaze position Ha of the video image displayed on the left display 103a and performs pattern matching between the cut image and the image displayed on the right display 103b. Through this pattern matching, the control unit 104 can estimate the gaze position Hc of the user viewing the right display 103b based on the position with the highest correlation. Furthermore, for example, when the user is using the HMD 100 to view a two-dimensional (2D) video image, the control unit 104 can consider the coordinates of the gaze position Ha of the left display 103a as the gaze position Hc of the user viewing the right display 103b. As in the first embodiment, when the user views a deep subject in an external video image (3D space) with both their left and right eyes, the gaze positions of the left and right eyes viewing the subject may be offset from each other. Therefore, the control unit 104 estimates the gaze position Hc of the user viewing the right display 103b separately from the coordinates of the gaze position Ha of the left display 103a.

[0087] In the first embodiment, the control unit 104 estimates the gaze position Hc of the user viewing the right display 103b based on the coordinates (Iax, Iay) of the left gaze position Ia. However, while setting the left gaze detection unit 106a as the primary gaze detection unit and the right gaze detection unit 106b as the secondary gaze detection unit, the control unit 104 can estimate the gaze position of the user viewing the left display 103a based on the coordinates (Ibx, Iby) of the right gaze position Ib.

[0088] In this way, when the user is steadily looking at the same subject Obj with both eyes, even if only the gaze detected by any one of the gaze detection units 106 is used, the power consumption of HMD 100 can be reduced without reducing the accuracy of performing predetermined processing based on gaze.

[0089] In the above description, it has been explained that when calculating the gaze position obtained when a user views a video image with both eyes, there are cases where the gaze detected by both gaze detection units 106 is used and cases where the gaze detected by any one of the gaze detection units 106 is used.

[0090] Next, the mode switching process for switching between normal mode and power saving mode will be described, with the case of using the gaze detected by both gaze detection units 106 referred to as normal mode and the case of using the gaze detected by any one of the gaze detection units 106 referred to as power saving mode. Figure 7 This is a flowchart regarding the mode switching process according to the first embodiment, and Figure 7 The process shown begins in response to the HMD 100 being activated and a video image being displayed on the monitor 103. Figure 7 In the middle, the left gaze detection unit 106a is set as the left primary gaze detection unit, and the right gaze detection unit 106b is set as the secondary gaze detection unit.

[0091] In step S700, the control unit 104 sets the operation of the gaze detection unit 106 to normal mode. Therefore, the control unit 104 sets the gaze detection cycles of the left gaze detection unit 106a and the right gaze detection unit 106b to the same gaze detection cycle (hereinafter referred to as the "normal detection cycle"). Then, the control unit 104 uses the gaze detected by the two gaze detection units 106 to calculate the gaze position obtained when the user views the subject in the external video image with both eyes. Here, the state in which the gaze detection units 106 detect gazes with the normal detection cycle is set to "first state".

[0092] In step S701, the control unit 104 determines, based on the normal detection cycle set in step S700, whether the timing for the left gaze detection unit 106a and the right gaze detection unit 106b to perform gaze detection processing has been reached. Specifically, the control unit 104 determines whether the timing for detecting an object (eye) approaching the eyepiece unit 102 has been reached. If it is determined that the timing for performing gaze detection processing has been reached ("Yes" in step S701), the control unit 104 proceeds the processing to step S702; otherwise ("No" in step S701), the control unit 104 proceeds the processing to step S705.

[0093] In step S702, the control unit 104 performs gaze detection processing through the user-preset main left gaze detection unit 106a. Based on Figure 5 The process shown in the flowchart is used to perform the gaze detection process in step S702. Therefore, the coordinates (Hax, Hay) of the gaze position Ha of the user's left eye 101a, which is viewing the left display 103a, are calculated. Figure 6A ).

[0094] In step S703, the control unit 104 performs gaze detection processing through the user-preset secondary right gaze detection unit 106b. Figure 5 The process shown in the flowchart is used to perform the gaze detection process in step S703. Therefore, the coordinates (Hbx, Hby) of the gaze position Hb of the user viewing the right display 103b on the right eye 101b are calculated. Figure 6A ).

[0095] In step S704, the control unit 104 calculates the left gaze position Ia, right gaze position Ib, and gaze position Io when the user views the subject in the external video image with both eyes, based on the gaze position Ha of the user's left eye 101a and the gaze position Hb of the user's right eye 101b. Figure 6A ).

[0096] In step S705, the control unit 104 determines whether to switch the operation of the gaze detection unit 106 from normal mode to power-saving mode. Specifically, if the distance between the left gaze position Ia and the right gaze position Ib obtained when the user views the subject with both eyes is less than or equal to a predetermined value, the control unit 104 determines that the user is steadily looking at the same subject Obj with both eyes, and thus performs the switch from normal mode to power-saving mode. Therefore, if it is determined that the operation of the gaze detection unit 106 should be switched from normal mode to power-saving mode ("Yes" in step S705), the control unit 104 proceeds the process to step S706; otherwise ("No" in step S705), the control unit 104 returns the process to step S701.

[0097] In step S706, the control unit 104 sets the operation of the gaze detection unit 106 to power-saving mode, and then proceeds to step S707. Since the gaze detected by any of the gaze detection units 106 is used if power-saving mode is set, neither the left gaze detection unit 106a nor the right gaze detection unit 106b needs to be set to the normal detection cycle. With the left gaze detection unit 106a set as the primary gaze detection unit 106 and the right gaze detection unit 106b set as another secondary gaze detection unit, the control unit 104 sets the left gaze detection unit 106a to the normal detection cycle.

[0098] Then, the control unit 104 sets the right gaze detection unit 106b to a detection cycle slower than the normal detection cycle (hereinafter referred to as the "power-saving detection cycle"). Here, if the state in which the gaze detection unit 106 detects gaze at a power-saving detection cycle slower than the normal detection cycle is set to the "second state", the right gaze detection unit 106b changes from the first state to the second state. In addition, the state in which gaze detection processing performed by the gaze detection unit 106 is stopped can be set to the second state.

[0099] In step S707, the control unit 104 determines, based on the normal detection cycle set in step S706, whether the timing for the main left gaze detection unit 106a to perform gaze detection processing has been reached. Specifically, the control unit 104 determines whether the timing for detecting an object (eye) approaching the eyepiece unit 102 has been reached. If it is determined that the timing for performing gaze detection processing has been reached ("Yes" in step S707), the control unit 104 proceeds the processing to step S708; otherwise ("No" in step S707), the control unit 104 proceeds the processing to step S710.

[0100] In step S708, the control unit 104 performs gaze detection processing through the main left gaze detection unit 106a. Based on Figure 5 The process shown in the flowchart is used to perform the gaze detection process in step S708. Therefore, the coordinates (Hax, Hay) of the gaze position Ha of the user's left eye 101a, which is viewing the left display 103a, are calculated. Figure 6B ).

[0101] In step S709, the control unit 104 calculates the gaze position Io obtained when the user views the subject with both eyes, based on the gaze position Ha of the user viewing the left display 103a as detected by the left gaze detection unit 106a. Figure 6B Furthermore, the control unit 104 estimates the coordinates (Hcx, Hcy) of the gaze position Hc of the user's right eye 101b when viewing the subject, based on the gaze position Io obtained when the user views the subject with both eyes.

[0102] In step S710, the control unit 104 determines, based on the power-saving detection cycle set in step S706, whether the timing for the right-side gaze detection unit 106b to perform gaze detection processing has been reached. Specifically, the control unit 104 determines whether the timing for detecting an object (eye) approaching the eyepiece unit 102 has been reached. If it is determined that the timing for performing gaze detection processing has been reached ("Yes" in step S710), the control unit 104 proceeds the processing to step S711; otherwise ("No" in step S710), the control unit 104 proceeds the processing to step S712.

[0103] In step S711, the control unit 104 performs gaze detection processing through the secondary right gaze detection unit 106b. Based on Figure 5 The process shown in the flowchart is used to perform the gaze detection process in step S711. Therefore, the coordinates (Hbx, Hby) of the gaze position Hb of the user viewing the right display 103b on the right eye 101b are calculated. Figure 6BThus, even using a detection cycle slower than the normal detection cycle, the gaze detection processing performed by the secondary gaze detection unit can compare the gaze position of the right eye 101b estimated in step S709 with the actual gaze position of the right eye 101b detected in step S711. This comparison then allows for a determination in step S712 whether to continue in power-saving mode.

[0104] In step S712, the control unit 104 determines whether to continue in power-saving mode. If it is determined to continue in power-saving mode ("Yes" in step S712), the control unit 104 returns to step S707; otherwise ("No" in step S712), the control unit 104 returns to step S700.

[0105] If the distance between the estimated gaze position of the right eye 101b and the actual detected gaze position of the right eye 101b is greater than a predetermined value, the control unit 104 determines that the estimation accuracy of the gaze position has decreased, therefore, it does not continue in power-saving mode and switches to normal mode. On the other hand, if the distance between the estimated gaze position of the right eye 101b and the actual detected gaze position of the right eye 101b is less than or equal to the predetermined value, the control unit 104 determines that the estimation accuracy of the gaze position has not decreased, therefore, it continues in power-saving mode.

[0106] [Timed execution of pre-defined processing based on gaze in each mode]

[0107] Figure 8A and Figure 8B This is a diagram illustrating the timing of performing a predetermined processing operation based on line of sight in a corresponding mode according to the first embodiment. Figure 8A and Figure 8B In this configuration, the left gaze detection unit 106a is designated as the primary gaze detection unit, and the right gaze detection unit 106b is designated as the secondary gaze detection unit. However, the right gaze detection unit 106b can be designated as the primary gaze detection unit, while the left gaze detection unit 106a can be designated as the secondary gaze detection unit.

[0108] Figure 8AThis diagram illustrates the timing of predefined processing operations performed based on gaze in normal mode. In normal mode, the normal detection period in the left gaze detection unit 106a and the right gaze detection unit 106b is set to n frames per second (fps), and the vertical axis is set to time (t0 to t5). The horizontal axis, from top to bottom, represents the timing of gaze detection processing in the left gaze detection unit 106a, the timing of gaze position calculation in the left display 103a, and the timing of pointer display based on the gaze position in the left display 103a. Subsequently, the horizontal axis represents the timing of gaze detection processing in the right gaze detection unit 106b and the timing of gaze position calculation in the right display 103b. Furthermore, the horizontal axis represents the timing of gaze position calculation performed when the user views the subject with both eyes, and the timing of subject selection based on the gaze position obtained when the user views the subject with both eyes.

[0109] exist Figure 7 In the above-described process, the gaze detection process is performed by the main left gaze detection unit 106a. Figure 7 After step S702 shown, the control unit 104 performs gaze detection processing via the secondary right gaze detection unit 106b. Figure 7 Step S703 (as described above). In Figure 8A In the timing shown, the control unit 104 simultaneously executes the gaze detection processing of the main left gaze detection unit 106a and the secondary right gaze detection unit 106b. In this case, the timing for calculating the gaze position in the left display 103a and the timing for calculating the gaze position in the right display 103b also become the same. Then, the control unit 104 calculates the gaze position obtained when the user views the subject with both eyes based on the gaze position in the respective display 103. Furthermore, after calculating the gaze position obtained when the user views the subject with both eyes, the control unit 104 performs pointer display based on the gaze position in the left display 103a, and simultaneously performs subject selection based on the gaze position obtained when the user views the subject with both eyes. Moreover, regarding the pointer display, the control unit 104 can perform pointer display based on the gaze position in the right display 103b.

[0110] Figure 8BThis diagram illustrates the timing of predefined processing operations performed based on gaze in power-saving mode. In power-saving mode, the normal detection cycle in the left gaze detection unit 106a is set to n (fps), and the power-saving detection cycle in the right gaze detection unit 106b is set to n / 3 (fps), with the vertical axis set to time (t0 to t5). The horizontal axis, from top to bottom, represents the timing of gaze detection processing in the left gaze detection unit 106a and the timing of gaze position calculation in the left display 103a. Subsequently, the horizontal axis represents the timing of gaze detection processing in the right gaze detection unit 106b, the timing of gaze position calculation in the right display 103b, the timing of gaze position estimation in the right display 103b, and the timing of pointer display based on the gaze position in the right display 103b. Furthermore, the horizontal axis represents the timing of gaze position calculation performed when the user views the subject with both eyes, and the timing of subject selection based on the gaze position obtained when the user views the subject with both eyes.

[0111] At time t0, control unit 104 begins gaze detection processing of left gaze detection unit 106a and calculates the gaze position in left display 103a based on the detected gaze. Then, control unit 104 calculates the gaze position obtained when the user views the subject with both eyes based on the gaze position in left display 103a, and estimates the gaze position in right display 103b based on the calculated gaze position obtained when the user views the subject with both eyes. Furthermore, control unit 104 performs pointer display based on the estimated gaze position in right display 103b, and performs subject selection based on the gaze position obtained when the user views the subject with both eyes.

[0112] At time t1, the control unit 104 simultaneously executes the gaze detection processing of the left gaze detection unit 106a and the right gaze detection unit 106b. In this case, the timing for calculating the gaze position in the left display 103a and the timing for calculating the gaze position in the right display 103b also become the same. Here, the gaze position in the right display 103b calculated based on the gaze detected by the right gaze detection unit 106b is used to determine whether to continue in power-saving mode.

[0113] Then, the control unit 104 calculates the gaze position obtained when the user views the subject with both eyes based on the gaze position in the left display 103a, and then estimates the gaze position in the right display 103b based on the calculated gaze position obtained when the user views the subject with both eyes. In addition, the control unit 104 performs pointer display based on the estimated gaze position in the right display 103b, and performs subject selection based on the gaze position obtained when the user views the subject with both eyes.

[0114] exist Figure 8BIn the power-saving mode shown, the control unit 104 performs pointer display based on the estimated viewing position in the right display 103b. In this case, even in normal mode, pointer display is performed in the right display 103b based on the same viewing position, so that pointer display can be performed in a position that does not feel out of place, regardless of the distance to the subject viewed by the user.

[0115] Furthermore, in the first embodiment, power consumption of the electronic device is reduced by stopping the operation of any of the gaze detection units 106 or slowing down the gaze detection cycle. Alternatively, power consumption can be reduced, for example, by reducing the number of light sources to be turned on to emit light in each gaze detection unit or by controlling the luminous intensity of each light source. Additionally, power consumption can be reduced, for example, by reducing the number of eye image sensors to be driven in each gaze detection unit, or by controlling, for example, the image capture sensitivity of each eye image sensor.

[0116] <Second Embodiment>

[0117] In the first embodiment described above, the primary and secondary gaze detection units 106 are preset by the user. The second embodiment allows switching between the primary and secondary gaze detection units 106 even after the user has preset them. Specifically, the second embodiment can switch between the primary and secondary gaze detection units 106 based on the characteristics of the user's eyes. This allows for more accurate calculation of the gaze position obtained when the user views the subject with both eyes.

[0118] Figure 9A and 9B This is a flowchart regarding the mode switching process according to the second embodiment, and... Figure 9A and Figure 9B The process described in the flowchart is related to Figure 7 The steps S900, S901, S902, and S903 shown in the flowchart are non-overlapping.

[0119] In step S900, the control unit 104 sets the primary and secondary gaze detection units according to instructions from the user. For example, the control unit 104 sets the left gaze detection unit 106a as the primary gaze detection unit and the right gaze detection unit 106b as the secondary gaze detection unit. When performing the setting, the control unit 104 can set the gaze detection unit with the smaller change in the correction coefficients Ax, Bx, Ay, and By as the primary gaze detection unit based on the change in the correction coefficients Ax, Bx, Ay, and By calculated during the calibration operation. The change can be calculated by adding the differences (absolute values) of the average gaze position of each frame relative to the number of selectable frames during calibration. Furthermore, based on information about the pre-identified dominant eye, the control unit 104 can set the gaze detection unit of the dominant eye as the primary gaze detection unit.

[0120] In step S901, the control unit 104 calculates the corresponding changes for the left and right eyes based on the gaze information about the left eye detected by the left gaze detection unit 106a and the gaze information about the right eye detected by the right gaze detection unit 106b. While the changes can be calculated by adding the absolute values ​​of the differences between the gaze positions of each frame and the average of a selectable number of frames at the user's gaze position on the viewing display 103, the second embodiment is not limited to this calculation. For example, the control unit 104 may calculate the changes based on the rotation angle θ in the gaze direction.

[0121] In step S902, the control unit 104 compares the changes in the left and right eyes. With the left gaze detection unit 106a set as the primary gaze detection unit and the right gaze detection unit 106b set as the secondary gaze detection unit, if the change in the left eye is less than the change in the right eye ("Yes" in step S902), the control unit 104 continues processing to step S704 while maintaining the left gaze detection unit 106a as the primary gaze detection unit and the right gaze detection unit 106b as the secondary gaze detection unit. Conversely, if the change in the left eye is greater than or equal to the change in the right eye ("No" in step S902), the control unit 104 continues processing to step S903.

[0122] In step S903, the control unit 104 executes a setting for switching between the primary and secondary line-of-sight detection units. The control unit 104 executes a setting to switch the primary line-of-sight detection unit previously set by the user from the left line-of-sight detection unit 106a to the right line-of-sight detection unit 106b, and to switch the secondary line-of-sight detection unit previously set by the user from the right line-of-sight detection unit 106b to the left line-of-sight detection unit 106a.

[0123] The various control operations described above, which are performed by the control unit 104 (specifically the CPU included therein), can be performed by a single piece of hardware, or the control of the entire device can be performed by multiple pieces of hardware (e.g., multiple processors or circuits) that share the processing operations.

[0124] Furthermore, while this disclosure has been described in detail based on advantageous embodiments, it is not limited to these specific embodiments, and includes various configurations without departing from the spirit of this disclosure. Moreover, the above embodiments are merely specific examples of this disclosure, and some or all of the embodiments may be appropriately combined.

[0125] According to one aspect of this disclosure, when performing gaze detection for a user, the power consumption of the electronic device can be reduced.

[0126] Other implementation methods

[0127] The embodiments of this disclosure can also be implemented by a computer in a system or apparatus that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transient computer-readable storage medium") to perform one or more functions of the above embodiments, and / or includes circuitry (e.g., application-specific integrated circuits (ASICs)) for performing one or more functions of the above embodiments, and by methods executed by the computer in the system or apparatus, such as reading and executing computer-executable instructions from the storage medium to perform one or more functions of the above embodiments and / or controlling one or more circuits to perform one or more functions of the above embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessor unit (MPU)) and may include a network of separate computers or separate processors to read and execute computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or storage medium. The storage medium may include one or more of, for example, a hard disk, random access memory (RAM), read-only memory (ROM), the memory of a distributed computing system, an optical disk (such as an optical disc (CD), a digital versatile optical disc (DVD), or a Blu-ray disc (BD)™), a flash memory device, a memory card, etc.

[0128] Embodiments of the present invention can also be implemented by means of providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU), microprocessor unit (MPU)) of the system or device reading out and executing the computer program.

[0129] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the following claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. An electronic device, comprising: The first detection unit is used to detect the gaze of the user's left eye when viewing the first display unit; The second detection unit is used to detect the gaze of the user's right eye when viewing the second display unit; The control unit is configured to perform control under predetermined conditions to change either the first detection unit or the second detection unit from a first state to a second state where the power consumption is less than that of the first state.

2. The electronic device according to claim 1, further comprising: A computing unit is configured to perform control to switch between a first mode and a second mode based on whether the predetermined conditions are met. If the predetermined conditions are not met, in the first mode, the detection results obtained by the first detection unit and the detection results obtained by the second detection unit are used to calculate the gaze position obtained when the user is viewing the subject displayed on the display unit with both eyes, and If the predetermined conditions are met, in the second mode, the detection results obtained by a detection unit other than the first detection unit or the second detection unit are used to calculate the gaze position when the user is looking at the subject displayed on the display unit with both eyes.

3. The electronic device according to claim 1, in, The calculation unit detects the gaze position of the user's left eye in the first display unit based on the detection result obtained by the first detection unit, and detects the gaze position of the user's right eye in the second display unit based on the detection result obtained by the second detection unit; as well as The control unit determines whether the predetermined condition is met based on the distance between the user's left eye gaze position and the user's right eye gaze position calculated by each of the computing units.

4. The electronic device according to claim 1, wherein, If the predetermined conditions are not met, the control unit performs control to cause both the first detection unit and the second detection unit to enter the first state.

5. The electronic device according to claim 1, in, The first state is the state of the gaze detected in the first detection cycle, and The second state is determined by detecting the gaze state based on a second detection cycle that is slower than the first detection cycle.

6. The electronic device according to claim 3, wherein, The control unit performs the first processing at a location where either the user's left eye gaze position or the user's right eye gaze position is calculated based on each of the computing units.

7. The electronic device according to claim 6, wherein, The first process is a process for displaying an item indicating the viewing position on a first display unit or a second display unit.

8. The electronic device according to claim 2, wherein, The control unit performs control to perform a second process at a position based on the line-of-sight position obtained when the user is viewing the subject displayed on the display unit with both eyes.

9. The electronic device according to claim 8, wherein, The second process is a process for selecting the subject.

10. A control method for an electronic device, the electronic device comprising a first detection unit for detecting the gaze of the left eye of a user viewing a first display unit and a second detection unit for detecting the gaze of the right eye of a user viewing a second display unit, the control method comprising: Control is executed when predetermined conditions are met to change either the first detection unit or the second detection unit from a first state to a second state where power consumption is less than that of the first state.

11. A computer program product comprising a computer program / instructions that, when executed by a computer, cause the computer to perform a control method for an electronic device, said electronic device including a first detection unit for detecting the gaze of the left eye of a user viewing a first display unit and a second detection unit for detecting the gaze of the right eye of a user viewing a second display unit, said control method comprising: Control is executed when predetermined conditions are met to change either the first detection unit or the second detection unit from a first state to a second state where power consumption is less than that of the first state.

12. A non-transient computer-readable storage medium storing computer-executable instructions, which, when executed by a computer, cause the computer to perform a control method for an electronic device, the electronic device including a first detection unit for detecting the gaze of the left eye of a user viewing a first display unit and a second detection unit for detecting the gaze of the right eye of a user viewing a second display unit, the control method comprising: Control is executed when predetermined conditions are met to change either the first detection unit or the second detection unit from a first state to a second state where power consumption is less than that of the first state.

Images