Electronic device, control method thereof, and recording medium

Abstract

The present invention relates to an electronic device, a control method thereof, and a recording medium. In an operation inputted by sight, a position (designated position) designated by a gaze point is easier to view after a user performs a determination operation than before the determination operation. An electronic device includes: a detection unit configured to detect a gaze point on a display unit based on a user's sight line; and a control unit configured to display an indicator indicating the gaze point on the display unit, wherein, when a user operates an operation unit, the control unit controls to display a display item at the gaze point, and displays the indicator and the display item in a changed display order.

Description

Technical Field

[0001] The present invention relates to an electronic device configured to detect a user's line of sight and operable through line of sight input, a control method of the electronic device, and a storage medium. Background Art

[0002] In recent years, electronic devices that operate based on the user's line of sight (hereinafter referred to as "line of sight input") have become known. Line of sight input is effective in particular when a user wants to input instructions to an electronic device (such as a digital camera or a game console) to quickly specify or move a desired position. Japanese Patent Laid-Open No. 9-114846 discusses a technology for giving priority, in display order, to images similar to the image displayed at a gaze point calculated based on the user's line of sight. According to Japanese Patent Laid-Open No. 9-114846, if dedicated software is installed, a pointer can be displayed at a gaze point calculated based on the user's line of sight, so that the user can watch the user's gaze point and the movement of the gaze point.

[0003] Since the technology discussed in Japanese Patent Application Laid-Open No. 9-114846 does not display the gaze point calculated based on the user's line of sight, it is not possible to check whether the gaze point corresponds to the position the user is looking at. If the gaze point does not correspond to the position the user is looking at, similar images that the user does not want may be prioritized. According to Tobii Technology AB, "Streaming", Tobii Ghost Software,<https: / / gaming.tobii.com / software / ghost / > (searched on the Internet on October 7, 2020), a pointer indicating the gaze point is displayed. However, since the pointer is always displayed above any displayed object, it is difficult to view the target displayed object, and as a result, the user may feel annoyed by the pointer. Summary of the Invention

[0004] The present invention relates to a technique by which, when an operation is performed by line-of-sight input, a position specified by a gaze point (specified position) is easier to view after a user performs a determination operation than before the determination operation.

[0005] According to various embodiments of the present invention, an electronic device includes: a detection unit configured to detect a gaze point on a display unit based on a user's viewing line of sight; and a control unit configured to display an indicator representing the gaze point on the display unit, wherein, when the user operates the operation unit, the control unit controls to display a display item at the gaze point, and displays the indicator and the display item in a changed display order.

[0006] A control method for controlling an electronic device, the control method comprising: detecting a gaze point on a display unit based on a user's viewing line of sight; and controlling to display an indicator representing the gaze point on the display unit, wherein, when the user operates an operating unit, control is performed to display a display item at the gaze point, and the indicator and the display item are displayed in a changed display order.

[0007] A computer-readable recording medium storing a program for causing a computer to execute a method for controlling an electronic device, the method comprising: detecting a gaze point on a display unit based on a user's viewing line of sight; and controlling to display an indicator representing the gaze point on the display unit, wherein, when the user operates an operating unit, control is performed to display a display item at the gaze point, and the indicator and the display item are displayed in a changed display order.

[0008] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A and Figure 1B is a diagram showing the external appearance of a digital camera according to an exemplary embodiment of the present invention.

[0010] Figure 2 is a block diagram illustrating a structure of a digital camera according to an exemplary embodiment of the present invention.

[0011] Figure 3 is a flowchart illustrating a process of controlling a digital camera according to an exemplary embodiment of the present invention.

[0012] Figures 4A1 to 4A3 and Figures 4B1 to 4B3 : is a flowchart illustrating a process of controlling display of a line-of-sight pointer indicating a user's gaze point and a frame in an image capture mode process according to an exemplary embodiment of the present invention.

[0013] Figure 5A and Figure 5B is a setting menu screen according to an exemplary embodiment of the present invention.

[0014] Figures 6A to 6J : are diagrams each showing a display example of a line-of-sight pointer and a frame indicating a user's gaze point according to an exemplary embodiment of the present invention.

[0015] Figure 7A and Figure 7B is a diagram illustrating a display order of objects displayed on a display unit according to an exemplary embodiment of the present invention.

[0016] Figure 8 is a diagram showing the external appearance of a personal computer (PC) according to an exemplary embodiment of the present invention.

[0017] 9A to 9D is a diagram showing a display example in the case where an exemplary embodiment of the present invention is applied to a PC. DETAILED DESCRIPTION

[0018] Various exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Note that the following exemplary embodiments are merely examples of implementation of the present invention, and the embodiments of the present invention may be appropriately modified or changed according to the individual configurations and various conditions of the devices to which the present invention is applied. Therefore, the present invention is by no means limited to the following exemplary embodiments.

[0020] <External View of Digital Camera 100>

[0021] Various exemplary embodiments of the present invention will be described below with reference to the accompanying drawings.

[0022] Figure 1A and Figure 1B is a diagram showing the external appearance of a digital camera 100 as an example of an apparatus to which the present invention is applicable. Figure 1A is a front perspective view showing the digital camera 100, and Figure 1B 1 is a perspective view showing the rear side of the digital camera 100. Figure 1A and Figure 1B In the digital camera 100, the display unit 28 is a display unit that displays images and various types of information and is provided on the back surface of the digital camera 100. The touch panel 70a is an operating member that can be touch-operated. The touch panel 70a detects touch operations on the display surface (operation surface) of the display unit 28. The viewfinder display unit 43 is a display unit provided outside the viewfinder of the digital camera 100 and displays various setting values of the digital camera 100, such as shutter speed and aperture.

[0023] The shutter button 61 is an operating unit for issuing an instruction to shoot. The mode selector switch 60 is an operating unit for selecting various modes. The terminal cover 40 is a cover that protects a connector (not shown) for connecting a connection cable of an external device to the digital camera 100. The main electronic dial 71 is a rotary operating member of the operating unit 70, and by rotating the main electronic dial 71, setting values such as shutter speed and aperture are changed. The power switch 72 is an operating member for turning the digital camera 100 on and off. The sub-electronic dial 73 is a rotary operating member of the operating unit 70 for moving the selected frame and feeding images. The four-way key 74 of the operating unit 70 is an operating member including buttons that can be pressed in four directions, and operates based on the direction in which the four-way key 74 is pressed. The set (SET) button 75 is a button of the operating unit 70, and is mainly used to confirm the selected item. The moving image button 76 is used to issue an instruction to start / stop the shooting (recording) of moving images. Pressing the autofocus-on (AF-ON) button 77 of the operation unit 70 starts an autofocus (AF) operation (automatic focus adjustment). Although AF operation is primarily initiated in response to pressing the shutter button 61, pressing the AF-ON button 77 can also issue an instruction to start AF operation. In a digital camera 100 that can be configured not to perform AF operation in response to pressing the shutter button 61, the AF start instruction and the image capture instruction can be issued separately. By pressing the AF-ON button 77 and then the shutter button 61, image capture is performed with the AF position fixed, or even when AF is not operable. Pressing the autoexposure (AE) lock button 78 of the operation unit 70 in the image capture standby state (a state in which image capture is started using the image capture unit 22 in image capture mode, preparations for image capture are not being made, or the digital camera 100 is waiting in image capture mode) fixes the exposure state. Specifically, image capture is performed using a fixed exposure value desired by the user. The playback button 79 of the operation unit 70 is an operation button for switching between image capture mode and playback mode. Pressing the playback button 79 during image capture mode switches to playback mode, and the display unit 28 displays the most recent image recorded on the recording medium 200. Pressing the focus button 84 of the operation unit 70 focuses the lens at the aperture value set when the focus button 84 was pressed to check the focus range (depth of field). Pressing the menu button 81 of the operation unit 70 displays a menu screen for various settings on the display unit 28. The zoom button 82 of the operation unit 70 is an operation button for enabling or disabling the zoom mode during live view display in image capture mode. By enabling the zoom mode and then operating the main electronic dial 71, the live view image is magnified or reduced. In playback mode, the zoom button 82 functions as a zoom button for magnifying the reproduced image and increasing the magnification ratio. The multi-controller 83 is an operation member including a directional key and buttons.The directional key can be operated in eight directions and can be pressed. Operations based on pushing the corresponding direction of the multi-controller 83 are performed. The user can intuitively set various settings on the menu screen displayed on the display unit 28 using the four-way key 74, the setting button 75, and the multi-controller 83. A framing assist button (not shown) is an operation button provided on the lens unit 150 or a lens adapter for connecting the digital camera 100 to the lens unit 150. The framing assist button is an operation member arranged at a position where the user can press the framing assist button with the user's left hand while holding the grip portion 90 with the user's right hand and the lens unit 150 with the user's left hand during image capture.

[0024] The operation unit 70 includes various operating members that serve as an input unit for receiving user operations. The operation unit 70 includes a push button, a rotary dial, and a touch sensor, and includes at least the shutter button 61, a touch panel 70a, a main electronic dial 71, a power switch 72, a sub-electronic dial 73, a four-way key 74, a set button 75, a moving image button 76, an AF-ON button 77, an AE lock button 78, a playback button 79, a menu button 81, a zoom button 82, a multi-controller 83, a focus button 84, and a framing assist button. The line of sight determination function is a function that updates the AF frame to a position based on the gaze point. The digital camera 100 may include a dedicated button for the line of sight determination function, or the line of sight determination function may be assigned to an operating member having another function. The operating members to which the line of sight determination function may be assigned are the AF-ON button 77, the AE lock button 78, the push button of the multi-controller 83, the focus button 84, and the framing assist button. The AF-ON button 77, AE lock button 78, and multi-controller 83 are located in positions that can be operated by the user while simultaneously operating the shutter button 61 with the user's right index finger without interfering with image capture. Specifically, when the user grips the grip 90 with their right hand, the AF-ON button 77, AE lock button 78, and multi-controller 83 are located above the center position (closer to the shutter button 61) of the rear portion of the digital camera 100 (which is on the side opposite the subject), and on the right-hand side of the electronic viewfinder (EVF) 29 (closer to the grip 90). The operating member to which the line of sight determination function can be assigned is not limited to the operating member provided on the rear portion of the digital camera 100 and can be any operating member that can be operated by the user while simultaneously operating the shutter button 61. For example, the line of sight determination function can be assigned to an operating member provided on the front portion (subject side) or the lens unit 150, such as the focus button 84 or the framing assist button. In this case, the location of the operating member is not limited to the back of the digital camera 100 and can be any location where the user can operate the operating member with a finger other than the right index finger that operates the shutter button 61. The operating member to which the user can assign the line of sight determination function and which has another function is a button having a function that does not cause the digital camera 100 to change from the image capture mode when operated while the shutter button 61 is being operated, or a button having a function that does not interfere with the execution of the image capture function by operating the shutter button 61. Furthermore, a button that can be assigned various functions and can be pressed can be employed. Alternatively, a function can be triggered in response to an operation of a leftward and rightward operating bar or a rotatable ring, or a touch panel 70a capable of detecting a pressing force, described below, can be pressed with a strong pressure instead of a button.

[0025] The communication terminal 10 is used by the digital camera 100 to communicate with the (removable) lens unit 150 described below. The eyepiece 16 is the eyepiece of a viewfinder (a see-through viewfinder), and the user can view the video image displayed on the EVF 29 of the viewfinder display unit through the eyepiece 16. The eye proximity detection unit 57 is an eye detection sensor that detects whether the user's eye is placed against the eyepiece 16 when capturing an image. The cover 202 is a cover for the slot that stores the recording medium 200. The grip 90 is a grip unit with a shape that is easily grasped by the user's right hand when the user grasps the digital camera 100. When the user grasps the grip 90 with the pinky, ring, and middle fingers of their right hand, the shutter button 61 and main electronic dial 71 are located in positions that are easily accessible to the user's right index finger. Furthermore, the sub-electronic dial 73 is located in a position that is easily accessible to the right thumb in this position.

[0026] Figure 2 1 is a block diagram showing an example of the configuration of the digital camera 100 according to the present exemplary embodiment. Figure 2 In FIG. 1 , the lens unit 150 is a lens unit on which an interchangeable camera lens is mounted. The lens 103 is generally composed of a plurality of lenses, but in Figure 2 1 is simply shown as a single lens. Communication terminal 6 is used by the lens unit 150 to communicate with the digital camera 100. The lens unit 150 communicates with the system control unit 50 via communication terminal 6 and communication terminal 10, and the lens system control circuit 4 in the lens unit 150 controls the diaphragm 1 via the diaphragm drive circuit 2. The lens unit 150 then adjusts the focus of the lens 103 by shifting the lens 103 via the AF drive circuit 3.

[0027] The shutter 101 is a focal plane shutter that freely controls the exposure time of the imaging unit 22 under the control of the system control unit 50 .

[0028] The imaging unit 22 is an image sensor including a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor, and converts an optical image into an electrical signal. The analog / digital (A / D) converter 23 is used to convert the analog signal output from the imaging unit 22 into a digital signal.

[0029] The image processing unit 24 performs predetermined pixel interpolation, resizing processing such as reduction, and color conversion processing on the data from the A / D converter 23 or the data from the memory control unit 15 described below. Furthermore, the image processing unit 24 uses the captured image data to perform predetermined calculation processing. Based on the calculation results obtained from the image processing unit 24, the system control unit 50 controls exposure and distance measurement. Thus, through-the-lens (TTL) AF processing, AE processing, and pre-flash (EF) processing are performed. The image processing unit 24 further performs predetermined calculation processing using the captured image data and uses the obtained calculation results to perform TTL auto white balance (AWB) processing.

[0030] The memory control unit 15 controls data transmission and reception between the A / D converter 23, the image processing unit 24, and the memory 32. Data output from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data that has been acquired by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as image data to be displayed on the display unit 28 or the EVF 29. The memory 32 has a capacity sufficient to store a predetermined number of still images and a predetermined amount of moving images and audio.

[0031] The memory 32 also functions as a memory (video memory) for image display. Image data for display written to the memory 32 is displayed on the display unit 28 or the EVF 29 via the memory control unit 15. The display unit 28 and the EVF 29 display data on a liquid crystal device (LCD) or an organic electroluminescent (EL) display based on a signal from the memory control unit 15. The data converted from analog data to digital data by the A / D converter 23 and stored in the memory 32 is sequentially transmitted to the display unit 28 or the EVF 29 and displayed for live view display (LV display). Hereinafter, images displayed in live view will be referred to as live view images (LV images).

[0032] The infrared light emitting diode 166 is a light emitting element for detecting the user's line of sight on the screen in the viewfinder, and emits infrared light toward the user's eyeball (eye) 161. The infrared light emitted from the infrared light emitting diode 166 is reflected by the eyeball (eye) 161, and the reflected infrared light travels to the dichroic mirror 162. The dichroic mirror 162 reflects only infrared light and allows visible light to pass through. The reflected infrared light whose optical path has been changed passes through the imaging lens 163 and is imaged on the imaging surface of the line of sight detection sensor 164. The imaging lens 163 is an optical component of the line of sight detection optical system. The line of sight detection sensor 164 includes an image sensor such as a CCD image sensor.

[0033] The line of sight detection sensor 164 photoelectrically converts the incident reflected infrared light into an electrical signal and outputs the electrical signal to the line of sight detection circuit 165. The line of sight detection circuit 165 includes at least one processor. The line of sight detection circuit 165 detects the user's line of sight based on the image or movement of the user's eyeball (eye) 161 based on the signal output from the line of sight detection sensor 164, and outputs the detected information to the system control unit 50. The dichroic mirror 162, the camera lens 163, the line of sight detection sensor 164, the infrared light emitting diode 166, and the line of sight detection circuit 165 form the line of sight detection block 160. The line of sight detection block 160 is one of the receiving units that receives the line of sight input.

[0034] In various embodiments of the present invention, the line of sight is detected by the corneal reflection method using the line of sight detection block 160. The corneal reflection method is a method of detecting the direction of the user's line of sight by detecting the movement of the eye based on the relationship between the infrared light emitted from the infrared light emitting diode 166 and reflected from the eyeball (eye) 161 (particularly the cornea) and the position of the pupil of the eyeball (eye) 161. There are various other line of sight detection methods such as the sclera reflection method that uses the difference in light reflectivity between the black eye and the white eye. Any line of sight detection method that can detect the line of sight other than the above-mentioned method may also be used.

[0035] The out-of-viewfinder display unit 43 displays various setting values of the digital camera 100 including the shutter speed and the aperture via the out-of-viewfinder display unit drive circuit 44 .

[0036] The nonvolatile memory 56 is an electrically erasable and programmable memory, and uses, for example, a flash read-only memory (Flash ROM). The nonvolatile memory 56 stores constants and programs used for the operation of the system control unit 50. The programs here are programs for executing the various flowcharts described below according to this exemplary embodiment.

[0037] The system control unit 50 is a control unit including at least one processor or circuit, and controls the entire digital camera 100. The system control unit 50 executes a program stored in the nonvolatile memory 56 to implement the processing described below according to this exemplary embodiment. The system memory 52 is, for example, a random access memory (RAM), and constants and variables used for the operation of the system control unit 50 and programs read from the nonvolatile memory 56 are loaded onto the system memory 52. In addition, the system control unit 50 controls the display by controlling the memory 32 and the display unit 28.

[0038] The system timer 53 is a time measuring unit that measures time used in various types of control and the time of a built-in clock.

[0039] The mode selection switch 60 is an operating member of the operating unit 70, and switches the operating mode of the system control unit 50 between a still image shooting mode and a moving image shooting mode. The still image shooting mode includes an automatic shooting mode, an automatic scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). The still image shooting mode also includes various scene modes with different shooting settings corresponding to different shooting scenes, as well as a custom mode and the like. The user can use the mode selection switch 60 to directly change the operating mode to any of the aforementioned modes. Alternatively, the user uses the mode selection switch 60 to change to a shooting mode list screen, selects a mode from the multiple modes displayed, and then uses another operating member to change the operating mode to the selected mode. Similarly, the moving image shooting mode may include multiple modes.

[0040] When the shutter button 61 of the digital camera 100 is halfway operated, that is, when the shutter button 61 is half-pressed (image capture preparation instruction), the first shutter switch 62 is turned on and generates a first shutter switch signal SW1. Image capture preparation operations such as AF processing, AE processing, AWB processing, and EF processing are started based on the first shutter switch signal SW1.

[0041] When the shutter button 61 is fully pressed (instructing image capture), the second shutter switch 64 is turned on and generates a second shutter switch signal SW2. Based on the second shutter switch signal SW2, the system control unit 50 starts a series of image processing operations, from reading a signal from the image capture unit 22 to writing a captured image as an image file to the recording medium 200. While the second shutter switch 64 is continuously turned on, the digital camera 100 continuously captures images (continuously captures images) at a speed determined by a predetermined continuous image capture speed.

[0042] The power supply control unit 80 includes a battery detection circuit, a direct current (DC)-DC converter, and a switching circuit for changing the block to which the current is applied. The power supply control unit 80 detects the attachment of the battery, the battery type, and the battery level. In addition, the power supply control unit 80 controls the DC-DC converter based on the detection result and the instruction from the system control unit 50, and feeds a predetermined voltage to each component including the recording medium 200 during a predetermined period of time. The power supply unit 30 includes a primary battery (such as an alkaline battery and a lithium battery), a secondary battery (such as a nickel-cadmium (NiCd) battery, a nickel-metal hydride (NiMH) battery, and a lithium (Li) battery), and an alternating current (AC) adapter.

[0043] The recording medium interface (recording medium I / F) 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card that records captured images and includes a semiconductor memory or a magnetic disk.

[0044] The communication unit 54 is connected via wireless or wired cables and transmits and receives video signals and audio signals. The communication unit 54 can also be connected to a wireless local area network (wireless LAN) and the Internet. In addition, the communication unit 54 can be connected via Bluetooth or Bluetooth Low Energy ( The communication unit 54 can transmit images captured by the camera unit 22 (including live view images) and images recorded in the recording medium 200, and can receive images and various types of information from external devices.

[0045] The posture detection unit 55 detects the posture of the digital camera 100 relative to the direction of gravity. Based on the posture detected by the posture detection unit 55, it is determined whether the image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. The system control unit 50 can add direction information based on the posture detected by the posture detection unit 55 to the image file of the image captured by the imaging unit 22, and can rotate the image to record the rotated image. An acceleration sensor or a gyro sensor can be used as the posture detection unit 55. An acceleration sensor or a gyro sensor can be used as a posture detection unit to detect the movement (pan, tilt, lift, or stationary state) of the digital camera 100.

[0046] The eye approach detection unit 57 is a proximity detection sensor that detects (proximity detection) the approach (eye approach) or separation (eye separation) of the eye (object) 161 relative to the viewfinder's eyepiece 16. Based on the state detected by the eye approach detection unit 57, the system control unit 50 switches the display unit 28 and EVF 29 between display (display state) and non-display (non-display state). More specifically, when at least the digital camera 100 is in the image capture standby state and the display destination selection setting for the live view image captured by the image capture unit 22 is set to the automatic selection setting, if eye approach is not detected, the display unit 28 is set as the display destination, the display on the display unit 28 is turned on, and the display on the EVF 29 is set to the non-display state. If eye approach is detected, the EVF 29 is set as the display destination, the display on the EVF 29 is turned on, and the display on the display unit 28 is set to the non-display state. The eye approach detection unit 57 can use, for example, an infrared proximity sensor and detects the approach of an object to the viewfinder's eyepiece 16, which includes the EVF 29. When an object approaches, infrared light projected from the light-projecting unit (not shown) of the eye proximity detection unit 57 is reflected, and the reflected infrared light is received by the light-receiving unit (not shown) of the infrared proximity sensor. Based on the amount of infrared light received, the distance of the approaching object relative to the eyepiece 16 (the eye proximity distance) is also determined. As described above, the eye proximity detection unit 57 performs eye proximity detection to detect an object approaching the eyepiece 16. According to this exemplary embodiment, the light-projecting unit and light-receiving unit of the eye proximity detection unit 57 are separate devices from the infrared light-emitting diode 166 and the line of sight detection sensor 164. Alternatively, the infrared light-emitting diode 166 can also serve as the light-projecting unit of the eye proximity detection unit 57. Furthermore, the line of sight detection sensor 164 can also serve as the light-receiving unit. If the eye 161 is not in the vicinity (non-proximity state) and an approaching object is detected within a predetermined distance relative to the eyepiece 16, the eye is determined to be in the vicinity. If the eye 161 is in the vicinity (proximity state) and the detected approaching object is separated by a predetermined distance or more, the eye is determined to be separated. The thresholds for eye proximity detection and eye separation detection can be set differently, for example, by setting a hysteresis. Furthermore, after detecting eye proximity, the proximity state continues until eye separation is detected. After detecting eye separation, the non-proximity state continues until eye proximity is detected. The infrared proximity sensor is merely an example, and a different sensor that detects the proximity of an eye or an object as eye proximity may be employed as the eye proximity detection unit 57.

[0047] The system control unit 50 performs one of the operations or states described below based on the output from the visual line detection block 160 .

[0048] The sight line detection block 160 detects a state in which the user's sight line is close to the eyepiece 16 , that is, a state in which a sight line input is received.

[0049] The visual line detection block 160 detects a state in which the eye approaches the visual line of the user of the eyepiece 16 and the user is gazing (described below).

[0050] A state in which the sight line detection block 160 detects the sight line of the user whose eyes approach the eyepiece 16 and thereafter no longer detects the sight line, that is, a state in which the sight line input ends.

[0051] A state in which the sight line detection block 160 does not detect the sight line of the user whose eyes are close to the eyepiece 16 , that is, a state in which no sight line input is received.

[0052] The "user is looking" state refers to a state in which the gaze detection block 160 identifies a position (gaze point) on the EVF 29 based on the detected gaze, and determines that the movement amount of the identified gaze point does not exceed a predetermined amount within a predetermined time. Specifically, the gaze point refers to a position on the EVF 29 (on the display unit) that is determined to be the position that the user is looking at.

[0053] The state of “line of sight input ended” is a case where the user's eye 161 is separated from the eyepiece 16 , or a case where the eye 161 remains near the eyepiece 16 but is not detected due to closed eyelids.

[0054] The touch panel 70a and the display unit 28 may be integrated. For example, the touch panel 70a is configured so that light transmittance does not interfere with the display on the display unit 28, and the touch panel 70a is attached to the upper layer of the display surface of the display unit 28. Input coordinates on the touch panel 70a are associated with display coordinates on the display screen of the display unit 28. This provides a display object (graphical user interface) that appears as if the user can directly operate the screen displayed on the display unit 28. The system control unit 50 detects one of the following operations on the touch panel 70a or one of the following states.

[0055] A finger or pen that has not touched the touch panel 70a previously touches the touch panel 70a. Specifically, a touch starts (hereinafter referred to as a "touchdown").

[0056] A state where the touch panel 70a is being touched with a finger or a pen (hereinafter referred to as "touch-on").

[0057] A state in which the finger or pen touching the touch panel 70a is moved (hereinafter referred to as "touch movement").

[0058] The finger or pen touching the touch panel 70a is separated. Specifically, the touch is ended (hereinafter referred to as "touch off").

[0059] A state in which the touch panel 70a is not touched (hereinafter referred to as "untouched").

[0060] When a touch is detected, touch persist is also detected. After a touch is detected, touch persist is generally detected until touch cessation is detected. A state where touch movement is detected also means touch persist is detected. Even if touch persist is detected, touch movement is not detected unless the touch position moves. After all touched fingers or pens have stopped touching, the state changes to untouched.

[0061] The aforementioned operation or status, as well as the coordinates of the position on the touch panel 70a touched by the finger or pen, are notified to the system control unit 50 via the internal bus. Based on the notified information, the system control unit 50 determines which operation (touch operation) has been performed on the touch panel 70a. Regarding touch movement, based on the change in the position coordinates, the movement direction of the finger or pen on the touch panel 70a can be determined for each vertical component and each horizontal component on the touch panel 70a. When a touch movement of a predetermined distance or greater is detected, it is determined that a sliding operation has been performed. The operation of suddenly moving the finger touching the touch panel 70a a certain distance and then releasing the finger is called a "flick." In other words, a flick is an operation of sliding a finger quickly on the touch panel 70a in a manner similar to flicking the surface of the touch panel 70a. When a touch movement of a predetermined distance or greater at a predetermined speed or higher is detected, and then the touch stops, it is determined that a flick has been performed (determined to have been performed after the sliding operation). In addition, a touch operation of touching multiple locations (for example, two points) at the same time and bringing the touched locations closer to each other will be referred to as a "pinch-in", and a touch operation of moving the touched locations away from each other will be referred to as a "pinch-out". Pinch-out and pinch-in will be collectively referred to as a "pinch operation" (or simply "pinch"). The touch panel 70a can be a touch panel of any of various methods such as the following: a resistive film method, a capacitive method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method. Some methods detect touch based on touching the touch panel, while some other methods detect touch based on the proximity of a finger or a pen to the touch panel, but any of these methods may be employed.

[0062] As AF settings, set Figure 5AItems 501 and 502 in the AF area (AF frame setting method) are shown. As shown in item 501, one of "Single-point AF" and "Entire Area AF" is set as the AF area (AF frame setting method). The single-point AF setting indicates that the AF execution target position in the LV image is a single point (one location). The user can set a single point as the AF execution target position, and AF is performed at the display position of the single-point AF frame displayed based on the user's instruction to perform AF. For example, when shooting a train traveling toward the user, the user may wish to perform AF on the train operator's seat and capture an image of the train with a composition that includes the entire train. In this case, with the AF frame fixed at a single point (one location), shooting is performed when the train operator's seat reaches that position, thereby capturing an image with the user's desired composition and focus on the user's desired location. On the other hand, the entire area AF setting shown in item 502 enables AF execution (focus detection) to be performed on the entire AF executable area displayed on the EVF 29 of the digital camera 100. When imaging a subject with unpredictable movement, the AF area setting is set to the entire area so that AF is performed on the subject regardless of where the subject is in the AF executable area.

[0063] like Figure 5A As shown in item 502 in the AF operation setting, the tracking setting is set to either "ON" or "OFF." The tracking function set to "ON" is suitable for shooting in situations where the tracking target is a subject with a continuously changing (moving) shooting distance, such as an athlete in a game, a moving child, or an animal. Tracking and continuous AF are performed on the AF target subject determined using the element conditions. Continuous AF means that AF is continuously performed on the tracking target subject.

[0064] The tracking function set to "off" is suitable for imaging of a still subject, and determines the AF frame position based on element conditions (priorities) described below when the first shutter switch 62 is turned on. At this time, the subject is not tracked.

[0065] When the tracking function is set to "On", if a person's face is detected from the LV image, that face is prioritized as the AF target subject. If multiple people's faces are detected, for example, one face is selected based on one of the following predetermined priority levels (conditions) and is set as the AF target subject.

[0066] Face is the face of a detected person.

[0067] ·The face is large in size.

[0068] The face is located closer to the digital camera 100 (closer to the side).

[0069] The face is located close to the center of the image.

[0070] The face is a face of a person registered in advance.

[0071] In a case where a person's face is not detected, a subject other than the face is selected based on, for example, one of the following predetermined priority levels (conditions), and is set as the AF target subject.

[0072] A subject that is close to the digital camera 100 (closer to the side).

[0073] Subjects with high contrast.

[0074] Subjects with high priority, such as animals or vehicles.

[0075] The subject is a moving object.

[0076] When the user specifies a tracking target subject, the tracking target subject is set as the AF target subject. Specifically, at least one of the above-mentioned element conditions is used as an example of the priority of the two groups for weighting, and the subject with a score greater than or equal to a predetermined threshold, or the subject with the highest score, is determined to be the AF target subject.

[0077] There are four modes of setting combinations of items 501 and 502 that can be set as AF settings. The user uses different settings for different imaging target subjects or imaging situations to set the user's optimal imaging settings, thereby performing imaging at a desired timing with a desired composition.

[0078] exist Figure 5A In the [Image capture] section, the AF area is set to "Single-point" AF, and tracking is set to "Off." This indicates that the user wishes to fix the AF frame to a single point. This setting combination often improves user operability when shooting still subjects such as flowers, for example.

[0079] Figure 5A An example of a settings menu screen is shown. By switching the line of sight input function in item 503 between "On" and "Off," it is possible to select whether to detect the user's line of sight, that is, whether to drive the line of sight detection block 160. Specifically, by switching the line of sight input function in item 503 between "On" and "Off," it is possible to set whether to accept line of sight input operations.

[0080] When the user selects item 504, the screen changes to Figure 5B The screen shown ( Figure 5A (the bottom screen of the setting menu screen in ). Figure 5B A screen showing detailed settings for the line of sight AF function can be set. The line of sight AF function of item 505 is a function of performing AF to focus on a subject displayed at a gaze point. When item 505 is set to "on", a subject is identified based on the movement of the user's line of sight, and AF is performed on the identified subject, so that the user can intuitively perform AF on a desired subject. Item 506 is a setting item related to whether to display a line of sight pointer at a gaze point. When item 506 is set to "on", the user can view the display of a line of sight pointer indicating a gaze point detected by the line of sight detection block 160 based on the direction in which the user's eyes are looking. According to this exemplary embodiment, the line of sight pointer is displayed as shown. Figure 6D The pointer 606 in FIG. displays a small circle centered on the gaze point, and a circle larger than the small circle is displayed around the small circle. Figure 6D In the display, the small circle of pointer 606 is displayed with a shaded interior, while the larger circle is displayed with a shaded frame. Each shaded portion is semi-transparent to prevent the subject from being completely obscured from view when the shaded portion is superimposed on the subject or AF frame in the LV image. Similarly, since the area between the small circle and the larger circle is not shaded, visibility of the LV image is not lost.

[0081] Item 507 is a setting item regarding whether to use the operation of turning on the first shutter switch SW1 (62) (i.e., the operation of half-pressing the shutter button 61) as the operation of determining the gaze point (line of sight determination operation). If item 507 is enabled, the point detected by the line of sight detection block 160 when the user half-presses the shutter button 61 is determined as the gaze point. If item 507 is disabled, the gaze point cannot be determined by half-pressing the shutter button 61.

[0082] For example, the sight line determination function is assigned to the AF-ON button 77 during initial setup (at factory shipment). If item 507 is set to "Enable," the sight line determination operation is performed by pressing the shutter button 61 to turn on the first shutter switch 62, rather than pressing the AF-ON button. Similarly, the sight line determination operation can be assigned to a button on the operation unit 70 that is assigned another function. The following example discusses a case where, after the user has determined the gaze point using a button with another function, they half-press the shutter button 61. The gaze point determined using the other button is different from the gaze point when the shutter button 61 was half-pressed. In this case, since the gaze point when the shutter button 61 was half-pressed can be determined, AF is performed on the subject at that gaze point (which is different from the user's desired gaze point determined using the button with the other function), resulting in poor operability. Therefore, item 507 on the settings menu screen is provided to allow the user to set whether the gaze point is determined by turning on the first shutter switch (SW1) 62.

[0083] The following will refer to Figure 3 and Figures 4A1 to 4A3 The control according to the first exemplary embodiment is explained by the flowchart shown in FIG. Figure 3 In step S302 (imaging mode processing), as a result of the line of sight determination operation, frames (items) such as the single-point AF frame and the subject frame displayed before the user's line of sight determination operation are displayed at the gaze point.

[0084] Figure 3 1 is a flowchart showing control processing by the digital camera 100 according to the present exemplary embodiment. The system control unit 50 implements this control processing by expanding the program stored in the nonvolatile memory 56 onto the system memory 52 and executing the expanded program. When the digital camera 100 is turned on (started up), the system starts Figure 3 Flowchart of Figure 3 If the camera mode process is to be executed in the flowchart ("Yes" in step S301), the process starts. Figures 4A1 to 4A3 In addition, Figures 6A to 6J It shows that Figure 3 FIG. 1 is a diagram showing an example of a display on the EVF 29 in the case of the control processing shown in the flowchart. 6A to 6I It is shown in Figure 5A 5 is a diagram of an example of display in a case where the AF area setting of the item 501 is set to “single point” and the tracking function setting of the item 502 is set to “on”. Figure 6J is a diagram showing an example of display in a case where the AF area setting is set to “entire area” and the tracking function is set to “off”.

[0085] In step S301, the system control unit 50 determines whether the imaging mode is set. If the imaging mode is set ("YES" in step S301), the process proceeds to step S302. If the imaging mode is not set ("NO" in step S301), the process proceeds to step S303.

[0086] In step S302, the system control unit 50 performs image capture mode processing. Figures 4A1 to 4A3 The camera mode processing will be described.

[0087] In step S303, the system control unit 50 determines whether the playback mode is set. If the playback mode is set ("Yes" in step S303), the process proceeds to step S304. If the playback mode is not set ("No" in step S303), the process proceeds to step S305.

[0088] In step S304 , the system control unit 50 performs a reproduction mode process. The reproduction mode process is a control process for reproducing a captured image, and by operating the operation unit 70 , the image reproduced on the display unit 28 can be fed or enlarged.

[0089] In step S305, the system control unit 50 performs other processing based on the determinations in steps S301 and S303. The other processing is, for example, processing for performing settings related to the digital camera 100 and imaging on a setting menu screen.

[0090] In step S306, the system control unit 50 determines whether to end the process. In the case of ending the process ("Yes" in step S306), Figure 3 The control process shown in the flowchart ends. If the process is not ended (NO in step S306), the process returns to step S301. Ending the process means, for example, turning off the digital camera 100.

[0091] Figures 4A1 to 4A3 The above is shown as Figure 3 Flowchart of the camera mode processing described in step S302. Figure 3 If it is determined in step S301 that the camera mode is set ( Figure 3 ), the processing of these flowcharts is started.

[0092] In step S401 , the system control unit 50 initializes flags and control variables.

[0093] In step S402, the system control unit 50 performs a control operation based on the signal acquired by the imaging unit 22, such as Figure 6A As shown in FIG. 6 , an LV image 601 is displayed on the EVF 29. The LV image display layer is as follows: Figure 7A and Figure 7B The layer 705 is shown. Specifically, according to the present exemplary embodiment, the LV image 601 is displayed on the layer with the lowest position (lowest priority level) among the layers displayed on the EVF 29.

[0094] In step S403, the system control unit 50 displays various types of imaging information (imaging parameters) related to imaging mode processing, such as various setting values, battery charge, and the number of recording pixels. Figure 6A As shown in FIG. 6 , information 602 is displayed on the EVF 29 together with the LV image 601. Figure 7A As shown in layer 703, imaging information such as information 602 is displayed on a layer higher than the LV image display layer.

[0095] In step S404, the system control unit 50 determines whether the area in which AF can be performed (hereinafter referred to as the "AF performable area") is smaller than the area determined by the main body of the digital camera 100. If the AF performable area is smaller ("YES" in step S404), the process proceeds to step S405. If the AF performable area is not smaller ("NO" in step S404), the process proceeds to step S406. The AF performable area is uniquely defined based on the performance of the main body of the digital camera 100, but depending on the type of lens unit 150 attached to the digital camera 100, the AF performable area may become smaller than the AF performable area determined by the performance of the main body of the digital camera 100. When a specific lens, such as a super-telephoto single-focus lens, is attached, a smaller AF performable area than the AF performable area determined by the performance of the main body of the digital camera 100 may be set. This is because sufficient light for AF execution may not be available in the area near the lens periphery with the super-telephoto single-focus lens. Therefore, when such a lens is mounted as the lens unit 150 on the digital camera 100, an AF-enabled area is set that is smaller than the area determined by the main body of the digital camera 100. While the user may be aware of the AF-enabled area determined by the main body of the digital camera 100, since the user may use various types of lenses when capturing images, when the user changes the lens mounted on the digital camera 100, the user may not be aware of the extent of the AF-enabled area determined by the currently mounted lens. This may result in a situation where the AF-enabled area, based on the performance of the digital camera 100, is the entire EVF 29, but this AF-enabled area is reduced as a result of mounting a specific lens, potentially confusing the user. Unless the user visually recognizes the reduced AF area, the user may not realize that AF is not enabled for the specific area, and the user may miss a photographic opportunity. To prevent this loss of opportunity, an AF-enabled area determination is performed in step S404, and in the subsequent step S405, the AF-enabled area is displayed. According to the present exemplary embodiment, the AF executable area depending on the main body of the digital camera 100 is the entire EVF 29 (aspect ratio: 100%×100%).

[0096] In step S405, the system control unit 50 displays an indicator indicating the AF executable area. Since the determination result of step S404 is "yes", it is understood that the attachment of a specific lens makes the AF executable area smaller than the AF executable area depending on the main body of the digital camera 100. Therefore, AF can be executed only in an area smaller than the entire area of the EVF 29. In order to notify the user of this situation, the AF executable area is superimposed and displayed on the LV image. Figure 6BThe AF executable area is displayed so as to be visually recognizable to the user as shown in a frame 603. The display form is not limited to a frame display such as the frame 603, and may be a display form in which an area outside the AF executable area is shaded gray.

[0097] In step S406, the system control unit 50 determines whether the AF area setting is set to an item other than "entire area". Figure 5A and Figure 5B As described above, the AF area setting is to set whether the entire area or a single point is set as the area to perform AF. If the AF area setting is set to an item other than "entire area", that is, a single point ("YES" in step S406), the process proceeds to step S407, and if the AF area setting is set to "entire area" ("NO" in step S406), the process proceeds to step S408.

[0098] In step S407 , the system control unit 50 displays a single-point AF frame. Figure 6B The frame 604 is an example of this display. By operating the direction key of the four-way key 74, the single-point AF frame can be moved to a position desired by the user.

[0099] In step S408, the system control unit 50 determines whether the AF tracking setting is set to "ON." If the tracking setting is set to "ON" ("YES" in step S408), the process proceeds to step S409, whereas if the tracking setting is set to "OFF" ("NO" in step S408), the process proceeds to step S411.

[0100] In step S409, the system control unit 50 determines whether a subject has been detected in the LV image acquired by the imaging unit 22. If a subject has been detected ("Yes" in step S409), the process proceeds to step S410. If a subject has not been detected ("No" in step S409), the process proceeds to step S411. Subject detection refers to, for example, the detection of a person's face, body, or organs (such as eyes). Subjects are not limited to people; animals, birds, or vehicles can also be detected.

[0101] In step S410 , the system control unit 50 displays a subject frame on the subject detected in step S409 . Figure 6CFrame 605 is an example of this display. Since the determination result in step S408 is "Yes," the user can attempt to capture an image of a moving subject. Therefore, a subject frame is displayed over the subject detected from the LV image by the system control unit 50, thereby indicating that the subject has been detected and allowing the user to easily visually identify the subject. According to this exemplary embodiment, the subject frame is displayed in a manner that includes triangle brackets at each of the four corners of the detected subject's position, surrounding the area where the subject is detected.

[0102] In step S411, the system control unit 50 determines whether the line of sight input function is set to "on". In the case where the line of sight input function is set to "on" ("yes" in step S411), the processing proceeds to step S412, and in the case where the line of sight input function is set to "off" ("no" in step S411), the processing proceeds to step S430. As described above, the line of sight input function is set by the user on the setting menu screen as desired. Without changing to the setting menu screen, the line of sight input function can be switched between "on" and "off" when a dedicated button for switching the line of sight input function is pressed or when a button to which the line of sight input function can be assigned is pressed.

[0103] In step S412, the system control unit 50 determines whether the user's line of sight has been detected using the line of sight detection block 160. If the line of sight has been detected ("Yes" in step S412), the process proceeds to step S413, while if the line of sight has not been detected ("No" in step S412), the process proceeds to step S430. According to this exemplary embodiment, if the user's eye is not near the eyepiece 16, the user's line of sight is not detected.

[0104] In step S413, the system control unit 50 determines whether the gaze pointer display setting for the gaze pointer indicating the gaze point is set to "ON." If the gaze pointer display setting is set to "ON" ("YES" in step S413), the process proceeds to step S414. If the gaze pointer display setting is set to "OFF" ("NO" in step S413), the process proceeds to step S415.

[0105] In step S414 , the system control unit 50 displays a gaze pointer at the detected gaze point. Figure 6D The pointer 606 is an example of this display. Figure 6D , a frame 605 is displayed above the detected subject, and a frame 604 indicating a single-point AF frame is displayed. The user is viewing a display position of a pointer 606 near the lower right of the LV image 601. Figure 6EAn example of display is shown in the case where the area the user is viewing changes so that the gaze point moves to a position near the display position of the frame 605. A line of sight pointer 606 is displayed on a display layer lower than the frame 604.

[0106] In step S415, the system control unit 50 determines whether the user has performed a line of sight determination operation. If a line of sight determination operation has been performed ("Yes" in step S415), processing proceeds to step S416. If a line of sight determination operation has not been performed ("No" in step S415), processing proceeds to step S427. A line of sight determination operation refers to an operation on a dedicated button having a line of sight determination function or an operation on a button assigned to the line of sight determination function. The line of sight determination function determines the gaze point detected during the line of sight determination operation and invalidates the user's line of sight detected after the determination and the gaze point calculated based on the line of sight detected after the determination. Displaying a frame display (item) such as an AF frame on the EVF 29 that follows the detected gaze point may be irritating to the user. Even if the user does not intend to move their line of sight, since the frame display (item) such as the AF frame follows the moved line of sight, processing such as AF may be performed at an unintended location. Therefore, before the sight line determination operation is performed, the frame display does not move to the gaze point and remains displayed at or above the position / subject determined based on the AF area setting, tracking setting, and the above-mentioned priority, and in the case of the sight line determination operation, the frame display moves to the gaze point and is displayed at the gaze point. Since the AF frame moves from a certain position to the gaze point and is displayed at the gaze point based on the sight line determination operation, the AF frame can be moved quickly and intuitively even when the AF frame position currently displayed on the EVF 29 and the user's desired position (gaze point) are far apart from each other. In addition, the user's intention is accurately read and reflected based on the user's sight line determination operation. Therefore, control such as AF execution is not performed at a gaze point that the user does not desire.

[0107] When the frame display before the line of sight determination operation is disabled (for example, the AF area setting is set to "entire area" and the tracking setting is set to "Off"), when the line of sight determination operation is performed, the display is controlled to display the frame display at the determined gaze point.

[0108] In step S416, similar to step S413, the system control unit 50 determines whether the sight pointer display setting is set to "ON." If the sight pointer display setting is set to "ON" ("YES" in step S416), the process proceeds to step S417. If the sight pointer display setting is set to "OFF" ("NO" in step S416), the process proceeds to step S426.

[0109] In step S417, as in step S408, the system control unit 50 determines whether the AF tracking setting is set to "ON." If the AF tracking setting is set to "ON" ("YES" in step S417), the process proceeds to step S418, whereas if the AF tracking setting is set to "OFF" ("NO" in step S417), the process proceeds to step S423.

[0110] In step S418, as in step S412, the system control unit 50 determines whether the user's line of sight is detected using the line of sight detection block 160. If the line of sight is detected ("YES" in step S418), the process proceeds to step S419, whereas if the line of sight is not detected ("NO" in step S418), the process proceeds to step S427.

[0111] In step S419, the system control unit 50 determines whether the specific subject is at the gaze point detected from the LV image in step S418. If the specific subject is at the detected gaze point ("YES" in step S419), the process proceeds to step S420, while if the specific subject is not at the detected gaze point ("NO" in step S419), the process proceeds to step S422.

[0112] In step S420, the system control unit 50 determines the subject determined to be at the gaze point in step S419 as the tracking target and performs continuous AF on the tracking target subject. In this case, even if the gaze point and the subject's positions do not completely coincide, if the gaze point is detected within a predetermined range relative to the subject, the system control unit 50 determines that the gaze point is above the subject, and AF on the tracking target subject continues.

[0113] In step S421 , the tracking AF frame for the subject determined as the tracking target in step S420 is superimposed and displayed on the line-of-sight pointer 606 . Figure 6FFrame 607 is an example of this display. The tracking AF frame indicates the position where continuous AF is performed on the subject determined based on the above-mentioned element conditions. The display format of the tracking AF frame includes double triangle brackets at each of the four corners of the position where continuous AF is performed, surrounding the area of the tracking target subject. If the result of the judgment in step S415 is "Yes," that is, if the user performs a line of sight confirmation operation, the tracking AF frame is displayed in a display format different from the frame (subject frame) 605 and frame (AF frame) 604 displayed before the line of sight confirmation operation. This allows the user to recognize that the tracking target subject was determined by the user's line of sight confirmation operation and to clearly visually identify the subject determined as the tracking target. The subject frame is simply a display to inform the user whether a specific subject (e.g., a person's face) is in the LV image displayed on the EVF 29 (whether the subject is detected). If multiple subjects are detected, multiple subject frames are displayed. In contrast, multiple tracking AF frames are not displayed in the LV image (a single tracking AF frame is displayed), and the display of the tracking AF frame indicates that the subject at the displayed position of the tracking AF frame will be focused upon pressing the shutter button 61. In other words, the subject frame displayed in step S410 and the tracking AF frame displayed in step S421 are different in nature. The subject frame is displayed by the system control unit 50 over a subject identified as a specific subject in the LV image, but does not indicate that the subject is a candidate for AF execution target. AF is not performed at the displayed position of the subject frame. In contrast, the tracking AF frame indicates that the subject has been identified as a specific subject by the system control unit 50 and is an AF execution target. Furthermore, continuous AF is performed at the displayed position of the tracking AF frame. Specifically, from the perspective of the importance of the user identifying the AF execution target, the tracking AF frame is more important to the user. Therefore, since the tracking AF frame is displayed over the user's gaze point (the position where the line of sight is detected) through the line of sight determination operation, the tracking AF frame quickly moves to the desired position at the user's desired timing.

[0114] In step S422, the system control unit 50 displays a multi-point AF frame at the display position of the sight pointer (i.e., the gaze point detected based on the user's sight line). The multi-point AF frame is an AF frame displayed at a position with the highest score obtained in the judgment based on the element condition when no subject is detected in the LV image displayed on the EVF 29. The display form of the multi-point AF frame is as follows: Figure 6JA small rectangle is displayed at a position where the obtained score is high, as shown in the frame 610 of FIG. 4 . In the case where the multi-point AF frame is displayed as in step S422, frames such as the AF frame and the subject frame are not displayed in the LV image until the line of sight confirmation operation is performed in step S415. Specifically, after the line of sight confirmation operation is performed, the multi-point AF frame is displayed as a frame display. The multi-point AF frame is displayed on a display layer higher than the display layer of the line of sight pointer. Since the multi-point AF frame is displayed as described above, when an AF execution instruction is issued, the user can visually check that AF will be performed on the subject for which the multi-point AF frame is displayed. Therefore, the user can determine the next operation, such as an operation to immediately issue an AF execution instruction, an operation to change the position of the AF frame, or an operation to change the composition or camera settings.

[0115] In step S423, the system control unit 50 determines whether the AF area setting is set to "entire area." If the AF area setting is set to "entire area" ("YES" in step S423), the process returns to step S418. If the AF area setting is not set to "entire area," that is, if the AF area setting is set to "single point" ("NO" in step S423), the process proceeds to step S424.

[0116] In step S424, as in steps S412 and S418, the system control unit 50 determines whether the user's line of sight is detected using the line of sight detection block 160. If the line of sight is detected ("YES" in step S424), the process proceeds to step S425, and if the line of sight is not detected ("NO" in step S424), the process proceeds to step S427.

[0117] In step S425, the system control unit 50 moves the single-point AF frame to the display position of the sight pointer, that is, the gaze point. The single-point AF frame is a frame indicating the position of a single point to be focused when an AF execution instruction is issued. The single-point AF frame is displayed in a manner such as Figure 6B The single-point AF frame is displayed in a display format such as the display format of the frame 604 in the image. When the user performs a sight confirmation operation, the single-point AF frame is moved from the display position set before the sight confirmation operation to the gaze point. Therefore, even if the display position before the sight confirmation operation is far away from the gaze point, the single-point AF frame is moved quickly. As a result, camera image loss is reduced and an image focused on the user's desired subject is captured and recorded. The single-point AF frame is displayed on a display layer higher than the display layer of the sight pointer 606.

[0118] In step S426, the system control unit 50 displays a frame display such as a tracking AF frame, a multi-point AF frame, or a single-point AF frame at the gaze point based on the details of the settings of items 501 and 502. Since the judgment result in step S416 is "No", the line of sight pointer is not displayed on the EVF 29. However, since the judgment results in steps S411 and S412 are both "Yes", the user's line of sight is detected using the line of sight detection block 160. As in steps S421, S422, and S425, the AF frame is moved quickly even in a case where the display position of the frame before the line of sight determination operation is far from the gaze point. In addition, even in a case where the frame is not displayed before the line of sight determination operation is performed, since the AF frame is quickly displayed at the gaze point, the position of the AF frame is intuitively determined.

[0119] In step S427, the system control unit 50 determines whether an AF execution instruction has been issued. If an AF execution instruction has been issued ("YES" in step S427), the process proceeds to step S428. If no AF execution instruction has been issued ("NO" in step S427), the process proceeds to step S430. The AF execution instruction is issued by, for example, pressing the AF-ON button 77 or turning on the first shutter switch 62. The AF execution instruction is not limited to those described above and may be an instruction issued by an operation assigned to a function that issues an instruction to execute AF.

[0120] In step S428, the system control unit 50 performs AF on the subject at the display position of the AF frame displayed above the LV image on the EVF 29. Figure 6H An example of this display is shown in . Figure 6H is Figure 6F Example of display when an AF execution instruction is issued in the state shown in FIG. 4 (Yes in step S427). When AF is executed on the subject at the AF frame based on the AF execution instruction, the display form of the AF frame changes from [0, 1, 2] to [1, 3] based on the focusing result. Figure 6F The display form of the frame 607 in the image is changed to Figure 6H At this time, if the sight pointer display setting is enabled, the frame 608 is displayed on a display layer higher than the display layer of the sight pointer. Figure 6G If the AF execution instruction in step S427 is issued at the display position of the frame 604 of the single-point AF frame displayed as shown, AF is executed on the subject below the frame 604. Then, the display form of the single-point AF frame changes from

[0054] to

[0055] based on the focusing result. Figure 6B The display form of the frame 604 in the image is changed to Figure 6IThe display form of the frame 609 in the AF frame is changed from the state set before AF execution or based on the focusing result (focusing success and focusing failure) to notify the user of the focus position and focusing result. Specifically, AF is performed on the subject at the display position of the AF frame, and in the case of successful focusing, Figure 6H The frame 608 in the image is displayed in green, while in the case of unsuccessful focusing, the frame 608 is displayed in gray. In addition, in the case of successful focusing, a beep sound is generated, while in the case of unsuccessful focusing, no sound is generated. In the case of issuing an AF execution instruction when the AF area setting is set to "entire area" and the tracking function setting is set to "off", a beep sound is generated. Figure 6J The multi-point AF frame is displayed in the form of a frame 610 in the image. The frame 610 is displayed after the user issues an AF execution instruction, and the frame is not displayed until the AF execution instruction is issued. Specifically, if the AF area setting is set to "Entire Area" and the tracking function setting is set to "Off," the AF frame is not displayed until the user issues an AF execution instruction.

[0121] In step S429, the system control unit 50 determines whether the AF execution instruction determined in step S427 was issued using the first shutter switch 62. If it is determined that the AF execution instruction was issued using the first shutter switch 62 ("YES" in step S429), the process proceeds to step S433. Otherwise ("NO" in step S429), the process proceeds to step S430.

[0122] In step S430, the system control unit 50 determines whether the first shutter switch 62 is on. If the first shutter switch 62 is on ("YES" in step S430), the process proceeds to step S431. If the first shutter switch 62 is not on ("NO" in step S430), the process proceeds to step S437. The first shutter switch 62 is on when the shutter button 61 is halfway pressed, as described above. Specifically, the user is expected to capture an image.

[0123] In step S431, the system control unit 50 determines whether the focus mode is set to AF mode. If the focus mode is set to AF mode ("YES" in step S431), the process proceeds to step S432. Otherwise (if the focus mode is set to manual focus (MF) mode) ("NO" in step S431), the process proceeds to step S434. The AF mode and MF mode are switched via the setting menu screen or using a switch provided on the exterior of the lens unit 150.

[0124] In step S432 , as in step S428 , the system control unit 50 performs AF processing based on the display position of the AF frame.

[0125] In step S433 , the system control unit 50 performs other imaging preparation processing such as AE and AWB.

[0126] In step S434, the system control unit 50 determines whether the second shutter switch 64 is on. If the second shutter switch 64 is on, that is, if the shutter button 61 is fully pressed ("YES" in step S434), the process proceeds to step S435. Otherwise ("NO" in step S434), the process proceeds to step S436.

[0127] In step S435 , the system control unit 50 performs a series of image capturing processes until the captured image is recorded as an image file in the recording medium 200 .

[0128] In step S436, the system control unit 50 determines whether the first shutter switch 62 is still on. If the first shutter switch 62 is still on (YES in step S436), the process returns to step S431, whereas if the first shutter switch 62 is no longer on (NO in step S436), the process proceeds to step S437.

[0129] In step S437, the system control unit 50 determines whether to end the image capture mode. In the case of ending the image capture mode ("YES" in step S437), Figures 4A1 to 4A3 The control flow graph in ends and processing returns to Figure 3 On the other hand, if the image capture mode is not ended (NO in step S437), the process returns to step S403. The image capture mode is ended by, for example, turning off the digital camera 100 or pressing the menu button 81 to change to the setting menu screen.

[0130] As described above, according to the first exemplary embodiment, in a case where an operation can be performed by line of sight input, when the user performs a line of sight confirmation operation, the AF frame is moved to the gaze point and displayed at the gaze point. Since AF frame tracking that follows the movement of the user's line of sight is not performed before the line of sight confirmation operation, the position of the AF frame does not move to follow the movement of the gaze point that the user does not expect. Therefore, the user is less likely to feel annoyed when checking the LV image. In addition, even in a case where the display position of the AF frame before the line of sight confirmation operation is far away from the position desired by the user, when the user performs the line of sight confirmation operation with the line of sight directed toward the desired position, the AF frame can be quickly moved to the gaze point. In addition, since a frame display such as the AF frame is displayed on a display layer higher than the display layer of the line of sight pointer when the line of sight pointer is displayed at the gaze point, the user can clearly recognize the position of the AF frame and can also easily recognize that the position of the AF frame has moved to the position pointed by the line of sight.

[0131] (Second embodiment)

[0132] The following will be described according to the second exemplary embodiment: Figure 3 In step S302 (camera mode processing), the display order of the frame display such as the tracking AF frame, the multi-point AF frame or the single-point AF frame as the display object and the line of sight pointer indicating the gaze point is changed before and after the line of sight determination operation. Figure 3 as well as Figures 4B1 to 4B3 The control according to the second example embodiment is explained with reference to the control flow chart in FIG. Figure 3 Similar to the first exemplary embodiment. Figure 3 In the case of the camera mode processing in the flowchart (the judgment result in step S301 is "yes"), the process starts. Figures 4B1 to 4B3 In the second exemplary embodiment, the flow chart of Figures 4B1 to 4B3 The flowchart according to the first exemplary embodiment Figures 4A1 to 4A3 The steps in the flowchart are redundant descriptions of similar steps.

[0133] In step S404, if the AF executable area is smaller than the area depending on the subject of the digital camera 100 (YES in step S404), the process proceeds to step S451. In step S451 according to the present exemplary embodiment, the system control unit 50 sets the AF executable area ( Figure 6B 603 in the figure) is displayed on a layer higher than the LV image and lower than the camera information. Figure 7A As shown, the AF executable area is displayed on a layer 704 that is higher than a layer 705 of the LV image and lower than a layer 703 of the imaging information.

[0134] In step S452, the system control unit 50 sets the single-point AF frame ( Figure 6B The frame 604 in FIG. 1 is displayed on a layer higher than the imaging information (imaging parameters). Specifically, Figure 7A As shown, the single-point AF frame is displayed on a layer 702 higher than a layer 703 of imaging information.

[0135] In step S453, the system control unit 50 sets the subject frame ( Figure 6C The frame 605 in step S409 is displayed above the subject detected in step S409. At this time, the subject frame is displayed on a layer higher than the image information. Specifically, Figure 7A As shown, similar to the single-point AF frame, the subject frame is displayed on a layer 702 higher than the layer 703 of the imaging information.

[0136] In step S454, the system control unit 50 displays the sight line pointer indicating the gaze point on a layer higher than the frame display such as the single-point AF frame or the subject frame. Figure 7A As shown, the gaze pointer is displayed on a layer 700 that is higher than the layer 702. Similarly, even in the case where the gaze point moves as a result of the user changing the viewing direction, the gaze pointer is displayed on a layer that is higher than the frame display.

[0137] Specifically, before the user performs the line of sight determination operation in step S415 , the line of sight pointer is displayed on a layer higher than the frame display such as the single-point AF frame or the subject frame.

[0138] According to this exemplary embodiment, in addition to setting the sight input function to "on" or "off", the user can also set the sight pointer display setting to "on" or "off". Figure 5B In a case where the sight pointer display setting of item 506 in is enabled, it can be considered that the user wishes to detect the viewing direction of the user to view the sight pointer indicating the gaze point identified based on the detected viewing direction. Therefore, before performing the sight determination operation and moving the frame display, the display of the sight pointer is prioritized and displayed on a layer higher than the frame display. In addition, the sight pointer according to the present exemplary embodiment is displayed in a display form that allows the user to check the LV image even in a state where the sight pointer is superimposed and displayed on the LV image as described above. Therefore, even in a case where the sight pointer is displayed on a layer higher than the frame display, the sight pointer is less likely to completely hide the display position of the frame display and is less likely to prevent camera preparation.

[0139] In step S456, the tracking AF frame is displayed on the tracking target object determined in step S420. Since this operation is performed after the sight line determination operation ("Yes" in step S415), Figure 7B As shown, the tracking AF frame is displayed on a layer 712 higher than the line of sight pointer layer 700. Specifically, according to this exemplary embodiment, before the user performs the line of sight determination operation in step S415, the frame display is displayed on a layer lower than the line of sight pointer (in Figure 7A On the contrary, after the user performs the sight line determination operation in step S415, a frame display is displayed on a layer higher than the sight line pointer (in Figure 7B (displayed in a display box in layer 712).

[0140] When the user performs a sight line determination operation, it can be considered that the user fully recognizes the gaze point corresponding to the user's viewing direction, and further, wishes to move (shift) the frame display to the gaze point and display the frame display at the gaze point. In addition, the display position of the frame display such as the single-point AF frame or the subject frame is a very important position, and the user should check it before executing the shooting instruction. Therefore, the frame display displayed and moved based on the sight line determination operation is displayed on a layer higher than the sight line pointer. This makes it easy for the user to view the frame display moved by the sight line determination operation and, for example, recognize the subject that will be the AF target when the AF execution instruction is issued.

[0141] In step S457, the system control unit 50 moves the multi-point AF frame to the detected gaze point based on the "No" judgment result in step S419. At this time, the multi-point AF frame is displayed on a layer higher than the gaze pointer. Similar to the tracking AF frame in step S456, the multi-point AF frame is also displayed on a layer higher than the gaze pointer. Specifically, Figure 7B As shown, the multi-point AF frame is displayed on a layer 712 higher than the layer 700 of the line-of-sight pointer.

[0142] In step S458, the system control unit 50 moves the single-point AF frame displayed on a layer lower than the line of sight pointer before the line of sight determination operation to the gaze point indicated by the line of sight pointer, and superimposes and displays the single-point AF frame on a layer higher than the line of sight pointer. Specifically, Figure 7B As shown, the single-point AF frame is displayed on a layer 712 higher than the line-of-sight pointer layer 700. Similar to the tracking AF frame in step S456 and the multi-point AF frame in step S457, the frame display of the single-point AF frame is moved to the detected gaze point by the line-of-sight confirmation operation and displayed on a layer higher than the line-of-sight pointer. This makes it easier for the user to visually recognize that the single-point AF frame has moved to the gaze point by the line-of-sight confirmation operation, and to identify the position of the subject for which AF is to be executed if an AF execution instruction is issued.

[0143] If it is determined in step S416 that the line of sight pointer display setting is set to “off”, then in step S459 , the system control unit 50 does not display the line of sight pointer but displays the AF frame on a higher layer than the imaging information.

[0144] As described above, according to the present exemplary embodiment, before the user performs a line of sight determination operation, the line of sight pointer indicating the gaze point is controlled to be superimposed and displayed on a layer higher than the AF frame, and after the line of sight determination operation is performed, the AF frame is controlled to be superimposed and displayed on a layer higher than the line of sight pointer. Therefore, before the user performs the line of sight determination operation, the user can easily visually identify the line of sight pointer indicating the gaze point corresponding to the user's viewing direction, so in the case where the user performs the line of sight determination operation, the user can predict to which position the frame display will move (shift). On the other hand, after the line of sight determination operation is performed, the user can easily identify the position of the frame display moved by the line of sight determination operation, and this makes it easy for the user to check the target of camera-related processing such as AF.

[0145] The present exemplary embodiment is also applicable to a case where, in electronic devices other than digital cameras such as PCs, tablet terminals, and smartphones, objects such as text files and folders are to be selected by a line-of-sight input operation.

[0146] Figure 8 and 9A to 9D : is a diagram showing a display example or a structure in a case where the present exemplary embodiment is applied to a PC. Figure 8 is a diagram showing a structure in the case where the present exemplary embodiment is applied to a PC, and 9A to 9D : is an example showing a display screen in the case where the present exemplary embodiment is applied to a PC. Figure 8 , a touch panel 805 corresponding to the touch panel 70a according to this exemplary embodiment is shown. A sight line detection device 810, which is a separate unit corresponding to the sight line detection block 160 for use in a PC, is connected to the PC. A camera 811 of the sight line detection device 810 and a built-in camera 812 of the PC are used to detect the user's eye movement and the user's eye direction (sight line). Based on the detected sight line, the gaze point on the display screen is identified.

[0147] exist Figure 9A , an icon 901 representing a text file, an icon 902 representing a folder, an icon 910, a folder name 903, a mouse cursor 904, and a sight pointer 905 are displayed. The sight pointer 905 represents a gaze point, and the display of the sight pointer 905 allows the user to identify the gaze point identified by the sight detection device 810 based on the user's viewing direction. At this time, the mouse cursor 904 is superimposed and displayed on the icon 910, and thus detailed information 906 related to the folder represented by the icon 910 is displayed. At this time, the sight pointer 905 is superimposed and displayed on the mouse cursor 904 and the detailed information 906. Specifically, Figure 9A The order in which the objects are displayed is as follows Figure 7AShown from the top are a layer 700 of a gaze pointer 905, a layer 701 of a mouse cursor 904, and a layer 702 of detailed information 906.

[0148] This will explain the user's line of sight from Figure 9A The position of the sight pointer 905 in the Figure 9B The position of the sight pointer 907 (on the icon 902) in the Figure 9B When the sight line confirmation operation is performed in the state shown, Figure 9C As shown, the mouse cursor 904 moves (shifts) to the display position of the sight pointer 907. As described above, in the case where the mouse cursor 904 moves to the display position of the sight pointer 907, the mouse cursor 904 moves to the icon 902 representing the folder, and detailed information 906 related to the display object such as the folder or file on which the mouse cursor 904 is superimposed and displayed is displayed. Specifically, detailed information 906 related to the object selected with the mouse cursor 904 as a result of executing a specific function executed by moving the mouse cursor 904 in response to the user's sight determination operation is displayed. At this time, the detailed information 906 is superimposed and displayed on a layer higher than the sight pointer 907. Specifically, Figure 9C The order in which the objects are displayed is as follows Figure 7B Shown are layers 711 of mouse cursor 904, layer 712 of detailed information 906, and layer 700 of gaze pointer 907, starting from the top.

[0149] If there is no display object at the position where the mouse cursor 904 is moved, detailed object information is not displayed. Detailed information 906 is displayed after a predetermined time of about 1 second from when the mouse cursor 904 is moved to the position of the icon 902 that is the display object. Figure 9C The detailed information 906 is not displayed in the image, but after a predetermined time from when the mouse cursor 904 is moved. Figure 9D Detailed information 906 is displayed in the . The sight line determination operation on the PC is not a button operation as described above with reference to the digital camera 100, but rather, for example, a condition where the annotation state continues for a predetermined time, a touch operation on the touch panel 805, a click operation on the mouse, or a predetermined operation on the keyboard. Therefore, the user can perform the sight line determination operation more naturally and intuitively.

[0150] According to the present exemplary embodiment, detailed information as a result of execution of the line of sight determination operation is displayed in a display layer (eg, Figure 7B712 in the image). Optionally, the order in which the mouse cursor 904 and the detailed information are displayed can be switched. Furthermore, the mouse cursor 904 and the detailed information can be on the same layer. Specifically, after the sight line confirmation operation, the detailed information will be displayed on a layer higher than the sight line pointer.

[0151] As described above, before the user confirms their gaze, the gaze pointer indicating the gaze point is superimposed and displayed on the detailed information, allowing the user to easily visually identify the gaze point corresponding to the user's viewing direction. On the other hand, after the gaze confirmation operation, the detailed information is superimposed and displayed on the gaze pointer, allowing the user to easily review the detailed information without being annoyed by the gaze pointer. Furthermore, after the gaze confirmation operation, the mouse cursor 904 is superimposed and displayed on the gaze pointer and the detailed information, allowing the user to visually recognize that the mouse cursor 904 has quickly moved a long distance to reach the gaze point in response to the gaze confirmation operation.

[0152] although Figure 8 A PC according to another exemplary embodiment is shown. This exemplary embodiment can also be implemented by performing a line of sight determination operation using a mouse, pointing device, or joystick instead of the touchpad 805 of the PC. Specifically, when a predetermined operation such as a click operation, a press operation, an operation of shaking the device horizontally, or an operation of drawing a circle is performed, it is determined that the line of sight determination operation is being performed. The touchpad, mouse, pointing device, and joystick do not need to be built into the PC and can be built into external hardware.

[0153] The display format of the gaze pointer is not limited to those described above. Instead of a combination of an inner circle (with shading) and an outer circular frame, a form including an inner circle, an outer circular frame, or a circle whose size can be changed as needed by the user can be used. For example, the color of the gaze pointer can be changed based on the time the gaze point is viewed. For example, if the gaze point is viewed for longer than a predetermined time, the color is red, while if the viewing time is shorter than the predetermined time, the color is blue.

[0154] A means for the user to set the display form may be provided. As the display form changes, the display order of the display layers may change.

[0155] Optionally, when the user performs a line of sight confirmation operation, the frame 604 representing the single-point AF frame moves to the position of the line of sight pointer 606, and then the line of sight pointer 606 is hidden. For example, while the line of sight confirmation operation is continued (pressing the first shutter switch 62), that is, during the execution of AF, the line of sight pointer 606 is hidden, and when the AF execution is completed, the line of sight pointer 606 is displayed again. The aforementioned control makes it easier to visually identify the display of the frame 604 representing the single-point AF in a situation where the user may want to check the subject at the AF execution position on the LV image immediately after the line of sight confirmation operation. In addition to the time during AF execution, the display can also be controlled to hide the line of sight pointer 606 until a predetermined time set by the user has passed. When processing enters with Figure 3 In the case of a mode processing different from the camera mode processing in step S302 of the control unit 100, the frame display such as the AF frame, etc. displayed on a layer higher than the line of sight pointer by the user's confirmation operation is reset to the display order before the confirmation operation. Specifically, the display order is reset when the user presses the menu button 81 to enter the setting menu mode processing, or when the digital camera 100 is turned off. The trigger for resetting the display order changed by the confirmation operation to the display order set before the confirmation operation is not limited to those described above, and in the case where a predetermined time has passed after the confirmation operation, the display order can be controlled to be reset and returned to the display order set before the confirmation operation.

[0156] In addition, even in Figures 4A1 to 4A3 During the processing of the control flow chart in FIG. 1 , even when the operation unit 70 is operated to move the AF frame or change various imaging parameters, the function corresponding to that operation is executed. For example, when the multi-controller 83 is pushed in any of eight directions, the AF frame moves based on the direction of operation. When the main electronic dial 71 is operated, the function assigned to the main electronic dial 71 (specifically, shutter speed change) is executed.

[0157] The present invention is also achieved by performing the following processing. Specifically, the software (program) for realizing the functions of the above-mentioned example embodiments is supplied to a system or device via a network or a recording medium, and the computer (or central processing unit (CPU) or microprocessing unit (MPU)) of the system or device reads the program code and executes the read program code. In this case, the program and the recording medium storing the program constitute the present invention.

[0158] The various types of control described above as being performed by the system control unit 50 may be performed by a single piece of hardware, or a plurality of pieces of hardware (eg, a plurality of processors or circuits) may share the processing to control the entire apparatus.

[0159] Furthermore, while the above exemplary embodiments illustrate the application of the present invention to a digital camera, the disclosed example is not restrictive, and the present invention can be applied to any electronic device in which a user can specify a position through line-of-sight input and operation of an operating member. Specifically, the present invention can be applied to PCs, personal digital assistants (PDAs), mobile phone terminals, mobile image viewers, and head-mounted displays. Furthermore, the present invention can also be applied to digital photo frames, music players, game consoles, e-book readers, tablet terminals, smartphones, projection equipment, home appliances including displays, and in-vehicle devices including displays.

[0160] In addition, although various exemplary embodiments of the present invention have been described in detail above, the present invention is not limited to specific exemplary embodiments and includes various forms within the scope and spirit of the present invention. Although the touch panel 70a is described as an example of a position movement indication member for use in combination with the line of sight input, another operating unit such as a button or a dial may be used. In addition, although the display position is indicated by the AF frame, an icon frame or a parameter setting frame may be used, or an indicator display other than the AF frame such as the mouse cursor 904 may be used. Although whether to display the line of sight pointer is determined based on the user setting, the determination may be based on whether the line of sight input function is set to "on" or "off" ( Figure 5A and Figure 5B 503 in ), or based on whether there is an eye approaching the eyepiece 16.

[0161] Furthermore, the present invention can be applied not only to the main body of an imaging device but also to a control device that communicates with an imaging device (including a webcam) via wired or wireless communication and remotely controls the imaging device. Examples of devices that remotely control an imaging device include smartphones, tablet PCs, and desktop PCs. The control device remotely controls the imaging device by notifying the imaging device of commands for various operations and settings based on operations and processing performed on the control device. Furthermore, a live view image captured by the imaging device can be received via wired or wireless communication and displayed on the control device.

[0162] The present invention is also achieved by performing the following processing. Specifically, the software (program) for realizing the functions of the above-mentioned example embodiments is supplied to a system or device via a network or a recording medium, and the computer (or central processing unit (CPU) or microprocessing unit (MPU)) of the system or device reads the program code and executes the read program code. In this case, the program and the recording medium storing the program constitute the present invention.

[0163] According to the present invention, when an operation is performed using line of sight input, the position specified by the gaze point (specified position) is displayed in such a manner that the specified position is more easily visually recognized after the user performs a confirmation operation than before the confirmation operation. This makes it easier to recognize the difference between the user's viewing position and the gaze point before the specified position is determined, and also makes it easier to check the determined specified position.

[0164] Other embodiments

[0165] The embodiments of the present invention can also be implemented by the following method, that is, providing software (program) that performs the functions of the above-mentioned embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0166] Although exemplary embodiments have been described, it is to be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.

Claims

1. An electronic device comprising: a detection unit configured to detect a user's viewing line of sight on the display unit; as well as a control unit configured to display a pointer indicating a gaze point on the display unit, the gaze point being based on the line of sight detected by the detection unit, and to move and display the pointer following movement of the gaze point based on the line of sight detected by the detection unit, When the user operates the operation unit, the control unit controls to display the display item at the gaze point based on the line of sight detected by the detection unit, and displays the pointer below the display item in a manner such that the pointer and the display item overlap each other.

2. The electronic device according to claim 1, wherein The control unit displays the display item at a position based on a predetermined condition before the user operates the operation unit, and controls to move and display the display item to the gaze point when the user operates the operation unit.

3. The electronic device according to claim 1, wherein Before a user operates the operation unit, the control unit controls to display the pointer below the display item in such a manner that the pointer and the display item overlap each other when the display item is displayed at a position based on a predetermined condition.

4. The electronic device according to claim 1, wherein Before the user operates the operation unit, the control unit controls so that when the display item is displayed at a position based on a predetermined condition, the pointer is displayed above the display item in such a manner that the pointer and the display item overlap with each other, and when the user operates the operation unit, the control unit controls so that the pointer is displayed below the display item in such a manner that the pointer and the display item overlap with each other.

5. The electronic device according to claim 1, wherein The control unit controls not to display the display item before the user operates the operation unit, and controls to display the display item at the display position of the pointer when the user operates the operation unit.

6. The electronic device according to claim 1, in, The electronic device is an imaging device configured to perform imaging processing using imaging parameters, and The control unit further controls to display information related to the imaging parameters on the display unit, and to display the pointer on the information in a manner that the pointer and the information related to the imaging parameters overlap with each other.

7. The electronic device according to claim 1, wherein The display item indicates a position where automatic focus adjustment is performed.

8. The electronic device according to claim 1, wherein The information about the position selected with the display item represents a result of automatic focus adjustment performed at the position where the display item is displayed.

9. The electronic device according to claim 1, wherein Information related to the position selected using the display item is displayed instead of the displayed display item.

10. The electronic device according to claim 1, wherein When the user holds the grip of the electronic device with his right hand, the operating unit is arranged at a position that is on the opposite side of the subject side and closer to the side of the shutter button configured to issue a shooting instruction at a central position on the surface relative to the opposite side, and is on the side closer to the grip.

11. The electronic device according to claim 1, wherein: The operation unit is a shutter button configured to issue an image capturing instruction.

12. The electronic device according to claim 1, further comprising: Viewfinder; as well as an imaging unit configured to capture an image of a subject, The display unit is a display unit inside the viewfinder.

13. The electronic device according to claim 1, wherein When the user operates the operation unit, after the display item moves and is displayed to the gaze point, the control unit controls so that regardless of the display setting related to the pointer set by the user, the pointer is not displayed during the period when the user issues an autofocus execution instruction, i.e., an AF execution instruction.

14. The electronic device according to claim 1, wherein The display item is a mouse cursor.

15. The electronic device according to claim 1, wherein The information about the position selected with the display item is information about an object at the position where the display item is displayed.

16. A control method for controlling an electronic device, the control method comprising: detecting a user's viewing line of sight on the display unit; as well as Control is performed to display a pointer indicating a gaze point on the display unit, the gaze point being based on a detected line of sight, and the pointer being moved and displayed following the movement of the gaze point based on the detected line of sight, wherein, when a user operates an operating unit, control is performed to display a display item at the gaze point based on the detected line of sight, and the pointer is displayed below the display item in a manner such that the pointer and the display item overlap with each other.

17. A computer-readable recording medium storing a program for causing a computer to execute a method for controlling an electronic device, the method comprising: detecting a user's viewing line of sight on the display unit; as well as Control is performed to display a pointer indicating a gaze point on the display unit, the gaze point being based on a detected line of sight, and the pointer being moved and displayed following the movement of the gaze point based on the detected line of sight, wherein, when a user operates an operating unit, control is performed to display a display item at the gaze point based on the detected line of sight, and the pointer is displayed below the display item in a manner such that the pointer and the display item overlap with each other.

18. A computer program product comprising computer program instructions, which implement the control method according to claim 16 when executed by a computer.

Images