Electronic device, control method of electronic device, and computer-readable medium

Abstract

The present invention relates to an electronic device, a control method for an electronic device, and a computer-readable medium. According to a movement operation performed on an operation unit, an indicator is moved from a first position based on a line-of-sight input to a second position, the operation unit receiving a user operation different from the line-of-sight input, (a) without performing an instruction operation for executing a specific process at the position of the indicator, calibration of the input position according to the line of sight based on the first position and the second position is not performed, and (b) in a state where there is no additional movement operation and a specific condition is satisfied, when an instruction operation for executing the specific process is performed, calibration of the input position according to the line of sight is performed based on the first position and the second position.

Description

Technical Field

[0001] The present invention relates to an electronic device, and more particularly to an electronic device capable of detecting a line of sight. Background Art

[0002] There are known electronic devices that are operated by a user's line of sight (hereinafter referred to as line-of-sight input). In particular, in a case where a user wants to immediately indicate an operation of an electronic device such as a digital camera and a game console, input of the line of sight is efficient.

[0003] In order to perform line-of-sight input with high precision, calibration (hereinafter referred to as CAL) is required. Although the precision of line-of-sight input is improved in a case where CAL is performed multiple times, generally, it is not convenient to perform CAL multiple times because each CAL operation requires effort. In order to solve this problem, a technique of performing CAL correction as needed during normal use of an electronic device (automatic CAL correction) has been proposed.

[0004] For example, Japanese Patent Application Laid-Open No. 2015-207290 discloses a technique of generating CAL correction data by calculating a deviation from a line-of-sight position based on a touch position / cursor position.

[0005] However, in the technique disclosed in Japanese Patent Application Laid-Open No. 2015-207290, a configuration is provided in which CAL correction is performed every time a touch position / cursor position is moved. Therefore, even when a user operates without intending to perform fine adjustment, CAL correction is performed, and thus the precision of line-of-sight input deteriorates. Summary of the Invention

[0006] The present invention provides an electronic device capable of performing calibration, so that the precision of line-of-sight input can be more reliably improved during normal use.

[0007] The electronic device according to the present invention includes: a line-of-sight input unit configured to receive a line-of-sight input, which is an input based on a position of a user's line of sight; and a control unit configured to perform control such that an indicator is displayed at a first position based on the line-of-sight input, and the indicator is moved from the first position to a second position according to a movement operation performed on an operation unit that receives a user operation different from the line-of-sight input, (a) calibration based on the first position and the second position of an input position according to the line of sight is not performed in a case where an instruction operation for performing a specific process is not performed at the position of the indicator, and (b) calibration based on the input position according to the line of sight is performed based on the first position and the second position in a state where there is no additional movement operation and a specific condition is satisfied and an instruction operation for performing the specific process is performed.

[0008] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A and 1B are external views of a digital camera;

[0010] Figure 2 is a block diagram of the digital camera;

[0011] Figures 3A to 3H is a diagram illustrating the setting of an AF frame according to the line of sight in single-point AF;

[0012] Figures 4A to 4H is a diagram illustrating the setting of an AF frame according to the line of sight in face + tracking priority AF;

[0013] Figure 5A and 5B is a flowchart of shooting mode processing;

[0014] Figure 6 is a flowchart of camera setting processing;

[0015] Figure 7 is a flowchart of touch operation response processing;

[0016] Figure 8 is a flowchart of relative position specifying processing when the line of sight is enabled;

[0017] Figure 9A and 9B is a flowchart of touch movement processing during single-point AF;

[0018] Figure 10A and 10B is a flowchart of touch movement processing during face + tracking priority AF; and

[0019] Figures 11A to 11C is a display example of a setting menu screen. DETAILED DESCRIPTION

[0020] External View of Digital Camera 100

[0021] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1A and 1B are external views of a digital camera 100, which is an example of a device to which the present invention can be applied. Figure 1A is a front perspective view of the digital camera 100, and Figure 1B is a back perspective view of the digital camera 100.

[0022] The display unit 28 is provided on the back surface of the digital camera 100 and displays images and various information. The touch panel 70a can detect a touch operation performed on the display surface (touch operation surface; touch operation member) of the display unit 28. The display unit 43 other than the viewfinder is provided on the top surface of the digital camera 100 and displays various setting values of the digital camera 100 including the shutter speed and the aperture. The shutter button 61 is an operation member for giving a shooting instruction (imaging instruction). The mode switching switch 60 is an operation member for switching between various modes. The terminal cover 40 is a cover for protecting a connector (not shown) that connects the digital camera 100 to an external device.

[0023] The main electronic dial 71 is a rotary operation member, and the setting values such as the shutter speed and the aperture are changed by rotating the main electronic dial 71. The power switch 72 is an operation member for switching between turning on and off the power supply of the digital camera 100. The sub-electronic dial 73 is a rotary operation member, and the movement of a selection box (cursor) and image transfer are performed by rotating the sub-electronic dial 73. The four-way key 74 is configured such that its upper part, lower part, left part, and right part can be pushed respectively, and processing corresponding to the pushed part of the four-way key 74 can be performed. The setting button 75 is a push button and is mainly used for determining a selected item or the like. The multi-controller (hereinafter referred to as MC) 65 can receive direction indications in eight directions and a center part push operation.

[0024] The moving image button 76 is used to indicate the start or stop of moving image shooting (recording). The AE lock button 77 is a push button, and by pushing the AE lock button 77 in the shooting standby state, the exposure state can be fixed. The zoom button 78 is an operation button for switching between on and off of the zoom mode in the live view display (LV display) as the shooting mode. After the zoom mode is set to on, the main electronic dial 71 can be operated to magnify or reduce the live view image (LV image). In the playback mode, the zoom button 78 serves as an operation button for magnifying the playback image or increasing its magnification. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 in the shooting mode, the camera can be switched to the playback mode, and the latest image among the images recorded on the recording medium 200 (to be described later) is displayed on the display unit 28. The menu button 81 is a push button for performing an indication operation for displaying the menu screen. When the menu button 81 is pressed, the menu screen is displayed on the display unit 28, and various settings can be made through this menu screen. The user can intuitively make various settings using the menu screen, 4-way key 74, setting button 75, or MC65 displayed on the display unit 28. The line-of-sight confirmation button 82 is an operation member included in the operation unit 70 and is a push button for indicating the selective execution or cancellation of the subject based on the position of the line-of-sight pointer (to be described later). The line-of-sight confirmation button 82 is arranged at a position where it can be easily manipulated even when the user is in the state of looking into the viewfinder (the state where the user's eyes are close to the eyepiece unit 16), and is set at a position where it can be manipulated by the thumb of the right hand that is holding the grip portion 90.

[0025] The communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the lens unit 150 (to be described later and is detachable). The eyepiece unit 16 is the eyepiece of the eyepiece viewfinder (internal viewfinder), and the user can visually recognize the video displayed on the internal EVF 29 (to be described later) through the eyepiece unit 16. The eyepiece detection unit 57 is an eyepiece detection sensor that detects whether the user's (photographer's) eyes are close to the eyepiece unit 16. The lid 202 is the lid of the slot that stores the recording medium 200 (to be described later). The grip portion 90 is a holder having a shape that can be easily held by the right hand when the user is ready to use the digital camera 100. The shutter button 61 and the main electronic dial 71 are arranged at positions where they can be manipulated by the index finger of the right hand in the state where the grip portion 90 is held by the little finger, ring finger, and middle finger of the right hand to hold the digital camera 100. In addition, in the same state, the sub-electronic dial 73 and the line-of-sight confirmation button 82 are arranged at positions where they can be manipulated by the thumb of the right hand.

[0026] Configuration block diagram of digital camera 100

[0027] Figure 2 is a block diagram showing an example of the configuration of digital camera 100. The lens unit 150 is a lens unit to which an interchangeable photographing lens is attached. Although the lens 103 generally consists of a plurality of lenses, Figure 2 only one lens is shown. The communication terminal 6 is a communication terminal for communication between the lens unit 150 and the digital camera 100 side, and the communication terminal 10 is a communication terminal for communication between the digital camera 100 and the lens unit 150 side. The lens unit 150 communicates with the system control unit 50 via these communication terminals 6 and 10. In addition, the lens unit 150 controls the aperture 1 via the aperture drive circuit 2 according to the lens system control circuit 4 provided therein. Further, the lens unit 150 focuses by shifting the lens 103 via the AF drive circuit 3 according to the lens system control circuit 4.

[0028] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 according to the control of the system control unit 50.

[0029] The imaging unit 22 is an imaging element configured as a CCD or CMOS element, etc., and the imaging unit 22 converts an optical image into an electrical signal. The imaging unit 22 may have an imaging surface phase difference sensor that outputs focus amount information to the system control unit 50.

[0030] The image processing unit 24 performs predetermined processing (such as pixel interpolation, size adjustment processing (such as reduction, etc.), color conversion processing, etc.) on the data from the A / D converter 23 or the data from the memory control unit 15. In addition, the image processing unit 24 performs predetermined arithmetic operation processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the arithmetic operation result obtained by the image processing unit 24. Therefore, through-the-lens (TTL) type autofocus (AF) processing, automatic exposure (AE) processing, and pre-flash (EF) processing, etc. are performed. In addition, the image processing unit 24 performs predetermined arithmetic operation processing using the captured image data, and performs TTL type automatic white balance (AWB) processing based on the obtained arithmetic operation result.

[0031] The memory control unit 15 controls the transmission and reception of data among the A / D converter 23, the image processing unit 24, and the memory 32. The output data from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, the output data from the A / D converter 23 is written into the memory 32 via the memory control unit 15 without passing through the image processing unit 24. The memory 32 stores the image data acquired by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as the image data to be displayed on the display unit 28 or the EVF 29. The memory 32 has a storage capacity sufficient to store a predetermined number of still images, as well as moving images and sounds within a predetermined time period.

[0032] In addition, the memory 32 also serves as a memory (video memory) for image display. The image data for display written into the memory 32 is displayed by the display unit 28 or the EVF 29 via the memory control unit 15. Each of the display unit 28 and the EVF 29 performs display on a display device such as an LCD or an organic EL in response to a signal from the memory control unit 15. The live view (LV) display can be performed in such a manner that the data A / D-converted by the A / D converter 23 and stored in the memory 32 is sequentially transmitted to the display unit 28 or the EVF 29 and displayed on the display unit 28 or the EVF 29. Hereinafter, the image displayed by the live view display will be referred to as a live view image (LV image).

[0033] The line-of-sight detection unit 160 (reception unit) detects the line of sight of a user's eye approaching the eyepiece unit 16 toward the EVF 29. The line-of-sight detection unit 160 includes a dichroic mirror 162, an imaging lens 163, a line-of-sight detection sensor 164, a line-of-sight detection circuit 165, and an infrared emitting diode 166.

[0034] The infrared emitting diode 166 is a light-emitting element for detecting the position of the user's line of sight in the viewfinder screen, and radiates infrared light onto the user's eyeball (eye) 161. The infrared light radiated from the infrared emitting diode 166 is reflected from the eyeball (eye) 161, and the infrared reflected light reaches the dichroic mirror 162. The dichroic mirror 162 only reflects infrared light and transmits visible light. The infrared reflected light with a changed optical path is imaged on the imaging surface of the line-of-sight detection sensor 164 by the imaging lens 163. The imaging lens 163 is an optical member constituting the line-of-sight detection optical system. The line-of-sight detection sensor 164 is constituted by an imaging device such as a CCD type image sensor.

[0035] The line-of-sight detection sensor 164 photoelectrically converts the incident infrared reflected light into an electrical signal and outputs the electrical signal to the line-of-sight detection circuit 165. The line-of-sight detection circuit 165 detects the user's line-of-sight position based on the output signal of the line-of-sight detection sensor 164 according to the movement of the user's eyeball (eye) 161 and outputs the detected information to the system control unit 50 and the fixation determination unit 170.

[0036] Based on the detection information received from the line-of-sight detection circuit 165, when the period during which the user's line of sight is fixed to a certain area exceeds a predetermined threshold, the fixation determination unit 170 determines that the user is fixating on that area. Therefore, this area can be referred to as the fixation position (fixation area), which is the position where fixation is taking place. On the other hand, "the line of sight is fixed to a certain area" may, for example, mean that the average position of the movement of the line of sight is within that area and the dispersion (variance) is less than a predetermined value until a predetermined period has elapsed. In addition, the system control unit 50 can arbitrarily change the predetermined threshold. Additionally, the fixation determination unit 170 may not be provided as an independent block, and the system control unit 50 can perform the same function as that of the fixation determination unit 170 based on the detected information received from the line-of-sight detection circuit 165.

[0037] In the present embodiment, the line-of-sight detection unit 160 uses a method (mode) called the corneal reflection method to detect the line of sight. The corneal reflection method is a method of detecting the direction and position of the line of sight based on the positional relationship between the reflected light obtained when infrared light emitted from the infrared emission diode 166 is reflected from the eyeball (eye) 161 (specifically, the cornea) and the pupil of the eyeball (eye) 161. On the other hand, the mode of detecting the line of sight (the direction and position of the line of sight) is not particularly limited, and a mode other than the above mode can be used. For example, a method (mode) called the heterochromatic boundary tracking method can be used, and the heterochromatic boundary tracking method uses the fact that the iris and sclera of the eye have different light reflectivities.

[0038] The display unit 43 other than the viewfinder displays various setting values of the camera including the shutter speed and aperture through the drive circuit 44 for the display unit other than the viewfinder.

[0039] The non-volatile memory 56 is an electrically erasable and recordable memory and can be, for example, a flash ROM or the like. The non-volatile memory 56 records constants for the operation of the system control unit 50, as well as programs and the like. Here, the program refers to a program for executing the various flowcharts to be described in the present embodiment.

[0040] The system control unit 50 is a control unit composed of at least one processor or circuit, and controls the entire digital camera 100. The system control unit 50 implements the respective processes of the present embodiment described later by executing the program recorded in the non-volatile memory 56. The system memory 52 can be, for example, a RAM, and the system control unit 50 expands constants and variables for the operation of the system control unit 50, as well as programs read from the non-volatile memory 56, etc. in the system memory 52. In addition, the system control unit 50 performs display control by controlling the memory 32, the display unit 28, etc.

[0041] The system timer 53 is a clock unit that measures the time for various types of control and the time of the embedded clock.

[0042] The power control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching the blocks supplied with power, etc., and detects the presence or absence of an installed battery, the battery type, the remaining battery level, etc. In addition, the power control unit 80 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies the required voltage to each unit including the recording medium 200 during the required period. The power supply unit 30 is composed of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery or a Li battery, and an AC adapter, etc.

[0043] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording the captured images, and is composed of a semiconductor memory, a magnetic disk, etc.

[0044] The communication unit 54 transmits / receives video signals and audio signals to / from an external device connected wirelessly or through a wired cable. The communication unit 54 can be connected to a wireless local area network (LAN) and the Internet. In addition, the communication unit 54 can communicate with an external device via Bluetooth (registered trademark) and Bluetooth Low Energy. The communication unit can transmit the images (including LV images) captured by the imaging unit 22 and the images recorded in the recording medium 200, and receive image data and various other types of information from the external device.

[0045] The posture detection unit 55 detects the posture of the digital camera 100 relative to the direction of gravity. Based on the posture detected by the posture detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 is an image captured by horizontally holding the digital camera 100 or an image captured by vertically holding the digital camera 100. The system control unit 50 can add posture information to the image file of the image captured by the imaging unit 22 or rotate the image in response to the posture detected by the posture detection unit 55, and record the image. As the posture detection unit 55, an acceleration sensor or a gyro sensor or the like can be used. An acceleration sensor or a gyro sensor can also be used as the posture detection unit 55 to detect the movement (panning, tilting, lifting, or whether it stops, etc.) of the digital camera 100.

[0046] The eyepiece detection unit 57 is an eyepiece detection sensor that detects the approach (eye approach) and separation (eye separation) (approach detection) of the eye (object) 161 relative to the eyepiece portion 16 of the eyepiece viewfinder (hereinafter simply referred to as "viewfinder"). The system control unit 50 switches between the display (display state) / non-display (non-display state) of the display unit 28 and the EVF 29 in response to the state detected by the eyepiece detection unit 57. More specifically, at least in the shooting standby state and when the switching of the display destination is automatically switched, when the eye is not approaching the eyepiece portion 16, the display unit 28 is set as the display destination and is turned on, and the EVF 29 is not displayed. In addition, when the eye approaches the eyepiece portion 16, the EVF 29 is set as the display destination and is turned on for display, and the display unit 28 is not displayed. For example, an infrared proximity sensor can be used as the eyepiece detection unit 57, and it can detect the approach of any object relative to the eyepiece portion 16 of the viewfinder including the EVF 29. When the object gets closer, the infrared rays projected from the light projection portion (not shown) of the eyepiece detection unit 57 are reflected from the object and received by the light receiving portion (not shown) of the infrared proximity sensor. The distance (eyepiece distance) between the object and the eyepiece portion 16 can be determined based on the amount of the received infrared rays. In this way, the eyepiece detection unit 57 performs eyepiece detection for detecting the approach distance of the object to the eyepiece portion 16. When an object within a predetermined distance from the eyepiece portion 16 that approaches the eyepiece portion 16 is detected in the non-eyepiece state (non-approach state), it is considered that the eye approach is detected. When an object that has been detected as approaching the eyepiece portion 16 in the eye approach state (approach state) separates from the eyepiece portion 16 by at least a predetermined distance, it is considered that the eye separation is detected. For example, the threshold for detecting the eye approach and the threshold for detecting the eye separation can be different by setting hysteresis. In addition, the state after the eye approach is detected is considered as the eye approach state until the eye separation is detected. The state after the eye separation is detected is considered as the non-eye approach state until the eye approach is detected. On the other hand, the infrared proximity sensor is an example, and other sensors can be used as the eyepiece detection unit 57 as long as they can detect the approach of the eye or object that is considered as the eye approach.

[0047] By controlling the line-of-sight detection unit 160, the system control unit 50 can detect the following line-of-sight states toward the EVF 29.

[0048] · The state where the line of sight that was not directed at the EVF 29 newly points at the EVF 29. That is, the start of the line-of-sight input.

[0049] · The state where the line-of-sight input is performed for the EVF 29.

[0050] ·The state where a certain position of the EVF 29 is being gazed at.

[0051] ·The state where the line of sight directed at the EVF 29 has moved away. That is, the input of the line of sight has ended.

[0052] ·The state where no line of sight input is made to the EVF 29 (the state where the EVF 29 is not being observed).

[0053] The system control unit 50 is notified of these operations / states and the position (direction) of the line of sight directed at the EVF 29 via the internal bus, and determines the ongoing line of sight input based on the notified information.

[0054] The operation unit 70 is an input unit that receives inputs of operations from the user (user operations), and is used to input various operation instructions to the system control unit 50. As Figure 2 shown, the operation unit 70 includes a mode switching switch 60, a shutter button 61, a power switch 72, a touch panel 70a, etc. In addition, as other operation members 70b, the operation unit 70 includes a main electronic dial 71, a sub - electronic dial 73, a 4 - way key 74, a setting button 75, a moving image button 76, an AE lock button 77, a zoom - in button 78, a playback button 79, a menu button 81, and an MC 65, etc.

[0055] The mode switching switch 60 switches the operation mode of the system control unit 50 to any one of a still - image shooting mode, a moving - image shooting mode, a playback mode, etc. As the modes included in the still - image shooting mode, there are an auto - shooting mode, an auto - scene judgment mode, a manual mode, an aperture - priority mode (Av mode), a shutter - speed - priority mode (Tv mode), and a program AE mode (P mode). In addition, there are various scene modes for shooting settings for each shooting scene, and a custom mode, etc. The user can directly switch the operation mode to any one of these modes via the mode switching switch 60. Alternatively, the shooting - mode list screen can be temporarily switched via the mode switching switch 60, and then any one of the multiple displayed modes can be selectively switched using other operation members. Similarly, the moving - image shooting mode can also include multiple modes.

[0056] The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on by a so-called half-press (shooting preparation instruction) to generate a first shutter switch signal SW1 during the operation of the shutter button 61. The system control unit 50 starts shooting preparation operations such as autofocus (AF) processing, autoexposure (AE) processing, auto white balance (AWB) processing, and pre-flash emission (EF) processing according to the first shutter switch signal SW1. When the operation of the shutter button 61 is completed, the second shutter switch 64 is turned on by a so-called full-press (shooting instruction) to generate a second shutter switch signal SW2. According to the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processing operations from reading the signal from the imaging unit 22 to writing the captured image as an image file to the recording medium 200.

[0057] The touch panel 70a and the display unit 28 can be integrally configured. For example, the touch panel 70a can be configured such that its light transmittance does not interfere with the display of the display unit 28 and is attached to the upper layer of the display surface of the display unit 28. Then, the input coordinates in the touch panel 70a are associated with the display coordinates on the display surface of the display unit 28. Therefore, a graphical user interface (GUI) can be provided as if the user can directly operate the screen displayed on the display unit 28.

[0058] The system control unit 50 can detect the following operations or states regarding the touch panel 70a.

[0059] · A finger or a pen that has not touched the touch panel 70a newly touches the touch panel 70a, that is, touch start (hereinafter referred to as "touch-down").

[0060] · A state where a finger or a pen touches the touch panel 70a (hereinafter referred to as "touch-on").

[0061] · Movement of a finger or a pen that touches the touch panel 70a (hereinafter, referred to as "touch-move").

[0062] · Separation (release) of a finger or a pen that touches the touch panel 70a from the touch panel 70a, that is, touch end (hereinafter referred to as "touch-up").

[0063] · A state where the touch panel 70a is not touched (hereinafter referred to as "touch-off").

[0064] When a touch is detected, touch continuation is also detected simultaneously. After the touch, as long as touch stop is not detected, touch continuation is generally detected continuously. When touch movement is detected, touch continuation is also detected simultaneously. Even if touch continuation is detected, touch movement is not detected unless the touch position is moved. After all fingers and pens touching the touch panel stop touching, the state changes to not touched.

[0065] These operations / statuses and the coordinates of the position where a finger or a pen touches the touch panel 70a are notified to the system control unit 50 via the internal bus. Then, the system control unit 50 determines the operation (touch operation) on the touch panel 70a based on the notified information. Even in the moving direction of a finger or a pen moving on the touch panel 70a, touch movement can be determined for each vertical component / horizontal component on the touch panel 70a based on the change of the position coordinates. When touch movement of at least a predetermined distance is detected, it is considered that a swipe operation is determined to be executed. An operation of quickly moving a finger a certain distance and then releasing the finger when the finger touches the touch panel 70a is called a flick. In other words, a flick is an operation of quickly tracing on the touch panel 70a while flicking the finger on it. When touch movement of at least a predetermined distance at at least a predetermined speed is detected and touch stop is detected in this state, it can be determined that a flick has been performed (it can be determined that a flick has been performed following the swipe operation). In addition, a touch operation of simultaneously touching (multi-touch) multiple positions (for example, two points) and bringing the touch positions closer to each other is called a pinch-in, and a touch operation of separating the touch positions from each other is called a pinch-out. Pinch-in and pinch-out are collectively referred to as pinch operations (or simply pinch). The touch panel 70a can be any one of various types of touch panels (such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type, etc.). Although there are types that detect touch based on the presence of contact with the touch panel and types that detect touch based on the approach of a finger or a pen to the touch panel, any type can be adopted.

[0066] On the other hand, the digital camera 100 may be provided with an audio input unit (not shown), which sends an audio signal obtained from an embedded microphone or an audio input device connected via an audio input terminal to the system control unit 50. In this case, the system control unit 50 selects the input audio signal as needed, performs analog-to-digital conversion on the input audio signal, and performs level optimization processing, specific frequency reduction processing, etc. to generate an audio signal.

[0067] In this embodiment, the user can set the type of the position index (e.g., AF frame) for the position in the case of touch movement in the eye proximity state to any one of an absolute position designation type and a relative position designation type. The absolute position designation type is a type in which the input coordinates on the touch panel 70a are associated with the display coordinates on the display surface of the EVF 29. In the case of the absolute position designation type, when the touch panel 70a is touched, even if there is no touch movement, the AF frame is set at a position associated with the touch position (the position of the input coordinates) (moved from the position before the touch). The position set by the absolute position designation type is independent of the position set before the touch and becomes a position based on the touch position. In addition, when there is touch movement after the touch, the position of the AF frame is also moved based on the touch position after the touch movement. The relative position designation type is a type in which the input coordinates on the touch panel 70a are not associated with the display coordinates on the display surface of the EVF 29. In the case of the relative position designation type, in a state where only the touch panel 70a is touched and there is no touch movement, the position of the AF frame does not move from the position before the touch. When there is touch movement later, the position of the AF frame moves a distance corresponding to the amount of movement of the touch movement in its movement direction from the current set position of the AF frame (the position set before the touch), regardless of the position of the touch.

[0068] On the other hand, as the AF type (AF frame setting type), any one of a plurality of AF types including "single-point AF" and "face + tracking priority AF" can be set. "Single-point AF" is a type in which the user designates a position as the position for which AF is to be performed by a single-point AF frame. "Face + tracking priority AF" is a type in which the AF position is automatically set based on automatic selection conditions when the user does not specify a tracking object. When automatically setting the AF position, if a person's face is detected from the LV image, that face is preferentially selected as the AF object subject. When multiple people's faces are detected, one face is selected according to priorities (such as a large face size, a face position close to the digital camera 100 (on the near side), a face position close to the center in the image, and a pre-registered person's face, etc.) and that face is set as the AF object subject. If no person's face is detected, a subject other than the face is selected according to priorities (such as a subject close to the digital camera 100 (on the near side), a subject with high contrast, a subject with high priority (such as an animal or a vehicle, etc.), and a moving object, etc.) and that subject is set as the AF object subject. When the user designates a subject as the tracking object, the subject as the tracking object is set as the AF object subject. That is, the automatic selection condition is a condition in which at least one of the following element conditions is used as an example for weighting, the score obtained thereby is at least a predetermined threshold, or the score obtained is the highest.

[0069] · The detected human face.

[0070] · Large face size.

[0071] · The face position is close to the digital camera 100 (on the near side).

[0072] · The face position is close to the center of the image.

[0073] · The face of a pre-registered person.

[0074] · Close to the digital camera 100 (on the near side).

[0075] · High contrast.

[0076] · Subjects with high priority, such as animals or vehicles, etc.

[0077] · A moving object.

[0078] Set the AF according to the line of sight in single-point AF

[0079] Will be used Figures 3A to 3H to describe the use of line-of-sight input to control the movement of the AF frame when the AF frame selection type (AF type) in the digital camera 100 is set to "single-point AF". Figures 3A to 3H Is a display example shown on the EVF 29 when the AF frame selection type (AF type) in the digital camera 100 is set to "single-point AF".

[0080] Figure 3AThis is a display example in a state where the gaze function (a function of specifying an AF position using gaze input) is set to enabled and the gaze detection unit 160 detects the user's gaze. Live view (hereinafter referred to as "LV") 301 is an LV image being captured by the imaging unit 22. When the AF frame selection type (AF type) is set to "single-point AF", the single-point AF frame 302 is an AF frame (an indicator indicating the position as the AF object, i.e., the focus adjustment position). The single-point AF frame is set at the center of the screen in the initial state. The gaze pointer 310 is a pointer (indicator, display item) indicating the position of the gaze input detected by the gaze detection unit 160. Although the gaze detection unit 160 can obtain the coordinates of a certain point as the position where the gaze input is applied, the gaze pointer 310 is displayed as an indicator for indicating a range with a predetermined size to a certain extent, with the position where the gaze input is applied as the center. Thus, even when the detected gaze input position does not exactly match the position of the object subject that the user wants to select, the object subject can be captured within the range indicated by the gaze pointer. That is, the approximate position can be specified by the gaze input. In addition, the gaze pointer 310 centered on the position obtained by averaging the gaze positions detected by the gaze detection unit 160 within a predetermined period (for example, a period of 30 ms) is displayed. Thereby, it is possible to prevent excessive movement of the gaze pointer due to changes in the gaze input position of the user within a very short time, and the visibility of the gaze pointer is improved. The human gaze has the characteristic that the eyeball will move slightly even when fixating on a certain point, which is called fixation movement. Therefore, when the user intends to specify an exact position only using gaze input, it is difficult to specify the position as intended by the user, bringing an uncomfortable operation feeling to the user. By displaying the gaze pointer 310 in a first size large enough based on the position averaged within a predetermined period, this uncomfortable feeling can be reduced.

[0081] Figure 3B This is a display example in the EVF 29 when the user moves the gaze to change the place being observed in the EVF 29 in a state where Figure 3A In Figure 3A the gaze pointer 310 is located in the upper right of the screen, but in Figure 3B it moves to the lower left of the screen. In this way, the position of the gaze pointer 310 also moves as the user's gaze moves. On the other hand, when only the gaze pointer 310 moves, the single-point AF frame 302 does not move. That is, Figure 3A and 3B the positions of the single-point AF frame 302 in are the same.

[0082] Figure 3C This is a display example in a state where Figure 3BAn example of the display in the EVF 29 when the sight confirmation button 82 is pressed in the state of Figure 3C When the sight confirmation button 82 is pressed in the state of Figure 3A or the display state of 3B.

[0083] Figure 3D is an example of the display when a touch on the touch panel 70a is detected in the state of Figure 3B When a touch is made in the state where the sight pointer 310 is displayed, the single-point AF frame 302 is set (moved) to the sight input position at that time point (the position obtained by averaging the positions detected by the sight detection unit 160 over a predetermined period). Then, the state changes to a state where the AF frame can be moved according to the touch & drag AF function (the state where the AF frame is moved according to the touch & drag AF function). The touch & drag AF function is a function of moving the AF frame displayed on the EVF 29 to a position different from the EVF 29 according to the touch movement on the touch panel 70a. Since the user can accurately specify the desired position by a touch operation, it is troublesome when the pointer for indicating the touch position (the single-point AF frame 302 in this example) is large and an accurate position cannot be specified. Therefore, a large pointer such as the sight pointer 310 is not displayed, and the position is specified by the single-point AF frame 302 that is smaller than the sight pointer 310.

[0084] Figure 3E is when Figure 3D a touch movement toward the lower left is detected on the touch panel 70a in the state of Figure 3D and the single-point AF frame 302 is moved to the lower left in response to the detected touch movement according to the relative position specification in the state of

[0085] Figure 3F This is a display example when touch stop is performed while in the Figure 3E state. When the movement of the single-point AF frame 302 based on touch & drag has ended, the display icon 303 is shown. In this way, the position (the selected position) of the AF frame can be specified according to the combination of line-of-sight input and touch operation. On the other hand, if a touch movement is performed by additionally touching the touch panel 70a while in the Figure 3F or 3C state, the single-point AF frame 302 can be further moved in response to the touch movement.

[0086] In the above example, it is assumed that the position where the user wants to adjust the focus is the license plate of the vehicle (subject) included in the LV image 301. In this case, the single-point AF frame 302 can be set at the position of the license plate as follows. First, as Figure 3B shown, by observing the front part of the vehicle in the LV image 301, the front part of the vehicle is roughly specified quickly with the line-of-sight pointer 310. Thereafter, the position of the single-point AF frame 302 set based on the line-of-sight pointer 310 ( Figure 3D the position of the single-point AF frame 302 in ) is moved and finely adjusted by a touch operation so that it can precisely correspond to the position of the license plate. The amount of movement of the touch movement at this time is reduced because the single-point AF frame 302 has been set near the license plate based on the line-of-sight input, and the amount of movement is the amount of movement starting from the single-point AF frame 302. In this way, according to this embodiment, the user can quickly and precisely specify the desired position.

[0087] Figure 3G This is a display example when the operation of the first shutter switch 62 (half-pressing the shutter button 61) is detected while in the Figure 3F state and AF is performed at the position of the single-point AF frame 302. The single-point AF frame 302 is switched to the focused single-point AF frame 304, and the focused state is indicated.

[0088] On the other hand, although an example in which the line-of-sight pointer 310 is not displayed when line-of-sight confirmation is performed has been described, the present invention is not limited to this. Figure 3H An example in which the line-of-sight pointer 310 is still displayed even after the single-point AF frame is moved to the position of the line-of-sight pointer 310 by pressing the line-of-sight confirmation button 82 is shown. Thus, even when the single-point AF frame 302 cannot be moved to the intended position by pressing the line-of-sight confirmation button 82, the position of the single-point AF frame 302 can be immediately reset by moving the line-of-sight pointer 310 again and pressing the line-of-sight confirmation button 82.

[0089] Setting the AF frame according to the line of sight in face + tracking priority AF

[0090] will utilizeFigures 4A to 4H This describes the use of gaze input to control the movement of the AF frame when the AF frame selection type (AF type) in the digital camera 100 is set to "Face + Tracking Priority". Figures 4A to 4H This is a display example shown on the EVF 29 when the AF frame selection type (AF type) is set to "Face + Tracking Priority".

[0091] Figure 4A This is a display example in a state where the gaze function is set to enabled and the gaze detection unit 160 detects the user's gaze. Components identical to those described in Figure 3A are denoted by the same reference numerals, and their descriptions are omitted. In Face + Tracking Priority, the gaze pointer 310 is displayed in a first size large enough based on the position averaged over a predetermined period, as in the case of single-point AF. The face frames 401 to 405 are indicators indicating the positions of the faces of the persons detected from the LV image. In Figure 4A this state, no face is selected.

[0092] Figure 4B This is Figure 4A a display example in the EVF 29 in a state where the user moves the gaze to change the place the user is observing in the EVF 29. Although the gaze pointer 310 is present on the left side of the screen in Figure 4A , in Figure 4B this gaze pointer 310 moves to the upper right of the screen.

[0093] Figure 4C This is Figure 4B a display example in the EVF 29 when the gaze confirmation button 82 is pressed in Figure 4C this state. When the gaze confirmation button 82 is pressed while the gaze pointer 310 is being displayed, within the range indicated by the gaze pointer 310 at that time point, a subject to be tracked (AF object) is automatically selected according to the above automatic selection conditions. In Figure 4C this example, the face indicated by the face frame 402 is selected (this face is completely included in Figure 4BThe face (the closest face among the faces (face frames 402 and 403) in the line-of-sight pointer 310 shown) is set as the tracking object. A tracking frame 412 is displayed on the subject that has become the tracking object, and the face frame is not displayed. Then, tracking starts. During tracking, even when the subject that is the tracking object moves, the tracking frame moves with the tracking object. Since the area of the subject selected by the line-of-sight pointer 310 is narrowed down, subjects outside the line-of-sight pointer 310 are not selected, and the faces and vehicles indicated by the face frame 401 or the face frame 405 are not selected. That is, when the tracking object is set within the range quickly and roughly specified by the user using the line of sight, a subject that better matches the user's intention can be selected compared to the subject selected by automatic selection without using the line of sight. Additionally, in Figure 4C when the state of the position specified according to the line of sight is confirmed, the line-of-sight pointer 310 is not displayed. When the Figure 4C line-of-sight confirmation button 82 is pressed in the Figure 4A state, the confirmation state is cancelled, and the state returns to the

[0094] Figure 4D display state of Figure 4B This is a display example when a touch on the touch panel 70a is detected in the

[0095] Figure 4E state. When a touch is made in the state where the line-of-sight pointer 310 is displayed, a touch pointer 406 is displayed at the line-of-sight input position at that time (the position obtained by averaging the positions detected by the line-of-sight detection unit 160 over a predetermined period). Then, the state changes to a state where the position can be specified by the touch & drag AF function (the state where the touch pointer 406 moves by the touch & drag AF function). Since the user can accurately specify the desired position by a touch operation, it is troublesome when the pointer for indicating the touch position (the touch pointer 406 in this example) is large and an accurate position cannot be specified. Therefore, large pointers such as the line-of-sight pointer 310 are not displayed, and the position is specified by the touch pointer 406 that is smaller than the line-of-sight pointer 310. Therefore, as shown in the figure, even when the faces are dense, the user can easily select the desired face.

[0095] Figure 4E This is the state where a touch movement diagonally upward to the right applied to the touch panel 70a is detected in the Figure 4D state and in the Figure 4DThe following is a display example in the state of Figure 4E wherein, in response to a detected touch movement, the touch pointer 406 is moved to the upper right according to the relative position. The touch pointer 406 is set at a position corresponding approximately to the position of the face frame 403 (more precisely, the position where the center of the touch pointer 406 is included within the range of the face frame 403). When a touch stop is performed in this state, the face frame 403 is designated as the tracking object based on the position of the touch pointer 406. On the other hand, in the changed display as shown in

[0096] Figure 4F a display example of the attachment display is shown. In response to a touch movement, the touch pointer 406 is moved, and when the touch pointer 406 reaches a position where the face frame 403 can be designated, the touch pointer 406 is not displayed and the face frame 403 is displayed in a display form different from other face frames. Thus, if the user performs a touch stop at this time point, the user can recognize that the face frame 403 is designated and can easily determine whether the touch has moved to the target position.

[0097] Figure 4G This is a display example when a touch stop is performed in the state of Figure 4E or 4F. Based on the position of the touch pointer 406 immediately before the touch stop, the face frame 403 is set as the tracking object, the tracking frame 413 is displayed, and tracking is started. In a state where the movement of the touch pointer 406 according to the touch & drag has ended, the icon 303 is displayed. On the other hand, if a touch movement is performed by additionally touching the touch panel 70a in the state of Figure 4G or 4C, the tracking is canceled and the touch pointer 406 is displayed at the position where the tracking object exists, so that the touch pointer 406 can be moved in response to the touch movement.

[0098] In the above example, it is assumed that the position where the user wants to adjust the focus is the face indicated by the face frame 403 included in the LV image 301. In this case, the tracking object (AF position) can be set at the position of the face frame 403 as follows. First, as shown in Figure 4BAs shown, by observing the portion near the face frame 403 in the LV image 301, the range is quickly and roughly specified. Thereafter, by a touch operation, the touch pointer 406 is moved from the position of the face frame 402 which is the tracking object set based on the line-of-sight pointer 310, and the touch pointer 406 is finely adjusted so that the touch pointer 406 can precisely correspond to the face frame 403. The amount of movement of the touch at this time is reduced because the face frame 402 near the face frame 403 has been set based on the line-of-sight input and the amount of movement is the amount of movement starting from the face frame 402. In this way, according to this embodiment, the user can quickly and precisely specify the desired position (subject).

[0099] Figure 4H is a display example in the case where the operation of the first shutter switch 62 (half-pressing the shutter button 61) is detected in the state of Figures 4E to 4G and AF is executed at the position of the tracking frame. When the touch pointer 406 is arranged at the position of the subject in the non-tracking state, the object is tracked and then AF is continued. The tracking frame and the adsorbed display face frame are switched to the focused face frame 414 and focus is indicated. On the other hand, when the same operation is performed in the state of Figure 4A and 4B where the line-of-sight pointer 310 is displayed, the subject near the center of the line-of-sight pointer 310 can be tracked and then AF can be continued.

[0100] On the other hand, as in the case of single-point AF, a configuration can be adopted in which the line-of-sight pointer 310 is also displayed even after the subject at the position of the line-of-sight pointer 310 is tracked by pressing the line-of-sight confirmation button 82 or the touch pointer 406 is displayed by a touch movement.

[0101] Shooting mode processing

[0102] Figure 5A and 5B are flowcharts of the shooting mode processing in the digital camera 100 in this embodiment. Figure 5A and 5B The processing is the processing in the case where the display destination is the EVF 29. Therefore, in the specific conditions for performing CAL calibration, an indicator such as a single-point AF frame is included in the display on the EVF 29, which will be described in detail later. Each process in the Figures 5A to 10B flowchart is implemented by the system control unit 50 expanding the program stored in the non-volatile memory 56 in the system memory 52 and executing the program. When the digital camera 100 is started in the shooting mode, the flags and control variables are initialized and the Figure 5A and 5B processing is started.

[0103] In S500, the system control unit 50 starts shooting a live view image (LV image) in the imaging unit 22 and displays the captured LV image on the EVF 29.

[0104] In S501, the system control unit 50 performs camera setting processing for executing various settings related to imaging in response to a user operation. The camera setting processing will be described later. Figure 6 for use.

[0105] In S502, the system control unit 50 determines whether the line-of-sight function is enabled (whether the setting of line-of-sight AF to be described later is enabled). If the line-of-sight function is enabled, the process proceeds to S503; otherwise (if the line-of-sight function is disabled), the process proceeds to S516.

[0106] In S503, the system control unit 50 determines whether the line-of-sight detection unit 160 has detected a line of sight. If a line of sight is detected, the process proceeds to S504; otherwise (if no line of sight is detected despite the line-of-sight function being enabled), the process proceeds to S516.

[0107] In S504, the system control unit 50 determines whether the line-of-sight pointer display is enabled. If the line-of-sight pointer display is enabled, the process proceeds to S505; otherwise (if the line-of-sight pointer display is disabled), the process proceeds to S507.

[0108] In S505, the system control unit 50 determines whether the line-of-sight confirmation flag stored in the system memory 52 is 0. The initial value is 0. The line-of-sight confirmation flag = 0 indicates that the above-mentioned line-of-sight confirmation state has been canceled and the state where the line-of-sight pointer can be moved by the line of sight. There is also a "coarse adjustment mode" in which the approximate position can be specified by the line of sight. On the other hand, the line-of-sight confirmation flag = 1 indicates the above-mentioned line-of-sight confirmation state and the state where the position cannot be specified by the line of sight after the approximate position has been specified by the line of sight. There is also a "fine adjustment mode" in which the position can be specified finely by touch movement. If the line-of-sight flag is 0, the process proceeds to S506; otherwise (if the line-of-sight flag = 1), the process proceeds to S507.

[0109] In S506, the system control unit 50 displays the gaze pointer 310 on the EVF 29 based on the gaze input position detected by the gaze detection unit 160. As described above, the gaze pointer 310 has a first size that is large enough, and is displayed based on the position obtained by averaging the gaze input positions within a predetermined period. If the average gaze detection position is not close to the edge of the EVF 29, the gaze pointer 310 is displayed within a range of the first size centered on the gaze input position. If the average gaze detection position is close to the edge of the EVF 29, the gaze pointer 310 is displayed within a range of the first size that contacts the screen edge close to the gaze input position. According to the process of S506, if the AF type is set to single-point AF, the above display as shown in Figure 3A or 3B is performed, and if the AF type is set to face + tracking priority, the above display as shown in Figure 4A or 4B is performed.

[0110] In S507, the system control unit 50 determines whether the gaze confirmation button 82 has been pressed (i.e., whether an operation to indicate the execution of position designation / designation cancellation according to the gaze has been performed). If the gaze confirmation button 82 has been pressed, the process proceeds to S508, otherwise the process proceeds to S516.

[0111] In S508, the system control unit 50 determines whether the gaze confirmation flag stored in the system memory 52 is 0. If the gaze confirmation flag = 0, the process proceeds to S512, otherwise (if the gaze confirmation flag = 1) the process proceeds to S509.

[0112] In S509, the system control unit 50 sets the gaze confirmation flag to 0. In addition, the system control unit 50 cancels the displayed icon 303 and returns to the display state for display in the state where the gaze confirmation has been cancelled.

[0113] In S510, the system control unit 50 determines whether the currently set AF type is face + tracking priority AF. If the currently set AF type is face + tracking priority AF, the process proceeds to S511, where tracking is cancelled, and then proceeds to S504. Therefore, for example, in the case of the above display as shown in Figure 4C or 4G, when the gaze confirmation button 82 is pressed, it transitions to the display state of Figure 4A or 4B. When it is determined in S510 that the currently set AF type is not face + tracking priority AF (i.e., it is determined that the currently set AF type is single-point AF), the process proceeds to S504. Therefore, for example, in the case of the above display as shown in Figure 3C or 3F, when the gaze confirmation button 82 is pressed, it transitions to the display state of Figure 3A or 3B.

[0114] In S512, the system control unit 50 sets the line-of-sight confirmation flag to 1. In addition, the system control unit 50 displays the icon 303 on the EVF 29 and displays the line-of-sight confirmation status.

[0115] In S513, the system control unit 50 determines whether the currently set AF type is face + tracking priority AF. If the currently set AF type is face + tracking priority AF, the process proceeds to S514; otherwise (i.e., if the currently set AF type is single-point AF), the process proceeds to S515.

[0116] In S514, based on the above automatic selection conditions, within the range of the first size indicated by the line-of-sight pointer 310 (even within the same range when the line-of-sight pointer 310 is not displayed), a subject to be tracked is selected. Then, a tracking frame is displayed on the selected subject (tracking object), and tracking is started. Therefore, for example, a display transition as shown in Figures 4B to 4C may occur.

[0117] In S515, the system control unit 50 sets the single-point AF frame 302 at the line-of-sight input position at the time point when the line-of-sight confirmation button 82 is pressed (a position obtained by averaging the positions detected by the line-of-sight detection unit 160 within a predetermined period). Therefore, for example, a display transition as shown in Figures 3B to 3C may occur. On the other hand, in the present embodiment, an example of setting the single-point AF frame at the line-of-sight input position detected by the line-of-sight detection unit 160 in the case of single-point AF is described. However, the present invention is not limited thereto, and even in the case of single-point AF, automatic selection based on the automatic selection conditions can be performed within the range of the line-of-sight pointer 310 as in the case of face + tracking priority AF, and the single-point AF frame 302 can be set at the position of the automatically selected subject.

[0118] In S532, the system control unit 50 stores the position where the single-point AF frame is set, that is, the line-of-sight input position at the time point when the line-of-sight confirmation button 82 is pressed.

[0119] In S533, the system control unit 50 sets the line-of-sight reflection flag to 1. The line-of-sight reflection flag is a flag for determining whether to perform the process (processing based on touch movement correction CAL data) described using Figure 9A , 9B , 10A, and 10B. The CAL data is data obtained through CAL (calibration; detailed CAL or CAL correction described later) of the line-of-sight input (based on the line-of-sight input position) and data for correcting the line-of-sight input position.

[0120] In S516, the system control unit 50 determines whether the touch panel 70a has been touched. If it has been touched, the process proceeds to S517; otherwise, the process proceeds to S518.

[0121] In S517, the system control unit 50 performs touch operation response processing in response to the touch operation on the touch panel 70a. The touch operation response processing will be described later using Figure 7 to describe the touch operation response processing.

[0122] In S518, the system control unit 50 determines whether any other operation has been performed on the operation unit 70. If any other operation has been performed, the process proceeds to S519; otherwise, the process proceeds to S520.

[0123] In S519, the system control unit 50 performs processing in response to the other operation. For example, the system control unit can change various shooting parameters such as the shutter speed, aperture value, and exposure correction value, or set the recording image quality or self-timer.

[0124] In S520, the system control unit 50 determines whether the first shutter switch 62 has been turned on to output the signal SW1, that is, whether the shutter button 61 has been half-pressed and a shooting preparation instruction has been given. If SW1 has been output, the process proceeds to S521; otherwise, the process proceeds to S531.

[0125] In S521, the system control unit 50 determines whether the gaze pointer 310 is being displayed, that is, whether the gaze function is enabled, a gaze has been detected, the display of the gaze pointer is enabled, and the gaze confirmation flag is 0. If the gaze pointer 310 is being displayed, the process proceeds to S522; otherwise, the process proceeds to S523. On the other hand, in the determination of whether the gaze pointer 310 is being displayed, it can be determined whether the gaze function is enabled, a gaze has been detected, and the gaze confirmation flag is 0. In this case, if the gaze function is enabled, a gaze has been detected, and the gaze confirmation flag = 0, the process proceeds to S522 even if the display of the gaze pointer is disabled (the gaze pointer 310 is not displayed).

[0126] In S522, the system control unit 50 selects a subject to be an AF object within the range of the first size indicated by the line-of-sight pointer 310 (even if the line-of-sight pointer 310 is not displayed, it is within the same range), according to the above automatic selection conditions. This is the same process as selecting a tracking object in S514. Then, AF is performed based on the selected subject (AF object, focus adjustment object). In the same way, processes such as AE or AWB can be performed based on the selected subject. On the other hand, when the AF type is single-point AF, the AF object is not selected based on the automatic selection conditions, but the range of a single-point AF frame centered on the line-of-sight input position at that time can be selected as the AF object.

[0127] In S523, the system control unit 50 determines whether the currently set AF type is face + tracking priority AF. If the currently set AF type is face + tracking priority AF, the process proceeds to S524; otherwise (if the currently set AF type is single-point AF), the process proceeds to S527.

[0128] In S524, the system control unit 50 determines whether subject tracking has been performed. If tracking has been performed, the process proceeds to S526; if tracking has not been performed, the process proceeds to S525.

[0129] In S525, the system control unit 50 selects a subject to be an AF object for the entire range of the LV image being captured based on the above automatic selection conditions. Then, the system control unit 50 performs AF based on the selected subject (AF object, focus adjustment object). In the same way, processes such as AE or AWB can be performed based on the selected subject. On the other hand, the selected object is not limited to the entire range of the LV image, and a range of the second size in the LV image, which is larger than the first size that is the size of the line-of-sight pointer 310, can be used as the object. For example, a range of 80% from the center of the LV image (larger than the first size) can be used as the range of the object for automatically selecting a subject based on the automatic selection conditions in S525. In this case, it is assumed that other edge regions are areas where it is less likely that a main subject requiring AF exists, so these other edge regions are excluded from the object for automatically selecting a subject based on the automatic selection conditions in S525.

[0130] In S526, the system control unit 50 performs AF within the tracking frame during tracking (i.e., for the tracking object). In the same way, processes such as AE or AWB can be performed based on the tracking object.

[0131] In S527, the system control unit 50 performs AF within the set single-point AF frame. In the same manner, processes such as AE or AWB can be performed based on the single-point AF frame.

[0132] In S528, the system control unit 50 determines whether the second shutter switch 64 has been turned on to output a signal SW2, that is, whether the shutter button 61 has been fully pressed and a shooting instruction has been given. If SW2 has been output, the process proceeds to S530; otherwise, the process proceeds to S529.

[0133] In S529, the system control unit 50 determines whether the on state of SW1 is maintained. If the on state of SW1 is maintained, the process proceeds to S528; otherwise (if SW1 is off), the process proceeds to S531.

[0134] In S530, the system control unit 50 performs a series of shooting processes (the above-mentioned shooting processes) from exposure to recording the captured image as an image file in the recording medium 200 through the imaging unit 22.

[0135] In S531, it is determined whether a shooting mode end event has occurred (such as a power-off operation or an instruction to transition to another operation mode such as a playback mode). If this end event has not occurred, the process returns to S500 and is repeated; if this end event has occurred, the shooting mode process ends.

[0136] Camera setting process

[0137] The camera setting process of S501 described above will be described. The camera setting process is a process of setting each shooting-related setting item displayed in the setting menu screen when the menu button 81 is pressed. Figure 5A The figures 11A, 11B, and 11C show display examples of the setting menu screen related to shooting displayed on the EVF 29 or the display unit 28. The setting item 1101 included in the menu screen

[0138] Figure 11A , 11B is an item for setting the AF type. The setting item 1102 is an item for setting touch & drag AF. The setting item 1103 is an item for setting the settings related to the eye view function. The setting item 1104 is an item for setting the operation when the center part of the press MC 65 is operated. Figure 11A

[0139] Figure 11B Figure 11A is a display example of the detailed setting menu screen of the eye view AF for setting the settings related to the eye view function. This screen is displayed when the setting item 1103 Figure 11A has been selected. InFigure 11B On the detailed setting menu screen of the line of sight AF, setting items 1105 to 1110 are displayed. Setting item 1105 is an item for setting whether to enable or disable the line of sight function. Setting item 1106 is an item for setting the display of the line of sight pointer to be enabled (displayed) or disabled (not displayed). Setting item 1107 is an item for setting the response (hereinafter referred to as sensitivity) of the display of the line of sight pointer with respect to the detected line of sight information. Setting item 1108 is an item for setting whether to enable or disable the function of making the AF frame jump to the line of sight detection position when SW1 is turned on. Setting item 1109 is an item for setting the CAL number to be described later. The CAL number 1109a, the user string 1109b, and the indication 1109c for indicating whether CAL is completed are also displayed together with setting item 1109. When the CAL calibration to be described later has been performed and thus the CAL data has been accumulated (updated) at least a predetermined number of times, that is, when the CAL calibration has been performed at least a predetermined number of times, the indication 1109c for indicating whether CAL is completed can be switched from the indication indicating that CAL has not been performed to the indication indicating that CAL is completed. Setting item 1110 is an item for transitioning to Figure 11C the screen of

[0140] Figure 11C is an example of the display of the detailed setting menu screen of the line of sight AF for performing settings / executions related to CAL. Setting item 1111 is an item for executing CAL. Setting item 1112 is an item for deleting CAL data. Setting item 1113 is an item for executing the operation of storing CAL data in an SD card or the like or reading CAL data from an SD card or the like. Setting item 1114 is an item for setting whether to execute the CAL calibration to be described later.

[0141] Figure 6 shows the above Figure 5A details of the camera setting process of S501.

[0142] In S601, the system control unit 50 determines whether an operation for switching the on / off (enable / disable) of the line of sight function (line of sight AF) has been performed on the operation unit 70. In the present embodiment, the operation for switching the on / off (enable / disable) of the line of sight function is the operation of opening the menu screen, selecting the corresponding setting item (setting item 1105), and switching the setting. In the present embodiment, when the line of sight AF is on, the function of inputting the user's line of sight is enabled, and when the line of sight AF is off, the function is disabled. If an operation for switching the on / off of the line of sight function has been performed, the process proceeds to S602, and otherwise the process proceeds to S603.

[0143] In S602, the system control unit 50 switches the on / off state of the line-of-sight function and records the details of the changed settings in the non-volatile memory 56.

[0144] In S603, the system control unit 50 determines whether an operation has been performed on the operation unit 70 to switch the on / off state (enable / disable) of the display of the line-of-sight pointer. In this embodiment, the operation of switching the on / off state (enable / disable) of the line-of-sight confirmation function is an operation of opening the menu screen, selecting the corresponding setting item (setting item 1106), and switching the setting. In this embodiment, when the display of the line-of-sight pointer is on, the line-of-sight pointer 310 as a GUI is displayed in response to the user's line-of-sight input, and when the display of the line-of-sight pointer is off, the line-of-sight pointer is not displayed. If an operation of switching the on / off state of the display of the line-of-sight pointer has been performed, the process proceeds to S604; otherwise, the process proceeds to S605.

[0145] In S604, the system control unit 50 switches the on / off state (enable / disable) of the display of the line-of-sight pointer and records the details of the changed settings in the non-volatile memory 56.

[0146] In S605, the system control unit 50 determines whether an operation has been performed on the operation unit 70 to switch the settings of the touch & drag AF function. In this embodiment, the operation of switching the settings of the touch & drag AF function is an operation of opening the menu screen, selecting the corresponding setting item (setting item 1102), and switching the setting. In this embodiment, either "absolute (the above absolute position specification type)" or "relative (the above relative position specification type)" can be selected as the setting of the touch & drag AF. If an operation of switching the touch & drag AF function has been performed, the process proceeds to S606; otherwise, the process proceeds to S607.

[0147] In S606, the system control unit 50 switches the settings of the touch & drag AF function and records the details of the changed settings in the non-volatile memory 56.

[0148] In S607, the system control unit 50 determines whether an operation has been performed on the operation unit 70 to switch the AF type. In this embodiment, the operation of switching the AF type is an operation of opening the menu screen, selecting the corresponding setting item (setting item 1101), and switching the setting. If an operation of switching the AF type has been performed, the process proceeds to S608; otherwise, the camera setting process ends. On the other hand, although an example in which any of face + tracking priority AF and single-point AF can be selected as the AF type is described in this embodiment, other AF types (area AF, multi-point AF, etc.) can be set.

[0149] In S608, the system control unit 50 switches the AF type and records the changed setting details in the non-volatile memory 56.

[0150] In S609, the system control unit 50 determines whether an operation to switch the CAL number has been performed on the operation unit 70. The CAL number is a number corresponding to each storage area in the case of ensuring a plurality of areas for storing CAL data. The CAL number is useful when multiple users use the same electronic device or when a single user wants to use the electronic device in multiple forms (such as the state of wearing glasses and the state of the naked eye, etc.). In the present embodiment, the operation to switch the CAL number is an operation of opening the menu screen, selecting the corresponding setting item (setting item 1109), and switching the setting. If the operation to switch the CAL number has been performed, the process proceeds to S610; otherwise, the process proceeds to S611.

[0151] In S610, the system control unit 50 switches the CAL number setting and records the changed setting details in the non-volatile memory 56.

[0152] In S611, the system control unit 50 determines whether an operation to switch the setting of whether to perform CAL correction has been performed on the operation unit 70. CAL correction is a process of correcting CAL data based on user operations, which will be described later. In the present embodiment, the operation to switch the setting of CAL correction is an operation of opening the menu screen, selecting the corresponding setting item (setting item 1114), and switching the setting. If the operation to switch the setting of CAL correction has been performed, the process proceeds to S612; otherwise, the process proceeds to S613.

[0153] In S612, the system control unit 50 switches the setting of CAL correction and records the changed setting details in the non-volatile memory 56.

[0154] In S613, the system control unit 50 determines whether an operation to execute CAL has been performed on the operation unit 70. Here, CAL is a process (detailed CAL) executed in a mode for generating the user's CAL data, which can generate detailed data but generally requires an arrangement for it. In the present embodiment, the operation to execute the detailed CAL is an operation of opening the menu screen and selecting the corresponding setting item (setting item 1111). If the operation to execute the detailed CAL has been performed, the process proceeds to S614; otherwise, the process proceeds to S615.

[0155] In S614, the system control unit 50 executes CAL and generates CAL data. The system control unit 50 associates the generated CAL data with the current CAL number and records the CAL data associated with the CAL number in the non-volatile memory 56.

[0156] In S615, the system control unit 50 determines whether any other operation has been performed on the operation unit 70. If any other operation has been performed, the process proceeds to S616; otherwise, the camera setting process ends. Here, other operations may include, for example, an operation to switch the sensitivity setting of the line-of-sight pointer (the setting of setting item 1107), and an operation to input a user string (user string 1109b) for identifying the CAL number, etc.

[0157] In S616, the system control unit 50 executes other processing.

[0158] Touch operation response processing

[0159] Figure 7 is the detailed flowchart of the touch operation response processing in S517 described above. Here, it is assumed that the position specification type of the position input applied to the touch panel 70a is the relative position specification type. Therefore, the fact that the position specification type is the relative position specification type is included in the specific conditions for performing CAL correction, which will be described in detail later. Figure 5A In S701, as in S502, the system control unit 50 determines whether the line-of-sight function is enabled. If the line-of-sight function is enabled, the process proceeds to S702; otherwise (if the line-of-sight function is disabled), the process proceeds to S706.

[0160] In S702, as in S503, the system control unit 50 determines whether a line of sight has been detected. If a line of sight has been detected, the process proceeds to S703; otherwise, the process proceeds to S708.

[0161] In S703, the system control unit 50 sets the line-of-sight setting flag to 1.

[0162] In S704, the system control unit 50 controls the line-of-sight pointer so that it is not displayed.

[0163] In S705, when the line-of-sight function is enabled, the system control unit 50 performs relative position specification processing. This processing will be described later using

[0164] In S706, when the line-of-sight function is disabled, the system control unit 50 performs relative position specification processing. Figure 8 to describe this processing.

[0165] In S706, when the line-of-sight function is disabled, the system control unit 50 performs relative position specification processing.

[0166] Relative position specification processing when the line of sight is enabled

[0167] Figure 8 is the detailed flowchart of the relative position specification processing when the line of sight is enabled in S705 described above. Figure 7 In S705, when the line-of-sight function is enabled, the system control unit 50 performs relative position specification processing. This processing will be described in detail later.

[0168] In S801, the system control unit 50 determines whether the AF type is "Face + Tracking Priority AF". If the AF type is "Face + Tracking Priority AF", the process proceeds to S805; otherwise (if the AF type is "Single Point AF" in this embodiment), the process proceeds to S802.

[0169] In S802, the system control unit 50 displays a single point AF frame at the line-of-sight input position at the time of touch (the position obtained by averaging the positions detected by the line-of-sight detection unit 160 over a predetermined period). Therefore, a transition occurs from the above Figure 3B display to Figure 3D display.

[0170] In S803, the system control unit 50 stores the position of the current line of sight. On the other hand, in the case of performing the processes of Figure 5A S515, S532, and S533, the processes of S802 and S803 can be omitted. The processes of S802 and S803 can be performed, and the position of the single point AF frame and the stored line-of-sight position can be updated from the processing results of S515 and S532 to the processing results of S802 and S803.

[0171] In S804, in response to a touch movement on the touch panel 70a, the system control unit 50 performs a touch movement process during single point AF. The touch movement process during single point AF will be described later using Figure 9A and 9B .

[0172] In S805, the system control unit 50 determines whether the subject is being tracked. If the subject is being tracked, the process proceeds to S810; otherwise, the process proceeds to S806.

[0173] In S806, the system control unit 50 displays a touch pointer 406 indicating the touch position at the line-of-sight input position at the time of touch (the position obtained by averaging the positions detected by the line-of-sight detection unit 160 over a predetermined period). Therefore, for example, a transition may occur from the Figure 4B display to Figure 4D display.

[0174] In S807, the system control unit 50 stores the position of the current line of sight.

[0175] In S808, in response to a touch movement on the touch panel 70a, the system control unit 50 performs a touch movement process during face + tracking priority AF. The touch movement process during face + tracking priority AF will be described later using Figure 10A and 10B .

[0176] In S809, the system control unit 50 selects the subject at the position of the touch pointer and starts tracking the subject. In this case, the selection is not based on the above automatic selection conditions. Additionally, the display icon 303 is shown. Thus, for example, a display transition such as from Figures 4E to 4G may occur.

[0177] In S810, the system control unit 50 displays the touch pointer 406 indicating the touch position at the position of the tracking object at the time of touch. Thus, for example, a display transition from Figure 4C to Figure 4D may occur.

[0178] In S811, in response to a touch movement (movement instruction operation) on the touch panel 70a, the system control unit 50 moves the touch pointer 406 indicating the touch position. This movement is based on relative position specification type movement. Thus, for example, a display transition from Figure 4D to Figure 4E may occur. The system control unit 50 can recognize that the display position of the touch pointer 406 moves in response to the touch movement, regardless of the line-of-sight input after the touch pointer 406 is displayed.

[0179] In S812, the system control unit 50 determines whether a touch stop has been made on the touch panel 70a. If a touch stop has been made, the process proceeds to S813; otherwise, it returns to S811.

[0180] S813 is the same as S809.

[0181] On the other hand, in the present embodiment, an example is described in which when a touch is made with the line of sight enabled, if tracking is in progress, the touch pointer 406 is displayed at the tracking position instead of the line-of-sight input position. However, the present invention is not limited to this, and if the line of sight is enabled, regardless of whether tracking is in progress, when a touch is made, the touch pointer 406 can be displayed at the line-of-sight input position (the position obtained by averaging the positions detected by the line-of-sight detection unit 160 over a predetermined period). In this case, when the determination result in S801 is "yes", the process proceeds to S806 without making the determination in S805.

[0182] Additionally, when performing subject tracking based on the relative position specification type according to touch stop (such as in S809 and S813, etc.), subject tracking can be started after a predetermined time has elapsed since the touch stop. Thus, in the case of the relative position specification type, when moving the touch pointer while repeating a series of operations of touch, touch movement, and touch stop, it is easy to move the touch pointer because the subject tracking process is not performed every time.

[0183] Touch movement processing during single-point AF

[0184] Figure 9A and 9B is a detailed flowchart of touch movement processing during single-point AF in S804 above Figure 8 of.

[0185] In S901, the system control unit 50 moves the single-point AF frame in response to a touch movement (movement instruction operation) on the touch panel 70a

[0186] In S902, the system control unit 50 determines whether the first shutter switch is turned on to output a signal SW1. If SW1 has been output, the process proceeds to S909; otherwise, the process proceeds to S903

[0187] In S903, the system control unit 50 determines whether touch has stopped on the touch panel 70a. If touch has stopped, the process proceeds to S904; otherwise, the process proceeds to S901

[0188] In S904, the system control unit 50 determines whether the gaze reflection flag is set to 1. If the gaze reflection flag = 1, the process proceeds to S912; otherwise, the process proceeds to S905

[0189] In S905, the system control unit 50 determines whether the shortest distance (ΔP1) from the stored gaze position to the current position of the single-point AF frame moved in S901 is greater than a first threshold (Th1) and less than a second threshold (Th2). If Th1 < ΔP1 < Th2, the process proceeds to S906; otherwise, the process proceeds to S912. When ΔP1 is less than Th1, the accuracy of the stored gaze position is high enough, and the user hardly needs to correct the position of the single-point AF frame based on touch movement. In addition, when ΔP1 is greater than Th2, this can be considered a case where the user wishes to move the position of the single-point AF frame to a large extent based on touch movement regardless of the accuracy of the stored gaze position. Therefore, it is assumed that if ΔP1 does not have a value that satisfies Th1 < ΔP1 < Th2, CAL correction described later is not performed. On the other hand, although it is assumed that the difference between the stored gaze position and the center position of the single-point AF frame is calculated as ΔP1, the present invention is not limited thereto. The method of determining ΔP1 can be freely changed to a method such as using the position of a specific subject detected within the single-point AF frame as a reference point

[0190] In S906, the system control unit 50 determines whether continuous AF (a mode in which AF is always performed at the position of the AF frame) is enabled. In the case of this mode, the process proceeds to S912; otherwise, the process proceeds to S907

[0191] In S907, the system control unit 50 determines whether other conditions are satisfied. If the other conditions are satisfied, the process proceeds to S908; otherwise, the process proceeds to S912. The other conditions may include at least any one of the following multiple conditions.

[0192] · When at least one touch movement is made before the touch movement in S901, each touch movement indicates the same direction (because it is considered that in this case, the intention is to correct the deviation of the line-of-sight position to a fixed direction).

[0193] · Multiple position inputs are not applied to the touch panel 70a, that is, at least two touch positions are not detected (because when the user looks into the EVF, the user's nose may touch the touch panel 70a, so an incorrect ΔP1 may be calculated based on the position of the nose).

[0194] · The longitudinal component of ΔP1 is at least a predetermined value (because the EVF of the digital camera is horizontally long, and when normal CAL (detailed CAL) is performed, data with relatively high horizontal reliability is obtained as CAL data).

[0195] When the condition of not detecting at least two touch positions is excluded from the conditions and multiple position inputs are applied to the touch panel 70a, the latest position input can be processed as the position specified by the touch movement. That is, ΔP1 can be calculated based on the latest touch position at the time point when at least two touch positions have been detected.

[0196] In S908, the system control unit 50 sets the line-of-sight reflection flag to 1 because it can be determined that CAL data correction based on the touch movement will be performed according to the conditions in S902 to S907.

[0197] In S909, the system control unit 50 determines whether the line-of-sight reflection flag is set to 1. If the line-of-sight reflection flag = 1, the process proceeds to S910; otherwise, the process proceeds to S527.

[0198] In S910, the system control unit 50 determines whether Th1 < ΔP1 < Th2 is satisfied. If it is satisfied, the process proceeds to S911; otherwise, the process proceeds to S527.

[0199] In S911, the system control unit 50 performs CAL correction based on ΔP1. By repeating this CAL correction, the accuracy of the line-of-sight input is gradually improved. As described above, since data with relatively high horizontal reliability is obtained as CAL data when normal CAL (detailed CAL) is performed, CAL correction can be performed only based on the longitudinal component (the component in the longitudinal direction) of ΔP1.

[0200] In S912, the system control unit 50 determines whether the touch panel 70a has been touched. If it has been touched, the process proceeds to S923; otherwise, the process proceeds to S913.

[0201] In S913, the system control unit 50 determines whether a predetermined time has elapsed since the touch stopped in S903. If the predetermined time has elapsed, the process proceeds to S914; otherwise, the process proceeds to S915.

[0202] In S914, the system control unit 50 assumes that the user has not performed a completion operation indicating the completion of the frame movement operation at the current position of the single-point AF frame within the predetermined time since the touch stopped in S903, and sets the sight reflection flag to 0 so that the user does not perform CAL calibration.

[0203] In S915, the system control unit 50 determines whether the sight confirmation button 82 has been pressed. If the sight confirmation button 82 has been pressed, the process proceeds to S508, and the single-point AF frame is moved to the position of the current sight. If the sight confirmation button 82 has not been pressed, the process proceeds to S916.

[0204] In S916, the system control unit 50 determines whether the first shutter switch 62 is turned on to output the signal SW1. If SW1 has been output, the process proceeds to S920; otherwise, the process proceeds to S917. Here, since the fact that the first shutter switch 62 is turned on can be considered that the user has completed the frame movement operation at the current position of the single-point AF frame, the process proceeds to the process including CAL calibration based on ΔP1 in S920 to S922.

[0205] In S917, the system control unit 50 determines whether any other operation has been performed on the operation unit 70. If any other operation has been performed, the process proceeds to S918; otherwise, the process proceeds to S912. Here, other operations may include, for example, an operation to end the live view shooting state (such as pressing the menu button 81, etc.).

[0206] In S918, the system control unit 50 sets the sight reflection flag to 0 to set a state where CAL calibration is not performed.

[0207] In S919, the system control unit 50 performs corresponding processing based on the other operation.

[0208] In S920, the system control unit 50 determines whether the sight reflection flag is set to 1. If the sight reflection flag = 1, the process proceeds to S921; otherwise, the process proceeds to S922.

[0209] In S921, the system control unit 50 performs CAL calibration based on ΔP1.

[0210] In S922, the system control unit 50 performs AF at the position of the single-point AF frame based on the operation of the first shutter switch 62 in S916. Therefore, for example, a transition may occur from the display of Figure 3F to the display of Figure 3G .

[0211] On the other hand, although a configuration for performing CAL calibration is described assuming that the single-point AF frame movement operation is completed according to the operation of the first shutter switch 62, the operation member for indicating the completion operation indicating the completion of the frame movement is not limited to the first shutter switch 62. For example, a pressing operation on a pressure-sensitive touch panel or a member (intelligent controller) having a touch detection member on the surface of a pressable button can be used as the above-mentioned completion operation.

[0212] In addition, an example of simultaneously performing CAL calibration and AF at the position of the single-point AF frame is described, but the present invention is not limited thereto. AF may not be performed, or an operation of selecting a subject existing at the position of the single-point AF frame (or a pointer similar to the single-point AF frame) may be performed. An operation of inserting an object (character, etc.) at the position of the pointer may be performed.

[0213] Furthermore, when performing CAL calibration, the system control unit 50 may display a guide on the EVF 29 for indicating whether to perform CAL calibration, and switch the execution / non-execution of CAL calibration according to the response from the user to this guide. The guide may be displayed at a timing when no shooting-related operation is performed so as not to hinder shooting.

[0214] In S923, the system control unit 50 determines whether a touch movement has been made on the touch panel 70a. If a touch movement has been made, the process proceeds to S926, otherwise the process proceeds to S924.

[0215] In S924, the system control unit 50 determines whether a touch stop has been made from the touch panel 70a. If a touch stop has been made, the process proceeds to S925, otherwise the process proceeds to S923.

[0216] In S925, the system control unit 50 determines whether the operation until the touch stop in S924 is a double tap. If the operation is a double tap, the process proceeds to S920, otherwise the process proceeds to S912. An operation of applying a touch to the touch panel 70a and separating the touch without a touch movement is a tap, and a series of operations of making a tap and making a tap again within a predetermined time is a double tap.

[0217] In S926, since the case of making multiple touch movements may be considered as a case where the user has not completed the frame movement operation or the user wishes to move the single-point AF frame to a large extent from the sight position, the system control unit 50 sets the sight reflection flag to 0.

[0218] In S927, the system control unit 50 moves the single-point AF frame in response to a touch movement (movement instruction operation) performed on the touch panel 70a.

[0219] In S928, the system control unit 50 determines whether the first shutter switch 62 is turned on to output a signal SW1. If SW1 has been output, the process proceeds to S527, and AF is performed at the current position of the single-point AF frame. If SW1 has not been output, the process proceeds to S929.

[0220] In S929, the system control unit 50 determines whether a touch stop has occurred on the touch panel 70a. If a touch stop has occurred, the touch movement process during single-point AF ends; otherwise, the process proceeds to S926.

[0221] Touch movement process during face + tracking priority AF

[0222] Figure 10A and 10B is the detailed flowchart of the touch movement process during face + tracking priority AF in S808 of the above Figure 8 above.

[0223] In S1001, the system control unit 50 moves the touch pointer 406 in response to a touch movement (movement instruction operation) performed on the touch panel 70a.

[0224] In S1002, the system control unit 50 determines whether an operation to complete the movement of the touch pointer 406 has been performed. The process of S1002 corresponds to Figure 9B the process of S916 of the above. The operation to complete the movement operation can be, for example, an operation of performing AF by pressing the first shutter switch 62 and completing the movement, or an operation of pressing a touch operation member capable of detecting a pressing operation, etc. If the completion operation has been performed, the process proceeds to S1009; otherwise, the process proceeds to S1003.

[0225] In S1003, the system control unit 50 determines whether a touch stop has occurred on the touch panel 70a. If a touch stop has occurred, the process proceeds to S1004; otherwise, the process proceeds to S1001.

[0226] In S1004, the system control unit 50 determines whether the shortest distance (Δcurrent) from the stored line-of-sight position to the current position of the touch pointer 406 moved in S1001 is greater than the first threshold (Th1) and less than the second threshold (Th2). If Th1 < ΔP2 < Th2, the process proceeds to S1005; otherwise, the process proceeds to S1008. On the other hand, ΔP1 based on the single-point AF frame and ΔP2 based on the touch pointer can be calculated by different calculation methods. For example, ΔP1 is calculated based on the position of the center of the single-point AF frame, while ΔP2 can be calculated based on a predetermined position (such as the center of the face, or the center of gravity, etc.) of the subject tracked in response to pressing the line-of-sight confirmation button 82.

[0227] In S1005, the system control unit 50 determines whether continuous AF is enabled. If continuous AF is enabled, the process proceeds to S1008; otherwise, the process proceeds to S1006. In a scene where shooting is performed by an AF operation of normal AF, in many cases, the subject is a moving object. In this case, the touch movement (AF frame movement operation) after moving the AF frame to near the subject according to the line of sight can be considered as an operation to simply make the AF frame track the subject for a change in the relative position of the subject displayed on the EVF (or display unit), rather than an operation for the accuracy of the line of sight. Therefore, in S906 and S1005, the line-of-sight reflection flag is not set to 1, that is, CAL correction is not performed in the case of continuous AF. On the other hand, the present invention is not limited to this. In the case of single-point AF, even when continuous AF is enabled, execution of CAL correction, etc. can be performed.

[0228] In S1006, the system control unit 50 determines whether other conditions are satisfied. If other conditions are satisfied, the process proceeds to S1007; otherwise, the process proceeds to S1008. The other conditions may include at least any one of the following multiple conditions.

[0229] · The change in the relative position of the subject displayed on the EVF (or display unit) is not greater than the threshold (because if the subject is stabilized to a certain extent, it is considered that the user has finely adjusted the position of the touch pointer 406).

[0230] · Organs (such as pupils) of a human subject are detected (because a scene where a person's pupils are detected is considered to have many stationary subjects).

[0231] · No subject other than a human is detected (because subjects other than humans, such as animals and vehicles, are moving objects in many cases).

[0232] In S1007, the system control unit 50 sets the line-of-sight reflection flag to 1.

[0233] In S1008, the system control unit 50 starts counting the subject confirmation timer. The subject confirmation timer is a timer for performing subject tracking processing after a specific time has elapsed since the user performs a touch stop operation.

[0234] In S1009, the system control unit 50 determines whether Th1 < ΔP2 < Th2 is satisfied. If it is satisfied, the process proceeds to S1026; otherwise, the process proceeds to S1010.

[0235] In S1010, the system control unit 50 performs movement completion processing in response to the operation in S1002. For example, when the operation in S1002 is an operation on the first shutter switch 62, the system control unit 50 tracks the subject near the touch pointer 406 and performs AF. Therefore, for example, a transition from the display of Figure 4E to the display of Figure 4H may occur.

[0236] In S1011, the system control unit 50 determines whether the touch panel 70a has been touched. If it has been touched, the process proceeds to S1014; otherwise, the process proceeds to S1012.

[0237] In S1012, the system control unit 50 determines whether the count of the subject confirmation timer has expired. If the count has expired, the process proceeds to S1013; otherwise, the process proceeds to S1021.

[0238] In S1013, the system control unit 50 sets the gaze reflection flag to 0.

[0239] In S1014, the system control unit 50 determines whether a touch movement has been made on the touch panel 70a. If a touch movement has been made, the process proceeds to S1017; otherwise, the process proceeds to S1015.

[0240] In S1015, the system control unit 50 determines whether a touch stop has been made from the touch panel 70a. If a touch stop has been made, the process proceeds to S1016; otherwise, the process proceeds to S1014.

[0241] In S1016, the system control unit 50 determines whether the operation until the touch stop in S1015 is a double tap. If the operation is a double tap, the process proceeds to S1025; otherwise, the process proceeds to S1021.

[0242] In S1017, the system control unit 50 sets the gaze reflection flag to 0.

[0243] S1018 is the same as S1001.

[0244] S1019 is the same as S1002. If the operation of completing the operation of moving the touch pointer 406 has been performed, the process proceeds to S1010; otherwise, the process proceeds to S1020.

[0245] In S1020, the system control unit 50 determines whether the touch has stopped from the touch panel 70a. If the touch has stopped, the process proceeds to S1008; otherwise, the process proceeds to S1018.

[0246] S1021 is the same as S1002. If the operation of completing the operation of moving the touch pointer 406 has been performed, the process proceeds to S1025; otherwise, the process proceeds to S1022.

[0247] S1022 is the same as S917. If other operations have been performed, the process proceeds to S1023; otherwise, the process proceeds to S1011.

[0248] In S1023, the system control unit 50 sets the line-of-sight reflection flag to 0.

[0249] S1024 is the same as S919.

[0250] In S1025, the system control unit 50 determines whether the line-of-sight reflection flag has been set to 1. If the line-of-sight reflection flag = 1, the process proceeds to S1026; otherwise, the process proceeds to S1010.

[0251] In S1026, the system control unit 50 performs CAL calibration based on ΔP2.

[0252] On the other hand, the electronic device of the present invention is not limited to an electronic device equipped with a touch panel, and the present invention can be applied to any device capable of detecting the movement of an operating body (finger or pen) and moving an indicator (selection position). For example, the present invention can be applied to the following situation: according to the sliding operation performed by a finger (operating body) on the touch panel of a touch notebook PC, the indicator (such as a pointing cursor or an item selection cursor, etc.) displayed on the display of the notebook PC is relatively moved. The present invention is not limited to touch operations, and can also be applied to the situation where the indicator relatively moves according to the operation of a joystick or a rotary dial that indicates a direction through a tilting member. The present invention can also be applied to a device equipped with only a small number of operating members (such as a wearable device, etc.). In addition, the present invention can also be applied to a device that detects the movement of a user's hand in a non-contact manner (such as a spatial gesture, etc.) and moves an indicator (such as an indicator displayed on a projector) according to the movement.

[0253] On the other hand, various types of control performed by the system control unit 50 in the above description can be carried out by a single piece of hardware, or multiple pieces of hardware (e.g., multiple processors or circuits) can control the overall operation of the device by processing in a distributed manner.

[0254] In addition, although the present invention has been described in detail based on suitable embodiments, the present invention is not limited to specific embodiments, and various forms without departing from the basic features of the present invention are also included in the present invention. Moreover, each of the above embodiments merely represents an embodiment of the present invention, and the embodiments can be appropriately combined.

[0255] In addition, although the application of the present invention to a imaging device (digital camera) has been illustrated in the above embodiments, the present invention is not limited thereto, and can be applied to any electronic device capable of receiving gaze input or movement operations. For example, the present invention can be applied to personal computers, PDAs, cellular phones, portable image viewers, printer devices, digital photo frames, music players, game consoles, and e-book readers, etc. Moreover, the present invention can be applied to video players, display devices (including projection devices), tablet terminals, smart phones, AI speakers, household electronic devices, and in-vehicle devices, etc.

[0256] Furthermore, the present invention is not limited to the imaging device main body, and can also be applied to a control device that communicates with an imaging device (including a network camera) via wired or wireless communication and remotely controls the imaging device. As a device for remotely controlling an imaging device, for example, devices such as smart phones, tablet PCs, or desktop PCs can be considered. Based on the operations performed on the control device side or the processing performed on the control device side, commands for causing the imaging device to perform various operations or settings can be notified to the imaging device by the control device to remotely control the imaging device. In addition, the control device can receive a live view image captured by the imaging device via wired or wireless communication and display the live view image on the control device side.

[0257] According to the present disclosure, an electronic device capable of performing calibration to more reliably improve the accuracy of gaze input during normal use can be provided.

[0258] Other Embodiments

[0259] Embodiments of the present invention can also be implemented by the following method, that is, software (program) that executes the functions of the above embodiments is provided to a system or device via a network or various storage media, and a method in which a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0260] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications as well as equivalent structures and functions.

Claims

1. An electronic device, comprising: A line-of-sight input unit configured to receive a line-of-sight input, which is an input based on the position of the user's line of sight; And A control unit configured to perform control such that: An indicator is displayed at a first position based on the line-of-sight input, The indicator is moved from the first position to a second position according to a movement operation performed on an operation unit, where the operation unit receives a user operation different from the line-of-sight input, (a) Calibration based on the first position and the second position according to the input position of the line of sight is not performed when an instruction operation for performing a specific process is not performed at the position of the indicator, and (b) Calibration based on the first position and the second position according to the input position of the line of sight is performed when an instruction operation for performing the specific process is performed in a state without an additional movement operation and satisfying a specific condition, Wherein, when the specific condition is not satisfied, the control unit performs control such that even when an instruction operation for performing the specific process is performed, calibration based on the first position and the second position according to the input position of the line of sight is not performed, and Wherein, the specific condition includes a condition that the distance between the first position and the second position is less than a first threshold.

2. The electronic device according to claim 1, wherein, The specific condition includes a condition that the distance between the first position and the second position is greater than a second threshold.

3. The electronic device according to claim 1 or 2, further comprising a detection unit configured to be able to detect a subject, Among them, The specific condition includes that the detection unit detects a specific subject.

4. The electronic device according to claim 1 or 2, further comprising a detection unit configured to be able to detect a subject, Among them, The specific condition includes at least one of the following: A condition that the position change of the subject detected by the detection unit is not greater than a threshold, A condition that the detection unit detects a person's pupil, and A condition that the detection unit does not detect an animal or a vehicle.

5. The electronic device according to claim 1 or 2, wherein, The specific condition includes that a mode of performing continuous autofocus, i.e., continuous AF, is not set.

6. The electronic device according to claim 1 or 2, wherein, The specific condition includes that multiple position inputs are not performed on the operation unit.

7. The electronic device according to claim 1 or 2, wherein, The specific process is at least one of autofocus and tracking.

8. The electronic device according to claim 1 or 2, wherein The instruction operation for performing the specific process includes a specific touch operation.

9. The electronic device according to claim 8, wherein, The specific touch operation is a double tap.

10. The electronic device according to claim 1 or 2, wherein, The control unit performs control such that: Calibration is performed based on the first position and the second position in a mode of performing autofocus at a position specified by the user, and Calibration is performed based on the first position and the position of the subject selected based on the second position in a mode of performing autofocus at the position of the subject selected based on the position specified by the user.

11. The electronic device according to claim 1 or 2, wherein, The control unit performs control such that when calibration based on the first position and the second position according to the input position of the line of sight is performed, calibration is performed only based on the longitudinal components of the first position and the second position.

12. The electronic device according to claim 1 or 2, wherein, The control unit further performs control such that when calibrating the input position according to the line of sight based on the first position and the second position, a guidance for indicating whether to perform calibration is displayed.

13. The electronic device according to claim 1 or 2, further comprising a setting unit configured to set whether to perform calibration of the input position according to the line of sight based on the first position and the second position according to a user operation.

14. The electronic device according to claim 1 or 2, wherein, The moving operation is an operation of moving the operating body while the operating body touches the operation surface.

15. A control method for an electronic device, comprising: Receiving a line-of-sight input, which is a position input according to the user's line of sight; And Performing control such that: An indicator is displayed at a first position based on the line-of-sight input, The indicator is moved from the first position to a second position according to a moving operation performed on an operation unit, where the operation unit receives a user operation different from the line-of-sight input, (a) When an instruction operation for performing a specific process is not performed at the position of the indicator, calibration of the input position according to the line of sight based on the first position and the second position is not performed, and (b) In a state where there is no additional moving operation and a specific condition is satisfied, when an instruction operation for performing the specific process is performed, calibration of the input position according to the line of sight is performed based on the first position and the second position, wherein, when the specific condition is not satisfied, control is performed such that even when an instruction operation for performing the specific process is performed, calibration of the input position according to the line of sight based on the first position and the second position is not performed, and wherein the specific condition includes a condition that the distance between the first position and the second position is less than a first threshold.

16. A computer-readable medium storing a computer program, which when executed by a processor implements the steps of the control method for the electronic device according to claim 15.

17. A computer program product comprising a computer program, which when executed by a processor implements the steps of the control method for the electronic device according to claim 15.

Images