Electronic device, method for controlling electronic device, program, and storage medium

Abstract

To provide an electronic device capable of making stable line-of-sight detection.SOLUTION: An electronic device (100) has approach detection means (217) capable of detecting the approach of an object to a viewfinder, gaze detection means (260) for detecting gaze information with respect to a first display unit (229) located inside the viewfinder, determination means (201a) for determining whether the state is an eyepiece state or a non-eyepiece state on the basis of the detection result of the gaze detection means, and control means (201b) for controlling to drive the proximity detection means from a stopped state when the determination means determines that the state is a non-eyepiece state and to stop the driving of the gaze detection means when the approach detection means does not detect the approach of an object.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method for an electronic device, a program, and a storage medium.

Background Art

[0002] Conventionally, an electronic device including an eye contact sensor (proximity detection means) for detecting eye contact and a gaze detection sensor (gaze detection means) for detecting a user's gaze is known. In such an electronic device, there is a possibility of false detection due to the light (sensor light) emitted from each sensor affecting each other. Patent Document 1 discloses an electronic device that drives the gaze detection means and stops the eye contact detection means when the eye contact detection means detects eye contact, and stops the gaze detection means and drives the proximity detection means when the gaze detection means cannot detect the gaze. By controlling the proximity detection means and the gaze detection means in this way, it is possible to prevent the sensor lights from affecting each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the electronic device disclosed in Patent Document 1, when determining eye departure, even if the eyes shift slightly beside the viewfinder, the gaze cannot be detected, so it is determined as eye departure, and the gaze detection means is stopped and the eye contact detection means is driven. Also, every time a blink occurs, the gaze cannot be detected, so the stop of the gaze detection means and the driving of the eye contact detection means are repeated every time a blink occurs. As a result, stable gaze detection cannot be performed.

[0005] Therefore, an object of the present invention is to provide an electronic device capable of performing stable line-of-sight detection while preventing the sensor light of the eyepiece detection means and the line-of-sight detection means from affecting each other.

Means for Solving the Problems

[0006] An electronic device according to an aspect of the present invention includes an approach detection means capable of detecting the approach of an object to a viewfinder, a line-of-sight detection means for detecting line-of-sight information with respect to a first display portion disposed inside the viewfinder, a determination means for determining whether it is in an eyepiece state or a non-eyepiece state based on a detection result of the line-of-sight detection means, and when the determination means determines that it is in the non-eyepiece state, driving the approach detection means from a stopped state, and control means for controlling to stop driving the line-of-sight detection means when the approach detection means does not detect the approach of the object.

[0007] Other objects and features of the present invention will be described in the following embodiments.

Effects of the Invention

[0008] According to the present invention, it is possible to provide an electronic device capable of performing stable line-of-sight detection while preventing the sensor light of the eyepiece detection means and the line-of-sight detection means from affecting each other.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Although a plurality of features are described in each embodiment, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. In each figure, the same members are denoted by the same reference numerals, and redundant descriptions are omitted. (First Embodiment) First, with reference to FIG. 1, the configuration and functions of an imaging device (digital camera, camera body) 100 as an electronic device in the first embodiment will be described. FIGS. 1(a) and (b) are external views of the imaging device 100, where FIG. 1(a) shows a front perspective view of the imaging device 100 and FIG. 1(b) shows a rear perspective view of the imaging device 100. The imaging device 100 is a digital camera capable of shooting still images or moving images.

[0011] The rear display unit 101 is a display device such as a liquid crystal panel or an organic EL panel provided on the rear surface of the imaging device 100 outside the viewfinder that displays images and various information so that the user can view them. Further, the rear display unit 101 has functions of still image playback after still image shooting, moving image display during recording, and live view display. A touch panel 271 is provided on the rear display unit 101. The touch panel 271 is a touch detection means capable of detecting contact (touch operation) with the display surface of the rear display unit 101 (the operation surface of the touch panel 271). The outside-the-viewfinder display unit 243 is a display device such as a liquid crystal panel or an organic EL panel provided on the upper surface of the camera body, and displays various setting values of the camera such as the shutter speed and aperture.

[0012] The shutter button 102 is a push-button type operation member for giving a shooting instruction. The mode switching switch 103 is a dial type operation member for switching various modes. The mode switching switch 103 switches the operation mode of the system control unit 201 to any one of a still image shooting mode, a moving image recording mode, and a playback mode. The still image shooting mode includes, for example, an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), a program AE mode (P mode). Also, the still image shooting mode includes, for example, various scene modes that are shooting settings according to the shooting scene, a program AE mode, a custom mode, etc. The mode switching switch 103 directly switches to any one of a plurality of modes included in the still image shooting mode. Alternatively, after once switching to the still image shooting mode with the mode switching switch 103, it may be possible to switch to any one of a plurality of modes included in the still image shooting mode using another operation member. Similarly, the moving image recording mode and the playback mode may also include a plurality of modes.

[0013] The terminal cover 104 is a cover member that protects a connector (not shown) for connecting the external device and the imaging device 100 via a cable such as USB. The main electronic dial 105 is a rotary operation member included in the operation unit 270 described later with reference to FIG. 2. By turning the main electronic dial 105, setting values such as the shutter speed and aperture can be changed. The power switch 106 is an operation member for switching the on / off of the power of the imaging device 100. The sub electronic dial 107 is a rotary operation member for performing operations such as moving the selection frame and image scrolling. The cross key 108 is a movement instruction member that can perform an operation according to the pressed portion of the cross key 108 by pressing any one of the four-direction buttons consisting of up, down, left, and right.

[0014] The SET button 109 is a push-button type operation member mainly used for determining selected items and the like. The REC button 110 is a push-button type operation member that switches the on / off of the live view display in the still image shooting mode and is used to instruct the start and stop of video shooting (recording) in the video recording mode. The ZOOM button 111 is a push-button type operation member used to turn on / off the enlarged display during live view and to change the magnification during enlarged display. Also, the ZOOM button 111 is used to enlarge the playback image and increase the magnification in the playback mode. After turning on the enlarged display, the main electronic dial 105 can be operated to enlarge or reduce the live view. Also, in the playback mode, it functions as a zoom button for enlarging the playback image and increasing the magnification.

[0015] The AE LOCK button 112 is a push-button type operation member that can fix the exposure state by pressing it in the shooting standby state. The PLAYBACK button 113 is a push-button type operation member used to switch between the shooting mode and the playback mode. By pressing the PLAYBACK button 113 during the shooting mode, the user can shift to the playback mode and display the latest image recorded on the recording medium 250 on the rear display unit 101. The MENU button 114 is a push-button type operation member that, when pressed, displays various configurable menu screens on the rear display unit 101. The user can intuitively perform various settings using the menu screens displayed on the rear display unit 101, the cross keys 108, and the SET button 109.

[0016] The rear display unit (second display unit) 101 and the in-finder display unit (first display unit disposed inside the finder) 229 described later are used as an electronic viewfinder (EVF) and are display-controlled by the system control unit 201 according to the various operation modes described above. The eyepiece unit 216 is a peering-type eyepiece finder. The user can visually recognize the image displayed on the in-finder display unit 229 through the eyepiece unit 216, and can check the focus and composition of the subject image captured through the lens device 200. The eye detection unit (eye detection means) 217 is disposed near the eyepiece unit 216 and can detect the approach of an object to the eyepiece unit 216 (the approach of an object to the finder). For example, an infrared proximity sensor is used as the eye detection unit 217.

[0017] The communication terminal 210 is an electrical contact for the imaging device 100 to communicate with the lens device 200 (see FIG. 2). The lid 116 is a member that opens and closes a slot for attaching and detaching the recording medium 250 to and from the imaging device 100. The grip portion 115 has a shape that is easy to hold with the right hand when the user holds the imaging device 100. With the grip portion 115 held by the little finger, ring finger, and middle finger of the right hand and the imaging device 100 held, the shutter button 102 and the main electronic dial 105 are disposed at positions operable with the index finger of the right hand. Also, in the same state, the sub electronic dial 107 is disposed at a position operable with the thumb of the right hand.

[0018] Next, with reference to FIG. 2, the internal configuration of the imaging system 10 in the present embodiment will be described. FIG. 2 is a block diagram of the imaging system 10. The imaging system 10 includes an imaging device (camera body) 100 and a lens device (interchangeable lens) 200 that is detachable from the imaging device. However, the present embodiment is not limited to this, and is also applicable to an imaging device in which the camera body and the lens device are integrally configured. In FIG. 2, the same reference numerals are given to the configurations common to FIG. 1.

[0019] The lens device 200 has an imaging optical system 207. The imaging optical system 207 is usually composed of a plurality of lenses, but here it is simply shown as only one lens. The communication terminal 206 is an electrical contact for the lens device 200 to communicate with the imaging device 100. The communication terminal 210 is an electrical contact for the imaging device 100 to communicate with the lens device 200. The lens device 200 communicates with the system control unit 201 via the communication terminal 206, and the built-in lens control unit 204 controls the aperture drive circuit 202 to drive the aperture 205, and controls the AF drive circuit 203 to displace the position of the imaging optical system 207 to focus.

[0020] The focal plane shutter 221 can freely control the exposure time in the imaging unit 222 according to the instruction of the system control unit 201. The imaging unit 222 is an image sensor composed of an imaging element (photoelectric conversion element) such as a CCD or CMOS that converts a subject image into an electrical signal. The A / D converter 223 converts an analog signal of one pixel output from the imaging unit 222 into a digital signal of, for example, 10 bits. The image processing unit 224 performs predetermined pixel interpolation, resizing processes such as reduction, and color conversion processes on the data from the A / D converter 223 or the data from the memory control unit 215. Also, in the image processing unit 224, predetermined arithmetic processing is performed using the captured image data, and the system control unit 201 performs exposure control and distance measurement control based on the arithmetic result. Thereby, TTL (through-the-lens) type AF (autofocus) processing, AE (auto exposure) processing, and EF (flash pre-emission) processing are performed. Also, the image processing unit 224 performs predetermined arithmetic processing using the captured image data, and performs TTL type AWB (auto white balance) processing based on the arithmetic result.

[0021] The memory control unit 215 controls the transfer of data to and from the A / D converter 223, the image processing unit 224, or the memory 232. The digital data output from the A / D converter 223 is written directly into the memory 232 via the image processing unit 224 and the memory control unit 215, or via the memory control unit 215. The memory 232 stores the image data obtained from the imaging unit 222 and the A / D converter 223, as well as the image display data for display on the rear display unit 101 or the in-finder display unit 229. The memory 232 has a storage capacity sufficient to store a predetermined number of still images, a video of a predetermined duration, and audio. Also, the memory 232 doubles as an image display memory (video memory). The D / A converter 219 converts the image display data stored in the memory 232 into an analog signal and supplies it to the rear display unit 101 or the in-finder display unit 229.

[0022] The image display data written into the memory 232 is displayed by the rear display unit 101 or the in-finder display unit 229 via the D / A converter 219. The rear display unit 101 and the in-finder display unit 229 perform display according to the analog signal from the D / A converter 219 on the display device. In this way, by converting the digital signal stored in the memory 232 into an analog signal and sequentially transferring it to the rear display unit 101 or the in-finder display unit 229 for display, it functions as an EVF that performs live view (LV) display (through-image display). Various setting values of the camera, such as the shutter speed and aperture, are displayed on the out-of-finder display unit 243 via the out-of-finder display unit drive circuit 244.

[0023] The non-volatile memory 256 is electrically erasable and recordable, such as an EEPROM. Constants for the operation of the system control unit 201, programs, etc. are stored in the non-volatile memory 256. Here, the program refers to the program for executing the flowchart described later. The system control unit 201 includes a CPU or MPU that comprehensively controls the entire imaging device 100, and realizes each process of the flowchart described later by executing the program stored in the non-volatile memory 256. The system memory 252 is a RAM or the like, and is also used as a work memory for expanding constants, variables for the operation of the system control unit 201, programs read from the non-volatile memory 256, etc. Further, the system control unit 201 also performs display control by controlling the memory 232, the D / A converter 219, the rear display unit 101, or the in-finder display unit 229, etc.

[0024] The system timer 253 is a timing unit that measures the time used for various controls and the time of the built-in clock. The first shutter switch 211 and the second shutter switch 212 input the following operation instructions to the system control unit 201. The first shutter switch 211 turns on during the operation of the shutter button 102 provided in the imaging device 100, that is, during a so-called half-press (shooting preparation instruction), and generates a first shutter switch signal SW1. The system control unit 201 receives the first shutter switch signal SW1 and starts AF processing, AE processing, AWB processing, EF processing, etc. by the image processing unit 224. The second shutter switch 212 turns on when the operation of the shutter button 102 is completed, that is, during a so-called full-press (shooting instruction), and generates a second shutter switch signal SW2. The system control unit 201 starts a series of shooting processes from the signal reading from the imaging unit 222 to writing the image data to the recording medium 250 by the second shutter switch signal SW2.

[0025] The operation unit 270 is an operation member such as various switches or buttons that accepts various operations from the user and notifies the system control unit 201. The operation members include, for example, the shutter button 102, the mode switching switch 103, the main electronic dial 105, the power switch 106, the sub - electronic dial 107, and the cross key 108. The imaging members also include the SET button 109, the recording button 110, the zoom - in button 111, the AE - lock button 112, the playback button 113, and the menu button 114. The power control unit 280 is composed of a battery detection circuit, a DC - DC converter, a switch circuit for switching the energized block, etc., and detects the presence or absence of battery installation, the type of battery, and the remaining battery level. Also, the power control unit 280 controls the DC - DC converter based on the detection result and the instruction of the system control unit 201, and supplies the necessary voltage to each part including the recording medium 250 for the necessary period. The power supply unit 230 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li - ion batteries, and AC adapters. The recording medium I / F 218 is an interface with a recording medium 250 such as a memory card or a hard disk.

[0026] The recording medium 250 is a recording medium such as a memory card for recording the captured images, and is composed of a semiconductor memory, a magnetic disk, etc. The communication unit 254 is communicably connected to external devices via a wireless antenna or a wired cable, and transmits and receives video and audio. The communication unit 254 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 254 can transmit the image data (including the live - view image) captured by the imaging unit 222 and the image files recorded on the recording medium 250 to external devices, and can also receive image data and other various information from external devices. Note that the communication unit 254 may use not only a wireless LAN, but also wireless communication modules such as infrared communication, Bluetooth (registered trademark), Bluetooth (registered trademark) Low Energy, WirelessUSB, etc. Or the communication unit 254 may use wired connection means such as a USB cable, HDMI (registered trademark), IEEE1394, etc.

[0027] The posture detection unit 255 detects the posture of the imaging device 100 with respect to the direction of gravity. Based on the posture detected by the posture detection unit 255, it is possible to determine whether the image captured by the imaging unit 222 is an image captured with the imaging device 100 held horizontally or an image captured with it held vertically. The system control unit 201 can add orientation information corresponding to the posture detected by the posture detection unit 255 to the image file of the image captured by the imaging unit 222, or rotate and record the image. The posture detection unit 255 has, for example, an acceleration sensor or a gyro sensor. By using the acceleration sensor or the gyro sensor, the posture detection unit 255 can also detect the movement (such as panning, tilting, lifting, and stationary) of the imaging device 100.

[0028] As one of the operation units 270, the imaging device 100 has a touch panel 271 capable of detecting a touch operation on the rear display unit 101. The touch panel 271 and the rear display unit 101 can be integrally configured. For example, the touch panel 271 is configured so that the light transmittance does not interfere with the display of the rear display unit 101, and is attached to the upper layer of the display surface of the rear display unit 101. Then, the input coordinates on the touch panel 271 and the display coordinates on the rear display unit 101 are associated with each other. Thereby, a GUI can be configured as if the user can directly operate the screen displayed on the rear display unit 101.

[0029] The system control unit 201 can detect the following operations or states on the touch panel 271. (1) A finger or pen that was not touching the touch panel 271 newly touches the touch panel 271. That is, the start of a touch (hereinafter referred to as a touch-down). (2) The state of touching the touch panel 271 with a finger or pen (hereinafter referred to as a touch-on). (3) Moving while touching the touch panel 271 with a finger or pen (hereinafter referred to as a touch-move). (4) Releasing the finger or pen that was touching the touch panel 271. That is, the end of the touch (hereinafter referred to as Touch-Up). (5) A state where nothing is touching the touch panel 271 (hereinafter referred to as Touch-Off).

[0030] When a Touch Down is detected, it is also detected that it is Touch On at the same time. After Touch Down, unless a Touch Up is detected, usually Touch On continues to be detected. When a Touch Move is detected, it is also in a state where Touch On is detected. Even if Touch On is detected, if the touch position has not moved, Touch Move is not detected. After it is detected that all the fingers and pens that were touching have performed a Touch Up, it becomes Touch Off. These operations / states and the position coordinates on the touch panel 271 where the finger or pen is touching are notified to the system control unit 201 through the internal bus. Then, the system control unit 201 determines what kind of operation (touch operation) has been performed on the touch panel 271 based on the notified information.

[0031] Regarding Touch Move, the moving direction of the finger or pen moving on the touch panel 271 can also be determined for each vertical and horizontal component on the touch panel 271 based on the change in position coordinates. When it is detected that a Touch Move has occurred over a predetermined distance or more, it is determined that a slide operation (drag) has been performed. An operation of quickly moving a finger a certain distance while touching the touch panel and then releasing it is called a flick. In other words, a flick is an operation of quickly tracing on the touch panel 271 as if flicking with a finger. When it is detected that a Touch Move has occurred over a predetermined distance or more at a predetermined speed or more and a Touch Up is detected immediately afterwards, it can be determined that a flick has been performed (it can be determined that there was a flick following a drag). Furthermore, a touch operation where multiple locations (for example, two points) are touched simultaneously and the touch positions are brought closer together is called a pinch-in, and a touch operation where the touch positions are moved apart from each other is called a pinch-out. The pinch-out and pinch-in are collectively referred to as a pinch operation (or simply a pinch).

[0032] The touch panel 271 may be of any type among various types of touch panels, such as a resistive film type, a capacitance type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, an optical sensor type, etc. Depending on the type, there are a method of detecting a touch when there is contact with the touch panel and a method of detecting a touch when there is an approach of a finger or a pen to the touch panel, and any method may be used.

[0033] The eye detection unit 217 detects the approach (eye contact) and separation (eye release) of the eye (object) to the eyepiece unit 216 (approach detection). The system control unit 201 switches the display (display state) / non-display (non-display state) of the rear display unit 101 and the in-finder display unit 229 according to the state detected by the eye detection unit 217. When at least in the shooting mode and the switching of the display destination is automatic, the system control unit 201 sets the display destination to the rear display unit 101 and the in-finder display unit 229 is not displayed when not in eye contact.

[0034] As a specific implementation method, when an object approaches, the infrared light irradiated from the light projecting unit (not shown) of the eye detection unit 217 is reflected and enters the light receiving unit (not shown) of the infrared proximity sensor. Based on the incident light amount of the infrared light received by the infrared proximity sensor, it is possible to determine the detection of the approach of the object to the eyepiece unit 216 and how close the object has approached the eyepiece unit 216 (eye contact distance). When detecting the approach of the object to the eyepiece unit 216, it is possible to transmit the approach information of the object to the eyepiece unit 216 to the system control unit 201, and the system control unit 201 can start the display of the in-finder display unit 229. Thereby, when the user looks into the eyepiece unit 216, the in-finder display unit 229 can be displayed with as little delay as possible.

[0035] Also, when an object that has been detected approaching from a state where the system control unit 201 determines an eye-contact state (approach state) moves away by a predetermined distance or more, information on the separation of the object from the eyepiece unit 216 is transmitted to the system control unit 201. Then, the system control unit 201 stops the display on the in-finder display unit 229 and performs the display on the rear display unit 101. Note that the threshold for detecting eye contact and the threshold for detecting eye separation may be different, for example, by providing hysteresis. Also, after detecting eye contact, it is assumed to be in the eye-contact state until eye separation is detected. After detecting eye separation, it is assumed to be in the non-eye-contact state until eye contact is detected. Thus, the system control unit 201 performs display control of the rear display unit 101 and the in-finder display unit 229 according to the eye-contact state or eye-separation state detected by the eye-contact detection unit 217. Note that the eye-contact detection unit 217 is not limited to an infrared proximity sensor, and other sensors may be used as long as they can detect the approach of an eye or an object that can be regarded as an eye contact.

[0036] The imaging device 100 of the present embodiment includes a gaze detection device (gaze detection means) 260 capable of detecting information on the user's line of sight (the line-of-sight position with respect to the in-finder display unit 229) when the user is looking into the eyepiece unit 216. The gaze detection unit 260 includes a dichroic mirror 262, an imaging lens 263, a gaze detection sensor 264, a gaze detection circuit 265, and an infrared light-emitting element 266, and detects not only the presence or absence of the user's line of sight but also the position and movement of the line of sight.

[0037] The infrared light-emitting element 266 is a diode that emits infrared light for detecting the position of the user's line of sight within the finder screen, and irradiates the user's eyeball (eye) 261 with infrared light toward the vicinity of the center of the eyepiece unit 216. The infrared light irradiated from the infrared light-emitting element 266 is reflected by the eyeball (eye) 261, and the reflected infrared light reaches the dichroic mirror 262. The dichroic mirror 262 has a function of reflecting only infrared light and transmitting visible light, and the reflected infrared light whose optical path has been changed forms an image on the imaging surface of the gaze detection sensor 264 via the imaging lens 263. The imaging lens 263 is an optical member that constitutes the gaze detection optical system.

[0038] The line-of-sight detection sensor 264 includes an image sensor such as a CCD or a CMOS. The line-of-sight detection sensor 264 photoelectrically converts the incident reflected infrared light into an electrical signal and outputs it to the line-of-sight detection circuit 265. The line-of-sight detection circuit 265 detects the user's line-of-sight position from the movement of the user's eyeball (eye) 261 and the position of the pupil based on the output signal of the line-of-sight detection sensor 264, and outputs the detected information (detection result) to the system control unit 201. Since the line-of-sight detection sensor 264 can detect the pupil of a person's eye, even if another object approaches or contacts the eyepiece 216, it does not detect that the person's line of sight is input. Thus, the eyepiece 216 has a function as a line-of-sight operation unit, but the line-of-sight detection unit may have a different configuration.

[0039] The line-of-sight detection unit 260 can detect the following information while being in contact with the eyepiece 216. (1) The detection state of at least one of the pupil edge 161a and the corneal reflex image (Purkinje image = P image) 161b input to the eyepiece 216 (2) The distance, change amount, and change direction between the P images 161b input to the eyepiece 216 These pieces of information are notified to the system control unit 201 through the internal bus, and the system control unit 201 determines the contact state or non-contact state with respect to the eyepiece 216 based on the information notified from the line-of-sight detection unit 260. As described above, in order to measure the eyeball information using the light (e.g., infrared light) from the light-emitting means, there is a possibility that correct line-of-sight information (line-of-sight position) cannot be detected when light enters from other light sources. Therefore, it is necessary to stop the light projection of the eye contact detection unit 217. Note that the validity / invalidity of the line-of-sight input function by the line-of-sight detection unit 260 can be set by the user via, for example, a menu screen.

[0040] In this embodiment, the system control unit 201 includes a determination means 201a and a control means 201b. The determination means 201a determines whether it is an eye contact state or a non-eye contact state (eye-off state) based on the detection result of the eye line detection unit 260. When the determination means 201a determines that it is a non-eye contact state, the control means 201b drives the eye contact detection unit 217 from a stopped state and controls to stop the drive of the eye line detection unit 260 when the eye contact detection unit 217 does not detect the approach of an object.

[0041] Next, with reference to FIGS. 3(a), (b) and 4, the control method (exclusive control of eye contact detection process and eye line detection process) in this embodiment will be described. Hereinafter, the exclusive control of the eye contact detection process and the eye line detection process according to the eye-off determination based on the disappearance of the P image and the pupil edge will be described. FIG. 3(a) is a flowchart showing the control method (exclusive control of eye contact detection process and eye line detection process) in this embodiment. FIG. 3(b) is a flowchart showing step S306 (eye line detection process) in FIG. 3(a). The processes in FIGS. 3(a) and (b) are realized by turning on the power of the imaging device 100, the system control unit 201 expanding and executing the program stored in the non-volatile memory 256 in the system memory 252, and controlling each functional block. Also, the processes in FIGS. 3(a) and (b) are started when the power of the imaging device 100 is turned on.

[0042] First, in step S301 of FIG. 3(a), the system control unit 201 switches the eye contact detection unit 217 to a detectable state. The system control unit 201 makes the infrared light projection unit of the eye contact detection unit 217 project infrared light outward so that the infrared light reflected by the objective can be received by the infrared light receiving sensor. Subsequently, in step S302, the system control unit 201 determines whether the user is in an eye contact state with respect to the eye contact part 216 based on the detection result of the eye contact detection unit 217. When the system control unit 201 determines that it is not in an eye contact state, step S302 is repeated. On the other hand, when the system control unit 201 determines that it is in an eye contact state, it proceeds to step S303.

[0043] In step S303, the system control unit 201 (control means 201b) stops (hides) the display on the rear display unit 101 and starts the display on the in-finder display unit 229. Subsequently, in step S304, the system control unit 201 determines whether the eye-gaze input function is enabled or not. If the eye-gaze input function is enabled, the process proceeds to step S305. On the other hand, if the eye-gaze input function is not enabled, the process returns to step S302. The enable / disable of the eye-gaze input function can be set by the user using the menu button 114 (on the menu screen).

[0044] In step S305, the system control unit 201 sets the eyepiece detection unit 217 to a non-detection state and stops the driving of the infrared light emitting unit and the infrared light receiving sensor of the eyepiece detection unit 217 (eyepiece detection off). Subsequently, in step S306, the system control unit 201 performs an eye-gaze detection process. That is, when the determination means 201a determines that the user is in an eye-attached state, the control means 201b stops the driving of the eyepiece detection unit 217 and then drives the eye-gaze detection unit 260.

[0045] Here, with reference to FIG. 3(b), the eye-gaze detection process will be described in detail. First, in step S3001, the system control unit 201 sets the eye-gaze detection unit 260 to a detectable state. The system control unit 201 irradiates the infrared light emitting element 266 of the eye-gaze detection unit 260 toward the eyepiece unit 216 and drives the eye-gaze detection sensor 264 to a state where it can detect infrared light. As a result, the infrared light emitting element 266 reaches the user's eyeball through the eyepiece unit 216, and further, the infrared light reaches the eye-gaze detection sensor 264 through the eyepiece unit 216.

[0046] Here, with reference to FIG. 4, the P image and the pupil edge will be described. FIG. 4 is an eyeball image of a person irradiated with infrared light. One or more (two in FIG. 4) bright spots called P images 161b appear as reflection images on the eyeball irradiated with infrared light. Also, the pupil is a hole in the center of the iris, and the edge 161a of the pupil can be detected from the luminance difference.

[0047] In step S3002 of FIG. 3(b), the system control unit 201 acquires an image (pupil image) of the user by the gaze detection sensor 264, and outputs the acquired image to the gaze detection circuit 265. Subsequently, in step S3003, the system control unit 201 acquires the edge 161a of the user's pupil by the gaze detection circuit 265. Subsequently, in step S3004, the system control unit 201 acquires the P image 161b of the user by the gaze detection circuit 265.

[0048] Subsequently, in step S3005, the system control unit 201 calculates the center position of the pupil from the edge 161a of the pupil, and calculates (acquires) the gaze position (gaze information) from the relationship with the P image 161b. Subsequently, in step S3006, the system control unit 201 makes various setting changes at the gaze position. For example, various setting changes include various setting changes in the shooting conditions, such as changing the focus adjustment position so as to focus the imaging device 100 on the fixation point (gaze position). Then, when the process of step S3006 is completed, the gaze detection process ends, and the process proceeds to step S307 in FIG. 3(a).

[0049] In step S307, the system control unit 201 (determination means 201a) determines whether at least one of the edge 161a of the pupil or the P image 161b is detected by the eye contact detection unit 217. When at least one of the edge 161a of the pupil or the P image 161b is detected, the system control unit 201 determines that the eye contact state is present (the user is in the process of making eye contact), and returns to step S305. In this embodiment, the determination is made based on whether the P image 161b is detected, but the present invention is not limited to this. For example, the determination may be made based on the number of P images. As a result, when performing gaze detection in subsequent steps, the gaze detection unit 260 can operate without being affected by the infrared light irradiated from the infrared light projection unit of the eye contact detection unit 217, and false detection during gaze detection can be prevented. In addition, by stopping the drive of the infrared light projection unit of the eye contact detection unit 217, power saving can be achieved.

[0050] On the other hand, if neither the pupil edge 161a nor the P image 161b is detected in step S307, the system control unit 201 determines that the user has removed their eyes (non-eye-contact state) and proceeds to step S308. In step S308, the system control unit 201 re-drives the infrared light-emitting unit and the infrared light-receiving sensor of the eye-contact detection unit 217 to switch the eye-contact detection unit 217 to a detectable state (turn on eye-contact detection). Subsequently, in step S309, the system control unit 201 determines whether the user is in an eye-contact state with respect to the eye-contact unit 216 based on the detection result of the eye-contact detection unit 217. If the system control unit 201 determines that the user is not in an eye-contact state, it proceeds to step S310. On the other hand, if the system control unit 201 determines that the user is in an eye-contact state, it returns to step S305.

[0051] In step S310, the system control unit 201 sets the gaze detection unit 260 to a non-detection state (turn off gaze detection). For this reason, the system control unit 201 stops driving the infrared light-emitting element 266 and the gaze detection sensor 264. As a result, even when the user blinks or their eyes shift horizontally and the gaze cannot be detected, the eye-contact detection unit 217 can detect the eye-contact state. Therefore, the system control unit 201 can prevent misjudging the user's eye-removed state (non-eye-contact state) and continue gaze detection.

[0052] Subsequently, in step S311, the system control unit 201 starts the display on the rear display unit 101 and stops the display on the in-finder display unit 229. Subsequently, in step S312, the system control unit 201 determines whether the power switch 106 has been turned off by the user. If the power switch 106 remains in the on state, it returns to step S302, and the system control unit 201 continues to monitor the detection state of the eye-contact detection unit 217. On the other hand, if the power switch 106 is in the off state, this flow ends.

[0053] According to the present embodiment, while preventing the light for detecting the eyepiece / non-eyepiece state and the light for detecting the line-of-sight position from affecting each other's sensors, the line-of-sight detection can be continued even when blinking or the eyes shift horizontally. Also, in the present embodiment, the system control unit 201 as the determination means determines the eyepiece state or the non-eyepiece state (non-eyepiece state) based on the detection result of the line-of-sight detection unit 260 (for example, the presence or absence of the pupil edge 161a and the P image 161b). Thereby, even if the eyepiece detection unit 217 is in a non-detection state, it is possible to determine whether the user is looking through the eyepiece or not, so that it becomes possible to exclusively control the eyepiece detection process and the line-of-sight detection process.

[0054] Also, even when the user intends to fixate on a single point, the eyes are slightly moving. Therefore, since the line-of-sight detection result also becomes unstable, generally, in order to suppress the slight movement of the eyes, smoothing processing is performed using the past line-of-sight detection history. At this time, if the line-of-sight detection sensor stops every time the user blinks or the eyes shift horizontally, it becomes impossible to perform line-of-sight detection at a stable cycle, the smoothing processing for suppressing the slight movement of the eyes becomes unstable, and it becomes difficult to suppress the slight movement of the line-of-sight detection. Therefore, according to the present embodiment, by entrusting the final determination to the eyepiece detection unit 217 in the non-eyepiece state, it is possible to continuously perform line-of-sight detection without stopping the line-of-sight detection even in a state where a line of sight such as blinking or horizontal eye shift cannot be detected. Thereby, line-of-sight detection can be performed at a stable cycle, and it becomes possible to stably suppress the slight movement of the line-of-sight detection.

[0055] (Second Embodiment) Next, with reference to FIG. 5, the second embodiment will be described. This embodiment relates to a method of performing control based on the interval of the P images in the exclusive control of the eyepiece detection process and the line-of-sight detection process. FIG. 5 is a flowchart showing the control method (exclusive control of the eyepiece detection process and the line-of-sight detection process) in the present embodiment. Note that steps S501 to S506 and S508 to S512 in FIG. 5 are the same as steps S301 to S306 and S308 to S312 in FIG. 3(a), respectively, and thus their descriptions are omitted.

[0056] In step S507, the system control unit 201 determines whether the interval (P-image interval) between two points of the P-image 161b acquired by the gaze detection sensor 264 is greater than a threshold value (P-image interval determination threshold value) D. If the P-image interval is greater than the threshold value D, the system control unit 201 determines that it is in the eye-contact state and returns to step S505. On the other hand, if the P-image interval is less than or equal to the threshold value D, the system control unit 201 determines that it is in the eye-separation state and proceeds to step S508. Note that the processing after step S508 overlaps with the description of the first embodiment, so those descriptions are omitted.

[0057] Here, with reference to FIG. 6, the processing of step S507 (a method for determining the eye-separation state based on the P-image interval) will be described. FIG. 6 is a diagram showing the relationship between the eye image of this embodiment and the distance to the eye (proximity distance of the eye to the imaging device 100). At the right end in FIG. 6, the change in the interval (P-image interval) of the P-image 161b acquired by the gaze detection sensor 264 with respect to the position of the user's face in the state where the eyepiece part 216 is arranged is shown.

[0058] The eye-contact position (i) is a position 65 mm away from the eyepiece part 216 and corresponds to the boundary determined to be in the eye-contact state by the eye-contact detection unit 217 (position boundary condition). The interval (P-image interval) of the P-image 161b at this position is narrow. For example, let the P-image interval be the value A. The eye-contact position (iii) is a position 23 mm away from the eyepiece part 216 and is the position in the state where the user holds a part of the face in contact with the eyepiece part 216. The P-image interval at this position is wide. For example, let the P-image interval be the value C. If the intermediate position between the eye-contact position (i) and the eye-contact position (iii) is defined as the eye-contact position (ii), the value B of the P-image interval satisfies the following relationship.

[0059] A < B < C According to this embodiment, while preventing the light for detecting the eyepiece / non-eyepiece state and the light for detecting the line-of-sight position from affecting each other's sensors, it is possible to continue detecting the line of sight even when blinking or the eyes shift horizontally. Further, in this embodiment, while preventing the influence of infrared light, it is possible to determine the eyepiece state or the non-eyepiece state based on the P-image interval. As a result, even when the eyepiece detection unit 217 is in a non-detection state, it is possible to determine eyepiece / non-eyepiece, so that it becomes possible to exclusively control the eyepiece detection process and the line-of-sight detection process. Also, in the non-eyepiece state, by entrusting the final determination to the eyepiece detection unit 217, it is possible to continuously detect the line of sight without stopping the line-of-sight detection even in a state where a line of sight such as blinking or the eyes shifting horizontally cannot be detected. As a result, it is possible to detect the line of sight at a stable cycle and stably suppress the fine movement of the line-of-sight detection.

[0060] (Third Embodiment) Next, with reference to FIG. 7, the third embodiment will be described. This embodiment relates to a method of controlling whether or not to continue line-of-sight detection based on the elapsed time since the non-eyepiece determination was first made in the exclusive control of the eyepiece detection process and the line-of-sight detection process. FIG. 7 is a flowchart showing the control method (exclusive control of the eyepiece detection process and the line-of-sight detection process) in this embodiment. Note that steps S701 to S708 and S714 to S716 in FIG. 7 are the same as steps S301 to S308 and S310 to S312 in FIG. 3(a), respectively, and thus the descriptions thereof are omitted. In FIG. 7, instead of step S707, step S507 in FIG. 5 may be inserted.

[0061] In step S709, after the non-eyepiece determination is made in step S707, the system control unit 201 determines whether or not the infrared light projection unit and the infrared light receiving sensor of the eyepiece detection unit 217 have been re-driven for the first time in step S708 (whether it is the first time the eyepiece detection is turned on). If it is the first re-drive, the process proceeds to step S710, and the system control unit 201 acquires the re-drive time from the system timer 253. On the other hand, if it is the second or subsequent re-drive, the process proceeds to step S711.

[0062] In step S711, the system control unit 201 acquires the current time from the system timer 253, and calculates the elapsed time (the elapsed time from the first eye contact detection on) from the time (the re-driving time) acquired in step S710. Subsequently, in step S712, the system control unit 201 determines whether the user is in an eye contact state with respect to the eye contact unit 216 based on the detection result of the eye contact detection unit 217. If the system control unit 201 determines that it is in an eye contact state, it proceeds to step S713. On the other hand, if the system control unit 201 determines that it is not in an eye contact state, it proceeds to step S714.

[0063] In step S713, the system control unit 201 determines whether the elapsed time (the elapsed time from the first eye contact detection on) acquired in step S511 is greater than the threshold value (elapsed time determination threshold value) T. If the elapsed time is greater than the threshold value T, it returns to step S705. On the other hand, if the elapsed time is less than or equal to the threshold value T, it proceeds to step S714.

[0064] In the present embodiment, when the control means 201b determines that the non-eye contact state has continued for a predetermined time after the determination means 201a determines the non-eye contact state, the control means 201b stops the driving of the line-of-sight detection unit 260. Therefore, according to the present embodiment, in addition to the effects of the first embodiment or the second embodiment, power saving can be achieved by stopping the driving of the line-of-sight detection sensor 264 in a state where the line of sight cannot be continuously detected.

[0065] Note that each of the above-described various controls described as being performed by the system control unit 201 may be performed by one piece of hardware, or the entire control of the apparatus may be performed by a plurality of pieces of hardware sharing the processing.

[0066] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0067] According to each embodiment, it is possible to provide an electronic device, an imaging device, a control method for an electronic device, and a program that can perform stable line-of-sight detection while preventing the sensor light of the eye detection means and the line-of-sight detection means from affecting each other.

[0068] The disclosure of each embodiment includes the following configurations and methods.

[0069] (Configuration 1) Proximity detection means capable of detecting the approach of an object to the viewfinder, Line-of-sight detection means for detecting line-of-sight information with respect to a first display unit disposed inside the viewfinder, Determination means for determining whether it is an eye-attached state or a non-eye-attached state based on the detection result of the line-of-sight detection means, Control means for driving the proximity detection means from a state where the driving is stopped when the determination means determines that it is the non-eye-attached state, and stopping the driving of the line-of-sight detection means when the proximity detection means does not detect the approach of the object. An electronic device characterized by comprising. (Configuration 2) The electronic device according to Configuration 1, wherein the control means drives the proximity detection means from a state where the driving is stopped when the determination means determines that the state has changed from the eye-attached state to the non-eye-attached state. (Configuration 3) The electronic device according to Configuration 1 or 2, wherein the detection result of the line-of-sight detection means is the edge of the pupil. (Configuration 4) The electronic device according to any one of Configurations 1 to 3, wherein the detection result of the line-of-sight detection means is the number of corneal reflection images. (Configuration 5) The electronic device according to any one of Configurations 1 to 4, wherein the detection result of the line-of-sight detection means is the interval between corneal reflection images. (Configuration 6) The electronic device according to any one of Configurations 1 to 5, wherein the control means stops driving the line-of-sight detection means when it is determined that the non-eye-contact state has continued for a predetermined time after the determination means determines the non-eye-contact state. (Configuration 7) The electronic device according to any one of Configurations 1 to 6, wherein the control means drives the line-of-sight detection means after stopping driving the proximity detection means when the determination means determines that it is in the eye-contact state. (Configuration 8) The control means when the proximity detection means detects the approach of the object, starts the display on the first display unit and stops the display on the second display unit arranged outside the viewfinder, The electronic device according to any one of Configurations 1 to 7, wherein when the proximity detection means does not detect the approach of the object, starts the display on the first display unit and stops the display on the second display unit. (Configuration 9) The electronic device according to any one of Configurations 1 to 8, wherein the electronic device is an imaging device. (Method 1) A control method for an electronic device having a proximity detection means capable of detecting the approach of an object to a viewfinder and a line-of-sight detection means for detecting line-of-sight information for a first display unit arranged inside the viewfinder, a step of determining whether it is in an eye-contact state or a non-eye-contact state based on the detection result of the line-of-sight detection means; a step of driving the proximity detection means from a state where driving is stopped when it is determined that it is in the non-eye-contact state based on the detection result of the line-of-sight detection means; When the proximity detection means does not detect the proximity of the object, a step of stopping the driving of the line-of-sight detection means, and a control method for an electronic device, characterized in that it has the above. (Configuration 10) A program characterized in that the control method according to Method 1 is executed by a computer. (Configuration 11) A computer-readable storage medium characterized by storing the program according to Configuration 10.

[0070] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist. Also, it is possible to appropriately combine each embodiment.

[0071] Also, in each embodiment, an imaging device (digital camera) has been described as an example of an electronic device. However, the present invention is not limited to this, and it is widely applicable to devices (electronic devices) having an eyepiece detection means and a line-of-sight detection means. For example, it is also applicable to personal computers, PDAs, mobile phone terminals, portable image viewers, printer devices, digital photo frames, music players, game machines, electronic book readers, tablet terminals, smartphones, projection devices, home appliance devices, in-vehicle devices, etc.

Explanation of reference numerals

[0072] 100 Imaging device (electronic device) 201a Determination means 201b Control means 217 Eyepiece detection unit (proximity detection means) 229 In-finder display unit (first display unit) 260 Line-of-sight detection unit (line-of-sight detection means)

Claims

1. Approach detection means capable of detecting the approach of an object to the viewfinder, Eye-gaze detection means for detecting eye-gaze information with respect to a first display unit disposed inside the viewfinder, Determination means for determining whether it is in an eyepiece state or a non-eyepiece state based on the detection result of the eye-gaze detection means, Control means for driving the approach detection means from a state where the driving is stopped when the determination means determines that it is in the non-eyepiece state, and for stopping the driving of the eye-gaze detection means when the approach detection means does not detect the approach of the object. An electronic device characterized by having the above.

2. The electronic device according to claim 1, wherein the control means drives the approach detection means from a state where the driving is stopped when the determination means determines that the state has changed from the eyepiece state to the non-eyepiece state.

3. The electronic device according to claim 1, wherein the detection result of the eye-gaze detection means is the edge of the pupil.

4. The electronic device according to claim 1, wherein the detection result of the eye-gaze detection means is the number of corneal reflection images.

5. The electronic device according to claim 1, wherein the detection result of the eye-gaze detection means is the interval between corneal reflection images.

6. The electronic device according to claim 1, wherein the control means stops the driving of the eye-gaze detection means when it is determined that the non-eyepiece state has continued for a predetermined time after the determination means determines that it is in the non-eyepiece state.

7. The electronic device according to claim 1, wherein the control means drives the approach detection means before driving the eye-gaze detection means, and if the approach detection means detects the approach of the object before driving the eye-gaze detection means, the control means stops the driving of the approach detection means and then drives the eye-gaze detection means.

8. The control means, when the approach detection means detects the approach of the object, starts the display of the first display unit and stops the display of a second display unit disposed outside the viewfinder, when the approach detection means does not detect the approach of the object, starts the display of the second display unit and stops the display of the first display unit. The electronic device according to claim 1 is characterized by the above.

9. The electronic device according to any one of claims 1 to 8, wherein the electronic device is an imaging device.

10. A control method for an electronic device having proximity detection means capable of detecting the approach of an object to a viewfinder and line-of-sight detection means for detecting line-of-sight information with respect to a first display unit disposed inside the viewfinder, a step of determining whether it is in an eyepiece state or a non-eyepiece state based on the detection result of the line-of-sight detection means, when it is determined that it is in the non-eyepiece state based on the detection result of the line-of-sight detection means, a step of driving from a state where the driving of the proximity detection means is stopped, a step of stopping the driving of the line-of-sight detection means when the proximity detection means does not detect the approach of the object, characterized in that it comprises a control method for an electronic device.

11. A program characterized in that the control method according to claim 10 is executed by a computer.

12. A computer-readable storage medium characterized by storing the program according to claim 11.

Images