Electronic device, control method thereof, program, and storage medium

Abstract

To prevent a sensor light for approach detection and a sensor light for sight line detection from affecting each other.SOLUTION: An electronic apparatus includes: an approach detection means configured to detect an object approaching with respect to a finder; sight line detection means configured to detect a sight line position to a display unit that is arranged in the finder; and control means configured to control the sight line detection means to a state in which the sight line can be detected in response to detection of approach of the object by the approach detection means, and to control the sight line detection means to switch from the state in which the sight line can be detected to a state in which the sight line is not detected on the basis of a detection result of the sight line detection means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for controlling approach detection and gaze detection. [Background technology]

[0002] There is a technology that detects the user's line of sight and accepts input according to the line of sight. Patent Document 1 describes a method for saving power by determining whether the user is looking at the rear display and controlling the drive / stop of the eye contact detection unit depending on the presence or absence of the line of sight. Patent Document 2 describes a method for saving power by having an optical viewfinder and a display for checking the captured image, and turning off the display when eye contact or line of sight is detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-120272 A [Patent Document 2] JP 2003-198889 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Documents 1 and 2, when a sensor for detecting eye proximity and a sensor for detecting line of sight are arranged close to each other, the light (sensor light) emitted from each sensor may affect each other, which may result in erroneous detection.

[0005] The present invention has been made in view of the above problems, and has an object to realize a technique for preventing the sensor light for approach detection and the sensor light for line of sight detection from affecting each other. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, an electronic device of the present invention comprises an approach detection means capable of detecting the approach of an object to a viewfinder, a gaze detection means for detecting the position of a gaze relative to a display unit arranged in the viewfinder, and a control means for controlling the gaze detection means to bring the gaze detection means into a state in which the gaze can be detected in response to the approach detection means detecting the approach of an object, and for controlling the gaze detection means to switch from a state in which the gaze can be detected to a state in which the gaze cannot be detected based on a detection result of the gaze detection means. The control means controls the approach detection means to switch from a state in which the approach of an object cannot be detected to a state in which the approach of an object can be detected, depending on a result of determining whether the eye is away from the object or in close contact with the object, based on information detected by the line of sight detection means. do. Effect of the Invention

[0007] According to the present invention, it is possible to prevent the sensor light for approach detection and the sensor light for line of sight detection from affecting each other. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an external view showing the device configuration of the first and second embodiments. [Diagram 2] FIG. 1 is a block diagram showing the device configuration of first and second embodiments. [Diagram 3] 6 is a flowchart showing exclusive control of eye proximity detection processing and gaze detection processing according to the first embodiment. [Figure 4] 11A and 11B are diagrams illustrating examples of eyeball images of a person when infrared light is projected thereon; [Diagram 5] 10 is a flowchart showing exclusive control of eye proximity detection processing and gaze detection processing according to the second embodiment. [Figure 6] FIG. 11A shows the relationship between the eyeball image and the distance to the eyeball in the second embodiment, and FIG. 11B shows the relationship between the P-image interval, the direction of change in the P-image interval, and the distance determination threshold value. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] [Embodiment 1] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which an electronic device of the present invention is applied to a digital camera, which is an image capturing device capable of capturing still images and moving images, will be described in detail below with reference to the accompanying drawings.

[0011] <Device Configuration> The configuration and functions of a digital camera 100 according to this embodiment will be described with reference to FIGS.

[0012] FIG. 1(a) is a front perspective view of a digital camera 100, and FIG. 1(b) is a rear perspective view of the digital camera 100. As shown in FIG.

[0013] In FIG. 1, the rear display unit 101 is a display device such as a liquid crystal panel or an organic EL panel provided on the rear surface of the camera body outside the viewfinder, which displays images and various information so that the user can view them. The rear display unit 101 also has a function of playing back still images after capturing still images, displaying moving images during recording, and displaying a live view. The rear display unit 101 is provided with a touch panel 271. The touch panel 271 is a touch detection means capable of detecting contact (touch operation) with the display surface (operation surface of the touch panel 271) of the rear display unit 101. The outside-of-viewfinder display unit 243 is a display device such as a liquid crystal panel or an organic EL panel provided on the top surface of the camera body, which displays various settings of the camera such as the shutter speed and aperture.

[0014] The shutter button 102 is a push button type operating member for issuing shooting instructions. The mode change switch 103 is a dial type operating member for switching between various modes. The mode change switch 103 switches the operation mode of the system control unit 201 to one of a still image shooting mode, a video recording mode, and a playback mode. The still image shooting modes include, for example, an auto shooting mode, an auto scene discrimination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). The still image shooting modes also include, for example, various scene modes, program AE mode, and custom mode, which are shooting settings according to shooting scenes.

[0015] The mode changeover switch 103 directly switches to one of the multiple modes included in the still image shooting mode. Alternatively, after the mode changeover switch 103 switches to the still image shooting mode once, another operating member may be used to switch to one of the multiple modes included in the still image shooting mode. Similarly, the moving image recording mode and the playback mode may each include multiple modes.

[0016] The terminal cover 104 is a cover member that protects a connector (not shown) for connecting an external device to the digital camera 100 via a cable such as a USB. The main electronic dial 105 is a rotary operation member included in the operation unit 270, which will be described later with reference to Fig. 2. By turning this main electronic dial 105, settings such as the shutter speed and aperture can be changed.

[0017] The power switch 106 is an operating member for switching the power of the digital camera 100 on and off. The sub electronic dial 107 is a rotary operating member for moving the selection frame, forwarding images, and the like. The cross key 108 is a movement instruction member that allows an operation corresponding to the part of the cross key 108 pressed by pressing one of the four directional buttons consisting of up, down, left, and right. The SET button 109 is a push button type operating member that is mainly used for confirming a selection item, etc. The record button 110 is a push button type operating member that switches the live view display on and off in the still image shooting mode, and is used to instruct the start and stop of video shooting (recording) in the video recording mode. The enlargement button 111 is a push button type operating member that is used to turn the enlargement display on and off during the live view, and to change the enlargement ratio during the enlargement display. The enlargement button 111 is also used to enlarge the playback image and increase the enlargement ratio in the playback mode. The live view can be enlarged or reduced by operating the main electronic dial 105 after the enlargement display is turned on. In the playback mode, the button 112 functions as a magnification button for enlarging a playback image and increasing the magnification ratio. The AE lock button 112 is a push button type operation member that can fix the exposure state by pressing it in the shooting standby state. The playback button 113 is a push button type operation member used to switch between the shooting mode and the playback mode. Pressing the playback button 113 in the shooting mode switches to the playback mode, and the latest image among the images recorded in the recording medium 250 can be displayed on the rear display unit 101. The menu button 114 is a push button type operation member that displays a menu screen on the rear display unit 101 on which various settings can be made when pressed. The user can intuitively perform various settings using the menu screen displayed on the rear display unit 101, the cross key 108, and the SET button 109.

[0018] The rear display unit 101 and a finder display unit 229 described later are controlled by the system control unit 201 as an electronic viewfinder (hereinafter, EVF) in accordance with the various operation modes described above.

[0019] The eyepiece 216 is a peer-in type eyepiece finder. A user can view an image displayed on a finder display unit 229 through the eyepiece 216, and can check the focus and composition of the subject image captured through the lens unit 200.

[0020] Eyepiece detection unit 217 is disposed near eyepiece unit 216 and is capable of detecting the approach of an object to eyepiece unit 216. Eyepiece detection unit 217 may be, for example, an infrared proximity sensor.

[0021] The communication terminal 210 is an electrical contact for the digital camera 100 to communicate with the lens unit 200 (FIG. 2). The cover 116 is a member that opens and closes a slot to insert and remove the recording medium 250 into the digital camera 100. The grip section 115 has a shape that is easy for the user to hold with the right hand when holding the digital camera 100. The shutter button 102 and main electronic dial 105 are located at positions that can be operated with the index finger of the right hand when the digital camera 100 is held by gripping the grip section 115 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 107 is located at a position that can be operated with the thumb of the right hand.

[0022] Next, the internal configuration of the digital camera 100 and the lens unit 200 of this embodiment will be described with reference to Fig. 2. In Fig. 2, components common to Fig. 1 are denoted by the same reference numerals.

[0023] 2, the lens unit 200 is equipped with a photographing lens 207 and is detachable from the digital camera 100. The photographing lens 207 is usually composed of multiple lenses, but for simplicity, only one lens is shown here. The communication terminal 206 is an electrical contact for the lens unit 200 to communicate with the digital camera 100. The communication terminal 210 is an electrical contact for the digital camera 100 to communicate with the lens unit 200. The lens unit 200 communicates with the system control unit 201 via the communication terminal 206, and the built-in lens control unit 204 controls the aperture drive circuit 202 to drive the aperture 205, and controls the AF drive circuit 203 to displace the position of the photographing lens 207 to adjust the focus.

[0024] The focal plane shutter 221 can freely control the exposure time of the imaging unit 222 in response to an instruction from the system control unit 201. The imaging unit 222 is an image sensor composed of imaging elements such as CCD or CMOS that convert a subject image into an electrical signal. The A / D converter 223 converts an analog signal of one pixel output from the imaging unit 222 into, for example, a 10-bit digital signal.

[0025] The image processing unit 224 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on the data from the A / D converter 223 or the data from the memory control unit 215. The image processing unit 224 also performs predetermined arithmetic processing using the captured image data, and the system control unit 201 performs exposure control and distance measurement control based on the arithmetic results. This allows TTL (through-the-lens) type AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing to be performed. The image processing unit 224 also performs predetermined arithmetic processing using the captured image data, and TTL type AWB (auto white balance) processing based on the arithmetic results.

[0026] The memory control unit 215 controls data transmission between the A / D converter 223, the image processing unit 224, and the memory 232. Digital data output from the A / D converter 223 is written to the memory 232 via the image processing unit 224 and the memory control unit 215, or directly via the memory control unit 215. The memory 232 stores image data obtained from the imaging unit 222 and the A / D converter 223, and image display data to be displayed on the rear display unit 101 or the in-finder display unit 229. The memory 232 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of video and audio. The memory 232 also serves as a memory for image display (video memory).

[0027] The D / A converter 219 converts image display data stored in the memory 232 into an analog signal and supplies it to the rear display unit 101 or the in-finder display unit 229. The display image data written to the memory 232 is displayed on the rear display unit 101 or the in-finder display unit 229 via the D / A converter 219. The rear display unit 101 or the in-finder display unit 229 performs display on a display device according to the analog signal from the D / A converter 219. In this way, by converting the digital signal stored in the memory 232 into an analog signal and sequentially transferring and displaying it on the rear display unit 101 or the in-finder display unit 229, it functions as an EVF that performs live view (LV) display (through image display).

[0028] Various settings of the camera, such as the shutter speed and aperture, are displayed on the outside viewfinder display section 243 via an outside viewfinder display section drive circuit 244 .

[0029] The non-volatile memory 256 is an electrically erasable and recordable memory, such as an EEPROM. Constants, programs, and the like for the operation of the system control unit 201 are stored in the non-volatile memory 256. The programs referred to here are programs for executing the flowcharts described later.

[0030] The system control unit 201 includes a CPU and MPU that control the entire digital camera 100, and executes programs stored in a nonvolatile memory 256 to realize each process in the flowcharts described below. A system memory 252 is a RAM or the like, and is also used as a work memory for expanding constants and variables for the operation of the system control unit 201, programs read from the nonvolatile memory 256, and the like. The system control unit 201 also performs display control by controlling the memory 232, the D / A converter 219, the rear display unit 101, the in-finder display unit 229, and the like. A system timer 253 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0031] The first shutter switch 211 and the second shutter switch 212 input the following operation instructions to the system control unit 201.

[0032] The first shutter switch 211 is turned on by half-pressing (instruction to prepare for shooting) during operation of the shutter button 102 provided on the digital camera 100, and generates a first shutter switch signal SW1. Upon receiving the first shutter switch signal SW1, the system control unit 201 starts AF processing, AE processing, AWB processing, EF processing, etc. by the image processing unit 224.

[0033] The second shutter switch 212 is turned on when the operation of the shutter button 102 is completed, that is, when the shutter button 102 is fully pressed (photographing instruction), and generates a second shutter switch signal SW2. The system control unit 201 starts a series of photographing processes from reading a signal from the imaging unit 222 to writing image data to the recording medium 250 in response to the second shutter switch signal SW2.

[0034] The operation unit 270 is made up of operation members such as various switches and buttons that accept various operations from the user and notify the system control unit 201, and includes at least the following operation members: a shutter button 102, a mode change switch 103, a main electronic dial 105, a power switch 106, a sub electronic dial 107, a cross key 108, a SET button 109, a record button 110, a magnification button 111, an AE lock button 112, a playback button 113, and a menu button 114.

[0035] The power supply control unit 280 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to be energized, and detects whether a battery is attached, the type of battery, and the remaining battery power. The power supply control unit 280 also controls the DC-DC converter based on the detection results and instructions from the system control unit 201, and supplies the necessary voltage for the necessary period to each unit including the recording medium 250.

[0036] The power supply unit 230 is composed of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li-ion battery, an AC adapter, etc. The recording medium I / F 218 is an interface with a recording medium 250 such as a memory card or a hard disk. The recording medium 250 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, etc.

[0037] The communication unit 254 is communicably connected to an external device via a wireless antenna or a wired cable, and transmits and receives video and audio. The communication unit 254 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 254 can transmit image data (including live view images) captured by the imaging unit 222 and image files recorded in the recording medium 250 to an external device, and can also receive image data and various other information from an external device. Note that the communication unit 254 is not limited to wireless LAN, and may use a wireless communication module such as infrared communication, Bluetooth (registered trademark), Bluetooth (registered trademark) Low Energy, or Wireless USB, or a wired connection means such as a USB cable, HDMI (registered trademark), or IEEE1394.

[0038] The attitude detection unit 255 detects the attitude of the digital camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 255, it is possible to determine whether the image captured by the imaging unit 222 was captured with the digital camera 100 held horizontally or vertically. The system control unit 201 can add orientation information corresponding to the attitude detected by the attitude detection unit 255 to the image file of the image captured by the imaging unit 222, or rotate and record the image. An acceleration sensor, a gyro sensor, or the like can be used as the attitude detection unit 255. By using an acceleration sensor or a gyro sensor, the attitude detection unit 255 can also detect the movement of the digital camera 100 (panning, tilting, lifting, standing still, etc.).

[0039] The operation unit 270 includes a touch panel 271 capable of detecting a touch operation on the rear display unit 101. The touch panel 271 and the rear display unit 101 can be configured as one unit. For example, the touch panel 271 is configured so that the light transmittance does not interfere with the display of the rear display unit 101, and is attached to the upper layer of the display surface of the rear display unit 101. Then, input coordinates on the touch panel 271 are associated with display coordinates on the rear display unit 101. This makes it possible to configure a GUI that makes it appear as if the user can directly operate the screen displayed on the rear display unit 101. The system control unit 201 can detect the following operations or states on the touch panel 271.

[0040] A finger or pen that has not been touching the touch panel 271 touches the touch panel 271 again, that is, the start of touching (hereinafter referred to as touch-down).

[0041] The touch panel 271 is in a state of being touched with a finger or a pen (hereinafter, referred to as Touch-On).

[0042] Touching the touch panel 271 with a finger or a pen and moving it (hereinafter referred to as Touch-Move).

[0043] The finger or pen that has been touching the touch panel 271 is released, that is, the touch ends (hereinafter, referred to as "touch-up").

[0044] A state in which nothing is being touched on the touch panel 271 (hereinafter referred to as Touch-Off).

[0045] When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. Touch-move is also detected while touch-on is detected. Even if touch-on is detected, touch-move will not be detected if the touch position has not moved. After it is detected that all fingers or pens that were touching have touched up, touch-off occurs.

[0046] These operations and states, and the position coordinates of the touch panel 271 touched by a finger or pen are notified to the system control unit 201 via the internal bus, and the system control unit 201 determines what kind of operation (touch operation) has been performed on the touch panel 271 based on the notified information. Regarding touch-move, the moving direction of the finger or pen moving on the touch panel 271 can also be determined for each vertical component and horizontal component on the touch panel 271 based on the change in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation (drag) has been performed. An operation in which a finger is touched on the touch panel, quickly moved a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced on the touch panel 271 as if flicking it. If a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is detected immediately, it can be determined that a flick has been performed (it can be determined that a flick has occurred following a drag). Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer to each other is called pinch in, and a touch operation in which the touch positions are moved away from each other is called pinch out. Pinch out and pinch in are collectively called pinch operation (or simply pinch). The touch panel 271 may be of any of various touch panel types, such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. Depending on the type, there are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or a pen to the touch panel, and either type may be used.

[0047] The eyepiece detection unit 217 detects the approach (eye approach) and departure (eye departure) of the eye (object) to the eyepiece unit 216 (approach detection). The system control unit 201 switches between display (display state) / non-display (non-display state) of the rear display unit 101 and the in-finder display unit 229 depending on the state detected by the eyepiece detection unit 217. At least in shooting mode and when the display destination switching is automatic, the system control unit 201 sets the display destination to the rear display unit 101 while the eye is not placed near the subject and sets the in-finder display unit 229 to a non-display state. Also, when the eye is placed near the subject, the display destination is set to the in-finder display unit 229 and the rear display unit 101 to a non-display state.

[0048] When an object approaches, infrared light emitted from a light-projecting section (not shown) of the eyepiece detection section 217 is reflected and enters a light-receiving section (not shown) of the infrared proximity sensor. Depending on the amount of incident infrared light received by the infrared proximity sensor, it is possible to detect the approach of an object to the eyepiece section 216 and to determine how close the object is to the eyepiece section 216 (eyepiece distance). When it detects the approach of an object to the eyepiece section 216, the system control section 201 can start displaying the in-finder display section 229. This makes it possible to display the in-finder display section 229 with as little delay as possible when the user looks through the eyepiece section 216.

[0049] In addition, when the eye contact detection unit 217 detects an object approaching within a predetermined distance from the eye contact unit 216 from a non-eye contact state (non-approach state), it determines that the eye contact has been detected, and transmits an eye contact detection notification to the system control unit 201. In addition, when the object that was detected as approaching from the eye contact state (approach state) moves away from the eye contact state by a predetermined distance or more, it determines that the eye contact has been detected, and transmits an eye contact detection notification to the system control unit 201. The threshold value for detecting the eye contact and the threshold value for detecting the eye contact may be different, for example, by providing a hysteresis. In addition, after the eye contact is detected, the eye contact state is assumed to remain until the eye contact is detected. After the eye contact is detected, the non-eye contact state is assumed to remain until the eye contact is detected. In this way, the system control unit 201 performs display control of the rear display unit 101 and the in-finder display unit 229 according to the eye contact state or the eye contact state detected by the eye contact detection unit 217.

[0050] The eye proximity detection unit 217 is not limited to an infrared proximity sensor, and any other sensor may be used as long as it can detect the approach of an eye or an object that can be regarded as an eye.

[0051] The gaze detection unit 260 includes a dichroic mirror 262, an imaging lens 263, a gaze detection sensor 264, a gaze detection circuit 265, and an infrared light emitting element 266, and detects not only the presence or absence of the user's gaze but also the position and movement of the gaze.

[0052] The infrared light emitting element 266 is a diode that emits infrared light for detecting the position of the user's line of sight within the finder screen, and irradiates the infrared light toward the center of the eyepiece 216 toward the user's eyeball (eye) 261. The infrared light irradiated from the infrared light emitting element 266 is reflected by the eyeball (eye) 261, and the reflected infrared light reaches the dichroic mirror 262. The dichroic mirror 262 has a function of reflecting only infrared light and transmitting visible light, and the reflected infrared light whose optical path has been changed forms an image on the imaging surface of the line of sight detection sensor 264 via the imaging lens 263.

[0053] The imaging lens 263 is an optical member that constitutes the line of sight detection optical system. The line of sight detection sensor 264 includes an image sensor made of a CCD, a CMOS, or the like. The line of sight detection sensor 264 photoelectrically converts the incident reflected infrared light into an electric signal and outputs it to the line of sight detection circuit 265. The line of sight detection circuit 265 detects the user's line of sight position from the movement of the user's eyeball (eye) 261 and the position of the pupil based on the output signal of the line of sight detection sensor 264, and outputs the detected information to the system control unit 201. Since the line of sight detection sensor 264 can detect the pupil of the human eye, it does not detect that a human line of sight is being input even if another object approaches or touches the eyepiece unit 216. In this way, the eyepiece unit 216 has a function as a line of sight operation unit, but the line of sight detection unit may have a different configuration.

[0054] The line of sight detection unit 260 can detect the following information when placed in contact with the eyepiece unit 216. At least one of the detection state of the pupil edge 161a and the corneal reflection image (Purkinje image = P image) 161b input to the eyepiece unit 216 Distance, amount of change, and direction of change between the P images 161b input to the eyepiece 216 These pieces of information are sent to the system control unit 201 via the internal bus, and the system control unit 201 determines whether the eye is in close contact with the eyepiece unit 216 or not, based on the information sent from the line-of-sight detection unit 260 .

[0055] The eye gaze input function by the eye gaze detection unit 260 can be enabled / disabled by the user via a menu screen, for example.

[0056] <Control Processing> Next, the exclusive control of the eye proximity detection processing and the line of sight detection processing in the first embodiment will be described with reference to Figs.

[0057] In the following, a description will be given of exclusive control of the eye proximity detection process and the gaze detection process in response to the eye separation determination based on the disappearance of the P image and the pupil edge.

[0058] 3(a) and (b) are flowcharts showing exclusive control of eye proximity detection processing and gaze detection processing in embodiment 1. The processing in Fig. 3(a) and (b) is realized when the power supply of the digital camera 100 is turned on and the system control unit 201 loads a program stored in the non-volatile memory 256 into the system memory 252, executes it, and controls the functional blocks of each unit. The processing in Fig. 3(a) and (b) is started when the power supply of the digital camera 100 is turned on.

[0059] 3A, in S302, the system control unit 201 switches the eye proximity detection unit 217 to a detectable state. The system control unit 201 switches the eye proximity detection unit 217 to a state in which the infrared light projecting unit of the eye proximity detection unit 217 projects infrared light toward the outside and the infrared light reflected by the object can be received by the infrared light receiving sensor.

[0060] In S303, the system control unit 201 determines whether or not the user has placed the eye of the eyepiece unit 216 close to the eye, based on the detection result of the eyepiece detection unit 217. If the system control unit 201 determines that the user has not placed the eye of the eyepiece unit 216 close to the eye, the process proceeds to S310, and if the system control unit 201 determines that the user has placed the eye of the eyepiece unit 216 close to the eye, the process proceeds to S304.

[0061] In S304, the system control unit 201 stops (hides) the display on the rear display unit 101, starts the display on the viewfinder display unit 229, and proceeds to S305.

[0062] In S305, the system control unit 201 determines whether the eye-gaze input function is enabled or not, and if the eye-gaze input function is enabled, the process proceeds to S306, and if the eye-gaze input function is not enabled, the process returns to S303. The user can enable / disable the eye-gaze input by using the menu button 114 (on the menu screen).

[0063] In S306, the system control unit 201 sets the eye proximity detection unit 217 to a non-detection state, and stops driving the infrared light projector and infrared light receiver of the eye proximity detection unit 217.

[0064] This allows the line of sight detection unit 260 to operate without being affected by the infrared light emitted from the infrared light projector of the eyepiece detection unit 217 when performing line of sight detection in the next step and thereafter, preventing erroneous detection during line of sight detection. Also, stopping the driving of the infrared light projector of the eyepiece detection unit 217 enables power saving.

[0065] In S307, the system control unit 201 performs gaze detection processing. Fig. 3B is a flowchart showing the details of the gaze detection processing in S307.

[0066] In S3002, the system control unit 201 puts the gaze detection unit 260 into a detectable state. The system control unit 201 irradiates the infrared light emitting element 266 of the gaze detection unit 260 toward the eyepiece unit 216, and drives the gaze detection sensor 264 to a detectable state of infrared light. As a result, the infrared light emitting element 266 reaches the user's eyeball via the eyepiece unit 216, and the infrared light reaches the gaze detection sensor 264 further via the eyepiece unit 216.

[0067] Here, the P image and the pupil edge will be described with reference to FIG.

[0068] Figure 4 shows an image of a person's eyeball illuminated with infrared light. When infrared light is illuminated on the eyeball, one or more bright spots (two in Figure 4) called P image 161b appear as reflected images. The pupil is a hole in the center of the iris, and the edge 161a of the pupil can be detected from the brightness difference or the amount of defocus.

[0069] In S3003, the system control unit 201 acquires an image of the user's pupils by the gaze detection sensor 264, and outputs the acquired image to the gaze detection circuit 265.

[0070] In S3004, the system control unit 201 causes the line of sight detection circuit 265 to acquire the edge 161a of the user's pupil.

[0071] In S3005, the system control unit 201 causes the line-of-sight detection circuit 265 to acquire the P image 161b of the user.

[0072] In S3006, the system control unit 201 calculates the center position of the pupil from the edge 161a of the pupil, and calculates the gaze position from the relationship with the P image 161b.

[0073] In S3007, the system control unit 201 changes various settings for the gaze position, ends the gaze detection process, and proceeds to S308. In S3007, various settings for the shooting conditions are changed, such as changing the focus adjustment position so that the camera is focused on the gaze point.

[0074] In S308, the system control unit 201 determines whether or not at least one of the pupil edge 161a and the P image 161b has been detected by the eye proximity detection unit 217. If at least one of the pupil edge 161a and the P image 161b has been detected, the system determines that the eye is in the near-eye state, and returns to S307 to continue gaze detection; if neither of these has been detected, the system determines that the eye is in the far-eye state, and proceeds to S309.

[0075] In S309, the system control unit 201 sets the gaze detection unit 260 to the non-detection state. The system control unit 201 stops driving the infrared light emitting element 266 and the gaze detection sensor 264.

[0076] In S310, the system control unit 201 drives the rear display unit 101 and stops the display on the finder display unit 229.

[0077] In S311, the system control unit 201 determines whether or not the power switch 106 has been turned off by the user. If the power switch 106 remains on, the process returns to S302 and the detection state of the eye proximity detection unit 217 continues. If the power switch 106 is off, the process proceeds to S312 and ends the process.

[0078] As described above, according to the first embodiment, it is possible to prevent interference between light for detecting the eye-closed / non-eye-closed state and light for detecting the gaze position. In the first embodiment, the eye-closed state or the gaze-away state is determined based on the output result of the gaze detection unit 260 (here, the presence or absence of the pupil edge 161a and the P image 161b). This allows the eye-closed / eye-away determination to be performed even when the eye-closed detection unit 217 is in the non-detection state, so that the eye-closed detection process and the gaze detection process can be controlled exclusively. The gaze detection sensor 264 of the gaze detection unit 260 can operate without being affected by the infrared light irradiated from the infrared light projector of the eye-closed detection unit 217, and erroneous detection during gaze detection can be prevented. In addition, the infrared light receiving sensor of the eye-closed detection unit 217 can operate without being affected by the infrared light irradiated from the infrared light emitting element 266 of the gaze detection unit 260, and erroneous detection during eye-closed detection can be prevented. Furthermore, the exclusive control of the eye-closed detection process and the gaze detection process can also achieve a power saving effect.

[0079] [Second embodiment] Next, a second embodiment will be described with reference to Figs.

[0080] The second embodiment is a method of performing control based on the results of determining the interval between P images and the direction of change in the P image interval in exclusive control of eye proximity detection processing and line of sight detection processing.

[0081] FIG. 5 is a flowchart showing exclusive control of eye proximity detection processing and gaze detection processing according to the second embodiment.

[0082] The processes from S502 to S507 and S513 to S516 in FIG. 5 are similar to the processes from S302 to S307 and S309 to S312 in FIG. 3(a) and FIG. 3(b), and therefore description thereof will be omitted.

[0083] In S508, the system control unit 201 causes the line-of-sight detection circuit 265 to calculate the amount of change per unit time for the distance between the two points of the P image 161b acquired by the line-of-sight detection sensor 264. If the distance between the two points of the P image 161b is not in the reduction direction (= there is no change in enlargement or distance), the process proceeds to S509, and if it is in the reduction direction, the process proceeds to S511. That is, if it is determined that the user's eyes are moving away from the eyepiece unit 216, the process proceeds to S511, and if not, the process proceeds to S509.

[0084] In S509, the system control unit 201 sets the P-image interval determination threshold value used for determining the P-image interval in the line of sight detection circuit 265 to α.

[0085] In S510, the system control unit 201 determines whether or not the P-image interval is equal to or less than the threshold value α using the gaze detection circuit 265. If it is equal to or less than the threshold value α, it determines that the eye is not focused and proceeds to S513, whereas if it exceeds the threshold value α, it proceeds to S507 and continues gaze detection.

[0086] In S511, the system control unit 201 sets the P-image interval determination threshold value used for determining the P-image interval in the line of sight detection circuit 265 to β.

[0087] In S512, the system control unit 201 determines whether the P-image interval is equal to or less than a threshold value β using the gaze detection circuit 265. If the P-image interval is equal to or less than the threshold value β, it determines that the eye is off and proceeds to S513, whereas if the P-image interval exceeds the threshold value β, it proceeds to S507 and continues gaze detection.

[0088] Regarding the above-mentioned processing from S508 to S512, a method of determining the eye distance state based on the P-image interval and the direction of change in the P-image interval will be described with reference to FIG.

[0089] Fig. 6(a) conceptually shows the relationship between the eyeball image in the second embodiment and the approach distance of the eyeball to the digital camera 100. Fig. 6(b) shows the relationship between the threshold for determining eye distance based on the P-image interval and the direction of change of the P-image interval in the second embodiment.

[0090] FIG. 6(a) shows the change in the interval of P images 161b acquired by the line of sight detection sensor 264 with respect to the position of the user's face when the eyepiece 216 is located at the right end of the figure. The eyepiece position (i) is 65 mm away from the eyepiece 216, and is a position boundary condition for the eyepiece detection unit 217 to determine that the eyepiece is in an eye-contact state. The interval of the P images at this position is narrow and is assumed to be a value A. The eyepiece position (iii) is 23 mm away from the eyepiece 216, and is a position in which the user holds the eyepiece 216 with part of his or her face in contact with it. The interval of the P images at this position is wide and is assumed to be a value C. If the intermediate position between the eyepiece positions (i) and (iii) is assumed to be (ii), and the value of the interval of the P images is assumed to be B, the magnitude of the value has the following relationship. A <B<C Next, the threshold for determining eye deviation based on the P-image interval in relation to the direction of change in the P-image interval will be explained using FIG. 6(b). As a premise, it is determined that the eye deviation is occurring when the P-image distance is below the threshold. The vertical axis indicates the value of the P-image interval, and the horizontal axis indicates the direction of change in the P-image interval (binary values: shrinking direction and other (= enlargement or no change)). When the user's face is at position X between eye position (ii) and eye position (iii), the value of the P-image interval will be below C and above B. The threshold for determining eye deviation is set to threshold α when the P-image interval is not shrinking, and threshold β when it is shrinking. The relationship between the values ​​of each threshold is as follows: α<A、β> C That is, α<β.

[0091] For example, when the user's face is at position X, if the P-image interval is not in the contraction direction (= enlargement or no change), the threshold value becomes α and the P-image interval exceeds α, so that it is determined that the eye is close to the object based on the detection result of the gaze detection unit 260. When the P-image interval is in the contraction direction, the threshold value becomes β and the P-image interval is equal to or smaller than β, so that it is determined that the eye is far from the object based on the detection result of the gaze detection unit 260.

[0092] By setting the threshold value α to be equal to or less than the value A, it is possible to reliably determine that the eye is in the close-focus state when the P-image interval is in the enlargement direction other than the reduction direction.

[0093] By setting the threshold value β to a value equal to or greater than C, it is possible to reliably determine that the eye is far from the target when the P-image interval is reduced.

[0094] As described above, according to the second embodiment, it is possible to prevent interference between light for detecting the eye-closed / non-eye-closed state and light for detecting the gaze position. In the second embodiment, the influence of infrared light is reduced, and the eye-closed state or the eye-away state can be determined based on the P image interval and the change direction of the P image interval. This makes it possible to reflect the user's intention in a state where the user's face is neither close to nor far from the eyepiece unit 216 and the determination cannot be made only from the P image. In other words, it becomes possible to determine whether the user wants to look at the rear display unit 101 by looking away from the eye or to maintain the eye-closed state based on the change direction of the distance of the P image and a threshold value, and it becomes possible to execute a function more desired by the user.

[0095] The various controls described above as being performed by the system control unit 201 may be performed by a single piece of hardware, or the entire device may be controlled by a plurality of pieces of hardware sharing the processing.

[0096] In addition, although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments merely shows one embodiment of the present invention, and each embodiment can be appropriately combined.

[0097] In the above-mentioned embodiment, the present invention is applied to a digital camera, but the present invention is not limited to this and can be applied to any device having an eye contact detection function and a gaze input function. That is, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printer devices equipped with displays, digital photo frames, music players, game consoles, electronic book readers, tablet terminals, smartphones, projection devices, home appliance devices equipped with displays, and in-vehicle devices.

[0098] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0099] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0100] 100: digital camera, 101: rear display unit (EVF), 201: system control unit, 216: eyepiece unit, 217: eyepiece detection unit, 229: in-finder display unit (EVF), 260: line-of-sight detection unit

Claims

1. an approach detection means capable of detecting an approach of an object to the viewfinder; a line of sight detection means for detecting a line of sight with respect to a display unit disposed within the viewfinder; controlling the line-of-sight detection means to be in a state in which the line of sight detection is possible in response to the approach of an object being detected by the approach detection means; a control means for controlling the gaze detection means to switch the gaze detection means from a state in which the gaze can be detected to a state in which the gaze cannot be detected based on a detection result of the gaze detection means; having The electronic device is characterized in that the control means controls the approach detection means to switch from a state in which the approach of an object cannot be detected to a state in which the approach of an object can be detected, depending on the result of determining whether the eye is away from the object or close to the object based on the information detected by the gaze detection means.

2. The electronic device according to claim 1, characterized in that the control means controls the approach detection means to switch from a state in which approach cannot be detected to a state in which approach can be detected in response to determining the eye-away state based on information detected by the gaze detection means.

3. 3. The electronic device according to claim 1, wherein the control means controls the approach detection means to switch the approach detection means from a state in which the approach can be detected to a state in which the approach cannot be detected when the approach detection means detects the approach of an object.

4. The electronic device according to claim 3, characterized in that, when the approach detection means detects the approach of an object, the control means switches the approach detection means from a state in which the approach of an object can be detected to a state in which the approach of an object cannot be detected based on information regarding gaze detection by the gaze detection means.

5. The electronic device described in claim 4, characterized in that the control means controls the approach detection means to switch to a state in which the approach of an object cannot be detected, and then, in response to determining a state of looking away based on information detected by the gaze detection means, to switch the approach detection means to a state in which the approach of an object can be detected.

6. 6. The electronic device according to claim 1, wherein the state in which the gaze detection means does not detect the gaze is a state in which the operation of an infrared light-emitting element and a gaze detection sensor constituting the gaze detection means is stopped.

7. 7. The electronic device according to claim 1, wherein the state in which the approach detection means is unable to detect the approach of an object is a state in which driving of an infrared light projecting section constituting the approach detection means is stopped.

8. 7. The electronic device according to claim 1, wherein the state in which the approach detection means is unable to detect the approach of an object is a state in which the operation of an infrared light projector and an infrared light receiver that constitute the approach detection means is stopped.

9. 9. The electronic device according to claim 1, wherein the control means determines whether the eye is in a far-away state or in a near-eye state based on a detection result relating to a pupil edge and a corneal reflection image by the line-of-sight detection means.

10. The electronic device according to claim 9, characterized in that the control means determines that the eye is away from the subject when the gaze detection means does not detect the edge of the pupil, and controls the approach detection means to switch from a state in which approach cannot be detected to a state in which approach can be detected.

11. The electronic device according to claim 9 or 10, characterized in that when the gaze detection means detects the edge of the pupil, the control means determines that the eye is in the close-contact state, controls the gaze detection means to continue in a state in which the gaze can be detected, and controls the approach detection means not to detect the approach of an object.

12. The electronic device according to any one of claims 9 to 11, characterized in that when the corneal reflection image is not detected by the gaze detection means, the control means determines that the eye is apart and controls the approach detection means to switch from a state in which approach cannot be detected to a state in which approach can be detected.

13. The electronic device according to any one of claims 9 to 12, characterized in that when the corneal reflection image is detected by the gaze detection means, the control means determines that the eye is in the close-contact state, controls the gaze detection means to continue in a state in which the gaze can be detected, and controls the approach detection means not to detect the approach of an object.

14. The electronic device according to any one of claims 9 to 13, characterized in that the control means determines that the eye is apart when the interval between the corneal reflection images detected by the gaze detection means is equal to or less than a predetermined value, and controls the approach detection means to switch from a state in which approach cannot be detected to a state in which approach can be detected.

15. The electronic device according to any one of claims 9 to 14, characterized in that when the interval between the corneal reflection images detected by the gaze detection means is greater than a predetermined value, the control means determines that the eye is in the close-contact state, controls the gaze detection means to continue in a state in which the gaze can be detected, and controls the approach detection means not to detect the approach of an object.

16. The control means When the approach detection means detects that an object has approached, a display unit disposed within the viewfinder is turned on and a first display unit disposed outside the viewfinder is turned off; 16. The electronic device according to claim 1, wherein the gaze detection means controls the display unit located within the viewfinder to be turned off and the first display unit located outside the viewfinder to be turned on when the gaze detection means detects no gaze.

17. an approach detection step capable of detecting an approach of an object to a viewfinder; a line of sight detection step for detecting a line of sight with respect to a display unit disposed within the viewfinder; In response to the approach of an object being detected in the approach detection step, the line of sight detection means is controlled to be in a state in which the line of sight can be detected; a control step of controlling the gaze detection means to switch from a state in which the gaze can be detected to a state in which the gaze cannot be detected based on a detection result of the gaze detection step; having A control method for an electronic device, characterized in that in the control step, the approach detection means is controlled to switch from a state in which the approach of an object cannot be detected to a state in which the approach of an object can be detected, depending on the result of determining whether the eye is away from the object or close to the object based on the information detected in the gaze detection step.

18. A program for causing a computer to function as the control means for the electronic device according to any one of claims 1 to 16.

19. A computer-readable storage medium storing a program for causing a computer to function as the control means for the electronic device according to any one of claims 1 to 16.

Images