Electronic device, its control method, and program

Abstract

To make a transition, in an electronic apparatus having a display, to a power mode according to the degree of a user's gaze on the display.SOLUTION: An electronic apparatus having a display has: a detection unit that detects the degree of a user's gaze on the display; a determination unit that determines, according to a result of detection performed by the detection unit, whether to make a transition to a normal mode indicating a normal drive state or a power saving mode consuming less power than the normal mode; and a control unit that causes the electronic apparatus to transition to a mode according to a result of determination made by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technology for reducing power consumption in electronic devices, particularly in electronic devices having a display unit that is gazed at by a user. [Background technology]

[0002] To extend the battery life of electronic devices such as smartphones and cameras, it is ideal to understand the user's intentions to operate the device and perform power-saving control when the user does not intend to operate the device.

[0003] One method for understanding an operator's intention is to detect the direction of the user's face and line of sight toward the electronic device and determine the operator's intention based on whether or not they are detected.

[0004] Research has been conducted into techniques for detecting the line of sight and facial direction of a user, and Patent Document 1 discloses a technique for detecting the facial direction using a ToF (Time of Flight) sensor.

[0005] In addition to power-saving control, various electronic devices equipped with devices that detect the user's line of sight and face direction have been released, with the aim of utilizing them for new user interfaces. However, if the detection devices are constantly running, the device's power consumption increases, which is an issue.

[0006] To address the above-mentioned problem, Patent Document 2 discloses a technology for detecting camera shake and ambient light of an electronic device and controlling the power supply of a gaze detection device and a face detection device. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2019-179386 [Patent Document 2] Japanese Patent Application Laid-Open No. 2014-142851 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the conventional technology disclosed in the above-mentioned Patent Document 1 can detect the direction of the face, but cannot detect where the user is looking. As a result, it may not be possible to accurately detect the user's intention to operate the device, such as when the user's face is facing the electronic device but their gaze is elsewhere.

[0009] Furthermore, the conventional technology disclosed in the above-mentioned Patent Document 2 keeps the power supply of the detection device on all the time depending on the movement of the electronic device and the brightness of the surroundings, which may make it impossible to solve the problem of the device consuming large amounts of power depending on the surrounding conditions.

[0010] The present invention has been made in view of the above problems, and aims to provide a technique for switching an electronic device having a display unit to a power mode according to the degree to which the user is gazing at the display unit. [Means for solving the problem]

[0011] In order to solve this problem, for example, an electronic device of the present invention has the following configuration: An electronic device having a display unit, a detection means for detecting the degree of gaze of a user on the display unit; a determining means for determining whether to transition to a normal mode indicating a normal operating state or a power saving mode which consumes less power than the normal mode, depending on the result of detection by the detecting means; The electronic device has a control means for shifting the electronic device to a mode according to the result of the determination by the determination means. [Effects of the Invention]

[0012] According to the present invention, an electronic device having a display unit can be shifted to a power mode according to the degree to which the user is gazing at the display unit. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is an external view of an electronic device to which an embodiment is applied. [Figure 2] FIG. 1 is a schematic block diagram illustrating an example of a hardware configuration of an electronic device according to an embodiment. [Figure 3] 4 is a flowchart showing a processing procedure in the first embodiment. [Figure 4] 10 is a flowchart showing a processing procedure in the second embodiment. [Figure 5] 10 is a flowchart showing a processing procedure in the third embodiment. [Figure 6] 10 is a flowchart showing a processing procedure in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0015] First, the configuration and operation of an electronic device, such as a digital camera, which is the premise of the embodiment, will be described. Note that the present invention is not limited to digital cameras, and can be applied to any electronic device that has a display unit that is viewed by a user.

[0016] 1(a) and 1(b) show external views of electronic device 100. FIG. 1(a) is a front perspective view of electronic device 100, and FIG. 1(b) is a rear perspective view of electronic device 100. In FIGS. 1(a) and 1(b), display unit 28 is a display unit provided on the rear of the camera that displays images and various information (including menu screens). A touch panel 70a is provided on the display surface of this display unit 28, and a user can input instructions to electronic device 100 by touching displayed menu items with a finger.

[0017] The viewfinder display 43 is a display provided on the top surface of the camera, and displays various camera settings such as shutter speed and aperture. The shutter button 61 is an operation unit for issuing shooting instructions. The mode selector switch 60 is an operation unit for switching between various modes. The terminal cover 40 is a cover that houses and protects a connector (not shown) for connecting with an external device. When actually connecting with an external device, this terminal cover is opened and a cable for connecting with the external device is connected to the housed connector. This connector includes a connection terminal for a USB cable (not shown) and an output I / F 20 (described below) that is an HDMI (registered trademark) output terminal.

[0018] The main electronic dial 71 is a rotary operation member included in the operation unit 70, and by turning this main electronic dial 71, settings such as shutter speed and aperture can be changed. The power switch 72 is an operation member that switches the power of the electronic device 100 on and off. The sub electronic dial 73 is a rotary operation member included in the operation unit 70, and is used to move the selection frame, advance images, etc. The cross key 74 is included in the operation unit 70, and is a cross key (four-way key) that can be pressed up, down, left, or right. Operations can be performed according to the part of the cross key 74 that is pressed. The SET button 75 is included in the operation unit 70, and is a push button that is mainly used to confirm selections, etc.

[0019] The video button 76 is used to start and stop video shooting (recording). The AE lock button 77 is included in the operation unit 70, and by pressing it in the shooting standby state, the exposure state can be fixed. The enlarge button 78 is included in the operation unit 70, and is an operation button for turning the enlargement mode on and off in the live view display in shooting mode. By turning the enlargement mode on and operating the main electronic dial 71, the live view image can be enlarged or reduced. In playback mode, it functions as an enlargement button for enlarging the playback image and increasing the magnification ratio. The playback button 79 is included in the operation unit 70, and is an operation button for switching between shooting mode and playback mode. Pressing the playback button 79 in shooting mode switches to playback mode, and the most recent image recorded on the recording medium 200 can be displayed on the display unit 28, EVF 29, or external device 210.

[0020] The menu button 81 is included in the operation unit 70, and pressing it displays a menu screen on the display unit 28, the EVF 29, or the external device 210, allowing various settings to be made. The user can intuitively configure various settings using the menu screen displayed on the display unit 28, the EVF 29, or the external device 210, along with the cross key 74 and the SET button 75. The communication terminal 10 is a communication terminal that enables the electronic device 100 to communicate with a detachable lens unit 150 (described later). The eyepiece 16 is the eyepiece of an eyepiece finder (a peer-type finder), and the user can view the image displayed on the internal EVF 29 through the eyepiece 16. The eyepiece detection unit 57 is an eyepiece detection sensor that detects whether the photographer has placed their eye on the eyepiece 16. The lid 202 is a lid for a slot that stores the recording medium 200. The grip unit 90 is a holding unit shaped to be easily gripped with the user's right hand when holding the electronic device 100. The shutter button 61 and main electronic dial 71 are positioned so that they can be operated with the index finger of the right hand when the digital camera is held by gripping the grip portion 90 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 73 is positioned so that it can be operated with the thumb of the right hand.

[0021] FIG. 2 is a block diagram showing an example of the configuration of electronic device 100 according to this embodiment. In FIG. 2, lens unit 150 is a lens unit equipped with an interchangeable photographic lens. Lens 103 is usually composed of multiple lenses, but for simplicity, only one lens is shown here. Communication terminal 6 is a communication terminal that enables lens unit 150 to communicate with electronic device 100. Lens unit 150 controls aperture 1 via aperture drive circuit 2 using an internal lens system control circuit 4, and focuses by displacing lens 103 via AF drive circuit 3.

[0022] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50.

[0023] The imaging unit 22 is an imaging element that converts an optical image into an electrical signal and is composed of a CCD, CMOS element, etc. The A / D converter 23 is used to convert the analog signal of the image output from the imaging unit 22 into digital image data.

[0024] The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on image data from the A / D converter 23 or image data from the memory control unit 15 (described later). The image processing unit 24 also performs predetermined arithmetic processing using the captured image data. The system control unit 50 performs exposure control and distance measurement control based on the arithmetic results obtained by the image processing unit 24. This results in TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the arithmetic results obtained.

[0025] The memory control unit 15 controls data transmission and reception between the A / D converter 23, the image processing unit 24, and the memory 32. Image data output from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores digital image data obtained by the imaging unit 22 and converted by the A / D converter 23, as well as image data to be displayed on the display unit 28, the EVF 29, or the external device 210. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 32 also serves as a memory for image display (video memory). The D / A converter 19 converts the image display data stored in the memory 32 into analog signals and supplies them to the display unit 28 and the EVF 29. The display image data written to the memory 32 is then displayed on the display unit 28 and the EVF 29 via the D / A converter 19.

[0026] The display unit 28 and the EVF 29 perform display on a display device such as an LCD or an organic EL display in accordance with the analog signal from the D / A converter 19. The output I / F 20 supplies the image display data stored in the memory 32 to the external device 210 as a digital signal. In this way, the display image data written to the memory 32 is displayed on the external device 210. A live view display (hereinafter referred to as LV display) can be performed by sequentially transferring and displaying the data stored in the memory 32 that has been A / D converted by the A / D converter 23 to the display unit 28, the EVF 29, or the external device 210. Hereinafter, an image captured by the imaging unit 22 and displayed as a live view will be referred to as a live view image or an LV image.

[0027] The outside viewfinder display 43 displays various camera settings such as shutter speed and aperture via an outside viewfinder display drive circuit 44 .

[0028] The nonvolatile memory 56 is an electrically erasable and recordable memory, and may be, for example, a Flash-ROM. The nonvolatile memory 56 stores constants, programs, etc. for the operation of the system control unit 50. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0029] The system control unit 50 is a control unit made up of at least one processor or circuit, and controls the entire electronic device 100. The system control unit 50 executes programs recorded in the nonvolatile memory 56 described above to realize each process of this embodiment, which will be described later. The system memory 52 uses, for example, a RAM, and stores constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like. The system control unit 50 also performs display control by controlling the memory 32, the display unit 28, the EVF 29, and the like.

[0030] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of a built-in clock.

[0031] The mode selector switch 60, first shutter switch 62, second shutter switch 64, and operation unit 70 are operating means for inputting various operational instructions to the system control unit 50. The mode selector switch 60 switches the operation mode of the system control unit 50 between still image capture mode, video capture mode, etc. Modes included in the still image capture mode include auto capture mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide capture settings for specific shooting scenes. The mode selector switch 60 allows the user to directly switch to one of these modes. Alternatively, the user may first switch to a list screen of shooting modes using the mode selector switch 60, then select one of the displayed modes and switch using other operation members. Similarly, the video capture mode may also include multiple modes.

[0032] The first shutter switch 62 is turned on and generates a first shutter switch signal SW1 when the shutter button 61 provided on the electronic device 100 is pressed halfway (a shooting preparation instruction) during operation. In response to receiving the first shutter switch signal SW1, the system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing.

[0033] The second shutter switch 64 is turned on when the shutter button 61 is fully pressed (photographing instruction) and generates a second shutter switch signal SW2. In response to receiving the second shutter switch signal SW2, the system control unit 50 starts a series of photographing processing operations, from reading out a signal from the imaging unit 22 to writing the captured image to the recording medium 200 as an image file.

[0034] The operation unit 70 is an input unit that accepts operations from the user and includes various operation members, such as the shutter button 61, the touch panel 70a, the main electronic dial 71, the power switch 72, the sub electronic dial 73, the cross key 74, the SET button 75, the movie button 76, the AE lock button 77, the enlarge button 78, the playback button 79, and the menu button 81.

[0035] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between powered blocks, etc., and detects whether a battery is installed, the battery type, and the remaining battery charge. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the required voltage for the required period to each unit, including the recording medium 200. The power supply unit 30 is composed of primary batteries such as alkaline batteries or lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, or Li batteries, an AC adapter, etc.

[0036] The I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a removable recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0037] The communication unit 54 is connected wirelessly or via a wired cable and transmits and receives video signals and audio signals. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can also communicate with external devices using Bluetooth (registered trademark) or Bluetooth (registered trademark) Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can also receive images and various other information from external devices.

[0038] The orientation detection unit 55 detects the orientation of the electronic device 100 with respect to the direction of gravity. Based on the orientation detected by the orientation detection unit 55, it is possible to determine whether an image captured by the imaging unit 22 was captured with the electronic device 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the orientation detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. An acceleration sensor, a gyro sensor, or the like can be used as the orientation detection unit 55. The acceleration sensor or gyro sensor of the orientation detection unit 55 can also be used to detect movement of the digital camera 100 (panning, tilting, lifting, whether the digital camera 100 is stationary, etc.).

[0039] The eyepiece detection unit 57 is an eyepiece detection sensor that detects (approach detection) the approach (eyepiece approach) and departure (eye separation) of the user's eyeball (eye) from the viewfinder eyepiece unit 16. The system control unit 50 switches the display unit 28 and the EVF 29 between display (display drive state) and non-display (non-display state) depending on the state detected by the eyepiece detection unit 57. More specifically, at least in the shooting standby state and when the display destination switching is automatic switching, when the eye is not in contact with the camera, the display is turned on as the display unit 28 and the EVF 29 is not displayed. When the eye is in contact with the camera, the display is turned on as the display unit with the EVF 29 and the display is not displayed. The eyepiece detection unit 57 can be, for example, an infrared proximity sensor, and can detect the approach of an object to the viewfinder eyepiece unit 16 incorporating the EVF 29. When an object approaches, infrared light emitted from the light-emitting unit (not shown) of the eyepiece detection unit 57 is reflected and received by the light-receiving unit (not shown) of the infrared proximity sensor. The amount of received infrared light can also determine the distance the object is approaching the eyepiece 16 (eyepiece distance). In this way, the eyepiece detection unit 57 performs eyepiece detection, which detects the proximity of an object to the eyepiece 16. When an object approaching within a predetermined distance from the eyepiece 16 is detected from a non-eyepiece state (non-approach state), it is detected as being in eye contact. When an object detected as approaching moves away from the eyepiece state (approach state) by more than a predetermined distance, it is detected as being away from the eye. The threshold for detecting eye contact and the threshold for detecting eye separation may be different, for example, by providing hysteresis. Furthermore, after eye contact is detected, the eyepiece remains in the eye contact state until eye separation is detected. After eye separation is detected, the eyepiece remains in the non-eye contact state until eye contact is detected. The infrared proximity sensor is just one example, and other sensors may be used for the eye proximity detection unit 57 as long as they can detect the approach of an eye or an object that can be considered as an eye proximity.

[0040] The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured so that its light transmittance does not interfere with the display of the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Then, input coordinates on the touch panel 70a are associated with display coordinates on the display screen of the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that allows the user to directly operate the screen displayed on the display unit 28.

[0041] The power saving control unit 160 detects the degree to which the user is gazing at the display unit 28, and performs control to change the drive state and power state of the electronic device 100. In the embodiment, the degree to which the user is gazing at the display unit 28 is detected by detecting the direction of the user's face and the direction of the user's line of sight. In addition, the power saving control unit 160 performs control to change the drive state or power state of a device that detects the direction of the face and the line of sight.

[0042] The face direction detection unit 161 detects the direction of the user's face. The face direction determination unit 163 calculates the angle of the user's face direction with respect to the display unit 28 from the detection result of the face direction detection unit 161 and determines whether the face is directed toward the display unit 28. The gaze detection unit 162 detects the direction of the user's gaze. The gaze determination unit 164 determines whether the gaze is directed toward the display unit 28 from the detection result of the gaze detection unit 162. The system control unit 50 performs control to switch the state of the digital camera 100 between the normal mode and the power saving mode according to the determination results of the face direction determination unit 163 and the gaze determination unit 164 of the power saving control unit 160. The face direction detection unit 161 and the gaze detection unit 162 may be arranged near the display unit 28 to enable detection of the gaze and face direction with respect to the display unit 28 even when the display unit 28 is configured to be movable.

[0043] Specific examples of the face direction detection unit 161 and the gaze detection unit 162 are as follows: The face direction detection unit 161 has an image sensor (not shown) installed near the display unit 28. This image sensor only needs to determine which of several preset orientations the user's face is facing; it needs to have a resolution significantly lower than that of the image sensor of the main image capture unit 22 and be a monochrome image sensor. As a result, the power consumption of the face direction detection unit 161 (and its image sensor) is much less than that of the main image capture unit 22. Based on the image from this low-resolution image sensor, the face direction detection unit 161 generates and outputs information about the face direction from the outline of the user's face and the arrangement of facial features such as the eyes, eyebrows, nose, and mouth. Based on this information, the face direction determination unit 163 determines which of several preset orientations the user's face is facing and outputs the information to the system control unit 50.

[0044] It should be noted that the face direction detection unit 161 and the face direction determination unit 163 use an imaging element. In this embodiment, the frequency of detection and determination of face direction and line of sight by the face direction detection unit 161 and the face direction determination unit 163 can be selected from two levels, with a high drive state (short cycle state) at intervals of 0.5 seconds and a low drive state (long cycle state) at intervals of, for example, 1 second. Naturally, the power consumption has the relationship: power off < low drive state < high drive state.

[0045] On the other hand, the gaze detection unit 162 has an imaging element (not shown) installed near the display unit 28. This imaging element is independent from the imaging element of the face direction detection unit 161. The imaging element of the gaze detection unit 162 also needs to be able to determine which of several preset directions the gaze direction corresponds to, and it can be a monochrome imaging element with a resolution significantly lower than that of the imaging element of the main imaging unit 22. Therefore, its power consumption is much lower than that of the main imaging unit 22. The gaze detection unit 162 outputs information indicating the user's gaze direction based on the image from this low-resolution imaging element and the arrangement of the user's facial features. Based on this information, the gaze determination unit 164 determines which of several preset gaze directions the user's gaze direction corresponds to, and outputs the information to the system control unit 50. Furthermore, the gaze detection unit 162 and the gaze determination unit 164 operate in one of three modes, namely, a high drive state (the gaze direction detection interval is, for example, 0.5 seconds), a low drive state (the gaze direction detection interval is, for example, 1 second), and power off (non-detection), as a frequency of gaze direction detection and determination per unit time using an image sensor. Naturally, the power consumption has the relationship: power off < low drive state < high drive state.

[0046] 3 shows a series of control flows from standby to shooting while the power is on for the digital camera 100. The processing of the system control unit 50 will be described below with reference to this figure.

[0047] When the digital camera 100 starts up, in S301 the system control unit 50 displays LV on the display unit 28 or EVF 29. In S302, the system control unit 50 determines whether or not a shutter switch signal SW1 has been received from the first shutter switch 62. That is, the system control unit 50 determines whether or not the user has half-pressed the shutter button 61. If the system control unit 50 determines that the shutter switch signal SW1 has not been received, the process returns to S301. On the other hand, if the system control unit 50 determines that the shutter switch signal SW1 has been received (that is, the user has half-pressed the shutter button 61), the process proceeds to S303.

[0048] In S303, the system control unit 50 performs preparatory operations before shooting, such as AF processing, etc. Then, the system control unit 50 advances the process to S304.

[0049] In S304, the system control unit 50 determines whether or not the shutter switch signal SW2 has been received. That is, the system control unit 50 determines whether or not the user has fully pressed the shutter button 61. If the system control unit 50 determines that the shutter switch signal SW2 has not been received, the process returns to S301. On the other hand, if the system control unit 50 determines that the shutter switch signal SW1 has been received (the user has fully pressed the shutter button 61), the process proceeds to S305.

[0050] In S305, the system control unit 50 performs a photographing operation, reads out a signal from the imaging unit 22, and writes the photographed image as an image file to the recording medium 200. Then, in S3056, the system control unit 50 displays the image captured this time on the display unit 28 or the EVF 29 for a certain period of time, and then returns the process to S301. In this way, while the digital camera 100 is on, LV display and photographing are repeated.

[0051] The above is the configuration of the digital camera and the photographing process in this embodiment.

[0052] [First embodiment] Hereinafter, a method for controlling the drive state and power supply of each detection unit using the detection results of the face direction detection unit 161 and the gaze detection unit 162 according to the first embodiment will be described with reference to the flowchart in Fig. 4. It should be understood that the flowchart in Fig. 4 is the control during LV display in step S301 in the flowchart in Fig. 3.

[0053] Before proceeding, the power saving modes of the digital camera of this embodiment will be described. The digital camera of this embodiment has three modes for reducing power consumption. The relationship of the amount of power consumed in each power saving mode, including the normal mode in which power consumption is not reduced, is as follows:

[0054] Power saving mode 3<Power saving mode 2<Power saving mode 1<Normal mode Details of each power saving mode are as follows: Power saving mode 1: The image capturing interval (frame rate) by the image capturing unit 22 is set lower than in normal mode. In other words, the frame rate of the LV display is set lower than in normal mode. For example, the frame rate by the image capturing unit 22 in normal mode is set to 30 fps, but this is reduced to, for example, 10 fps. Naturally, this reduces the drive rates per unit time of not only the image capturing unit 22 but also the A / D converter 22, image processing unit 24, display unit 28, etc., and therefore power consumption is lower than in normal mode. Power saving mode 2: In addition to the above power saving mode 1, the display unit 28 and EVF 29 are stopped from being driven. If the display unit 28 is, for example, a liquid crystal display, the power supply to the backlight is also turned off. Power saving mode 3: The entire electronic device 100 is powered off and put into a sleep state. However, in order to be able to return from the sleep state to the normal mode, part of the operation unit 70 and part of the system control unit 50 are kept operational to receive a return instruction input from the user.

[0055] Based on the above, the processing of the system control unit 50 will be described with reference to the flowchart of Fig. 4. This processing is realized by the system control unit 50 expanding a program stored in the nonvolatile memory 56 into the system memory 52, executing it, and controlling each functional block. The flowchart of Fig. 4 starts when the electronic device 100 is started.

[0056] When the electronic device 100 is started up, in S401 the system control unit 50 performs control to transition the driving state of the face direction detection unit 161 to a high driving state with a short detection interval, and then ends the process.

[0057] In S402, the system control unit 50 determines whether the face direction determination unit 163 has detected the face direction of the user. If it is determined that the face direction has been detected, the system control unit 50 proceeds to S408. However, if it is determined that the face direction has not been detected, the system control unit 50 proceeds to S403.

[0058] In S403, the system control unit 50 controls to turn off the power supply of the gaze detection unit 162, and transitions the electronic device 100 to power saving mode 2. As described above, power saving mode 2 includes power saving mode 1, and further includes processing to turn off driving of the display unit 28 and the EVF 29. After this, the system control unit 50 advances the processing to S404.

[0059] In S404, the system control unit 50 determines whether the state in which the face direction cannot be detected continues for a preset first predetermined time. If it is determined that the state in which the face direction cannot be detected continues for the first predetermined time, the system control unit 50 advances the process to S405. If it is determined that the state in which the face direction cannot be detected continues for less than the first predetermined time, the system control unit 50 advances the process to S406. S401 Return to.

[0060] In S405, the system control unit 50 performs control to switch the electronic device 100 to power off and transition to power saving mode 3, which reduces power consumption. Then, in S406, the system control unit 50 performs control to wait for a third predetermined time, and proceeds to S407. In S407, the system control unit 50 determines whether an operation on the operation unit 70 has been accepted, and if an operation has been accepted, the system control unit 50 switches back to normal mode and returns to S402. On the other hand, if it is determined that no operation has been accepted from the operation unit 70, the system control unit 50 returns to S405.

[0061] In S408, the system control unit 50 determines whether the face direction determination unit 163 determines whether the face of the user is facing the display unit 28. If the system control unit 50 determines that the face of the user is facing the display unit 28, the process proceeds to S413; if the system control unit 50 determines that the face is not facing the display unit 28, the process proceeds to S409.

[0062] In S409, the system control unit 50 determines whether the angle difference of the user's facial direction with respect to the line connecting the user and (the center of) the display unit 28 is equal to or greater than a certain value. If it is determined that the angle difference is smaller than the certain value, the system control unit 50 proceeds to S410, and if it is determined that the angle difference is equal to or greater than the certain value, the system control unit 50 proceeds to S411.

[0063] In S410, the system control unit 50 controls the face direction detection unit 161 to be in a high drive state and the line of sight detection unit 162 to be in a low drive state with a long detection interval. Furthermore, the system control unit 50 controls the image capture unit 22 to transition to a low frame rate power saving mode 1, which slows down the readout period of the signal, and then performs processing. S401 Return to.

[0064] In S411, the system control unit 50 controls the face direction detection unit 161 to transition to a high drive state and the gaze detection unit S162 to power off. Furthermore, the system control unit 50 controls the electronic device 100 to transition to power saving mode 2, and then proceeds to S412.

[0065] In S412, the system control unit 50 determines whether the angle of the user's face relative to the display unit 28 has remained greater than a certain value for a second predetermined time. If it is determined that the angle of the user's face relative to the display unit 28 has remained greater than a certain value for the second predetermined time, the system control unit 50 proceeds to S405; otherwise, the system control unit 50 proceeds to S405. S401 Return to.

[0066] In S413, the system control unit 50 performs control to transition the gaze detection unit 162 to a low drive state, and then the process proceeds to S414.

[0067] In S414, the system control unit 50 determines whether the line of sight determination unit 164 has detected the line of sight of the user. If it is determined that the line of sight of the user has been detected, the system control unit 50 proceeds to S416, and if it is determined that the line of sight of the user has not been detected, the system control unit 50 proceeds to S415.

[0068] In S415, the system control unit 50 transitions the face direction detection unit 161 to a high drive state and the gaze detection unit 162 to a low drive state. Furthermore, the system control unit 50 performs control to transition the electronic device 100 to the normal mode, which is the state at startup. Thereafter, the system control unit 50 returns the process to S408.

[0069] In S416, the line-of-sight determination unit 164 determines whether the user's line of sight is directed toward the display unit 28. If the system control unit 50 determines that the user's line of sight is directed toward the display unit 28, the process proceeds to S418, and if the system control unit 50 determines that the user's line of sight is not directed toward the display unit 28, the process proceeds to S417.

[0070] In step S417, the system control unit 50 performs control to transition the face direction detection unit 161 to a low drive state and the gaze detection unit 162 to a high drive state. Furthermore, the system control unit 50 performs control to transition the electronic device 100 to power saving mode 1. Thereafter, the system control unit 50 returns the process to S414.

[0071] In S418, the system control unit 50 turns off the power of the face direction detection unit 161 and transitions the gaze detection unit 162 to a high drive state. Furthermore, the system control unit 50 performs control to transition the electronic device 100 to the normal mode. Thereafter, the system control unit 50 returns the process to S414.

[0072] As described above, according to the first embodiment, by transitioning the power supply or driving state of each detection unit depending on the detection results of the face direction detection unit 161 and the gaze detection unit 162, it is possible to reduce power consumption compared to when the units are always driven.

[0073] In the above embodiment, the face direction detection unit 161 and the gaze detection unit 162 each have an image sensor, but the image sensor for face direction and gaze detection may be shared. In this case, the imaging cycle of the image sensor may be fixed to the high drive state described above, and the face direction detection process and gaze detection process may not be performed every time an image is captured, but may be performed at intervals corresponding to the "high drive state" and "low drive state," thereby optimizing the power consumption related to the calculation process.

[0074] Furthermore, when the eye proximity detection unit 57 detects that the user has placed their eye close to the camera, that is, when it is estimated that the user is looking into the EVF, the distance between the user's face and the image sensors of the face direction detection unit 161 and the gaze detection unit 162 is too close, making it impossible to detect the face direction or gaze direction. When the eye proximity detection unit 57 detects that the user has placed their eye close to the camera, the system control unit 50 may stop driving the face direction detection unit 161 and the gaze detection unit 162 and drive the electronic device in normal mode (however, the display unit 28 may be in a non-driven state).

[0075] [Second embodiment] A second embodiment will be described below with reference to the flowchart of FIG. 5. In this second embodiment, a method for controlling the state of the electronic device 100 in accordance with the detection result of the face direction detection unit 161 will be described. The flowchart of FIG. 5 should be understood as the control during LV display in step S301 of the flowchart of FIG. 3. Each process in the flowchart is realized by the system control unit 50 loading a program stored in the non-volatile memory 56 into the system memory 52, executing it, and controlling each functional block. The flowchart starts when the electronic device 100 is started. Note that steps identical to those in the flowchart of FIG. 4 are indicated by the same symbols, and their description will be omitted. Unlike the control in FIG. 4, this flowchart illustrates control in a configuration in which the electronic device 100 does not have the gaze detection unit 162 but only has the face direction detection unit 161. Except for the absence of the gaze detection unit 162, the device configuration is the same as that in FIG. 2, so please refer to FIG. 2 for the device configuration.

[0076] When the electronic device 100 is started up, the system control unit 50 executes the same process as S402 in the flowchart of FIG. 4, and then advances the process to S408 or S501.

[0077] In S408, the system control unit 50 determines whether the face direction of the user determined by the face direction determination unit 163 is facing the display unit 28. If the system control unit 50 determines that the face direction of the user is facing the display unit 28, the process proceeds to S503, and if the system control unit 50 determines that the face direction is not facing the display unit 28, the process proceeds to S409.

[0078] In S409, the system control unit 50 determines whether the angle difference of the user's face direction with respect to the line connecting the user and the display unit 28 (the center of the display unit 28) is equal to or greater than a certain value. If it is determined that the angle difference is smaller than the certain value, the system control unit 50 terminates the process. S402 If it is determined that the value is equal to or greater than a certain value, the process proceeds to S502.

[0079] In S501, the system control unit 50 controls the electronic device 100 to transition to power saving mode 1, and then returns the process to S402.

[0080] In addition, in S502, the system control unit 50 controls the electronic device 100 to transition to the power saving mode 2, and then returns the process to S402.

[0081] In S503, the system control unit 50 controls the electronic device 100 to transition to the normal mode, and returns the process to S402.

[0082] As described above, according to the second embodiment, power consumption can be reduced compared to when the electronic device 100 is always used in the normal mode by transitioning the state of the electronic device 100 in accordance with the detection result of the face direction detection unit 161. Unlike the first embodiment, the driving state of the detection unit is not changed, so the obtained power saving effect is smaller compared to the first embodiment, but the cost can be reduced because the number of components is reduced.

[0083] [Third embodiment] The third embodiment will be described below with reference to the flowchart of Fig. 6. In this third embodiment, the state of the electronic device 100 is controlled in accordance with the detection result of the face detection unit 161. It should be understood that the processing shown in the flowchart of Fig. 6 is the control processing during LV display in step S301 in the flowchart of Fig. 3.

[0084] 6 is realized by the system control unit 50 loading a program stored in the nonvolatile memory 56 into the system memory 52, executing the program, and controlling each functional block. The flowchart of FIG. 6 starts when the electronic device 100 is started. Note that the same steps as those in the flowchart of FIG. 4 are indicated by the same symbols, and their explanations will be omitted. Unlike the control of FIG. 4, this flowchart shows control in a configuration in which the electronic device 100 does not have a face direction detection unit 161 but only has a gaze detection unit 162. Except for the absence of the face direction detection unit 161, the device configuration is the same as that of FIG. 2, so please refer to FIG. 2 for the device configuration.

[0085] When electronic device 100 is started up, in S414, system control unit 50 determines whether gaze determination unit 164 has detected the gaze of the user, similar to step S414 in the flowchart of Fig. 4. If it is determined that the gaze of the user has been detected, system control unit 50 proceeds to S416, and if it is determined that the gaze has not been detected, system control unit 50 proceeds to S601.

[0086] 4, the system control unit 50 determines whether the determined line of sight is directed toward the display unit 28. If the system control unit 50 determines that the user's line of sight is directed toward the display unit 28, the process proceeds to S602; if the system control unit 50 determines that the user's line of sight is not directed toward the display unit 28, the process proceeds to S601.

[0087] In S601, the system control unit 50 controls the electronic device 100 to transition to the normal mode, and then returns the process to S414.

[0088] In S602, the system control unit 50 controls the electronic device 100 to transition to power saving mode 1, and then returns the process to S414.

[0089] As described above, according to the third embodiment, power consumption can be reduced compared to when the electronic device 100 is always used in the normal mode by transitioning the state of the electronic device 100 in accordance with the detection result of the gaze detection unit 162. Also, as in the second embodiment, the driving state of the detection unit is not changed, so the obtained power saving effect is smaller compared to the first embodiment, but the cost can be reduced because the number of components is reduced.

[0090] [Fourth embodiment] A fourth embodiment will be described. In this fourth embodiment, a method for controlling the power supply of the face direction detection unit 161 and the gaze detection unit 162 according to the state of the electronic device 100 will be described. Note that the device configuration is the same as that in FIG. 2, and a description thereof will be omitted.

[0091] In the fourth embodiment, system control unit 50 controls to turn off the power of face direction detection unit 161 and gaze detection unit 162 when electronic device 100 is in a state where settings can be changed during LV display, when a captured moving image is being displayed or played back, or when in menu mode. In states other than these, system control unit 50 drives face direction detection unit 161 and gaze detection unit 162 and performs the same processing as in the first embodiment according to the detection results.

[0092] As described above, according to the fourth embodiment, by controlling the power supply of the face direction detection unit 161 and the gaze detection unit 162 to be turned off depending on the state of the electronic device 100, it is possible to reduce power consumption compared to when the detection units are always driven.

[0093] [Fifth embodiment] A fifth embodiment will be described. In this fifth embodiment, a method for preventing the electronic device 100 from transitioning to a power saving mode in accordance with the detection result of the gaze detection unit 162 will be described. Here, it is assumed that the electronic device 100 has a control for transitioning to a state in which power consumption is lower than at startup if no operation is performed with an operation member for a certain period of time.

[0094] When a certain period of time has passed without any operation of the electronic device 100, and the gaze determination unit 164 determines that the gaze is directed at the electronic device 100, the system control unit 50 continues the state at the time of startup, but when it determines that the gaze is not directed at the electronic device 100, it transitions to a power saving mode (any of power saving modes 1 to 3) previously selected by the user.

[0095] As described above, according to the fifth embodiment, by controlling the state of the electronic device 100 in accordance with the detection state of the gaze detection unit 162, it is possible to prevent the electronic device 100 from transitioning to a state contrary to the user's intention.

[0096] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and alterations are possible within the scope of the gist thereof. Even if the facial direction cannot be detected, if it is detected that the gaze is directed toward the electronic device 100, the electronic device 100 may transition to the normal mode. While FIG. 4 illustrates a configuration in which the output result of the gaze detection unit 162 is referenced in accordance with the output of the facial direction detection unit 161, the output result of the facial direction detection unit 161 may also be referenced in accordance with the output of the gaze detection unit 162. Although FIGS. 5 and 6 only describe state transitions of the electronic device 100, the driving state and power supply of the facial direction detection unit 161 or the gaze detection unit 162 may also be transitioned. The control contents in power saving modes 1 to 3 may be changed to different controls.

[0097] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0098] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0099] 28...LCD, 29...EVF, 50...system control unit, 100...electronic device, 100a...electronic device, 160...power saving control unit, 161...face direction detection unit, 162...gaze detection unit, 164...face direction determination unit, 164...gaze determination unit

Claims

1. An electronic device having a display unit, an imaging means for capturing an image to be displayed on the display unit; a detection means for detecting the degree of gaze of a user on the display unit; a determining means for determining whether to transition to a normal mode indicating a normal operating state or a power saving mode which consumes less power than the normal mode, depending on the result of detection by the detecting means; a control means for causing the electronic device to transition to a mode according to the result of the determination by the determination means; and The power saving mode is a first power saving mode in which the imaging period of the imaging means is made longer than that in the normal mode; a second power saving mode in which power to the display unit is turned off in addition to the first power saving mode; A third power saving mode that puts the electronic device into a sleep state. An electronic device characterized by:

2. The detection means A face direction detection means for detecting the direction of the user's face; a gaze detection means for detecting a gaze direction of a user, The determination means determines the mode to be transitioned to as follows: When the face direction detection means is unable to detect the face direction, the second power saving mode is determined; determining a second power saving mode when the face direction detected by the face direction detection means is not directed toward the display unit and exceeds a threshold value with respect to the direction toward the display unit, and determining a first power saving mode when the face direction is equal to or less than the threshold value with respect to the direction toward the display unit; When the face direction detected by the face direction detection means is directed toward the display unit and the line of sight detection means cannot detect the line of sight, the normal mode is determined; when the line of sight detected by the line of sight detection means is not directed toward the display unit, the first power saving mode is determined; and when the line of sight detected by the line of sight detection means is directed toward the display unit, the normal mode is determined.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

3. The determining means determines the third power saving mode as a mode to transition to when the operation in the second power saving mode continues for a preset time by the control means.

3. The electronic device according to claim 2.

4. The control means When transitioning to the first power saving mode, a cycle of face direction detection by the face direction detection means is set to a higher cycle of two preset stages, and a cycle of gaze detection by the gaze detection means is set to a lower cycle of two preset stages, When transitioning to the second power saving mode, the cycle of face direction detection by the face direction detection means is set to the higher cycle of two preset stages, and the line of sight detection means is stopped. When the mode is shifted to the normal mode in a state where the gaze detection means is unable to detect the gaze, the cycle of face direction detection by the face direction detection means is set to a high cycle, and the cycle of the gaze detection means is set to a low cycle, When the normal mode is entered in a state where the line of sight detected by the line of sight detection means is directed to the display unit, the face direction detection means is stopped and the line of sight detection means is set to a high cycle.

4. The electronic device according to claim 2 or 3.

5. the detection means includes face direction detection means for detecting a face direction of the user, The determination means determines the mode to be transitioned to as follows: If the face direction detection means is unable to detect the face direction of the user, a first power saving mode is determined to be set in advance; determining the first power saving mode when the orientation of the user's face detected by the face orientation detection means is not larger than a preset angle with respect to a direction toward the display unit, and determining the second power saving mode when the orientation of the user's face is larger than the preset angle; When the direction of the user's face detected by the face direction detection means is directed toward the display unit, the normal mode is determined.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

6. the detection means includes a gaze detection means for detecting a gaze direction of the user, The determination means determines the mode to be transitioned to as follows: If the line of sight of the user cannot be detected by the line of sight detection means, the normal mode is determined; When the line of sight of the user detected by the line of sight detection means is not directed toward the display unit, a first power saving mode is determined to be set in advance; When the line of sight of the user detected by the line of sight detection means is directed toward the display unit, the normal mode is determined.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

7. A control method for an electronic device having a display unit and an imaging means for capturing an image to be displayed on the display unit, a detection step of detecting a degree of gaze of a user on the display unit; a determination step of determining whether to transition to a normal mode indicating a normal operating state or a power saving mode which consumes less power than the normal mode, based on the detection result of the detection step; a control step of causing the electronic device to transition to a mode according to the result of the determination step, The power saving mode is a first power saving mode in which the imaging period of the imaging means is made longer than that in the normal mode; a second power saving mode in which power to the display unit is turned off in addition to the first power saving mode; A third power saving mode that puts the electronic device into a sleep state. A method for controlling an electronic device.

8. A program that, when read and executed by a computer, causes the computer to execute each step of the method according to claim 7.

Images