Electronic apparatus and control method for the same, and program
Patent Information
- Application Number
- JP2021196466
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-12-02
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2041-12-02
AI Technical Summary
Conventional technologies for detecting user intention through face and line of sight in electronic devices lead to increased power consumption due to continuous operation of detection devices, and they fail to accurately determine user operation intent when face orientation and line of sight diverge.
An electronic device equipped with detection means for determining the degree of user gaze on a display, allowing it to shift between normal and power-saving modes based on the detection results, thereby optimizing power consumption.
The device effectively reduces power consumption by dynamically adjusting operational modes based on user gaze, ensuring accurate detection of operation intent and minimizing unnecessary power usage.
Smart Images

Figure 00000000_0001_ABST 
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0006] , ,
[0007] , , , , ,
[0001] The present invention relates to a technique for reducing power consumption in an electronic device, particularly an electronic device having a display portion that a user gazes at.
Background Art
[0002] In order to prolong the battery life of an electronic device such as a smartphone or a camera, it is ideal to grasp the user's operation intention and perform power saving control when there is no operation intention.
[0003] As a method for grasping the operation intention, there is a method of detecting the user's face orientation or line of sight with respect to the electronic device and determining the operation intention from the presence or absence thereof.
[0004] So far, research has been conducted on technologies for detecting the user's line of sight and face orientation. In Patent Document 1, a technology for detecting the face orientation using a ToF (Time of Flight) sensor is disclosed.
[0005] In addition, from the viewpoint of utilization in a new user interface as well as power saving control, various electronic devices equipped with a device for detecting the user's line of sight or a device for detecting the face orientation have appeared. However, if the detection device is always driven, there is a problem that the power consumption of the device increases.
[0006] Regarding the above-described problem, Patent Document 2 discloses a technology for detecting the hand shake or ambient light of an electronic device and controlling the power supply of the line of sight detection device and the face detection device.
Prior Art Documents
Patent Documents
[0007]
Patent Document 1
Patent Document 2
Summary of the Invention
[0008] However, while the prior art disclosed in Patent Document 1 mentioned above can detect the orientation of the face, it cannot detect where the user is looking. Therefore, there are cases where the user's intention to operate the device cannot be accurately detected, such as when the face is facing the electronic device but the gaze is directed elsewhere.
[0009] Furthermore, the prior art disclosed in Patent Document 2 mentioned above keeps the detection device powered on at all times depending on the movement of the electronic device and the ambient light. Therefore, it may not be able to solve the problem of the device consuming a large amount of power depending on the surrounding conditions.
[0010] This invention has been made in view of the above problems, and aims to provide a technology for electronic devices having a display unit that switches to a power mode according to the degree to which the user is paying attention to the display unit. [Means for solving the problem]
[0011] To solve this problem, for example, the electronic device of the present invention has the following configuration. That is, Electronic device having a display unit, A detection means for detecting the degree to which the user is paying attention to the display unit, A determination means determines, based on the detection result of the detection means, whether to switch to a normal mode indicating a normal operating state or a power-saving mode that consumes less power than the normal mode. The electronic device has a control means that causes it to switch to a mode corresponding to the result of the determination by the determination means. [Effects of the Invention]
[0012] According to the present invention, an electronic device having a display unit can switch to a power mode corresponding to the degree to which the user is paying attention to the display unit. [Brief explanation of the drawing]
[0013] [Figure 1] External view of an electronic device to which the embodiment applies. [Figure 2] A schematic block diagram showing an example of the hardware configuration of the electronic device of the embodiment. [Figure 3] A flowchart illustrating the processing procedure in the first embodiment. [Figure 4] A flowchart illustrating the processing procedure in the second embodiment. [Figure 5] A flowchart illustrating the processing procedure in the third embodiment. [Figure 6] A flowchart illustrating the processing procedure in the fourth embodiment. [Modes for carrying out the invention]
[0014] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.
[0015] First, we will explain the configuration and operation of electronic devices, such as digital cameras, which are the premise of the embodiments. Note that this is applicable to any electronic device with a display that the user views, and is not limited to digital cameras.
[0016] Figures 1(a) and 1(b) show external views of the electronic device 100. Figure 1(a) is a front perspective view of the electronic device 100, and Figure 1(b) is a rear perspective view of the electronic device 100. In Figures 1(a) and 1(b), the display unit 28 is a display unit located on the back of the camera that displays images and various information (including menu screens). A touch panel 70a is provided on the display surface of this display unit 28, and the user can input instructions to the electronic device 100 by touching the displayed menu items with their finger.
[0017] The external viewfinder display unit 43 is a display unit provided on the upper surface of the camera, and various setting values of the camera, such as shutter speed and aperture, are displayed. The shutter button 61 is an operation unit for giving a shooting instruction. The mode switch 60 is an operation unit for switching various modes. The terminal cover 40 is a cover that houses and protects a connector (not shown) for connecting to an external device. When actually connecting to an external device, this terminal cover is opened, and a cable for connecting to the external device is connected to the housed connector. This connector includes a connection terminal (not shown) for a USB cable and an output I / F 20 (described later), which is an HDMI (registered trademark) output terminal.
[0018] The main electronic dial 71 is a rotary operation member included in the operation unit 70. By turning this main electronic dial 71, setting values such as shutter speed and aperture can be changed. The power switch 72 is an operation member for switching the power of the electronic device 100 on and off. The sub - electronic dial 73 is a rotary operation member included in the operation unit 70 and can be used to move the selection frame and scroll the image. The cross - key 74 is included in the operation unit 70 and is a cross - key (4 - direction key) that can be pressed in the upper, lower, left, and right parts respectively. Operations corresponding to the pressed part of the cross - key 74 are possible. The SET button 75 is included in the operation unit 70 and is a push - button, mainly used for determining selected items and the like.
[0019] The video button 76 is used to instruct the start and stop of video shooting (recording). The AE lock button 77 is included in the operation unit 70, and by pressing it in the shooting standby state, the exposure state can be fixed. The zoom button 78 is included in the operation unit 70 and is an operation button for turning on and off the zoom mode in the live view display of the shooting mode. By operating the main electronic dial 71 after turning on the zoom mode, the live view image can be enlarged and reduced. In the playback mode, it functions as a zoom button for enlarging the playback image and increasing the magnification. The playback button 79 is included in the operation unit 70 and is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 during the shooting mode, the user can switch to the playback mode and display the latest image recorded on the recording medium 200 on the display unit 28, EVF 29, or external device 210.
[0020] The menu button 81 is included in the operation unit 70, and by pressing it, various configurable menu screens are displayed on the display unit 28, EVF 29, or external device 210. The user can intuitively perform various settings using the menu screen displayed on the display unit 28, EVF 29, or external device 210, and the cross keys 74 and SET button 75. The communication terminal 10 is a communication terminal for the electronic device 100 to communicate with the lens unit 150 (detachable) described later. The eyepiece part 16 is the eyepiece part of the viewfinder (peeping type finder), and the user can view the video displayed on the internal EVF 29 through the eyepiece part 16. The eye detection unit 57 is an eye detection sensor for detecting whether the photographer is looking through the eyepiece part 16. The lid 202 is the lid of the slot that stores the recording medium 200. The grip part 90 is a holding part with a shape that is easy to hold with the right hand when the user holds the electronic device 100. With the grip part 90 held by the little finger, ring finger, and middle finger of the right hand and the digital camera held, the shutter button 61 and main electronic dial 71 are arranged at positions that can be operated by the index finger of the right hand. Also, in the same state, the sub electronic dial 73 is arranged at a position that can be operated by the thumb of the right hand.
[0021] Figure 2 is a block diagram showing an example configuration of the electronic device 100 according to this embodiment. In Figure 2, the lens unit 150 is a lens unit equipped with an interchangeable photographic lens. The lens 103 is usually composed of multiple lenses, but here it is shown simply as a single lens. The communication terminal 6 is a communication terminal for the lens unit 150 to communicate with the electronic device 100. The lens unit 150 controls the aperture 1 via the aperture drive circuit 2 using the internal lens system control circuit 4, and focuses by displacing the lens 103 via the AF drive circuit 3.
[0022] The shutter 101 is a focal-plane shutter that allows the exposure time of the imaging unit 22 to be freely controlled by the system control unit 50.
[0023] The imaging unit 22 is an image sensor composed of a CCD or CMOS element, which converts optical images into electrical signals. The A / D converter 23 is used to convert the analog signal of the image output from the imaging unit 22 into digital image data.
[0024] The image processing unit 24 performs resizing and color conversion processing, such as predetermined pixel interpolation and reduction, on the image data from the A / D converter 23 or the image data from the memory control unit 15 (described later). The image processing unit 24 also performs predetermined calculations using the captured image data. Based on the calculation results obtained by the image processing unit 24, the system control unit 50 performs exposure control and distance measurement control. This enables TTL (through-the-lens) AF (autofocus), AE (automatic exposure), and EF (flash pre-flash) processing. The image processing unit 24 further performs predetermined calculations using the captured image data and performs TTL AWB (auto white balance) processing based on the obtained calculation results.
[0025] The memory control unit 15 controls the transmission and reception of data between the A / D converter 23, the image processing unit 24, and the memory 32. Image data output from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores digital image data obtained by the imaging unit 22 and converted by the A / D converter 23, as well as image data for display on the display unit 28, EVF 29, or external device 210. The memory 32 has sufficient storage capacity to store a predetermined number of still images, a predetermined amount of video footage, and audio. The memory 32 also serves as a memory for image display (video memory). The D / A converter 19 converts the image display data stored in the memory 32 into analog signals and supplies them to the display unit 28 and EVF 29. In this way, the display image data written to the memory 32 is displayed by the display unit 28 and EVF 29 via the D / A converter 19.
[0026] The display unit 28 and EVF 29 display information on a display device such as an LCD or OLED screen, corresponding to the analog signal from the D / A converter 19. The output I / F 20 supplies the image display data stored in the memory 32 to the external device 210 as a digital signal. In this way, the display image data written to the memory 32 is displayed on the external device 210. Live view display (hereinafter referred to as LV display) can be performed by sequentially transferring the data stored in the memory 32, which has been A / D converted by the A / D converter 23, to the display unit 28, EVF 29, or external device 210 for display. Hereinafter, the image obtained by imaging by the imaging unit 22 that is displayed as a live view will be referred to as a live view image or LV image.
[0027] The external viewfinder display unit 43 displays various camera settings, including shutter speed and aperture, via the external viewfinder display unit drive circuit 44.
[0028] The non-volatile memory 56 is an electrically erasable and recordable memory, such as Flash-ROM. The non-volatile memory 56 stores constants for the operation of the system control unit 50, programs, etc. The program referred to here is a program for executing various flowcharts described later in this embodiment.
[0029] The system control unit 50 is a control unit consisting of at least one processor or circuit, and controls the entire electronic device 100. The system control unit 50 implements each of the processes of this embodiment, which will be described later, by executing the program recorded in the non-volatile memory 56. For example, RAM is used in the system memory 52, and constants, variables for the operation of the system control unit 50, the program read from the non-volatile memory 56, etc. are stored there. The system control unit 50 also performs display control by controlling the memory 32, the display unit 28, the EVF 29, etc.
[0030] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of the built-in clock.
[0031] The mode switch 60, first shutter switch 62, second shutter switch 64, and operation unit 70 are operating means for inputting various operation instructions to the system control unit 50. The mode switch 60 switches the operating mode of the system control unit 50 to one of the following: still image shooting mode, video shooting mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene detection mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The user can directly switch to any of these modes using the mode switch 60. Alternatively, the user can switch to a list screen of shooting modes using the mode switch 60, select one of the displayed modes, and then switch using other operating components. Similarly, the video shooting mode may also include multiple modes.
[0032] The first shutter switch 62 turns ON during the operation of the shutter button 61 on the electronic device 100, specifically when it is half-pressed (instruction to prepare for shooting), and generates the first shutter switch signal SW1. In response to receiving the first shutter switch signal SW1, the system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing.
[0033] The second shutter switch 64 turns ON when the shutter button 61 is fully pressed (shooting instruction), generating the second shutter switch signal SW2. In response to receiving the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processes, from reading the signal from the imaging unit 22 to writing the captured image as an image file to the recording medium 200.
[0034] The operation unit 70 consists of various operating components that act as an input unit for receiving operations from the user. The operation unit 70 includes at least a shutter button 61, a touch panel 70a, a main electronic dial 71, a power switch 72, a sub electronic dial 73, a directional pad 74, a SET button 75, a video button 76, an AE lock button 77, a zoom button 78, a playback button 79, a menu button 81, and the like.
[0035] The power control unit 80 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, supplying the necessary voltage to each part, including the recording medium 200, for the required period. The power supply unit 30 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li batteries, and an AC adapter.
[0036] I / F18 is an interface to a recording medium 200, such as a memory card or hard disk. The recording medium 200 is a removable recording medium, such as a memory card, for recording captured images, and is composed of semiconductor memory, magnetic disks, etc.
[0037] The communication unit 54 is connected wirelessly or via a wired cable and transmits and receives video and audio signals. The communication unit 54 can also connect to a wireless LAN (Local Area Network) or the internet. Furthermore, the communication unit 54 can communicate with external devices using Bluetooth® or Bluetooth® Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can also receive images and other various information from external devices.
[0038] The attitude detection unit 55 detects the orientation of the electronic device 100 relative to the direction of gravity. Based on the orientation detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was taken with the electronic device 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate the image before recording. An acceleration sensor or a gyro sensor can be used as the attitude detection unit 55. It is also possible to detect the movement of the digital camera 100 (pan, tilt, lift, whether it is stationary or not, etc.) using the acceleration sensor or gyro sensor in the attitude detection unit 55.
[0039] The eyepiece detection unit 57 is an eyepiece detection sensor that detects the approach (eye-gazing) and departure (eye-separation) of the user's eyeball (eye) to the eyepiece section 16 of the viewfinder (proximity detection). The system control unit 50 switches the display (display driven state) / hidden (hidden state) of the display unit 28 and EVF 29 according to the state detected by the eyepiece detection unit 57. More specifically, at least in the shooting standby state and when the display destination switching is automatic, when the user is not using an eyepiece, the display destination is set to the display unit 28 and the display is turned on, and the EVF 29 is hidden. When the user is using an eyepiece, the display destination is set to the EVF 29 and the display is turned on, and the display unit 28 is hidden. The eyepiece detection unit 57 can use, for example, an infrared proximity sensor to detect the approach of any object to the eyepiece section 16 of the viewfinder which has a built-in EVF 29. When an object approaches, infrared light emitted from the light-emitting unit (not shown) of the eyepiece detection unit 57 is reflected and received by the light-receiving unit (not shown) of the infrared proximity sensor. The amount of infrared light received allows for the determination of how close the object is to the eyepiece (eyepiece distance). In this way, the eyepiece detection unit 57 performs eyepiece detection to detect the proximity distance of an object to the eyepiece 16. When an object is detected approaching the eyepiece 16 within a predetermined distance from a non-eyepiece state (non-approach state), it is detected as eyepiece use. When the object that was detected approaching moves beyond a predetermined distance from an eyepiece state (approach state), it is detected as eye-seated. The threshold for detecting eyepiece use and the threshold for detecting eye-seated eye-seating may be different, for example, by providing hysteresis. After eyepiece use is detected, the eyepiece remains in the eyepiece state until eye-seated eye-seating is detected. After eye-seated eye-seating is detected, the eyepiece remains in the non-eyepiece state until eyepiece use is detected again. Note that the infrared proximity sensor is just one example; the eyepiece detection unit 57 may use any other sensor that can detect the approach of an eye or object that can be considered an eyepiece.
[0040] The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured such that its light transmittance does not interfere with the display of the display unit 28, and is mounted on the upper layer of the display surface of the display unit 28. Then, the input coordinates on the touch panel 70a are associated with the display coordinates on the display screen of the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that makes it seem as if the user can directly operate the screen displayed on the display unit 28.
[0041] The power saving control unit 160 detects the degree to which the user is gazing at the display unit 28 and controls the operation state and power supply state of the electronic device 100. In this embodiment, the degree to which the user is gazing at the display unit 28 is detected by detecting the direction of the user's face and the direction of the user's gaze. In addition, it controls the operation state or power supply state of the device that detects the face direction and gaze.
[0042] The face orientation detection unit 161 detects the orientation of the user's face. The face orientation determination unit 163 calculates the angle of the user's face orientation relative to the display unit 28 from the detection result of the face orientation detection unit 161 and determines whether the face is facing the display unit 28 or not. The gaze detection unit 162 detects the direction of the user's gaze. The gaze determination unit 164 determines whether the gaze is directed towards the display unit 28 from the detection result of the gaze detection unit 162. The system control unit 50 controls the state of the digital camera 100 to switch between normal mode and power saving mode according to the determination results of the face orientation determination unit 163 and the gaze determination unit 164 of the power saving control unit 160. Even when the display unit 28 is configured to be movable, the face orientation detection unit 161 and the gaze detection unit 162 may be placed near the display unit 28 in order to enable detection of gaze and face orientation relative to the display unit 28.
[0043] Specific examples of the face orientation detection unit 161 and gaze detection unit 162 are as follows. The face orientation detection unit 161 has an image sensor (not shown) installed near the display unit 28. This image sensor only needs to be able to determine which of several pre-set orientations the user's face is facing, and it is sufficient to have a resolution lower than the image sensor of the main imaging unit 22 and to be a monochrome image sensor. As a result, the power consumption of the face orientation detection unit 161 (and its image sensor) is far less than that of the main imaging unit 22. Based on the image from this low-resolution image sensor, the face orientation detection unit 161 generates and outputs information about the face orientation from the contour of the user's face and the arrangement of parts that make up the face, such as eyes, eyebrows, nose, and mouth. Based on this information, the face orientation determination unit 163 determines which of several pre-set orientations the user's face is facing and outputs it to the system control unit 50.
[0044] The face orientation detection unit 161 and the face orientation determination unit 163 utilize an image sensor. In this embodiment, the frequency of detection and determination of face orientation and gaze by the face orientation detection unit 161 and the face orientation determination unit 163 can be selected from two stages: the high-drive state (short cycle) is 0.5-second intervals, and the low-drive state (long cycle) is, for example, 1-second intervals. Naturally, the power consumption follows the relationship: power off < low-drive state < high-drive state.
[0045] On the other hand, the gaze detection unit 162 has an image sensor (not shown) installed near the display unit 28. This image sensor is independent of the image sensor in the face orientation detection unit 161. The image sensor in the gaze detection unit 162 only needs to be able to determine which of several pre-set orientations the gaze direction corresponds to, and it can have a resolution significantly lower than the image sensor in the main imaging unit 22, and can also be a monochrome image sensor. Therefore, its power consumption is far less than that of the main imaging unit 22. Based on the image from this low-resolution image sensor, the gaze detection unit 162 outputs information indicating the user's gaze direction based on the arrangement of the user's body parts. Based on this information, the gaze determination unit 164 determines which of several pre-set gaze directions the user's gaze direction corresponds to and outputs it to the system control unit 50. Furthermore, the gaze detection unit 162 and the gaze determination unit 164 operate in one of three modes based on the frequency of detecting and determining the gaze direction per unit time using the image sensor: high drive state (gaze direction detection interval is, for example, 0.5 seconds), low drive state (gaze direction detection interval is, for example, 1 second), or power off (no detection). Naturally, the power consumption follows the relationship: power off < low drive state < high drive state.
[0046] Figure 3 shows the sequence of control steps from standby to shooting while the digital camera 100 is powered on. The processing of the system control unit 50 will be explained below with reference to this figure.
[0047] When the digital camera 100 is started up, in S301 the system control unit 50 displays LV on the display unit 28 or EVF 29. In S302, the system control unit 50 determines whether or not it has received the shutter switch signal SW1 from the first shutter switch 62. In other words, the system control unit 50 determines whether or not the user has performed the operation of half-pressing the shutter button 61. If the system control unit 50 determines that it has not received the shutter switch signal SW1, it returns to processing in S301. On the other hand, if the system control unit 50 determines that it has received the shutter switch signal SW1 (the user has performed the operation of half-pressing the shutter button 61), it proceeds to processing in S303.
[0048] In S303, the system control unit 50 performs preparatory operations before shooting, such as AF processing. Then, the system control unit 50 proceeds to S304.
[0049] In S304, the system control unit 50 determines whether or not it has received the shutter switch signal SW2. In other words, the system control unit 50 determines whether or not the user has fully pressed the shutter button 61. If the system control unit 50 determines that it has not received the shutter switch signal SW2, it returns to S301. On the other hand, if the system control unit 50 determines that it has received the shutter switch signal SW1 (that the user has fully pressed the shutter button 61), it proceeds to S305.
[0050] In S305, the system control unit 50 performs a shooting operation, reads the signal from the imaging unit 22, and writes the captured image as an image file to the recording medium 200. Then, in S3056, the system control unit 50 displays the image obtained from the current capture on the display unit 28 or EVF 29 for a certain period of time, and then returns to processing in S301. In this way, LV display and shooting are repeated while the digital camera 100 is turned on.
[0051] The above describes the configuration of the digital camera and the shooting process in the embodiment.
[0052] [First Embodiment] The following describes a method for controlling the drive state and power supply of each detection unit using the detection results of the face orientation detection unit 161 and the gaze detection unit 162, according to the first embodiment, with reference to the flowchart in Figure 4. Furthermore, please understand that the flowchart in Figure 4 represents the control during the LV display in step S301 of the flowchart in Figure 3.
[0053] Before proceeding, the power-saving modes in the digital camera of this embodiment will be described. The digital camera of this embodiment has three modes for suppressing power consumption. The relationship of power consumption for each power-saving mode, including the normal mode which does not reduce power consumption, is as follows.
[0054] Power saving mode 3 < Power saving mode 2 < Power saving mode 1 < Normal mode Details of each power saving mode are as follows: • Power saving mode 1: The imaging interval (frame rate) by the imaging unit 22 is lowered compared to normal mode. In other words, the frame rate of the LV display is lower compared to normal mode. For example, if the frame rate by the imaging unit 22 is 30fps in normal mode, it is reduced to, for example, 10fps. Naturally, not only the imaging unit 22, but also the drive rate per unit time of the A / D converter 22, image processing unit 24, display unit 28, etc. will decrease, resulting in lower power consumption than in normal mode. • Power saving mode 2: In addition to power saving mode 1 described above, the operation of the display unit 28 and EVF 29 is further stopped. If the display unit 28 is, for example, a liquid crystal display, the power supply to the backlight is also turned off. • Power saving mode 3: The entire power supply to the electronic device 100 is turned off, putting it into sleep mode. However, in order to allow it to return from sleep mode to normal mode, a portion of the operation unit 70 and a portion of the system control unit 50 are kept operational to receive a wake command input from the user.
[0055] Based on the above, the processing of the system control unit 50 will be explained with reference to the flowchart in Figure 4. This processing is achieved by the system control unit 50 loading the program stored in the non-volatile memory 56 into the system memory 52 and executing it, thereby controlling each functional block. The flowchart in Figure 4 starts when the electronic device 100 is powered on.
[0056] When the electronic device 100 is started up, in S401, the system control unit 50 controls the drive state of the face orientation detection unit 161 to transition to a high drive state with a short detection interval, and then terminates the process.
[0057] In S402, the system control unit 50 determines whether the face orientation determination unit 163 was able to detect the user's face orientation. If it determines that the face orientation was detected, the system control unit 50 proceeds to S408. If it determines that the face orientation was not detected, the system control unit 50 proceeds to S403.
[0058] In S403, the system control unit 50 controls the power supply of the gaze detection unit 162 to turn off, and transitions the electronic device 100 to power saving mode 2. As explained earlier, power saving mode 2 includes power saving mode 1 and further turns off the drive of the display unit 28 and EVF 29. After this, the system control unit 50 proceeds to S404.
[0059] In S404, the system control unit 50 determines whether the state in which the face orientation cannot be detected has continued for a predetermined first time. If it determines that the state in which the face orientation cannot be detected has continued for the predetermined first time, the system control unit 50 proceeds to S405. If it determines that the state in which the face orientation cannot be detected has not reached the predetermined first time, the system control unit 50 returns to S402.
[0060] In S405, the system control unit 50 performs control to switch to power-saving mode 3, which reduces power consumption by turning off the power to the electronic device 100. Then, in S406, the system control unit 50 performs control to wait for a third predetermined time and proceeds to S407. In S407, the system control unit 50 determines whether it has received an operation from the operation unit 70. If it has received an operation, it returns to normal mode and proceeds to S402. On the other hand, if it determines that there has been no operation from the operation unit 70, the system control unit 50 returns to S405.
[0061] In S408, the system control unit 50 determines whether the face orientation determination unit 163 is facing the user's face orientation display unit 28. If it determines that the user's face is facing the display unit 2, the system control unit 50 proceeds to S413; otherwise, it proceeds to S409.
[0062] In S409, the system control unit 50 determines whether the angle difference between the user's face direction and the line connecting the user and the center of the display unit 28 is greater than or equal to a certain value. If it determines that the difference is less than the certain value, the system control unit 50 proceeds to S410; if it determines that the difference is greater than or equal to the certain value, it proceeds to S411.
[0063] In S410, the system control unit 50 controls the face orientation detection unit 161 to a high-drive state and the gaze detection unit 162 to a low-drive state with a longer detection interval. Furthermore, the system control unit 50 controls the system to switch to a low-frame-rate power-saving mode 1, which slows down the signal readout cycle of the imaging unit 22, and then returns the process to S402.
[0064] In S411, the system control unit 50 controls the face orientation detection unit 161 to a high-drive state and the gaze detection unit S162 to a power-off state. Furthermore, the system control unit 50 controls the electronic device 100 to power-saving mode 2, and then proceeds to S412.
[0065] In S412, the system control unit 50 determines whether the angle at which the user's face is directed toward the display unit 28 is greater than a certain value for a second predetermined time. If it determines that the angle at which the user's face is directed toward the display unit 28 is greater than a certain value for a second predetermined time, the system control unit 50 proceeds to S405; otherwise, it returns to S402.
[0066] In S413, the system control unit 50 controls the gaze detection unit 162 to transition to a low-drive state, and then proceeds to S414.
[0067] In S414, the system control unit 50 determines whether the gaze detection unit 164 was able to detect the user's gaze. If it determines that the user's gaze was detected, the system control unit 50 proceeds to S416; if it determines that it was not detected, it proceeds to S415.
[0068] In S415, the system control unit 50 transitions the face orientation detection unit 161 to a high-drive state and the gaze detection unit 162 to a low-drive state. Furthermore, the system control unit 50 controls the electronic device 100 to transition to the normal mode, which is the state at startup. After that, the system control unit 50 returns the process to S408.
[0069] In S416, the gaze determination unit 164 determines whether the user's gaze is directed towards the display unit 28. If the system control unit 50 determines that the user's gaze is directed towards the display unit 28, it proceeds to S418; otherwise, it proceeds to S417.
[0070] In step S417, the system control unit 50 controls the face orientation detection unit 161 to a low-drive state and the gaze detection unit 162 to a high-drive state. Furthermore, the system control unit 50 controls the electronic device 100 to power-saving mode 1. After that, the system control unit 50 returns the process to S414.
[0071] In S418, the system control unit 50 turns off the power to the face orientation detection unit 161 and switches the gaze detection unit 162 to a high-drive state. Furthermore, the system control unit 50 controls the electronic device 100 to return to normal mode. After that, the system control unit 50 returns the process to S414.
[0072] As described above, according to the first embodiment, by transitioning the power supply or driving state of each detection unit according to the detection results of the face orientation detection unit 161 and the gaze detection unit 162, power consumption can be reduced compared to when they are always driven.
[0073] In the above embodiment, the face orientation detection unit 161 and the gaze detection unit 162 were described as each having an image sensor. However, the image sensors for face orientation and gaze detection may be shared. In this case, the image acquisition cycle of the image sensor is fixed to the high-drive state described above. Instead of performing the face orientation detection process and gaze detection process each time an image is captured, the time intervals between these processes may be adjusted according to the "high-drive state" and "low-drive state" cycles to optimize the power consumption related to the calculation process.
[0074] Furthermore, if the eyepiece detection unit 57 detects that the user is looking through the EVF, that is, if it is presumed that the user is looking through the EVF, the distance between the image sensors of the face orientation detection unit 161 and the gaze detection unit 162 and the user's face is too close, making it impossible to detect the face orientation and gaze direction. In this case, if the eyepiece detection unit 57 detects that the user is looking through the EVF, the system control unit 50 may stop driving the face orientation detection unit 161 and the gaze detection unit 162 and drive the electronic devices in normal mode (however, the display unit 28 may be kept in a non-driven state).
[0075] [Second Embodiment] The second embodiment will be described below with reference to the flowchart in Figure 5. In this second embodiment, a method for controlling the state of the electronic device 100 according to the detection result of the face orientation detection unit 161 will be described. Here, the flowchart in Figure 5 should be understood as the control during the LV display at step S301 in the flowchart of Figure 3. Each process in the figure is realized by the system control unit 50 unpacking the program stored in the non-volatile memory 56 into the system memory 52 and executing it, thereby controlling each functional block. The flowchart also starts when the electronic device 100 is started up. Steps identical to those in the flowchart of Figure 4 are indicated by the same symbols, and their explanations are omitted. Unlike the control in Figure 4, this flowchart describes the control in a configuration where the electronic device 100 does not have a gaze detection unit 162, but only a face orientation detection unit 161. Except for the absence of the gaze detection unit 162, the device configuration is the same as in Figure 2, so please refer to Figure 2 for the device configuration.
[0076] When the electronic device 100 is started up, the system control unit 50 performs the same process as in S402 of the flowchart in Figure 4, and then proceeds to S408 or S501.
[0077] In S408, the system control unit 50 determines whether the user's face direction, as determined by the face direction determination unit 163, is facing the display unit 28. If the system control unit 50 determines that the user's face direction is facing the display unit 28, it proceeds to S503; otherwise, it proceeds to S409.
[0078] In S409, the system control unit 50 determines whether the angle difference between the user's face direction and the line connecting the user and the center of the display unit 28 is greater than or equal to a certain value. If it determines that the difference is less than the certain value, the system control unit 50 proceeds to S501; if it determines that the difference is greater than or equal to the certain value, it proceeds to S502.
[0079] In S501, the system control unit 50 controls the electronic device 100 to switch to power saving mode 1, and then returns the process to S402.
[0080] Furthermore, in S502, the system control unit 50 controls the electronic device 100 to transition to power-saving mode 2, and then returns to processing in S402.
[0081] In S503, the system control unit 50 controls the electronic device 100 to transition to normal mode and returns the process to S402.
[0082] As described above, according to the second embodiment, by transitioning the state of the electronic device 100 according to the detection result of the face orientation detection unit 161, power consumption can be reduced compared to when it is always used in normal mode. Unlike the first embodiment, the power saving effect obtained is smaller compared to the first embodiment because the driving state of the detection unit is not changed, but the cost can be reduced because the number of parts is reduced.
[0083] [Third Embodiment] The third embodiment will now be described with reference to the flowchart in Figure 6. In this third embodiment, the state of the electronic device 100 is controlled according to the detection result of the face detection unit 161. Furthermore, the process shown in the flowchart in Figure 6 should be understood as the control process during the LV display in step S301 of the flowchart in Figure 3.
[0084] Furthermore, each process in the flowchart of Figure 6 is realized by the system control unit 50 loading the program stored in the non-volatile memory 56 into the system memory 52 and executing it, thereby controlling each functional block. The flowchart of Figure 6 starts when the electronic device 100 is powered on. Steps identical to those in the flowchart of Figure 4 are indicated by the same symbols, and their explanations are omitted. Unlike the control in Figure 4, this flowchart describes the control in a configuration where the electronic device 100 does not have a face orientation detection unit 161 and only has a gaze detection unit 162. Except for the absence of the face orientation detection unit 161, the device configuration is the same as in Figure 2, so please refer to Figure 2 for the device configuration.
[0085] When the electronic device 100 is started up, in step S414, the system control unit 50 determines whether the gaze detection unit 164 was able to detect the user's gaze, similar to step S414 in the flowchart of Figure 4. If it determines that the user's gaze was detected, the system control unit 50 proceeds to S416; if it determines that it was not detected, it proceeds to S601.
[0086] In S416, the system control unit 50 determines whether the determined gaze is directed towards the display unit 28, similar to step S416 in the flowchart of Figure 4. If the system control unit 50 determines that the user's gaze is directed towards the display unit 28, it proceeds to S602; otherwise, it proceeds to S601.
[0087] In S601, the system control unit 50 controls the electronic device 100 to transition to normal mode, and then returns the process to S414.
[0088] In S602, the system control unit 50 controls the electronic device 100 to transition to power saving mode 1, and then returns the process to S414.
[0089] As described above, according to the third embodiment, by transitioning the state of the electronic device 100 according to the detection result of the gaze detection unit 162, power consumption can be reduced compared to when it is always used in normal mode. Also, similar to the second embodiment, since the driving state of the detection unit is not changed, the power saving effect obtained is smaller compared to the first embodiment, but the cost can be reduced because the number of parts is reduced.
[0090] [Fourth Embodiment] A fourth embodiment will now be described. In this fourth embodiment, a method for controlling the power supply of the face orientation detection unit 161 and the gaze detection unit 162 according to the state of the electronic device 100 will be described. The device configuration is the same as in Figure 2, and its description will be omitted.
[0091] In this fourth embodiment, the system control unit 50 controls the power off of the face orientation detection unit 161 and the gaze detection unit 162 when the electronic device 100 is in a state where settings can be changed while LV is displayed, when it is displaying or playing back captured video images, or when it is in menu mode. In all other states, the face orientation detection unit 161 and the gaze detection unit 162 are driven, and the same processing as in the first embodiment is performed according to their detection results.
[0092] As described above, according to the fourth embodiment, by controlling the power supply of the face orientation detection unit 161 and the gaze detection unit 162 to be turned off according to the state of the electronic device 100, power consumption can be reduced compared to when the detection units are always driven.
[0093] [Fifth Embodiment] A fifth embodiment will now be described. In this fifth embodiment, a method will be described to prevent the electronic device 100 from transitioning to a power-saving mode in response to the detection result of the gaze detection unit 162. Here, it is assumed that the electronic device 100 has a control that transitions to a state with lower power consumption than at startup if there is no operation on the operating member for a certain period of time.
[0094] When the system control unit 50 determines that the user's gaze is directed towards the electronic device 100 after a certain period of inactivity, it continues the startup state. If the gaze determination unit 164 determines that the user's gaze is directed towards the electronic device 100, it transitions to a power-saving mode (one of power-saving modes 1 to 3) that the user has selected in advance.
[0095] As described above, according to the fifth embodiment, by controlling the state of the electronic device 100 according to the detection state of the gaze detection unit 162, it is possible to prevent the electronic device 100 from transitioning to a state contrary to the user's intention.
[0096] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. Even if face orientation cannot be detected, if it can be detected that the gaze is directed towards the electronic device 100, the electronic device 100 may transition to normal mode. In Figure 4, the output result of the gaze detection unit 162 is referenced according to the output of the face orientation detection unit 161, but the output result of the face orientation detection unit 161 may be referenced according to the output of the gaze detection unit 162. In Figures 5 and 6, only the state transitions of the electronic device 100 are mentioned, but the drive state and power supply of the face orientation detection unit 161 or the gaze detection unit 162 may also be transitioned. The control contents in power saving modes 1 to 3 may be changed to different controls.
[0097] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.
[0098] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]
[0099] 28...LCD, 29...EVF, 50...System control unit, 100...Electronic equipment, 100a...Electronic equipment, 160...Power saving control unit, 161...Face orientation detection unit, 162...Gaze detection unit, 164...Face orientation determination unit, 164...Gaze determination unit
Claims
1. An electronic device having a display unit, A detection means for detecting a degree of gaze of a user on the display unit; a determination means for determining whether to transition to a normal mode indicating a normal operating state or a power saving mode which consumes less power than the normal mode, depending on a result of detection by the detection means; a control means for causing the electronic device to transition to a mode corresponding to the result of the determination by the determination means; 1. An electronic device comprising:
2. The display unit further includes an image capturing unit configured to capture an image to be displayed on the display unit. The power saving mode is a first power saving mode in which a period of imaging by the imaging means is made longer than that in the normal mode; a second power saving mode in which power to the display unit is turned off in addition to the first power saving mode; A third power saving mode that puts the electronic device into a sleep state.
2. The electronic device according to claim 1 .
3. The detection means includes: A face direction detection means for detecting a direction of a user's face; a gaze detection means for detecting a gaze direction of a user, The determination means determines a mode to be transitioned to as follows: When the face direction detection means is unable to detect the face direction, the second power saving mode is determined; determining a second power saving mode when the face direction detected by the face direction detection means is not directed toward the display unit and exceeds a threshold value with respect to the direction toward the display unit, and determining a first power saving mode when the face direction is equal to or smaller than the threshold value with respect to the direction toward the display unit; When the face direction detected by the face direction detection means is directed toward the display unit and the line of sight detection means cannot detect the line of sight direction, a normal mode is determined; when the line of sight detection means detects that the line of sight direction is not directed toward the display unit, a first power saving mode is determined; and when the line of sight detection means detects that the line of sight direction is directed toward the display unit, a normal mode is determined.
3. The electronic device according to claim 2.
4. The determination means determines, when the operation in the second power saving mode continues for a preset time period by the control means, that the third power saving mode is the mode to be shifted to.
4. The electronic device according to claim 3.
5. The control means When transitioning to the first power saving mode, a cycle of face direction detection by the face direction detection means is set to a higher cycle of two preset stages, and a cycle of gaze detection by the gaze detection means is set to a lower cycle of two preset stages, When the power saving mode is switched to the second power saving mode, a cycle of face direction detection by the face direction detection means is set to a higher cycle of two preset stages, and the line of sight detection means is stopped. When the mode is changed to the normal mode in a state where the gaze detection means is unable to detect the gaze, the cycle of face direction detection by the face direction detection means is set to a high cycle and the cycle of the gaze detection means is set to a low cycle; When the display device is switched to the normal mode in a state where the line of sight detected by the line of sight detection means is directed toward the display unit, the face direction detection means is stopped and the line of sight detection means is set to a high cycle.
5. The electronic device according to claim 3 or 4.
6. the detection means includes face direction detection means for detecting a direction of the face of the user, The determination means determines a mode to be transitioned to as follows: if the face direction detection means cannot detect the direction of the user's face, a first power saving mode is determined to be set; determining the first power saving mode when the direction of the user's face detected by the face direction detection means is not greater than a preset angle with respect to a direction toward the display unit, and determining a preset second power saving mode when the direction of the user's face detected by the face direction detection means is greater than the preset angle; When the direction of the user's face detected by the face direction detection means is directed toward the display unit, the normal mode is determined.
3. The electronic device according to claim 2.
7. the detection means includes a gaze detection means for detecting a gaze direction of the user, The determination means determines a mode to be transitioned to as follows: if the line of sight of the user cannot be detected by the line of sight detection means, the normal mode is determined; When the line of sight of the user detected by the line of sight detection means is not directed toward the display unit, a first power saving mode that is set in advance is determined; When the line of sight of the user detected by the line of sight detection means is directed to the display unit, the normal mode is determined.
3. The electronic device according to claim 2.
8. A method for controlling an electronic device having a display unit, comprising: A detection step of detecting a degree of gaze of a user on the display unit; a determination step of determining whether to transition to a normal mode indicating a normal operating state or a power saving mode which consumes less power than the normal mode, based on a result of the detection step; a control step of causing the electronic device to transition to a mode corresponding to the result of the determination step; 13. A method for controlling an electronic device comprising:
9. A program for causing a computer to execute each step of the method according to claim 8 when the program is read and executed by the computer.
