Electronic device and control method

Abstract

The present invention aims to accurately detect a person using an electronic device. The electronic device includes a memory for temporarily storing image data of images captured by a camera, and a processor for processing the image data stored in the memory. The processor includes: a face detection unit for processing image data of multiple images captured by the camera at predetermined time intervals and stored in the memory, and detecting a facial region in which a face is captured from the multiple images; an activity determination unit for calculating a change in the position of the facial region detected from the multiple images, and determining whether the captured face has moved within a predetermined time interval based on the change; and a person determination unit for determining that a user is present if the activity determination unit determines that the face has moved.

Description

Technical Field

[0001] The present invention relates to an electronic device and a control method. Background Art

[0002] There is an electronic device that switches to a usable state when a person approaches, and switches to a standby state with some functions disabled when the person moves away. For example, Patent Document 1 uses an infrared sensor to detect whether a person approaches or moves away.

[0003] In recent years, advances in computer vision and other technologies have led to higher accuracy in detecting faces from images. Consequently, facial detection has begun to be used instead of infrared sensors for person detection. While infrared sensors reflect infrared light regardless of whether it is a person or an object, facial detection can prevent false detection of simple objects as people.

[0004] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-148895.

[0005] However, even when face detection is used, faces of people in photographs, posters, etc. are detected, and thus there is a possibility that a person using the electronic device is mistakenly detected as existing even though they do not exist. Summary of the Invention

[0006] The present invention has been made in view of the above-mentioned circumstances, and one object of the present invention is to provide an electronic device and a control method for detecting a person in use with high accuracy.

[0007] The present invention is completed to solve the above-mentioned problems. The electronic device involved in the first embodiment of the present invention comprises: a memory for temporarily storing image data of images captured by a camera; and a processor for processing the image data stored in the above-mentioned memory, and the above-mentioned processor comprises: a face detection unit for processing the image data of multiple images captured by the above-mentioned camera at specified time intervals and stored in the above-mentioned memory, and detecting a facial area in which a face is captured from the above-mentioned multiple images; an activity determination unit for calculating the change in the position of the above-mentioned facial area detected from the above-mentioned multiple images, and determining whether the captured face is active within the above-mentioned specified time interval based on the above-mentioned change; and a person determination unit for determining that a user is present when the above-mentioned activity determination unit determines that the face is active.

[0008] In the electronic device, the person determination unit may be configured such that, when the motion determination unit determines that the captured face is not moving, the person determination unit determines that the face is not the user's face and determines that the user does not exist.

[0009] In the electronic device, the activity determination unit may register an area of ​​the face detected from each of the images, the area being determined by the activity determination unit to be inactive, and subsequently exclude the registered area from determination targets.

[0010] The electronic device may further include a sensor for detecting movement of the electronic device, and the movement determination unit may return the registered area to a target area for determination when movement of the electronic device is detected using the sensor.

[0011] In the above-mentioned electronic device, it can also be configured that the above-mentioned image data includes image data of a first resolution, the above-mentioned face detection unit performs processing of the above-mentioned image data of the first resolution to detect a low-resolution mode of the above-mentioned facial area, and the above-mentioned activity determination unit determines whether the face has activity based on the above-mentioned facial area detected by the above-mentioned face detection unit in the above-mentioned low-resolution mode.

[0012] In the above-mentioned electronic device, it can also be configured that the above-mentioned image data includes image data of a second resolution having a higher resolution than the above-mentioned first resolution, and the above-mentioned face detection unit executes a high-resolution mode of processing the above-mentioned second-resolution image data to detect the above-mentioned facial area when the above-mentioned activity determination unit determines that the face detected by processing the above-mentioned first-resolution image data has no activity, and the above-mentioned activity determination unit determines whether the face has activity based on the above-mentioned facial area detected by the above-mentioned face detection unit in the above-mentioned high-resolution mode.

[0013] In the electronic device, the face detection unit may be configured to detect the face area in the low-resolution mode if the face area is not detected in the high-resolution mode.

[0014] The electronic device may also be configured to further include: a processing unit that performs system processing based on the system; and an activity control unit that, when migrating from a state where the user is determined to be present by the person determination unit to a state where the user is determined not to be present, moves the activity state of the system to an activity state where at least a portion of the system processing is restricted.

[0015] In addition, the control method of the electronic device involved in the second embodiment of the present invention includes the following steps: the above-mentioned electronic device has a memory for temporarily storing image data of images captured by the camera device, and a processor for processing the image data stored in the above-mentioned memory, and the above-mentioned steps are: in the above-mentioned processor, the face detection unit processes the image data of multiple images captured by the above-mentioned camera device at a specified time interval and stored in the above-mentioned memory, and detects the facial area where the face is captured from the above-mentioned multiple images; the activity judgment unit calculates the change in the position of the above-mentioned facial area detected from the above-mentioned multiple images, and judges whether the captured face is active within the above-mentioned specified time interval based on the above-mentioned change; the person judgment unit judges that the user is present when the above-mentioned activity judgment unit determines that the face is active.

[0016] According to the above aspect of the present invention, a person using an electronic device can be detected with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram illustrating an overview of HPD processing by the electronic device according to the first embodiment.

[0018] Figure 2 FIG. 1 is a diagram showing an example of a captured image.

[0019] Figure 3 FIG. 1 is a diagram showing another example of a captured image.

[0020] Figure 4 This is a diagram showing a comparison of the presence or absence of facial movement.

[0021] Figure 5 It is a perspective view showing a configuration example of the appearance of the electronic device according to the first embodiment.

[0022] Figure 6 This is a schematic block diagram showing a configuration example of an electronic device according to the first embodiment.

[0023] Figure 7 This is a block diagram showing an example of the configuration of a person detection unit according to the first embodiment.

[0024] Figure 8 This is a diagram showing an example of a captured image used for detecting facial movements according to the first embodiment.

[0025] Figure 9 This is a flowchart showing an example of the HPD process according to the first embodiment.

[0026] Figure 10 This is a flowchart showing an example of the determination area reset process according to the first embodiment.

[0027] Figure 11 This is a flowchart showing an example of the startup process according to the first embodiment.

[0028] Figure 12 This is a flowchart showing an example of the standby state transition process according to the first embodiment.

[0029] Figure 13 This is a diagram showing an example of a person detection result in the low-resolution mode.

[0030] Figure 14 This is a flowchart showing an example of the HPD process according to the second embodiment.

[0031] Description of Reference Numerals

[0032] 1…electronic device; 10…first housing; 20…second housing; 15…hinge mechanism; 110…display unit; 120…shooting unit; 130…accelerometer; 140…power button; 150…input device; 151…keyboard; 153…touchpad; 200…EC; 210…person detection unit; 211…face detection unit; 212…activity determination unit; 213…person determination unit; 220…motion control unit; 300…system processing unit; 302…CPU; 304…GPU; 306…memory controller; 308…I / O controller; 310…system memory; 350…communication unit; 360…storage unit; 400…power supply unit. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0034] <First embodiment>

[0035] [summary]

[0036] First, the electronic device 1 of the first embodiment will be briefly described. The electronic device 1 of this embodiment is, for example, a notebook PC (Personal Computer). Alternatively, the electronic device 1 may be any electronic device such as a desktop PC, a tablet terminal device, or a smartphone.

[0037] The electronic device 1 is capable of migrating between at least a normal active state (power-on state) and a standby state as the active state of the system. The normal active state refers to an active state in which processing can be performed without particular restrictions, for example, equivalent to the S0 state specified by ACPI (Advanced Configuration and Power Interface). The standby state is a state in which at least a portion of the system processing is restricted. For example, the standby state may be a standby state, a sleep state, etc., or may be a modern standby in Windows (registered trademark), a state equivalent to the S3 state (sleep state) specified by ACPI, etc. For example, the standby state is an active state in which power consumption is lower than that of the normal active state.

[0038] Hereinafter, shifting the system activity state from the standby state to the normal active state may be referred to as booting. In the standby state, the activity level is generally lower than in the normal active state, so booting the system of the electronic device 1 is an activity that activates the system in the electronic device 1.

[0039] Figure 1 This is a diagram illustrating an overview of the HPD processing of the electronic device 1 of this embodiment. The electronic device 1 detects a person (i.e., a user) that exists near the electronic device 1. The process of detecting the presence of the person is called HPD (Human Presence Detection) processing. The electronic device 1 detects the presence of the person through the HPD processing and controls the activity state of the system of the electronic device 1 based on the detection result. For example, Figure 1 As shown in (A), when the electronic device 1 detects a change from a state (Absence) where no person exists in front of the electronic device 1 to a state (Presence) where a person approaches the electronic device 1 (Approach), it determines that the user is approaching, automatically activates the system, and shifts it to the normal active state. Figure 1 As shown in (B), the electronic device 1 determines that a user is present in the presence of a person in front of the electronic device 1, and continues the normal activity state. Figure 1 As shown in (C), when the electronic device 1 detects a change from a state where a person exists in front of (front of) the electronic device 1 (Presence) to a state where no person exists (Absence), that is, when the person leaves the electronic device 1 (Leave), it determines that the user has left and moves the system to standby mode.

[0040] For example, the electronic device 1 determines whether there is a user in front of (from) the electronic device 1 by detecting a facial area in which a face is captured from an image captured from the front (front side). When the electronic device 1 detects a facial area from the captured image, it determines that the user is present. On the other hand, when the electronic device 1 does not detect a facial area from the captured image, it determines that the user is not present. Here, when the user uses the electronic device 1, there is a usage situation in which there are photos or posters of people behind the user. In this usage situation, if the electronic device 1 captures the front (front side), it is obtained. Figure 2 The captured image shown.

[0041] Figure 2 is a diagram showing an example of a captured image. The illustrated captured image G1 includes an image of a user U, positioned in front of the electronic device 1, and an image of a poster P depicting a person. The electronic device 1 detects two facial regions from captured image G1: the face of the user U and the face of the poster P. Detection region FD1 represents the facial region of the user U detected from captured image G1. Furthermore, detection region FD2 represents the facial region of the poster P detected from captured image G1. Because detection region FD1 is larger than detection region FD2, the electronic device 1 detects the face of the user U as the face of a first person (the main character), determining that a person (i.e., the user) is present. If the electronic device 1 detects the face of the poster P as the face of a second person (i.e., a person other than the user), the face of the poster P may be mistakenly detected as a person. For example, if the electronic device 1 has a function for detecting the presence of another person peeking from behind (sholder surfing), detecting the face of the poster P may mistakenly detect the presence of another person peeking.

[0042] Figure 3 is another example of a captured image. The captured image G2 shown in the figure represents the image captured by the user U from the Figure 2 The image shown in captured image G1 is an example of a situation where the subject has disappeared (moved away). In this example, electronic device 1 detects only the face of poster P from captured image G2. In this case, electronic device 1 may mistakenly detect the presence of a person (i.e., a user) even though the face is on poster P.

[0043] Therefore, the electronic device 1 of the present embodiment determines whether the face detected from the captured image has movement in order to distinguish whether it is the face on the poster P or the face of the user U.

[0044] Figure 4This is a diagram showing a comparison of the presence or absence of facial movement. In this figure, image G3 shows the facial area detected from the captured images at time t(1), time t(2), time t(3), and time t(4). The captured images at time t(1), time t(2), time t(3), and time t(4) are images captured by the camera 120 at predetermined time intervals (e.g., 15-second intervals, 10-second intervals, etc.). As shown in image G3, the user U rarely remains completely still. Therefore, there is a difference in the detection area FD1 of the user U's face at each of time t(1), time t(2), time t(3), and time t(4). On the other hand, the poster P is completely still, so the detection area FD2 of the poster P's face at each of time t(1), time t(2), time t(3), and time t(4) is consistent. When the face detected from the captured image moves, the electronic device 1 determines that the user U is present. On the other hand, if the face detected from the captured image is motionless, the electronic device 1 determines that it is the face of a person in a photograph or poster, and determines that the user U does not exist. This allows the person (i.e., user) using the electronic device 1 to be detected with higher accuracy.

[0045] Next, the configuration of the electronic device 1 according to the present embodiment will be described in detail.

[0046] [Appearance and structure of electronic equipment]

[0047] Figure 5 It is a perspective view showing a configuration example of the appearance of the electronic device 1 according to the present embodiment.

[0048] Electronic device 1 includes a first housing 10, a second housing 20, and a hinge mechanism 15. First housing 10 and second housing 20 are coupled together using hinge mechanism 15. First housing 10 is rotatable relative to second housing 20 about a rotation axis defined by hinge mechanism 15. The opening angle formed by the rotation of first housing 10 and second housing 20 is shown as "θ."

[0049] The first housing 10 is also referred to as the A cover or the display housing. The second housing 20 is also referred to as the C cover or the system housing. In the following description, the sides of the first housing 10 and the second housing 20 having the hinge mechanism 15 are referred to as the side surfaces 10c and 20c, respectively. The sides of the first housing 10 and the second housing 20 opposite to the side surfaces 10c and 20c are referred to as the side surfaces 10a and 20a, respectively. In the figures, the direction from the side surface 20a toward the side surface 20c is referred to as the "rear," and the direction from the side surface 20c toward the side surface 20a is referred to as the "front." The right and left relative to the rear are referred to as the "right" and "left," respectively. The left sides of the first housing 10 and the second housing 20 are referred to as the side surfaces 10b and 20b, respectively, and the right sides are referred to as the side surfaces 10d and 20d, respectively. In addition, the state in which the first housing 10 and the second housing 20 overlap and are completely closed (the state in which the opening angle θ = 0°) is referred to as the "closed state." The surfaces of the first housing 10 and the second housing 20 that face each other in the closed state are referred to as their "inner surfaces," and the surfaces opposite to the inner surfaces are referred to as their "outer surfaces." Furthermore, the state in which the first housing 10 and the second housing 20 are opened relative to the closed state is referred to as the "open state."

[0050] Figure 5 The appearance of the electronic device 1 shown shows an example of an open state. The open state is a state in which the side surface 10a of the first shell 10 is separated from the side surface 20a of the second shell 20. In the open state, the inner surfaces of the first shell 10 and the second shell 20 are respectively visible. The open state is one of the states when the user uses the electronic device 1, and typically it is used in a state where the open angle θ is about 100 to 130 degrees. In addition, the range of the open angle θ that becomes the open state can be arbitrarily determined based on the range of the angle that can be rotated by the hinge mechanism 15, etc.

[0051] A display unit 110 is provided on the inner surface of the first shell 10. The display unit 110 is configured to include a liquid crystal display (LCD), an organic EL (Electro Luminescence) display, and the like. In addition, a camera unit 120 is provided in an area surrounding the display unit 110 on the inner surface of the first shell 10. For example, the camera unit 120 is arranged on the side surface 10a of the area surrounding the display unit 110. The position where the camera unit 120 is arranged is an example, and other locations may be used as long as it can face the inner surface of the first shell 10 (front).

[0052] When the camera unit 120 is in the open position, it captures a predetermined imaging range in the direction facing (frontward) the inner surface of the first housing 10. The predetermined imaging range refers to the range of the viewing angle determined by the imaging element of the camera unit 120 and the optical lens disposed in front of the imaging surface of the imaging element. For example, the camera unit 120 can capture an image including a person in front of (frontal view of) the electronic device 1.

[0053] In addition, a power button 140 is provided on the side surface 20b of the second shell 20. The power button 140 is an operating member for the user to instruct the system to start (migrate from the standby state to the normal operation state) and to migrate from the normal operation state to the standby state. In addition, a keyboard 151 and a touchpad 153 are provided on the inner surface of the second shell 20 as input devices. In addition, as an input device, a touch sensor may be included, or a mouse or an external keyboard may be connected instead of the keyboard 151 and the touchpad 153. In the case of a structure in which a touch sensor is provided, it may be configured as a touch panel that receives operations in an area corresponding to the display surface of the display unit 110. In addition, a microphone for inputting sound may be included in the input device.

[0054] In addition, when the first shell 10 and the second shell 20 are closed, the display unit 110 and the shooting unit 120 arranged on the inner surface of the first shell 10 and the keyboard 151 and the touchpad 153 arranged on the inner surface of the second shell 20 are covered by the shell surface of the other party and become unable to function.

[0055] [Structure of electronic equipment]

[0056] Figure 6 This is a schematic block diagram illustrating an example configuration of electronic device 1 according to this embodiment. Electronic device 1 includes a display unit 110, an imaging unit 120, an acceleration sensor 130, a power button 140, an input device 150, an embedded controller (EC) 200, a human detection unit 210, a system processing unit 300, a communication unit 350, a storage unit 360, and a power supply unit 400. The display unit 110 displays display data (images) generated based on system processing executed by the system processing unit 300 and processing by applications running on the system processing.

[0057] The photographing unit 120 photographs an image of an object within a predetermined viewing angle in a direction facing the inner surface of the first shell 10 (front), and outputs the photographed image to the system processing unit 300 and the person detection unit 210. The photographing unit 120 can be an infrared camera or a conventional camera. An infrared camera is a camera having an infrared sensor as a photographing element. A conventional camera is a camera having a visible light sensor that receives visible light as a photographing element (for example, an RGB camera). In addition, in the case of a conventional camera, the photographed image used for face detection can also be an image with a reduced number of colors (for example, a monochrome image).

[0058] Acceleration sensor 130 detects the movement of electronic device 1 and outputs a detection signal indicating the detection result to EC 200. For example, acceleration sensor 130 outputs a detection signal based on the movement of electronic device 1 when electronic device 1 is moving or when electronic device 1 is being held and moved unsteadily. Alternatively, a gyroscopic sensor, tilt sensor, geomagnetic sensor, or the like may be provided in place of acceleration sensor 130.

[0059] Power button 140 outputs an operation signal to EC 200 in response to a user operation. Input device 150 is an input unit that receives user input and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to receiving an operation on keyboard 151 and touchpad 153, input device 150 outputs an operation signal indicating the operation to EC 200.

[0060] The power supply unit 400 supplies power to each component of the electronic device 1 via a power supply system for supplying power to each component, depending on the operating status of each component. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of DC power supplied from an AC (Alternate Current) / DC adapter or battery pack into the voltage required by each component. The power after the voltage conversion by the DC / DC converter is supplied to each component via each power supply system. For example, the power supply unit 400 supplies power to each component via each power supply system based on a control signal according to the operating status of each component input from the EC200.

[0061] The EC200 is a microcomputer comprised of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and I / O (Input / Output) logic circuits. The EC200 CPU reads the control program (firmware) pre-stored in the ROM and executes it to perform its functions. The EC200 operates independently of the system processing unit 300, controlling the operation of the system processing unit 300 and managing its operational status. The EC200 is also connected to the acceleration sensor 130, the power button 140, the input device 150, the human detection unit 210, and the power supply unit 400.

[0062] For example, the EC200 communicates with the power supply unit 400 to obtain information on the battery status (remaining capacity, etc.) from the power supply unit 400, and outputs control signals for controlling the supply of power corresponding to the operating status of each part of the electronic device 1 to the power supply unit 400. In addition, the EC200 obtains operation signals from the power button 140 and the input device 150, and outputs the operation signals related to the processing of the system processing unit 300 among the obtained operation signals to the system processing unit 300. In addition, the EC200 detects the movement of the electronic device 1 based on the detection signal from the acceleration sensor 130. For example, the EC200 detects whether the electronic device 1 is in a stationary state or in a moving state based on the detection signal from the acceleration sensor 130. In addition, the EC200 has an action control unit 220, which controls the movement of the system based on the detection result of the person detection unit 210.

[0063] The person detection unit 210 is a processor that processes image data from images captured by the imaging unit 120. For example, the person detection unit 210 obtains images captured by the imaging unit 120 via the system processing unit 300. Alternatively, the person detection unit 210 may obtain images directly from the imaging unit 120. The person detection unit 210 detects the presence of a user by detecting a facial region in the captured image and performs HPD processing based on the detection result.

[0064] The person detection unit 210 detects a facial area based on the image captured by the camera unit 120, thereby detecting whether a user is in front of the electronic device 1. For example, when a user approaches the electronic device 1, the person detection unit 210 changes the detection state from a state in which the user is not in front of the electronic device 1 to a state in which the user is present. Furthermore, when a user is using the electronic device 1 in front of the electronic device 1, the person detection unit 210 continues to detect the state in which the user is in front of the electronic device 1. Furthermore, when a user leaves the electronic device 1, the person detection unit 210 changes the detection state from a state in which the user is in front of the electronic device 1 to a state in which the user is not present. In this way, by detecting whether a user is in front of the electronic device 1, the person detection unit 210 can detect the user approaching the electronic device 1 (Approach), the user's presence in front of the electronic device 1 (Presence), the user leaving the electronic device 1 (Leave), and the user's absence in front of the electronic device 1 (Absence). The structure of the person detection unit 210 will be described in detail later.

[0065] The motion control unit 220 controls the operating state of the system based on the HPD processing. For example, when the person detection unit 210 detects a change from a state where there is no user in front of the electronic device 1 to a state where there is a user (i.e., the user approaches the electronic device 1) in the standby state, the motion control unit 220 activates the system in the standby state. Specifically, when the person detection unit 210 detects that a user is approaching the electronic device 1, the motion control unit 220 instructs the system processing unit 300 to activate the system. More specifically, when activating the system, the motion control unit 220 outputs a control signal to the power supply unit 400 for supplying the power required for the operation of each part of the electronic device 1. Then, the motion control unit 220 outputs a startup signal to the system processing unit 300 for instructing the activation of the system. If the system processing unit 300 receives the startup signal, it activates the system and moves it from the standby state to the normal operation state.

[0066] Furthermore, when the person detection unit 210 continuously detects the presence of a user in front of the electronic device 1, the motion control unit 220 controls the system processing unit 300 to prevent the system from transitioning to the standby state and maintain the normal operating state. Furthermore, when the person detection unit 210 continuously detects the presence of a user, the motion control unit 220 can also transition from the normal operating state to the standby state under predetermined conditions. Predetermined conditions include, for example, a predetermined period of inactivity or a transition to the standby state.

[0067] Furthermore, during normal operation, if the person detection unit 210 detects a change from a state in which a user is present in front of the electronic device 1 to a state in which the user is absent (i.e., the user has left the electronic device 1), the motion control unit 220 instructs the system processing unit 300 to shift the system from the normal operating state to the standby state. More specifically, the motion control unit 220 outputs a standby signal to the system processing unit 300 to instruct the system to shift from the normal operating state to the standby state. Upon receiving the standby signal, the system processing unit 300 shifts the system from the normal operating state to the standby state. The motion control unit 220 then outputs a control signal to the power supply unit 400 to stop the supply of power that is not required in the standby state.

[0068] The system processing unit 300 includes a CPU (Central Processing Unit) 302, a GPU (Graphic Processing Unit) 304, a memory controller 306, an I / O (Input-Output) controller 308, and system memory 310. Through system processing based on the OS (Operating System), it can execute various application programs on the OS. The CPU 302 and GPU 304 are sometimes collectively referred to as a processor.

[0069] CPU 302 executes processing based on the OS and processing based on applications running on the OS. Furthermore, CPU 302 transitions the system's operating state based on the control of the system's operating state by EC 200 (operation control unit 220). For example, when the CPU 302 is in the standby state and a startup signal is input from EC 200, the CPU 302 executes startup processing to transition from the standby state to the normal operating state. After completing the startup processing, CPU 302 begins executing system processing based on the OS. For example, when the CPU 302 is in the standby state and a startup signal is input from EC 200, the CPU 302 resumes execution of a stopped application, etc.

[0070] During the startup process, CPU302 executes a login process to determine whether to allow the use of the OS. If CPU302 starts the startup process based on the OS, it executes the login process before allowing the use of the OS, and temporarily stops the transition to the normal operation state until login is allowed during the login process. During the login process, a user authentication process is performed to determine whether the person using the electronic device 1 is a pre-registered regular user. Authentication methods include password authentication, facial authentication, fingerprint authentication, etc. If the authentication result is successful, CPU302 allows login and resumes the execution of the temporarily stopped system process. On the other hand, if the authentication result fails, login is not allowed and the state of stopping the execution of the system process remains.

[0071] The GPU 304 is connected to the display unit 110. Under the control of the CPU 302, the GPU 304 performs image processing to generate display data. The GPU 304 outputs the generated display data to the display unit 110. Furthermore, the CPU 302 and the GPU 304 may be integrated into a single core, or the load may be shared between the CPU 302 and the GPU 304, each formed as a core. The number of processors is not limited to one and may also be multiple.

[0072] The memory controller 306 controls the CPU 302 and the GPU 304 to read and write data from the system memory 310 , the storage unit 360 , and the like.

[0073] The I / O controller 308 controls input and output of data from the communication unit 350 , the display unit 110 , and the EC 200 .

[0074] The system memory 310 is used as a read area for the execution program of the processor and a work area for writing processing data. In addition, the system memory 310 temporarily stores image data of an image captured by the imaging unit 120.

[0075] The communication unit 350 is connected to other devices via a wireless or wired communication network to enable communication and transmit and receive various data. For example, the communication unit 350 is configured to include a wired LAN interface such as Ethernet (registered trademark) or a wireless LAN interface such as Wi-Fi (registered trademark).

[0076] The storage unit 360 includes storage media such as a hard disk drive (HDD), a solid state drive (SDD), RAM, and ROM. The storage unit 360 stores various programs such as the OS, device drivers, and applications, as well as various data acquired by program execution.

[0077] [Structure of the person detection unit]

[0078] Next, the structure of the person detection unit 210 will be described in detail. The person detection unit 210 detects a user in front of the electronic device 1 by detecting a face region from each of images captured by the imaging unit 120 at predetermined time intervals.

[0079] Figure 7 2 is a block diagram showing an example of the configuration of the person detection unit 210 according to the present embodiment. The person detection unit 210 shown in the figure includes a face detection unit 211 , a motion determination unit 212 , and a person determination unit 213 .

[0080] The face detection unit 211 detects facial regions containing faces from each of the captured images captured at predetermined time intervals. For example, the face detection unit 211 processes image data from a plurality of images captured at predetermined time intervals by the camera unit 120 and stored in the system memory 310, and detects facial regions containing faces from these images. Face detection methods can employ any method, including a face detection algorithm that detects faces based on facial feature information, learning data (learned models) machine-learned based on facial feature information, and a face detection library. The predetermined time intervals can be, for example, 15-second intervals or 10-second intervals, but can be set to any other time interval. Furthermore, in the case of the shortest time interval, detection is performed for all consecutive frames. The face detection unit 211 detects a facial region from each of the captured images and outputs facial detection information containing the detected facial region and the coordinates of its center.

[0081] The movement determination unit 212 determines whether the face has moved based on the facial region detected by the face detection unit 211 from the multiple captured images. For example, the movement determination unit 212 determines whether the face has moved based on the facial region detected from each of the captured images captured at predetermined time intervals during a predetermined period. Specifically, the movement determination unit 212 calculates the amount of change in the position of the facial region detected from the multiple captured images and, based on this change, determines whether the captured face has moved within the predetermined time interval.

[0082] Figure 8 This figure shows an example of captured images used by the activity determination unit 212 to detect facial activity. The captured images from time t(1) to time t(8) are arranged in chronological order and shown. For example, the activity determination unit 212 uses the past four frames of captured images to determine facial activity. In this case, the prescribed period corresponds to the period of four frames. For example, when using images captured at 15-second intervals, the prescribed period (the period of four frames) becomes 45 seconds.

[0083] At the time of acquiring the captured image at time t(4), the activity determination unit 212 determines the activity of the face based on the facial area detected from the four frames of captured images from time t(1) to time t(4). At the time of acquiring the captured image at time t(5), the activity determination unit 212 determines the activity of the face based on the facial area detected from the four frames of captured images from time t(2) to time t(5). At the time of acquiring the captured image at time t(6), the activity determination unit 212 determines the activity of the face based on the facial area detected from the four frames of captured images from time t(3) to time t(6). At the time of acquiring the captured image at time t(7), the activity determination unit 212 determines the activity of the face based on the facial area detected from the four frames of captured images from time t(4) to time t(7). At the time of acquiring the captured image at time t(8), the activity determination unit 212 determines the activity of the face based on the facial area detected from the four frames of captured images from time t(5) to time t(8).

[0084] For example, the motion determination unit 212 calculates the deviation of the center coordinates of the facial region detected from the past four frames of captured imagery. If the deviation is greater than a predetermined threshold, the motion determination unit 212 determines that the face has moved. On the other hand, if the deviation is less than the predetermined threshold, the motion determination unit 212 determines that the face has not moved. Alternatively, the standard deviation or variance may be used as the deviation value. Furthermore, when the center coordinates of the facial region are expressed in an XY two-dimensional coordinate system, both the X and Y coordinates may be used, or either one may be used. Furthermore, motion determination may use the deviation of the facial region or the deviation of the size of the facial region, rather than the deviation of the center coordinates of the facial region. Using the deviation of the size of the facial region also enables determination of forward and backward facial motion.

[0085] Furthermore, the activity determination unit 212 may register areas of faces detected in the captured image as inactive and subsequently exclude these registered areas from the target of determination. For example, even if the activity determination unit 212 detects a face in an area excluded from determination, it may ignore the face and not determine the movement of the face. Furthermore, if the acceleration sensor 130 detects movement of the electronic device 1, the activity determination unit 212 resets the determination area. In other words, the activity determination unit 212 returns the area registered as excluded from determination to the target of determination.

[0086] If the motion determination unit 212 determines that the face detected from each captured image is moving, the person determination unit 213 determines that the user is present. If the motion determination unit 212 determines that the face detected from each captured image is not moving, the person determination unit 213 determines that the face is not that of the user and determines that the user is not present.

[0087] [HPD processing operation]

[0088] Next, refer to Figure 9 , the operation of the HPD process (human detection process) performed by the human detection unit 210 will be described.

[0089] Figure 9 This is a flowchart showing an example of the HPD process according to this embodiment.

[0090] (Step S101) The person detection unit 210 detects a face region from the image captured by the image capture unit 120 at time t(n). First, the person detection unit 210 sets n=1 and detects a face region from the image captured by the image capture unit 120 at time t(1).

[0091] (Step S103) The person detection unit 210 determines whether a facial region was detected from the captured image in step S101. If the person detection unit 210 was unable to detect a facial region from the captured image in step S101 (No), the process proceeds to step S105. On the other hand, if the person detection unit 210 was able to detect a facial region from the captured image in step S101 (Yes), the process proceeds to step S107.

[0092] (Step S105) The person detection unit 210 determines that the user is not present. Then, the person detection unit 210 sets n to 1, returns to the process of step S101, and again detects a face area from the image captured by the imaging unit 120 at time t(1).

[0093] (Step S107) The person detection unit 210 determines whether n is equal to or greater than 4. That is, the person detection unit 210 determines whether a face region is detected from the captured image within 4 frames. If the person detection unit 210 determines that n is less than 4 (No), it increments n by 1, returns to the process of step S101, and detects a face region from the captured image captured by the capturing unit 120 at the next time t(n). For example, if the person detection unit 210 detects a face region from the captured image at time t(1), it detects a face region from the captured image at time t(2), then, if the person detection unit 210 detects a face region from the captured image at time t(2), then, if the person detection unit 210 detects a face region from the captured image at time t(3), then, if the person detection unit 210 detects a face region from the captured image at time t(3), then, if the person detection unit 210 detects a face region from the captured image at time t(4). On the other hand, when the person detection unit 210 determines that n is 4 or greater (Yes), that is, when it determines that the face region is detected from the captured image within 4 frames, the process proceeds to step S109 .

[0094] (Step S109) When the person detection unit 210 acquires the captured image at time t(n), it determines whether the face has moved based on the facial area detected from the four frames of captured images from time t(n-3) to time t(n). For example, when the person detection unit 210 acquires the captured image at time t(4), it determines whether the face has moved based on the facial area detected from the four frames of captured images from time t(1) to time t(4). For example, the person detection unit 210 determines whether the face has moved based on the deviation of the center coordinates of the facial area detected from the captured images at each time t(1) to time t(4). If the person detection unit 210 determines that the face has moved (Yes), the process proceeds to step S111. On the other hand, if the person detection unit 210 determines that the face has not moved (No), the process proceeds to step S113.

[0095] (Step S111) The person detection unit 210 determines that the moving face is the user's face and determines that the user is present. In other words, the person detection unit 210 determines that the user is present in front of the electronic device 1. The person detection unit 210 then increments n by 1 and returns to the process of step S101 to detect a facial region from the image captured by the imaging unit 120 at the next time t(n).

[0096] (Step S113) The person detection unit 210 determines that the inactive face is not the user's face and determines that the user does not exist. In other words, the person detection unit 210 determines that the user does not exist in front of the electronic device 1. Then, the process proceeds to step S115.

[0097] (Step S115) The person detection unit 210 registers the facial region determined to be inactive as a determination exclusion region. Subsequently, the registered determination exclusion region is excluded from the determination targets. The person detection unit 210 then increments n by 1 and returns to the process of step S101 to detect a facial region from the image captured by the imaging unit 120 at the next time t(n). Subsequently, even if the person detection unit 210 detects a face in the determination exclusion region, it is deemed not to be the user's face and does not determine facial activity (i.e., whether the face is the user's).

[0098] In the present embodiment, the period for determining facial movement is set to a period corresponding to four frames, but the present invention is not limited thereto and may be set to any period.

[0099] Next, refer to Figure 10 , to reset in Figure 9 The operation of the determination area reset processing of the determination exclusion area registered in step S115 will be described.

[0100] Figure 10 This is a flowchart showing an example of the determination area reset process according to the present embodiment.

[0101] (Step S151) The person detection unit 210 determines whether the electronic device 1 has moved based on the detection results of the acceleration sensor 130. If the person detection unit 210 determines that the electronic device 1 has moved (Yes), the process proceeds to step S153. On the other hand, if the person detection unit 210 determines that the electronic device 1 has not moved (No), the process ends without resetting the detection area.

[0102] (Step S153) The human detection unit 210 resets the registered determination exclusion area and ends the process. In other words, the human detection unit 210 returns the area excluded from the determination target (determination exclusion area) to the determination target.

[0103] For example, the human detection unit 210 executes the determination area reset processing at a predetermined period. Figure 9 If an area excluded from determination is registered in step S115, the determination area reset process is executed.

[0104] [Actions of the action state control process]

[0105] Next, the operation of the operation state control process for controlling the operation state of the system based on the result of the above-mentioned HPD process will be described. First, the operation of the operation control unit 220 detecting the user approaching the electronic device 1 through the HPD process and starting the system startup process will be described.

[0106] Figure 11 1 is a flowchart showing an example of the startup process of this embodiment. Here, the electronic device 1 is placed on a desk or the like in an open state and is in a standby state.

[0107] (Step S201) The motion control unit 220 determines whether a user's approach to the electronic device 1 has been detected. If the motion control unit 220 determines that the person detection unit 210 has detected a change from a state in which no user is present to a state in which a user is present in front of the electronic device 1 (i.e., a user's approach to the electronic device 1) (Yes), the process proceeds to step S203. On the other hand, if the motion control unit 220 determines that the person detection unit 210 has detected a state in which no user is present (i.e., a user has not approached the electronic device 1) (No), the process proceeds to step S201 again.

[0108] (Step S203) The motion control unit 220 activates the system processing unit 300. Specifically, when activating the system processing unit 300, the motion control unit 220 outputs a control signal to the power supply unit 400 to supply the power required for the operation of each component of the electronic device 1. Furthermore, the motion control unit 220 outputs a startup signal to the system processing unit 300 instructing it to activate the system. Upon receiving the startup signal, the system processing unit 300 begins the startup process. The process then proceeds to step S205.

[0109] (Step S205) The system processing unit 300 executes a login process (authentication process). For example, the system processing unit 300 executes a login process based on password authentication, face authentication, fingerprint authentication, etc., and then proceeds to the process of step S207.

[0110] (Step S207) The system processing unit 300 determines whether the authentication result is successful. If the system processing unit 300 determines that the authentication result is successful (Yes), the process proceeds to step S209. On the other hand, if the system processing unit 300 determines that the authentication result is unsuccessful (No), the process proceeds to step S213.

[0111] (Step S209) If the authentication result is successful, the system processing unit 300 notifies the user of the successful login (e.g., by displaying it on the display unit 110) and continues the startup process. The process then proceeds to step S211. (Step S211) The system processing unit 300 completes the login process and returns the system to its normal operating state.

[0112] (Step S213) If the authentication result fails, the system processing unit 300 notifies the user of the login failure (e.g., by displaying it on the display unit 110) and returns to the authentication process of step S205. Alternatively, if the authentication process fails a predetermined number of times, the system processing unit 300 may terminate the authentication process and enter a login-disabled state.

[0113] Next, the operation of the standby state transition process in which the operation control unit 220 shifts the system from the normal operation state to the standby state by detecting that the user has left the electronic device 1 will be described.

[0114] Figure 12 1 is a flowchart showing an example of the standby state transition process of this embodiment. Here, the electronic device 1 is placed on a desk or the like in an open state and is in a normal operating state.

[0115] (Step S251) The motion control unit 220 determines whether the user has been detected leaving the electronic device 1. If the human detection unit 210 detects a change from a state where the user is present to a state where the user is not present (i.e., the user has left the electronic device 1) (Yes), the motion control unit 220 proceeds to step S253. On the other hand, if the human detection unit 210 determines that the user is present (i.e., the user has not left the electronic device 1) (No), the motion control unit 220 repeats step S251.

[0116] (Step S253) The operation control unit 220 shifts the system operation state of the system processing unit 300 from the normal operation state to the standby state. Specifically, the operation control unit 220 outputs a standby signal to the system processing unit 300, instructing the system to shift to the standby state. Upon receiving the standby signal, the system processing unit 300 shifts the system operation state from the normal operation state to the standby state. Furthermore, the operation control unit 220 outputs a control signal to the power supply unit 400 to stop supplying power that is not required in the standby state.

[0117] [Summary of the First Embodiment]

[0118] As described above, the electronic device 1 of this embodiment includes a system memory 310 (an example of a memory) that temporarily stores image data of images (captured images) captured by the camera unit 120 (an example of a camera device), and a person detection unit 210 (an example of a processor) that processes the image data stored in the system memory 310. The person detection unit 210 processes the image data of a plurality of captured images captured by the camera unit 120 at predetermined time intervals and stored in the system memory 310, and detects a facial region in which a face is captured from the plurality of captured images. In addition, the person detection unit 210 calculates the amount of change in the position of the facial region detected from the plurality of captured images, and based on this amount of change, determines whether the captured face has moved within a predetermined time interval. Furthermore, if the person detection unit 210 determines that the captured face has moved, it determines that a user is present.

[0119] As a result, the electronic device 1 determines only the moving faces among the detected faces as the user, and can thus detect the presence of the user with high accuracy.

[0120] For example, the person detection unit 210 determines whether the face is moving based on the face region detected from each of the images captured at predetermined time intervals within a predetermined period.

[0121] Thus, the electronic device 1 determines whether the face has moved based on the facial movement over a certain period of time, and can therefore detect the presence of the user with high accuracy.

[0122] Furthermore, when the person detection unit 210 determines that the captured face is not moving, it determines that the face is not the user's face and determines that the user does not exist.

[0123] As a result, the electronic device 1 determines that any detected faces that are inactive are not the user, thereby preventing faces in photos, posters, etc. from being mistakenly identified as the user. This prevents the electronic device 1 from mistakenly detecting non-existent users. Furthermore, even if a face in a photo, poster, etc. is behind the user, the electronic device 1 can prevent the mistaken detection of another person peeping in.

[0124] Furthermore, the person detection unit 210 registers a face region determined to be inactive among faces detected from each captured image, and subsequently excludes the registered region from the targets of determination.

[0125] As a result, the electronic device 1 detects a facial area that is not the user's face and subsequently excludes it from the target of determination, thereby reducing the processing load and suppressing erroneous detection.

[0126] Furthermore, the electronic device 1 includes an acceleration sensor 130 (an example of a sensor) for detecting movement of the electronic device 1. When the person detection unit 210 detects movement of the electronic device 1 using the acceleration sensor 130, it returns the registered area to the target of the determination.

[0127] As a result, when the electronic device 1 moves, the area excluded from the target of determination also changes, and thus the area returns to the target of determination, thereby enabling appropriate detection of the user's presence.

[0128] Furthermore, the electronic device 1 includes a system processing unit 300 (an example of a processing unit) that executes system processing based on the system. When the state changes from a state where the user is determined to be present to a state where the user is determined to be absent, the electronic device 1 changes the operating state of the system to a standby state (an operating state in which at least a portion of the system processing is restricted).

[0129] This allows electronic device 1 to accurately detect when the user has left the device and appropriately switch to the standby state.

[0130] Furthermore, the control method of the electronic device 1 of this embodiment includes the following steps. The electronic device 1 includes a system memory 310 (an example of a memory) that temporarily stores image data of images (captured images) captured by the camera unit 120 (an example of a camera device), and a person detection unit 210 (an example of a processor) that processes the image data stored in the system memory 310. The steps are as follows: the person detection unit 210 processes the image data of a plurality of captured images captured by the camera unit 120 at predetermined time intervals and stored in the system memory 310, detects a facial region in which a face is captured from the plurality of captured images; calculates a change in the position of the facial region detected from the plurality of captured images, and determines whether the captured face has moved within a predetermined time interval based on the change; and determines that a user is present if the captured face is determined to be moving.

[0131] As a result, the electronic device 1 determines only the moving faces among the detected faces as the user, and can thus detect the presence of the user with high accuracy.

[0132] <Second embodiment>

[0133] Next, a second embodiment of the present invention will be described.

[0134] While the first embodiment described a method for detecting the presence of a user by detecting a facial region from a captured image, considering factors such as power consumption and the impact on other processing, it is preferable to minimize the processing load associated with facial detection. Therefore, in this embodiment, electronic device 1 uses either a high-resolution image or a low-resolution image when detecting a facial region from a captured image, depending on the situation. Hereinafter, the detection mode that uses a low-resolution image to detect a facial region will be referred to as "low-resolution mode," and the detection mode that uses a high-resolution image to detect a facial region will be referred to as "high-resolution mode."

[0135] Various methods are contemplated for acquiring high-resolution and low-resolution images. When using an imaging unit 120 capable of outputting both high-resolution and low-resolution images, the system may instruct the imaging unit 120 which image to output based on the circumstances. Alternatively, when using an imaging unit 120 capable of simultaneously outputting both high-resolution and low-resolution images, the system may select one of the images output from the imaging unit 120 as the image to be processed based on the circumstances. Additionally or alternatively, the system may acquire the low-resolution image to be processed by pre-processing the image data output from the imaging unit 120 to reduce its resolution.

[0136] Here, in the low-resolution mode, the processing load is small and power consumption is suppressed. However, due to the low resolution, subtle facial movements may not be detected, and the presence of the user may not be accurately detected.

[0137] Figure 13 This is a diagram showing an example of a person detection result in low-resolution mode. In the example shown in the figure, the user is captured in all the captured images from time t(1) to t(12). A face is detected from each captured image, but if there is a period when the user is almost motionless, subtle facial movements may not be detected. In the example shown in the figure, it is shown that the face detected at time t(1) to t(5) and time t(10) to t(12) is determined to have movement, but the face detected at time t(6) to t(8) is determined to have no movement. For example, in the case of capturing images at 5-second intervals, if it is determined that the user is almost motionless for more than 20 seconds and there is no facial movement for more than a specified period (for example, 4 frames), it is determined that the user does not exist.

[0138] Therefore, if electronic device 1 determines that there is no facial movement detected in low-resolution mode, it ends low-resolution mode. That is, if electronic device 1 determines that there is no facial movement detected in low-resolution mode, it detects facial movement in high resolution, thereby confirming facial movement in more detail. The resolution of high-resolution mode is higher than that of low-resolution mode, so even if facial movement cannot be detected in low-resolution mode, facial movement may be detected in high-resolution mode.

[0139] The basic structure of the electronic device 1 of this embodiment is similar to Figures 5-7 The structure of the first embodiment shown is the same, and its description is omitted. This embodiment differs from the first embodiment in that the face detection unit 211 detects the face area in either low-resolution mode or high-resolution mode. Here, the functions and processes that differ from the first embodiment are described.

[0140] The face detection unit 211 detects a facial area from a captured image in either low-resolution mode or high-resolution mode. Initially, the face detection unit 211 is set to low-resolution mode, processes image data from a low-resolution captured image, and detects a facial area. The activity determination unit 212 determines whether the face is moving based on the facial area detected by the face detection unit 211 in each captured image in low-resolution mode.

[0141] Furthermore, if the motion determination unit 212 determines that a face detected in low-resolution mode is not moving, the face detection unit 211 switches from low-resolution mode to high-resolution mode. Specifically, if the motion determination unit 212 determines that a face detected in low-resolution mode is not moving, the face detection unit 211 processes image data captured at a higher resolution than that captured in low-resolution mode and detects a facial area. The motion determination unit 212 determines whether the face is moving based on the facial area detected in each captured image in high-resolution mode. Furthermore, if the face detection unit 211 fails to detect a facial area in high-resolution mode, it returns from high-resolution mode to low-resolution mode and performs facial area detection in low-resolution mode.

[0142] Next, refer to Figure 14 , the operation of the HPD process (human detection process) performed by the human detection unit 210 will be described.

[0143] Figure 14 This is a flowchart showing an example of HPD processing in this embodiment. In this figure, the processing of steps S301 to S311 is the same as Figure 9 The processes of steps S101 to S111 shown are the same, and their description is omitted.

[0144] First, the person detection unit 210 is set to low-resolution mode (step S300) and proceeds to step S301. In steps S301 to S309, the person detection unit 210 detects a face area from the captured image in low-resolution mode and determines whether the detected face is moving.

[0145] If it is determined in step S309 that the face is not moving (No), the person detection unit 210 determines whether the current detection mode is the high-resolution mode (step S313). In this case, the person detection unit 210 determines No in step S313 and proceeds to step S315.

[0146] In step S315, the person detection unit 210 switches to high-resolution mode and returns to step S301. This switches from low-resolution mode to high-resolution mode. In steps S301 to S309, the person detection unit 210 detects facial areas from the captured image in high-resolution mode and determines whether the detected face is moving. If the person detection unit 210 determines that the face is not moving in low-resolution mode but is moving in high-resolution mode (step S309: Yes), it determines that a user is present (step S311).

[0147] On the other hand, if the face is determined to be motionless even in the high-resolution mode (step S309: No), the person detection unit 210 determines whether the current detection mode is the high-resolution mode (step S313). In this case, the person detection unit 210 determines Yes in step S313 and proceeds to step S317.

[0148] In step S317, the person detection unit 210 determines that the inactive face is not a person's face and determines that the user is not present. In other words, the person detection unit 210 determines that there is no user in front of the electronic device 1. Then, the process proceeds to step S319.

[0149] In step S319, the person detection unit 210 registers the facial region determined to be inactive as a determination exclusion region. Subsequently, the registered determination exclusion region is excluded from the determination targets. Subsequently, in either low-resolution mode or high-resolution mode, if the person detection unit 210 detects a facial region in the determination exclusion region, it is determined that the facial region is not the user's face and no further determination of facial activity (i.e., whether the facial region is the user's face) is made. The process then returns to step S300, where the person detection unit 210 is set to low-resolution mode. This switches from high-resolution mode to low-resolution mode.

[0150] In addition, if the person detection unit 210 cannot detect a facial area from the captured image in high-resolution mode (step S303: No), it determines that the user does not exist (step S305). Then, the process proceeds to step S321, where the person detection unit 210 determines whether the current detection mode is high-resolution mode. If it is determined in step S321 that it is in high-resolution mode (Yes), the process returns to step S300, and the person detection unit 210 sets it to low-resolution mode. Thus, the process switches from high-resolution mode to low-resolution mode. On the other hand, if it is determined in step S321 that it is in low-resolution mode (No), the person detection unit 210 returns to the process of step S301, and in steps S301 to S309, the facial area is detected from the captured image in low-resolution mode, and it is determined whether the detected face is moving.

[0151] [Summary of the Second Embodiment]

[0152] As described above, in the electronic device 1 of this embodiment, the image data of the image captured by the imaging unit 120 includes low-resolution (first-resolution) image data. Furthermore, the person detection unit 210 executes a low-resolution mode for processing the low-resolution image data to detect a facial region, and determines whether the face is moving based on the facial region detected in the low-resolution mode.

[0153] Thus, the electronic device 1 reduces power consumption and the impact on other processes, and can accurately detect the presence of a user by determining only active faces among the detected faces as the user.

[0154] Furthermore, in the electronic device 1 of this embodiment, the image data of the image captured by the imaging unit 120 includes high-resolution image data (second resolution) having a higher resolution than the low-resolution image data (first resolution). The person detection unit 210 executes a high-resolution mode for processing the high-resolution image data to detect a facial region, and determines whether the face is moving based on the facial region detected in the high-resolution mode.

[0155] Thus, when electronic device 1 does not detect facial movement in low-resolution mode, it determines whether there is facial movement in high-resolution mode, thereby reducing power consumption, the impact on other processing, etc., and also preventing false detection.

[0156] Furthermore, when the electronic device 1 fails to detect a facial region in the high-resolution mode, it performs facial region detection in the low-resolution mode.

[0157] Thus, electronic device 1 returns to low-resolution mode when detecting that a user is not present in high-resolution mode, thereby suppressing wasteful power consumption and influence on other processes.

[0158] The first and second embodiments of the present invention have been described in detail with reference to the accompanying drawings. However, the specific structure is not limited to the above-mentioned embodiments and also includes design changes within the scope of the present invention. For example, the various structures described in the above-mentioned embodiments can be arbitrarily combined.

[0159] Furthermore, in the above embodiment, an example configuration in which the camera unit 120 is built into the electronic device 1 is described, but the present invention is not limited thereto. For example, the camera unit 120 may not be built into the electronic device 1, but may be configured to be attachable to the electronic device 1 (e.g., any of the side surfaces 10a, 10b, 10c, etc.) as an external accessory to the electronic device 1, and to be connected to the electronic device 1 wirelessly or via a wired connection.

[0160] Furthermore, in the above embodiment, electronic device 1 detects the presence of a user by detecting a facial region in a captured image. However, this is not limited to faces; the presence of a user may also be detected by detecting a region in which at least a portion of a body is captured. Furthermore, electronic device 1 may also utilize a distance sensor (e.g., a proximity sensor) that detects the distance to an object. For example, the distance sensor is disposed on the inner surface of first housing 10 and detects objects (e.g., people) within a detection range facing (forward of) the inner surface of first housing 10. As an example, the distance sensor may be an infrared distance sensor comprising a light-emitting portion that emits infrared light and a light-receiving portion that receives the reflected light when the emitted infrared light reflects off the surface of an object. Furthermore, the distance sensor may be a sensor that uses infrared light emitted by a light-emitting diode (LED) or an infrared laser that emits light with a narrower wavelength than the infrared light emitted by a light-emitting diode (LED). Furthermore, the distance sensor is not limited to an infrared distance sensor; as long as it detects the distance to an object, it may also be an ultrasonic sensor, a sensor using a UWB (Ultra Wide Band) radar, or other sensors. Furthermore, the distance sensor may not be built into the electronic device 1, but may be configured to be attachable to the electronic device 1 (e.g., any of the side surfaces 10a, 10b, 10c, etc.) as an external accessory to the electronic device 1, and be connected to the electronic device 1 via wireless or wired communication. Furthermore, the imaging unit 120 and the distance sensor may be integrally formed.

[0161] Furthermore, in the above embodiment, an example is shown in which the person detection unit 210 is provided separately from the EC 200, but part or all of the person detection unit 210 may be provided in the EC 200. Furthermore, in the above embodiment, an example is shown in which the EC 200 includes the motion control unit 220, but part or all of the motion control unit 220 may be provided in a processing unit other than the EC 200 (e.g., the system processing unit 300).

[0162] In the above embodiment, the EC 200 operating independently from the system processing unit 300 may be any processing unit such as a sensor hub or a chipset, and a processing unit other than the EC 200 may execute the above processing instead of the EC 200.

[0163] In addition, the above-mentioned standby state may also include a sleep state, a power-off state, etc. The sleep state is equivalent to, for example, the S4 state specified by ACPI. The power-off state is equivalent to, for example, the S5 state specified by ACPI (the shutdown state). In addition, the standby state may also include at least a state in which the display of the display unit is turned off (the screen is turned off), or a state in which the screen is locked. Screen locking refers to a state in which a predetermined image (for example, an image for screen locking) is displayed on the display unit so that the content being processed cannot be visually confirmed, and the device cannot be used until the lock is released (for example, user authentication).

[0164] In addition, the electronic device 1 mentioned above has a computer system inside. Moreover, a program for realizing the functions of each structure possessed by the electronic device 1 mentioned above can also be recorded on a computer-readable recording medium, and the processing in each structure possessed by the electronic device 1 mentioned above can be performed by making the computer system read and execute the program recorded on the recording medium. Here, "making the computer system read and execute the program recorded on the recording medium" is included in the computer system installation program. The "computer system" mentioned here includes hardware such as OS and peripheral devices. In addition, the "computer system" can also include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. In addition, the "computer-readable recording medium" refers to a storage medium such as a portable medium such as a floppy disk, a magneto-optical disk, a ROM, and a CD-ROM, and a hard disk built into the computer system. In this way, the recording medium storing the program can also be a non-temporary recording medium such as a CD-ROM.

[0165] In addition, the recording medium also includes an internal or external recording medium that can be accessed from a distribution server in order to distribute the program. In addition, the program can also be divided into multiple parts, and the structure formed by the combination of the various structures possessed by the electronic device 1 after downloading at different times, and the distribution server for each of the divided programs is different. Moreover, the so-called "computer-readable recording medium" also includes a structure that keeps the program for a certain period of time, such as a server in the case of sending a program via a network, and a volatile memory (RAM) inside a computer system that becomes a client. In addition, the above-mentioned program can also be a structure for realizing a part of the above-mentioned functions. Furthermore, it can also be a so-called differential file (differential program) that can realize the above-mentioned functions by combining them with the programs already recorded in the computer system.

[0166] Furthermore, some or all of the functions of the electronic device 1 in the above-described embodiment may be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each function may be processed independently, or some or all of them may be integrated. Furthermore, the method of integrated circuitization is not limited to LSI; implementation may also be achieved using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that replaces LSI emerges with advances in semiconductor technology, integrated circuits based on such technology may also be used.

[0167] In addition, the electronic device 1 of the above embodiment is not limited to PCs, tablet terminal devices, smartphones, etc., and can also be applied to household appliances and commercial appliances. As household appliances, it can be applied to televisions, refrigerators with display units, microwave ovens, etc. For example, according to the approach or departure of a person, the screen of a television can be turned on / off, or the screen of the display unit of a refrigerator, microwave oven, etc. can be turned on / off. In addition, as commercial appliances, it can be applied to vending machines, multimedia terminals, etc. For example, according to the approach or departure of a person, the action state can be controlled, such as turning the lighting of a vending machine on / off, or turning the screen of the display unit of a multimedia terminal on / off.

Claims

1. An electronic device, wherein: The above-mentioned electronic equipment has: a memory for temporarily storing image data of an image captured by the imaging device; and a processor for processing the image data stored in the memory; The above processors have: a face detection unit that processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects a facial area in which a face is captured from the plurality of images; a motion determination unit that calculates a change in the position of the facial region detected from the plurality of images and determines whether the photographed face has moved within the predetermined time interval based on the change; and The person determination unit determines that the user is present when the motion determination unit determines that the face has motion. The motion determination unit registers a face region determined by the motion determination unit as being inactive among the faces detected from each of the images, and subsequently excludes the registered region from targets of determination.

2. The electronic device according to claim 1, wherein It also has a sensor for detecting the activity of the electronic device. The movement determination unit returns the registered area to the target of determination when the movement of the electronic device is detected using the sensor.

3. The electronic device according to claim 1, wherein When the motion determination unit determines that the captured face is not moving, the person determination unit determines that the face is not the user's face and determines that the user does not exist.

4. An electronic device, wherein: The above-mentioned electronic equipment has: a memory for temporarily storing image data of an image captured by the imaging device; and a processor for processing the image data stored in the memory; The above processors have: a face detection unit that processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects a facial area in which a face is captured from the plurality of images; a motion determination unit that calculates a change in the position of the facial region detected from the plurality of images and determines whether the photographed face has moved within the predetermined time interval based on the change; and a person determination unit that determines that a user is present when the motion determination unit determines that the face has motion, wherein the image data includes image data of a first resolution and image data of a second resolution higher than the first resolution; The face detection unit performs a low-resolution mode of processing the image data of the first resolution to detect the face area. The movement determination unit determines whether the face has movement based on the face area detected by the face detection unit in the low-resolution mode, and if it is determined that the face detected by processing the image data at the first resolution has not moved, executes the high-resolution mode to detect the face area by processing the image data at the second resolution. The movement determination unit determines whether the face has movement based on the face area detected by the face detection unit in the high-resolution mode.

5. The electronic device according to claim 4, wherein The face detection unit detects the face area in the low-resolution mode if the face area is not detected in the high-resolution mode.

6. The electronic device according to any one of claims 1 to 5, wherein: Also features: a processing unit that performs system-based system processing; and The activity control unit shifts the activity state of the system to an activity state in which at least a portion of the system processing is restricted when the state transitions from the state determined by the person determination unit as the user being present to the state determined as the user not being present.

7. A control method is a control method of an electronic device, wherein: The electronic device includes a memory for temporarily storing image data of an image captured by a camera, and a processor for processing the image data stored in the memory. The above control method comprises the following steps: In the above processors, The face detection unit processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects a facial area in which a face is captured from the plurality of images; The activity determination unit calculates a change in the position of the facial region detected from the plurality of images, and determines whether the captured face has moved within the predetermined time interval based on the change. The person determination unit determines that the user is present when the motion determination unit determines that the face has motion. In the step of determining by the motion determination unit, regions of the face detected from each of the images, which are determined by the motion determination unit to be inactive, are registered, and the registered regions are subsequently excluded from determination targets.

8. A control method is a control method of an electronic device, wherein: The electronic device includes a memory for temporarily storing image data of an image captured by a camera, and a processor for processing the image data stored in the memory. The above control method comprises the following steps: In the above processors, The face detection unit processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects a facial area in which a face is captured from the plurality of images; The activity determination unit calculates a change in the position of the facial region detected from the plurality of images, and determines whether the captured face has moved within the predetermined time interval based on the change. The person determination unit determines that the user is present when the motion determination unit determines that the face has motion. The image data includes image data of a first resolution and image data of a second resolution higher than the first resolution. In the step of detecting by the face detection unit, a low-resolution mode is executed to process the image data of the first resolution to detect the face area. In the step of determining by the activity determination unit, whether the face is moving is determined based on the face area detected by the face detection unit in the low-resolution mode. When the activity determination unit determines that the face detected by processing the image data of the first resolution has no movement, in the step of detection by the face detection unit, a high-resolution mode of processing the image data of the second resolution to detect the face area is executed, and in the step of determination by the activity determination unit, whether the face has movement is determined based on the face area detected by the face detection unit in the high-resolution mode.

Images