Electronic devices and control methods

Abstract

This invention relates to an electronic device and a control method. It enables high-precision detection of a person using the electronic device. The electronic device includes a memory that temporarily stores image data of images captured by an imaging device, and a processor that processes the image data stored in the memory. The processor processes image data of multiple images captured by the imaging device at predetermined time intervals and stored in the memory. Based on image data of a first resolution and image data of a second resolution, it detects facial regions of the face captured from the multiple images and determines whether the facial regions are detected consecutively in the multiple images. Furthermore, while processing is in progress to detect facial regions based on image data of the first resolution, if it is determined that the state changes between a state of consecutive detection of facial regions and a state of discontinuous detection of facial regions, the processor detects facial regions from the multiple images based on image data of the second resolution.

Description

Technical Field

[0001] This invention relates to electronic devices and control methods. Background Technology

[0002] There exists an electronic device that transitions to an active state when a person approaches, and to a standby state where all functions are disabled when the person moves away. For example, in Patent Document 1, an infrared sensor is used to detect whether a person is approaching or moving away.

[0003] In recent years, advancements in computer vision and other technologies have led to increased accuracy in facial detection based on images. Consequently, facial detection has begun to replace infrared sensors for detecting people. While infrared sensors reflect infrared light back from both people and other objects, facial detection prevents the misidentification of simple objects as people.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2016-148895

[0005] However, due to factors such as facial orientation, face detection can sometimes be difficult. Even when a person is present, the face may be detected intermittently, making the detection unstable. For example, when a user is using a second display connected to an electronic device, the user's profile may be detected from the electronic device side. In such cases, although a user is present using the electronic device, face detection may become unstable, potentially leading to false detections that the user is not present. Summary of the Invention

[0006] The present invention was made in view of the above circumstances, and one of its objectives is to provide an electronic device and control method for high-precision detection of the person in use.

[0007] The present invention was made to solve the aforementioned problems. The electronic device according to a first aspect of the present invention includes: a memory that temporarily stores image data of images captured by an imaging device; and a processor that processes the image data stored in the memory. The processor includes: a face detection unit that processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects facial regions of a captured face from the plurality of images based on image data of a first resolution and image data of a second resolution; and a detection state determination unit that determines whether the facial regions are continuously detected in the plurality of images. When processing is in progress to detect the facial regions based on the image data of the first resolution, if the detection state determination unit determines that the state changes between a state of continuous detection of the facial regions and a state of discontinuous detection of the facial regions, the face detection unit detects the facial regions from the plurality of images based on image data of the second resolution.

[0008] In the aforementioned electronic device, it may also be configured such that, based on a predetermined proportion of the plurality of images captured during a predetermined period at the predetermined time intervals, the detection state determination unit determines that the facial region has not been detected continuously.

[0009] In the aforementioned electronic device, the first resolution may be lower than the second resolution. When processing is being performed in a low-resolution mode for detecting the facial region based on image data of the first resolution, if the detection state determination unit determines that the state has changed from continuously detecting the facial region to not continuously detecting the facial region, the facial detection unit performs processing in a high-resolution mode. In the high-resolution mode, the facial region is detected based on image data of the second resolution in a specific region, and the specific region corresponds to the position of the facial region detected in the low-resolution mode processing.

[0010] In the aforementioned electronic device, the face detection unit may also be configured to perform face region detection in a low-resolution mode if it fails to detect the face region from the specific region in the high-resolution mode.

[0011] In the aforementioned electronic device, the detection range of the face detection unit when detecting the face region from the plurality of images may be set to a range smaller than the range of the image region of the captured image, and a detection range setting unit may also be provided, wherein the detection range setting unit moves the detection range according to the position of the face region detected by the face detection unit.

[0012] In the aforementioned electronic device, it may also be configured such that, in the initial state, the detection range setting unit is set such that the center position of the detection range corresponds to the center position of the image area of ​​the captured image.

[0013] Alternatively, the electronic device may further include a motion determination unit that determines whether the motion of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit from the plurality of images. If the motion determination unit determines that the motion of the facial region is above the predetermined threshold, the face detection unit invalidates the detection of the facial region. If the motion determination unit determines that the motion of the facial region is below the predetermined threshold, the face detection unit makes the detection of the facial region valid.

[0014] Furthermore, the electronic device according to the second aspect of the present invention includes: a memory that temporarily stores image data of images captured by an imaging device; and a processor that processes the image data stored in the memory. The processor includes: a face detection unit that processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects facial regions of a face captured from the plurality of images; and a movement amount determination unit that determines whether the movement amount of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit from the plurality of images. If the movement amount determination unit determines that the movement amount of the facial region is above the predetermined threshold, the face detection unit invalidates the detection of the facial region; if the movement amount determination unit determines that the movement amount of the facial region is less than the predetermined threshold, the face detection unit enables the detection of the facial region.

[0015] In the aforementioned electronic device, it may also be configured to include a sensor for detecting the activity of the electronic device, wherein the movement determination unit considers the activity of the electronic device detected by the sensor to determine the movement of the facial region detected by the facial detection unit.

[0016] Alternatively, the electronic device may further include: a processing unit that performs system-based processing; a person determination unit that determines that a user exists if the face region detected by the face detection unit is valid, and determines that a user does not exist if the face region is not detected by the face detection unit or if the face region detected by the face detection unit is invalid; and an action control unit that, when the determination result of the person determination unit shifts from a state of no user to a state of user existence, causes the action state of the system to shift from a first action state in which at least a portion of the system processing is restricted to a second action state in which the action of the system processing is activated compared to the first action state.

[0017] Furthermore, the control method of an electronic device comprising a memory for temporarily storing image data of images captured by an imaging device and a processor for processing the image data stored in the memory, according to the third aspect of the present invention, includes: a face detection unit processing image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and a step of detecting a facial region of a face captured from the plurality of images based on image data of a first resolution and image data of a second resolution; and a detection state determination unit determining whether the facial region is continuously detected in the plurality of images, wherein, in the step of the face detection unit detecting the facial region, if the face detection unit is performing processing based on the image data of the first resolution and determines that the state changes between a state of continuously detecting the facial region and a state of not continuously detecting the facial region, the face region is detected from the plurality of images based on image data of the second resolution.

[0018] Furthermore, the control method of an electronic device comprising a memory for temporarily storing image data of images captured by an imaging device and a processor for processing the image data stored in the memory, according to the fourth aspect of the present invention, includes: a step in which a face detection unit processes image data of a plurality of images captured by the imaging device at predetermined time intervals and stored in the memory, and detects a facial region of a face captured from the plurality of images; and a step in which a motion determination unit determines whether the motion amount of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit from the plurality of images, wherein in the step in which the face detection unit detects the facial region, the detection of the facial region is invalidated if the motion determination unit determines that the motion amount of the facial region is above the predetermined threshold, and the detection of the facial region is valid if the motion determination unit determines that the motion amount of the facial region is less than the predetermined threshold.

[0019] According to the above-described method of the present invention, it is possible to detect people using electronic devices with high precision. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating the outline of HPD processing of the electronic device according to the first embodiment.

[0021] Figure 2 This is a diagram illustrating an example of unstable facial detection.

[0022] Figure 3 This is a diagram showing an example of the detection area of ​​the face according to the first embodiment.

[0023] Figure 4 This is a perspective view showing a structural example of the appearance of the electronic device according to the first embodiment.

[0024] Figure 5 This is a simplified block diagram illustrating a structural example of the electronic device according to the first embodiment.

[0025] Figure 6 This is a block diagram illustrating an example of the structure of the person detection unit according to the first embodiment.

[0026] Figure 7 This is a flowchart illustrating a first example of the face detection processing involved in the first embodiment.

[0027] Figure 8 This is a flowchart illustrating a second example of the face detection processing involved in the first embodiment.

[0028] Figure 9 This is a flowchart illustrating an example of the startup process involved in the first embodiment.

[0029] Figure 10 This is a flowchart illustrating an example of the standby state transition process according to the first embodiment.

[0030] Figure 11 This is a flowchart illustrating a first example of the face detection processing involved in the second embodiment.

[0031] Figure 12 This is a flowchart illustrating a second example of the face detection processing involved in the second embodiment.

[0032] Figure 13 This is a diagram illustrating an example of the relationship between the detection range (DR) and the user's position.

[0033] Figure 14 This is a diagram illustrating the movement of the detection range involved in the third embodiment.

[0034] Figure 15 This is a diagram illustrating an example of the structure of the person detection unit according to the third embodiment.

[0035] Figure 16 This is a flowchart illustrating an example of the detection range control process involved in the third embodiment.

[0036] Figure 17 This is a diagram representing an example of facial detection of a person other than the user.

[0037] Figure 18 This is a diagram illustrating an example of the structure of the person detection unit according to the fourth embodiment.

[0038] Figure 19 This is a flowchart illustrating an example of the person detection processing involved in the fourth embodiment.

[0039] Explanation of reference numerals in the attached figures

[0040] 1…Electronic device; 10…First frame; 20…Second frame; 15…Hinge mechanism; 110…Display unit; 120…Camera unit; 130…Acceleration sensor; 140…Power button; 150…Input device; 151…Keyboard; 153…Touchpad; 200…EC; 210, 210A, 210B…Person detection unit; 211, 211A, 211B…Face detection unit; 21A…Detection range setting unit; 21B…Motion determination unit; 212…Detection status determination unit; 213, 213B…Person determination unit; 220…Motion control unit; 300…System processing unit; 302…CPU; 304…GPU; 306…Memory controller; 308…I / O controller; 310…System memory; 350…Communication unit; 360…Storage unit; 400…Power supply unit. Detailed Implementation

[0041] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0042] <First Implementation>

[0043] [summary]

[0044] First, an overview of the electronic device 1 according to the first embodiment will be described. The electronic device 1 according to this embodiment is, for example, a notebook PC (Personal Computer). In addition, the electronic device 1 may also be any type of electronic device such as a desktop PC, a tablet terminal device, or a smartphone.

[0045] Electronic device 1 is capable of transitioning between at least a normal operating state (power-on state) and a standby state as the system's operating states. A normal operating state refers to an operating state that allows for processing without particular restrictions, for example, equivalent to the S0 state defined by ACPI (Advanced Configuration and Power Interface). A standby state is a state that restricts at least a portion of the system's processing. For example, a standby state can be a standby state, a sleep state, or even the modern standby state in Windows, equivalent to the S3 state (sleep state) defined by ACPI. For example, a standby state is an operating state that consumes less power compared to a normal operating state.

[0046] Hereinafter, the process of transitioning the system's operating state from standby to normal operating state is sometimes referred to as startup. In standby state, the activity level is generally lower compared to normal operating state; therefore, startup of electronic device 1 is the action of activating the system in electronic device 1.

[0047] Figure 1 This diagram illustrates the general outline of HPD processing in the electronic device 1 according to this embodiment. The electronic device 1 detects a person (i.e., a user) present in its vicinity. The process of detecting the presence of this person is called HPD (Human Presence Detection) processing. The electronic device 1 detects the presence or absence of a person through HPD processing and controls the system's operational state based on the detection result. For example, as... Figure 1 As shown in (A), when electronic device 1 detects a change from a state where there is no person in front of it (Absence) to a state where there is a person (Presence), i.e., when a person approaches electronic device 1 (Approach), it determines that a user is approaching, automatically activates the system, and transitions it to its normal operating state. Additionally, as... Figure 1 As shown in (B), when a person is present in front of electronic device 1, it is determined that a user is present, and the normal operation state continues. Furthermore, as... Figure 1 As shown in (C), when electronic device 1 detects a change from a state where a person is present in front of electronic device 1 to a state where no person is present, i.e., the person leaves electronic device 1, it determines that the user has left and causes the system to switch to standby mode.

[0048] For example, electronic device 1 determines whether a user is present in front of (front of) it by detecting facial regions of the captured face in an image obtained from the front (front side) of the camera. If a facial region is detected in the captured image, electronic device 1 determines that a user is present. Conversely, if no facial region is detected in the captured image, electronic device 1 determines that no user is present. However, when detecting facial regions in the captured image, it can be difficult to perform the detection due to factors such as the orientation of the face. Even if a person is present, the facial region may be detected intermittently, making the facial detection unstable. (See reference...) Figure 2 Examples of unstable facial detection are illustrated.

[0049] Figure 2 This is a diagram illustrating an example of unstable face detection. The example shown is an image captured by electronic device 1 at predetermined time intervals, in the order of time series t(1) to t(16). Here, when the user is using a second display connected to electronic device 1, the image captured by electronic device 1 reflects the user's profile. The profile has less information about facial features compared to the frontal view, making it difficult to detect. Therefore, the images captured at times t(1) to t(16) include images where the facial region is detected and images where the facial region is not detected. In this diagram, images captured at times t(1) to t(2), t(6), t(9), and t(12) to t(16), enclosed by solid lines, show that the facial region is detected, while images captured at times t(3) to t(5), t(7) to t(8), and t(10) to t(11), enclosed by dashed lines, show that the facial region is not detected. In this case, although a user is present using electronic device 1, face detection becomes unstable and may be falsely detected as the user not existing.

[0050] Therefore, in this embodiment, if the electronic device 1 detects face regions continuously from the state of continuously detecting face regions in the captured image to the state of not detecting face regions (or if face detection becomes unstable), face detection may fail. Therefore, in subsequent face detection, the detection area is narrowed down to the area where the face was previously detected. Specifically, the electronic device 1 estimates the position of the captured face based on the face region when the face was previously detected, and magnifies the image region based on the estimated position to detect the face region.

[0051] Figure 3This diagram illustrates an example of the face detection region according to this embodiment. Reference numeral GR1 denotes the entire image region of the captured image. Reference numeral DR denotes the detection range when detecting a face from the captured image. Furthermore, the squares within the detection range DR schematically represent the resolution during detection, and do not determine the actual number of squares (resolution) during detection. Additionally, reference numeral FD1 denotes the facial region detected from the image region corresponding to the detection range DR within the captured image. Here, when the electronic device 1 continuously detects facial regions from the captured image to the point where no facial regions are detected, it estimates the position of the captured face based on previously detected facial regions and magnifies the image region GR2 based on the estimated position to detect the facial region. The electronic device 1 applies the detection range DR to the magnified image region GR2 to detect the facial region, thus detecting the facial region at a higher resolution. Therefore, the electronic device 1 can improve the detection accuracy of faces that are difficult to detect, such as side profiles. Furthermore, in the following description, applying the detection range DR to the entire image region GR1 of the captured image to detect the facial region will only be referred to as "detecting the facial region from the captured image." Additionally, the application of detection range DR to the image region GR2 after magnification of the captured image to detect the facial region is simply referred to as "Detecting facial regions from image region GR2".

[0052] For example, image region GR2 is set to be a region wider than the face region FD1, which includes the face region FD1 detected from the captured image. As an example, image region GR2 can also be set to 120% of the face region FD1. Furthermore, image region GR2 can be set based on the face region FD1 of only the last (immediately preceding) frame in which the face region was detected, or it can be set based on the face region FD1 of multiple frames in which the face region was detected. Multiple frames refer to, for example, multiple frames from the last (immediately preceding) frame in which the face region was detected to a previously defined frame (e.g., the first 3 frames). When using face region FD1 of multiple frames, image region GR2 is set in a way that includes the region that incorporates multiple face region FD1s.

[0053] Next, the structure of the electronic device 1 according to this embodiment will be described in detail.

[0054] [External structure of electronic devices]

[0055] Figure 4 This is a perspective view showing a structural example of the appearance of the electronic device 1 according to this embodiment.

[0056] Electronic device 1 includes a first frame 10, a second frame 20, and a hinge mechanism 15. The first frame 10 and the second frame 20 are connected by the hinge mechanism 15. The first frame 10 is capable of rotating relative to the second frame 20 about a rotation axis formed by the hinge mechanism 15. The opening angle formed by the rotation of the first frame 10 and the second frame 20 is illustrated as "θ".

[0057] The first frame 10 is also referred to as the A cover or the display frame. The second frame 20 is also referred to as the C cover or the system frame. In the following description, the sides of the first frame 10 and the second frame 20 that have the hinge mechanism 15 are referred to as sides 10c and 20c, respectively. The sides of the first frame 10 and the second frame 20 opposite to sides 10c and 20c are referred to as sides 10a and 20a, respectively. In the illustration, the direction from side 20a toward side 20c is referred to as "rear", and the direction from side 20c toward side 20a is referred to as "front". The right and left sides relative to the rear are referred to as "right" and "left", respectively. The left side of the first frame 10 and the second frame 20 are referred to as sides 10b and 20b, respectively, and the right side is referred to as sides 10d and 20d, respectively. In addition, the state in which the first frame 10 and the second frame 20 overlap and are completely closed (the state with an opening angle θ = 0°) is referred to as the "closed state". The faces of the first frame 10 and the second frame 20 facing each other in the closed state are called their "inner faces," and the faces opposite to their inner faces are called their "outer faces." Furthermore, the state in which the first frame 10 and the second frame 20 are opened relative to the closed state is called the "open state."

[0058] Figure 4 The electronic device 1 shown is an example of an open state. The open state is the state where the side 10a of the first frame 10 is separated from the side 20a of the second frame 20. In the open state, the inner surfaces of both the first frame 10 and the second frame 20 are visible. The open state is one of the states in which the user uses the electronic device 1, typically mostly used with an opening angle θ of approximately 100 to 130°. Furthermore, the range of the opening angle θ for the open state can be arbitrarily determined based on factors such as the range of angles that can be rotated via the hinge mechanism 15.

[0059] A display unit 110 is provided on the inner surface of the first frame 10. The display unit 110 is configured to include a liquid crystal display (LCD), an organic EL (electroluminescence) display, or the like. Furthermore, a camera unit 120 is provided in the area surrounding the display unit 110 on the inner surface of the first frame 10. For example, the camera unit 120 is disposed on the side 10a side of the area surrounding the display unit 110. Moreover, the position of the camera unit 120 is just one example; it can be placed in any other location as long as it faces the inner surface of the first frame 10 (front).

[0060] In the open state, the imaging unit 120 captures a predetermined shooting range in the direction facing (front) the inner surface of the first frame 10. The predetermined shooting range refers to the range of the viewing angle determined by the imaging element of the imaging unit 120 and the optical lens disposed in front of the imaging surface of the imaging element. For example, the imaging unit 120 can capture an image including a person present in front of (front of) the electronic device 1.

[0061] Additionally, a power button 140 is provided on the side 20b of the second housing 20. The power button 140 is an operating element used by the user to indicate system startup (transition from standby state to normal operation state) and transition from normal operation state to standby state. Furthermore, a keyboard 151 and a touchpad 153 are provided as input devices on the inner surface of the second housing 20. Moreover, as input devices, the keyboard 151 and touchpad 153 can be replaced, or a touch sensor can be included in addition to the keyboard 151 and touchpad 153; a mouse or an external keyboard can also be connected. In the case of a structure with a touch sensor, it can also be configured as a touch panel that receives operation in an area corresponding to the display surface of the display unit 110. Additionally, a microphone for inputting sound can also be included in the input device.

[0062] Furthermore, in the closed state where the first frame 10 and the second frame 20 are closed, the display unit 110 and the camera unit 120 disposed on the inner surface of the first frame 10, and the keyboard 151 and the touchpad 153 disposed on the inner surface of the second frame 20 are covered by the other frame surface, and thus cannot function.

[0063] [Structure of electronic devices]

[0064] Figure 5This is a simplified block diagram illustrating an example of the structure of the electronic device 1 according to this embodiment. The electronic device 1 is configured to include: a display unit 110, a camera unit 120, an accelerometer sensor 130, a power button 140, an input device 150, an EC (Embedded Controller) 200, a person detection unit 210, a system processing unit 300, a communication unit 350, a storage unit 360, and a power supply unit 400. The display unit 110 displays display data (images) generated based on system processing executed by the system processing unit 300 and the processing of application programs operating on the system processing.

[0065] The imaging unit 120 captures an image of an object within a predetermined viewing angle facing (front) the inner surface of the first frame 10, and outputs the captured image to the system processing unit 300 and the person detection unit 210. The imaging unit 120 can be an infrared camera or a conventional camera. An infrared camera is one that uses an infrared sensor as its imaging element. A conventional camera is one that uses a visible light sensor (e.g., an RGB camera) as its imaging element. Furthermore, in the case of a conventional camera, the image captured for face detection can also be a reduced-color image (e.g., a monochrome image).

[0066] Accelerometer 130 detects the activity of electronic device 1 and outputs a detection signal indicating the detection result to EC200. For example, when electronic device 1 is moving, or when electronic device 1 is being moved unstablely by hand, accelerometer 130 outputs a detection signal based on its activity. Alternatively, gyroscope sensor, tilt sensor, geomagnetic sensor, etc., may be added instead of accelerometer 130 or based on accelerometer 130.

[0067] The power button 140 outputs an operation signal to the EC200 based on the user's operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to receiving operations on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC200.

[0068] The power supply unit 400 supplies power to each component of the electronic device 1 via a power supply system for supplying power to each component, based on the operating status of each component. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of the DC power supplied from the AC (Alternate Current) / DC adapter or battery pack to the voltage required by each component. The power, after voltage conversion by the DC / DC converter, is supplied to each component via the respective power supply system. For example, the power supply unit 400 supplies power to each component via the respective power supply system based on control signals corresponding to the operating status of each component input from the EC200.

[0069] The EC200 is a microcomputer configured to include a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and I / O (Input / Output) logic circuits. The EC200's CPU reads the control program (firmware) pre-stored in its ROM and executes it to perform its functions. The EC200 operates independently of the system processing unit 300, controlling the operation of the system processing unit 300 and managing its operating state. Furthermore, the EC200 is connected to the accelerometer sensor 130, power button 140, input device 150, human detection unit 210, and power supply unit 400.

[0070] For example, EC200 communicates with power supply unit 400 to obtain information about the battery status (remaining capacity, etc.) from power supply unit 400, and outputs control signals, such as control signals for controlling the power supply corresponding to the operating status of each part of electronic device 1, to power supply unit 400. Additionally, EC200 obtains operation signals from power button 140 and input device 150, and outputs operation signals related to the processing of system processing unit 300 from the obtained operation signals to system processing unit 300. Furthermore, EC200 detects the operation of electronic device 1 based on detection signals from acceleration sensor 130. For example, EC200 detects whether electronic device 1 is stationary or moving based on detection signals from acceleration sensor 130. EC200 also includes a motion control unit 220, which controls the operation of the system based on the detection results of person detection unit 210.

[0071] The person detection unit 210 is a processor that processes image data of images captured by the camera unit 120. For example, the person detection unit 210 acquires images captured by the camera unit 120 via the system processing unit 300. Alternatively, the person detection unit 210 may directly acquire images captured by the camera unit 120. The person detection unit 210 detects the presence of a user by detecting facial regions from the captured images and performs HPD processing based on the detection results.

[0072] The person detection unit 210 detects the presence of a user in front of the electronic device 1 by detecting facial regions from images captured by the imaging unit 120. For example, when a user approaches the electronic device 1, the detection state changes from "no user in front of electronic device 1" to "user present". Furthermore, the person detection unit 210 continuously detects the presence of a user in front of the electronic device 1 while the user is using the electronic device 1. Conversely, when the user leaves the electronic device 1, the detection state changes from "user present in front of electronic device 1" to "user absent". Thus, by detecting whether a user is in front of the electronic device 1, the person detection unit 210 can detect when a user approaches, when a user is present, when a user leaves, and when a user is absent. The structure of this person detection unit 210 will be described in detail later.

[0073] The motion control unit 220 controls the system's operating state based on HPD processing. For example, in standby mode, if the person detection unit 210 detects a change from a state where no user is in front of the electronic device 1 to a state where a user is present (i.e., a user is approaching the electronic device 1), the control unit 220 starts the system from standby mode. Specifically, when the person detection unit 210 detects a user approaching the electronic device 1, the motion control unit 220 instructs the system processing unit 300 to start the system. More specifically, when starting the system, the motion control unit 220 outputs a control signal to the power supply unit 400 to supply the power required for the operation of each part of the electronic device 1. Then, the motion control unit 220 outputs a start signal to the system processing unit 300 to instruct the system to start. If the system processing unit 300 receives the start signal, it starts the system and transitions it from standby mode to normal operating mode.

[0074] Furthermore, when the person detection unit 210 continuously detects the presence of a user in front of the electronic device 1, the motion control unit 220, through the system processing unit 300, prevents the system from transitioning to a standby state, thus maintaining normal operation. Moreover, even when the person detection unit 210 continuously detects the presence of a user, the motion control unit 220 can transition from the normal operation state to the standby state based on predetermined conditions. These predetermined conditions may include, for example, a pre-set period of inactivity or an operation to transition to the standby state.

[0075] Furthermore, during normal operation, if the person detection unit 210 detects a change from a state where a user is present in front of the electronic device 1 to a state where the user is not present (i.e., the user leaves the electronic device 1), the motion control unit 220 instructs the system processing unit 300 to switch the system from the normal operation state to the standby state. More specifically, the motion control unit 220 outputs a standby signal to the system processing unit 300 to instruct the system to switch the system from the normal operation state to the standby state. If the system processing unit 300 receives the standby signal, it switches the system from the normal operation state to the standby state. Then, the motion control unit 220 outputs a control signal to the power supply unit 400 to stop the supply of power that is not needed in the standby state.

[0076] The system processing unit 300 is configured to include: a CPU (Central Processing Unit) 302, a GPU (Graphics Processing Unit) 304, a memory controller 306, an I / O (Input-Output) controller 308, and system memory 310. It is capable of executing various applications on the OS through OS-based system processing. Sometimes, the CPU 302 and GPU 304 are collectively referred to as the processor.

[0077] CPU 302 executes OS-based processing and processing of applications that operate on the OS. Furthermore, CPU 302 controls the system's operating state based on the control of the system's operating state by EC 200 (Motion Control Unit 220). For example, if the operating state is standby and a start signal is input from EC 200, CPU 302 executes startup processing to transition from standby to normal operating state. After completing the startup processing, CPU 302 begins executing OS-based system processing. For example, if the operating state is standby and a start signal is input from EC 200, CPU 302 resumes execution of applications that were previously suspended.

[0078] During the startup process, CPU 302 executes a login process to determine whether OS access is permitted. If CPU 302 initiates OS-based startup, it performs the login process before granting OS access, and temporarily suspends the transition to normal operation until login is granted. During the login process, user authentication is performed to determine whether the user of electronic device 1 is a pre-registered legitimate user. Authentication methods include password authentication, facial recognition, and fingerprint authentication. If authentication is successful, CPU 302 allows login and resumes the temporarily suspended system processing. Conversely, if authentication fails, login is not permitted, and the suspended system processing remains in place.

[0079] GPU 304 is connected to display unit 110. GPU 304 performs image processing to generate display data based on the control of CPU 302. GPU 304 outputs the generated display data to display unit 110. Furthermore, CPU 302 and GPU 304 can be integrated into a single core, or the load can be shared among the individual CPU 302 and GPU 304 cores. The number of processors is not limited to one; multiple processors are also possible.

[0080] The memory controller 306 controls the CPU 302 and GPU 304 to read and write data from the system memory 310, storage unit 360, etc.

[0081] I / O controller 308 controls the input and output of data from communication unit 350, display unit 110 and EC200.

[0082] The system memory 310 is used as a reading area for the processor's executable program and a working area for writing processing data. In addition, the system memory 310 temporarily stores image data of images captured by the imaging unit 120.

[0083] The communication unit 350 connects to other devices via a wireless or wired communication network to transmit and receive various types of data. For example, the communication unit 350 is configured to include a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark).

[0084] The storage unit 360 is configured to include storage media such as HDD (Hard Disk Drive), SDD (Solid State Drive), RAM, and ROM. In addition to various programs such as the OS, device drivers, and applications, the storage unit 360 also stores various data acquired through program actions.

[0085] [Structure of the Personnel Detection Department]

[0086] Next, the structure of the person detection unit 210 will be described in detail. The person detection unit 210 detects a user present in front of the electronic device 1 by detecting facial regions from each captured image taken by the imaging unit 120 at predetermined time intervals.

[0087] Figure 6 This is a block diagram illustrating an example of the structure of the person detection unit 210 according to this embodiment. The person detection unit 210 shown includes: a face detection unit 211, a detection state determination unit 212, and a person determination unit 213.

[0088] The face detection unit 211 detects facial regions from each captured image taken at predetermined time intervals. For example, the face detection unit 211 processes image data of multiple images captured by the capturing unit 120 at predetermined time intervals and stored in the system memory 310, and detects facial regions of the face from these multiple images. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects the face based on facial feature information, learning data (learned model) used for machine learning based on facial feature information, or a face detection library. Furthermore, the predetermined time interval can be set to, for example, a 15-second interval or a 10-second interval, but can be set to any time interval. Moreover, in the case of the shortest time interval, detection is performed on a unit of all consecutive frames.

[0089] The face detection unit 211 detects facial regions from each captured image and outputs face detection information containing the detected facial regions. Furthermore, the center coordinates of the facial regions may also be included in the face detection information. When a facial region is detected from an captured image, the face detection unit 211 stores the face detection information containing the facial regions in association with the time information (frame information) of that captured image. Conversely, when no facial region is detected from an captured image, the face detection unit 211 stores undetected facial information indicating that no facial regions were detected in association with the time information (frame information) of that captured image. In other words, the face detection unit 211 stores a history of face region detection results.

[0090] Furthermore, when the face detection unit 211 detects face regions continuously from the captured image to the point where no face regions are detected, it magnifies the image region GR2 (refer to) based on the face region FD1 previously detected from the captured image when detecting face regions in subsequent captured images. Figure 3The face detection unit 211 can detect facial regions by magnifying the image region GR2 when detecting facial regions from captured images in the future, if the detection state determination unit 212 determines that the face detection is unstable.

[0091] Furthermore, the face detection unit 211 detects face regions at a low resolution (first resolution) from each captured image. From a state where face regions are continuously detected at low resolution from the captured images to a state where no face regions are detected, it then detects face regions at a high resolution (second resolution) when subsequently detecting face regions from a magnified image region GR2 in the captured images. In other words, the face detection unit 211 performs face detection using two modes: a detection mode that detects faces at low resolution from captured images, and a detection mode that detects face regions at high resolution from a magnified image region GR2. Generally, by detecting face regions at low resolution from captured images, power consumption can be reduced and the impact on other processing can be suppressed.

[0092] Furthermore, if the face detection unit 211 cannot detect a face region from the image region GR2, it assumes that the person has moved, ends the detection of the face region from the image region GR2, and returns to the detection of the face region from the captured image. For example, if the face detection unit 211 fails to detect a face region from the image region GR2 at high resolution, it ends the detection of the face region from the image region GR2 and returns to the detection of the face region from the captured image at low resolution.

[0093] The detection state determination unit 212 determines whether face detection is unstable based on the history of face region detection results (face detection information and face non-detection information). Here, a stable face detection state (i.e., a non-unstable face detection state) refers to a state in which face regions are continuously detected in multiple images captured by the imaging unit 120 at predetermined time intervals and stored in the system memory 310. On the other hand, an unstable face detection state refers to a state in which face regions are not continuously detected in multiple images captured by the imaging unit 120 at predetermined time intervals and stored in the system memory 310. This "non-continuous detection of face regions" means that the face regions are not continuously detected (i.e., the detection of face regions is intermittent), and does not mean that the state of not detecting face regions is continuous. For example, a "non-continuous detection of face regions" is a state in which images captured at predetermined time intervals contain both images in which face regions are detected by the face detection unit 211 and images in which face regions are not detected, in a predetermined proportion.

[0094] When processing facial region detection based on low-resolution image data, if the state changes between a state of continuous facial region detection and a state of discontinuous facial region detection, the detection state determination unit 212 may also determine that facial detection is unstable. For example, the detection state determination unit 212 may also determine facial detection is unstable based on a change from a state of continuous facial region detection in the captured image to a state of discontinuous facial region detection. For example, the detection state determination unit 212 may also determine facial detection is unstable based on the fact that the captured images taken at predetermined time intervals within a predetermined period contain both images of facial regions detected by the facial detection unit 211 and images of facial regions not detected. In this case, the detection state determination unit 212 may determine facial detection is unstable based on the ratio of images of facial regions detected to images of facial regions not detected within the predetermined period, or it may determine facial detection is unstable based on the repetition of images of facial regions detected and images of facial regions not detected within the predetermined period.

[0095] Furthermore, the face detection unit 211 can detect the face region at a low resolution from each captured image and determine that the face detection from the captured image at a low resolution is unstable, and then detect the face region at a high resolution from the captured image by magnifying the image region GR2.

[0096] [Facial detection and processing actions]

[0097] Here, refer to Figure 7 The procedure for facial detection and processing is explained.

[0098] Figure 7 This is a flowchart illustrating a first example of the face detection processing according to this embodiment. Here, the operation of detecting face regions by magnifying the image region GR2, triggered by the fact that no face region was detected in the captured image, will be described.

[0099] (Step S101) The face detection unit 211 detects face regions from each captured image taken by the imaging unit 120 at predetermined time intervals. For example, the face detection unit 211 detects face regions from the captured image of the nth frame (n is an integer greater than or equal to 1). Then, the process proceeds to step S103. At this time, the face detection unit 211 detects face regions from the captured images at low resolution.

[0100] (Step S103) The face detection unit 211 determines whether a face region was detected from the captured image in step S101. If the face detection unit 211 determines that a face region was detected from the captured image (yes), it stores face detection information including the detected face region and the center coordinates of the face as a history of face detection results and returns to the processing in step S101. Then, in step S101, the face detection unit 211 detects the face region from the captured image of the next frame (the captured image of frame n+1). On the other hand, if the face detection unit 211 determines in step S103 that no face region was detected from the captured image (no), it proceeds to the processing in step S107.

[0101] (Step S107) The face detection unit 211 obtains the face region FD1 previously detected from the captured image from the history of face region detection results, and estimates the position of the face (the position of the captured face). Then, it proceeds to the processing in step S109.

[0102] (Step S109) In subsequent frames, the face detection unit 211 magnifies the image region GR2 based on the estimated face position (the position of the captured face) to detect the face region. At this time, the face detection unit 211 detects the face region from the captured image at high resolution. Then, the process proceeds to step S111.

[0103] (Step S111) The face detection unit 211 determines whether a face region was detected from the captured image in step S109. If the face detection unit 211 determines that a face region was detected from the captured image (yes), it stores face detection information, including the detected face region and the center coordinates of the face, as a history of face detection results. Furthermore, the face detection unit 211 returns to step S109 and zooms in on the image region GR2 from the captured image of the next frame to detect the face at high resolution. On the other hand, if the face detection unit 211 determines that no face region was detected in step S111 (no), it returns to the processing of step S101. That is, if the face detection unit 211 fails to detect a face region from the image region GR2 at high resolution, it ends the detection of the face region from the image region GR2 and returns to the detection of the face region from the captured image at low resolution.

[0104] Next, refer to Figure 8 This paper explains the process of using GR2 to magnify the image region to detect the face region when the face detection is unstable.

[0105] Figure 8 This is a flowchart illustrating a second example of the face detection processing according to this embodiment. In this figure, for... Figure 7 The corresponding processing steps are labeled with the same reference numerals in the attached figures, and their descriptions are omitted. Figure 8 In the face detection process shown, if it is determined in step S103 that no face region is detected from the captured image (No), the process proceeds to step S105.

[0106] (Step S105) The detection state determination unit 212 determines whether the face detection is unstable. For example, the detection state determination unit 212 may determine that the face detection is unstable based on the state from continuously detecting face regions in the captured image to not detecting face regions. Alternatively, the detection state determination unit 212 may determine that the face detection is unstable based on the fact that the captured images taken at predetermined time intervals within a predetermined period contain both images of face regions detected by the face detection unit 211 and images of face regions not detected. In this case, the detection state determination unit 212 may determine that the face detection is unstable based on the ratio of images of face regions detected to images of face regions not detected within the predetermined period, or it may determine that the face detection is unstable based on the repetition of images of face regions detected and images of face regions not detected within the predetermined period.

[0107] If the detection state determination unit 212 determines in step S105 that the face detection is unstable (yes), it proceeds to step S107. Then, the face detection unit 211 estimates the position of the face (the position of the captured face) (step S107), and in subsequent frames, it magnifies the image region GR2 based on the estimated position and detects the face at high resolution (step S109). On the other hand, if the detection state determination unit 212 determines in step S105 that the face detection is stable (no), it returns to step S101.

[0108] Return to Figure 6 The person determination unit 213 determines whether a user is in front of the electronic device 1 based on whether the face detection unit 211 detects a facial region in the captured image. For example, if the face detection unit 211 detects a facial region in the captured image, the person determination unit 213 determines that a user is in front of the electronic device 1. On the other hand, if the face detection unit 211 does not detect a facial region in the captured image, the person determination unit 213 determines that no user is in front of the electronic device 1. Furthermore, if the face detection unit 211 detects a facial region in the captured image, the person determination unit 213 may also determine whether the face detected in each captured image taken at predetermined time intervals is active, and if there is activity, determine that a user is present. Alternatively, if the person determination unit 213 determines that the detected face is not active, it may treat that face as the face of a poster, photograph, etc., and determine that no user is present.

[0109] With this structure, the person detection unit 210 detects a user present in front of the electronic device 1. Furthermore, by detecting whether a user is present in front of the electronic device 1, the person detection unit 210 detects a change from a state where no user is present to a state where a user is present (i.e., a user approaches the electronic device 1). Additionally, by detecting whether a user is present in front of the electronic device 1, the person detection unit 210 detects a change from a state where a user is present in front of the electronic device 1 to a state where no user is present (i.e., a user leaves the electronic device 1).

[0110] [Actions handled by action state control]

[0111] Next, the operation of the motion state control process, which controls the system's motion state based on the results of the HPD processing using facial detection described above, will be explained. First, the operation of the motion control unit 220 in starting the system by detecting the user's approach to the electronic device 1 using HPD processing will be explained.

[0112] Figure 9 This is a flowchart illustrating an example of the startup process involved in this embodiment. Here, the electronic device 1 is placed in an open state on a table or similar location, and is set to a standby state.

[0113] (Step S201) The motion control unit 220 determines whether it has detected a user approaching the electronic device 1. If the motion control unit 220 determines that the person detection unit 210 has detected a change from a state where there is no user in front of the electronic device 1 to a state where there is a user (i.e., the user is approaching the electronic device 1) (Yes), it proceeds to step S203. On the other hand, if the motion control unit 220 determines that the person detection unit 210 has detected that there is no user (i.e., the user is not approaching the electronic device 1) (No), it performs step S201 again.

[0114] (Step S203) The motion control unit 220 starts the system in the system processing unit 300. Specifically, when starting the system in the system processing unit 300, the motion control unit 220 outputs a control signal to the power supply unit 400 to supply the power required for the operation of each part of the electronic device 1. Additionally, the motion control unit 220 outputs a start signal to the system processing unit 300 to indicate system startup. If the system processing unit 300 receives the start signal, it begins the startup process. Then, it proceeds to step S205.

[0115] (Step S205) The system processing unit 300 performs login processing (authentication processing). For example, the system processing unit 300 performs login processing based on password authentication, facial authentication, fingerprint authentication, etc., and proceeds to step S207.

[0116] (Step S207) The system processing unit 300 determines whether the authentication result is successful. If the system processing unit 300 determines that the authentication result is successful (Yes), it proceeds to step S209. On the other hand, if the system processing unit 300 determines that the authentication result is unsuccessful (No), it proceeds to step S213.

[0117] (Step S209) If the authentication result is successful, the system processing unit 300 notifies the system of successful login (for example, displayed on the display unit 110) and continues the process. Then, the process proceeds to step S211.

[0118] (Step S211) The system processing unit 300 completes the login process and transfers the system's operating state to the normal operating state.

[0119] (Step S213) If the authentication result fails, the system processing unit 300 notifies the system of login failure (for example, by displaying the message on the display unit 110) and returns to the authentication process in step S205. Furthermore, if the system processing unit 300 fails the authentication process a predetermined number of times, it may also suspend the authentication process and transition to a state where login is not possible.

[0120] Next, the operation of the standby state transition process, in which the motion control unit 220 detects that the user has left the electronic device 1 and causes the system to switch from the normal operation state to the standby state, will be described.

[0121] Figure 10 This is a flowchart illustrating an example of the standby state transition process according to this embodiment. Here, the electronic device 1 is placed on a table or similar surface in an open state, which is considered its normal operating state.

[0122] (Step S251) The motion control unit 220 determines whether a user has been detected leaving the electronic device 1. If the person detection unit 210 detects a change from a state where a user is present to a state where no user is present (i.e., the user has left the electronic device 1) (Yes), the motion control unit 220 proceeds to step S253. On the other hand, if the motion control unit 220 determines that the person detection unit 210 has detected a state where a user is present (i.e., the user has not left the electronic device 1) (No), the motion control unit 220 performs step S251 again.

[0123] (Step S253) The motion control unit 220 transitions the operating state of the system in the system processing unit 300 from the normal operating state to the standby state. Specifically, the motion control unit 220 outputs a standby signal to the system processing unit 300 to instruct it to transition the system to the standby state. If the system processing unit 300 receives the standby signal, it transitions the operating state of the system from the normal operating state to the standby state. Additionally, the motion control unit 220 outputs a control signal to the power supply unit 400 to stop the supply of power that is not needed in the standby state.

[0124] [Summary of the First Implementation]

[0125] As explained above, the electronic device 1 of this embodiment detects facial regions from each captured image taken at predetermined time intervals, and from a state where facial regions are continuously detected from captured images to a state where facial regions are not detected, when detecting facial regions from captured images in subsequent captures, it magnifies an image region GR2 (an example of a specific region) based on facial regions previously detected from captured images to detect facial regions.

[0126] Therefore, even for faces that are difficult to detect, such as side profiles, electronic device 1 can improve the detection rate, thus enabling high-precision detection of users using electronic device 1. Furthermore, electronic device 1 only magnifies the image area GR2 to detect facial regions when the state changes from continuously detecting facial regions to not detecting facial regions, thereby suppressing increased power consumption and reducing the load on other processing.

[0127] Alternatively, the electronic device 1 can determine that face detection is unstable based on the state from when face regions are continuously detected to when face regions are not detected in the captured image. If face detection is determined to be unstable, the image region GR2 can be enlarged to detect face regions when face regions are detected from captured images in the future.

[0128] Therefore, electronic device 1 can improve the detection rate when face detection becomes unstable, thus enabling high-precision detection of the person (user) using electronic device 1. In addition, electronic device 1 only enlarges the image area GR2 to detect the face region when it is determined that face detection is unstable, thus suppressing the increase in power consumption and the load on other processing.

[0129] For example, electronic device 1 can also determine that face detection is unstable based on the fact that the captured images taken at specified time intervals during a specified period contain both images with detected facial regions and images without detected facial regions. Thus, electronic device 1 can improve the detection rate when face detection becomes unstable.

[0130] In addition, if the electronic device 1 is unable to detect the face region from the image region GR2, it will end the detection of the face region from the image region GR2 and return to the detection of the face region from the captured image.

[0131] Therefore, when electronic device 1 cannot detect the face region from image region GR2, it assumes that the person has moved and resets the detection area of ​​the face, thus enabling high-precision detection of the person (user) using electronic device 1.

[0132] In addition, the electronic device 1 can detect facial regions at a lower resolution (first resolution) from each captured image, and when the state of continuously detecting facial regions from the captured images at a lower resolution to the state of not detecting facial regions, detect facial regions at a higher resolution (second resolution) when detecting facial regions from the enlarged image region GR2 of the captured images in a later period.

[0133] Therefore, even for faces that are difficult to detect, such as side profiles, electronic device 1 can improve the detection rate, thus enabling high-precision detection of the person (user) using electronic device 1. In addition, electronic device 1 detects facial regions at high resolution only when the state changes from continuous detection of facial regions to no detection of facial regions, thus suppressing the increase in power consumption and the load on other processing.

[0134] In addition, if the electronic device 1 fails to detect the face region from the image region GR2 at high resolution, it ends the detection of the face region from the image region GR2 and returns to the detection of the face region from the captured image at low resolution.

[0135] Therefore, when the electronic device 1 fails to detect the facial region even at high resolution from the image region GR2, it assumes that the user has moved and resets the detection area of ​​the face, thus enabling high-precision detection of the person (user) using the electronic device 1.

[0136] Furthermore, the control method of the electronic device 1 according to this embodiment includes: a step of detecting a facial region from each captured image taken at predetermined time intervals; and a step of detecting a facial region by magnifying an image region GR2 based on the facial region previously detected from the captured image when detecting a facial region from a captured image in a state where a facial region is continuously detected from the captured image to a state where a facial region is not detected, when detecting a facial region from a captured image in a subsequent instance.

[0137] Therefore, even for faces that are difficult to detect, such as side profiles, electronic device 1 can improve the detection rate, thus enabling high-precision detection of the person (user) using electronic device 1. In addition, electronic device 1 only magnifies the image area GR2 to detect the face area when the state from continuously detecting the face area to not detecting the face area, thus suppressing the increase in power consumption and the load on other processing.

[0138] Furthermore, the electronic device 1 according to this embodiment includes: a system memory 310 (an example of a memory) that temporarily stores image data of images (captured images) captured by the imaging unit 120 (an example of an imaging device); and a person detection unit 210 (an example of a processor) that processes the image data stored in the system memory 310. The person detection unit 210 processes the image data of multiple captured images captured by the imaging unit 120 at predetermined time intervals and stored in the system memory 310, and detects facial regions of the captured face from the multiple captured images based on image data of a first resolution and image data of a second resolution. In addition, the person detection unit 210 determines whether a facial region is continuously detected in the multiple captured images. When processing the detection of facial regions based on image data of the first resolution is in progress, if it is determined that the state changes between a state of continuous detection of facial regions and a state of discontinuous detection of facial regions, it detects facial regions of the captured face from the multiple captured images based on image data of the second resolution.

[0139] Therefore, the electronic device 1 changes the resolution for detection based on the detection status of the face, thereby improving the detection rate and enabling high-precision detection of the person (user) using the electronic device 1.

[0140] For example, based on the fact that among multiple images captured at the specified time intervals during a specified period, there are images containing detected facial regions and images not containing detected facial regions in a specified proportion, the person detection unit 210 determines that the state is one in which facial regions are not detected continuously.

[0141] Therefore, even when faces, such as side profiles, are difficult to detect and face detection becomes unstable, electronic device 1 can improve the detection rate by changing the resolution, thus enabling high-precision detection of the person (user) using electronic device 1.

[0142] Furthermore, the aforementioned first resolution is a lower resolution than the aforementioned second resolution. When processing is being performed in a low-resolution mode for detecting facial regions based on image data of the first resolution, and it is determined that the state has changed from a state of continuous detection of facial regions to a state of discontinuous detection of facial regions, the person detection unit 210 performs processing in a high-resolution mode. In this high-resolution mode, facial regions are detected based on image data of the second resolution in a specific region (e.g., image region GR2), which corresponds to the location of the facial regions detected in the low-resolution mode processing.

[0143] Therefore, in situations where facial features, such as profile views, are difficult to detect and facial detection becomes unstable, electronic device 1 can perform detection in high-resolution mode, thereby improving the detection rate and enabling high-precision detection of the person (user) using electronic device 1. Furthermore, since electronic device 1 only performs detection in high-resolution mode when facial detection becomes unstable, it can suppress increased power consumption and the load on other processing methods.

[0144] In addition, if the person detection unit 210 fails to detect the face region from a specific region (e.g., image region GR2) in high-resolution mode, it performs face region detection in low-resolution mode.

[0145] Thus, when electronic device 1 fails to detect a facial region from a specific area (e.g., image region GR2) in high-resolution mode, it assumes that the user has moved and returns to processing in low-resolution mode, thereby suppressing wasteful power consumption and the load on other processing.

[0146] <Second Implementation>

[0147] Next, the second embodiment of the present invention will be described.

[0148] In the first embodiment, a method was described where the face region is magnified to detect it at high resolution when the face detection becomes unstable and no face region is detected from the captured image. However, it is also possible to detect the face region at high resolution without magnifying it. In this embodiment, the electronic device 1 typically uses a low-resolution image for face region detection when detecting the face region from the captured image, and uses a high-resolution image for face region detection when the face detection becomes unstable and no face region is detected from the captured image.

[0149] Hereinafter, the detection mode that uses low-resolution images to detect facial regions will be referred to as "low-resolution mode." In low-resolution mode, the processing load is smaller and power consumption is suppressed. Conversely, due to the lower resolution, it may sometimes be unable to detect difficult facial regions, such as side profiles. On the other hand, the detection mode that uses high-resolution images to detect facial regions will be referred to as "high-resolution mode." High-resolution mode has a higher resolution compared to low-resolution mode, therefore, there are cases where facial regions that could not be detected in low-resolution mode can be detected in high-resolution mode.

[0150] Furthermore, various methods can be considered for acquiring high-resolution and low-resolution images. When using an imaging unit 120 capable of outputting both high-resolution and low-resolution images, it can be configured to instruct the imaging unit 120 which image to output based on the situation. Alternatively, when using an imaging unit 120 capable of simultaneously outputting both high-resolution and low-resolution images, it can be configured to select either image output from the imaging unit 120 as the image to be processed, depending on the situation. It is also possible to acquire the low-resolution image of the image to be processed by pre-processing the image data output from the imaging unit 120 to reduce its resolution, based on these or different methods.

[0151] The basic structure of the electronic device 1 involved in this embodiment is the same as... Figures 4-6 The structure of the first embodiment shown is the same, and its description is omitted. In this embodiment, the process of switching between low-resolution mode and high-resolution mode by the face detection unit 211 will be described.

[0152] The face detection unit 211 detects facial regions from captured images in either low-resolution or high-resolution mode. Initially, the face detection unit 211 is set to low-resolution mode and detects facial regions from each captured image in low-resolution mode. If the face detection unit 211 detects facial regions in high-resolution mode after a period of continuous detection in low-resolution mode followed by a period of no detection, it then detects facial regions in high-resolution mode in subsequent captured images. Conversely, if the face detection unit 211 fails to detect facial regions in high-resolution mode, it reverts to low-resolution mode.

[0153] Figure 11 This is a flowchart illustrating a first example of the face detection processing according to this embodiment. Here, the process of switching from a low-resolution mode to a high-resolution mode to detect face regions, triggered by the fact that no face region was detected in the captured image, will be described.

[0154] (Step S301) The face detection unit 211 sets the detection mode to low resolution mode and proceeds to step S303.

[0155] (Step S303) The face detection unit 211 detects the face region in low-resolution mode from each captured image taken by the imaging unit 120 at predetermined time intervals. For example, the face detection unit 211 detects the face region from the captured image of the nth frame (n is an integer greater than or equal to 1). Then, the process proceeds to step S305.

[0156] (Step S305) The face detection unit 211 determines whether a face region was detected from the captured image in step S303. If the face detection unit 211 determines that a face region was detected from the captured image (yes), it stores face detection information including the detected face region and the center coordinates of the face region as a history of face detection results and returns to the processing in step S303. Then, in step S303, the face detection unit 211 detects the face region from the captured image of the next frame (the captured image of frame n+1) in low-resolution mode. On the other hand, if the face detection unit 211 determines in step S305 that no face region was detected from the captured image (no), it proceeds to the processing in step S309.

[0157] (Step S309) The face detection unit 211 sets the detection mode to high resolution mode and proceeds to the processing step S311.

[0158] (Step S311) In subsequent frames, the face detection unit 211 detects facial regions from the captured image in high-resolution mode. Then, the process proceeds to step S313.

[0159] (Step S313) The face detection unit 211 determines whether a face region was detected from the captured image in step S311. If the face detection unit 211 determines that a face region was detected from the captured image (yes), it stores face detection information, including the detected face region and the center coordinates of the face region, as a history of face detection results. Then, the face detection unit 211 returns to step S311 and detects the face region from the captured image of the next frame in high-resolution mode. On the other hand, if the face detection unit 211 determines that no face region was detected in step S313 (no), it returns to the processing of step S301. That is, if the face detection unit 211 fails to detect a face region in high-resolution mode, it ends the high-resolution mode and returns to the low-resolution mode.

[0160] Next, refer to Figure 12 This paper explains the process of switching from low-resolution mode to high-resolution mode to detect facial regions when face detection is unstable.

[0161] Figure 12 This is a flowchart illustrating a second example of the face detection processing according to this embodiment. In this figure, for... Figure 11 The corresponding processing steps are labeled with the same reference numerals in the attached figures, and their descriptions are omitted. Figure 12 In the face detection process shown, if it is determined in step S305 that no face region is detected from the captured image (No), the process proceeds to step S307.

[0162] (Step S307) The detection state determination unit 212 determines whether the face detection is unstable. For example, the detection state determination unit 212 may determine that the face detection is unstable based on the state from continuously detecting face regions in the captured image to not detecting face regions. Alternatively, the detection state determination unit 212 may determine that the face detection is unstable based on the fact that the captured images taken at predetermined time intervals within a predetermined period contain both images of face regions detected by the face detection unit 211 and images of face regions not detected. In this case, the detection state determination unit 212 may determine that the face detection is unstable based on the ratio of images of face regions detected to images of face regions not detected within the predetermined period, or it may determine that the face detection is unstable based on the repetition of images of face regions detected and images of face regions not detected within the predetermined period.

[0163] If the detection state determination unit 212 determines in step S307 that face detection is unstable (yes), it proceeds to step S309 and sets the detection mode to high-resolution mode. Then, in subsequent frames, the face detection unit 211 detects facial regions from the captured image in high-resolution mode (step S311). On the other hand, if the detection state determination unit 212 determines in step S307 that face detection is stable (no), it returns to step S303.

[0164] [Summary of the Second Implementation]

[0165] As explained above, the electronic device 1 of this embodiment detects facial regions in a low-resolution mode when detecting facial regions from each captured image, and from a state where facial regions are continuously detected from captured images in low-resolution mode to a state where facial regions are not detected, detects facial regions in a high-resolution mode when detecting facial regions from captured images in subsequent capture images.

[0166] Therefore, even for faces that are difficult to detect, such as side profiles, electronic device 1 can improve the detection rate, thus enabling high-precision detection of the person (user) using electronic device 1. In addition, electronic device 1 detects facial regions only in high-resolution mode when the state changes from continuous detection of facial regions to no detection of facial regions, thus suppressing the increase in power consumption and the load on other processing.

[0167] In addition, if the electronic device 1 determines that face detection is unstable in low-resolution mode, it can detect the face region in high-resolution mode when detecting the face region from the captured image in a later time.

[0168] Therefore, electronic device 1 can improve the detection rate when face detection becomes unstable, thus enabling high-precision detection of the person (user) using electronic device 1. In addition, electronic device 1 only detects the facial region in high-resolution mode when it is determined that face detection is unstable, thus suppressing the increase in power consumption and the load on other processing.

[0169] In addition, if the electronic device 1 fails to detect the facial region from the captured image in high-resolution mode, it ends the facial region detection in high-resolution mode and returns to the facial region detection in low-resolution mode.

[0170] Therefore, when the face area cannot be detected even in high-resolution mode, the electronic device 1 assumes that the person has left in front of the electronic device 1 and returns to low-resolution mode. Thus, power consumption is minimized and the impact on other processing is suppressed.

[0171] Furthermore, the electronic device 1 according to this embodiment includes: a system memory 310 (an example of a memory) that temporarily stores image data of images (captured images) captured by the imaging unit 120 (an example of an imaging device); and a person detection unit 210 (an example of a processor) that processes the image data stored in the system memory 310. The person detection unit 210 processes the image data of multiple captured images captured by the imaging unit 120 at predetermined time intervals and stored in the system memory 310, and detects facial regions of the captured face from the multiple captured images based on image data of a first resolution and image data of a second resolution. In addition, the person detection unit 210 determines whether a facial region is continuously detected in the multiple captured images. When processing the detection of facial regions based on image data of the first resolution is in progress, if it is determined that the state changes between a state of continuous detection of facial regions and a state of discontinuous detection of facial regions, it detects facial regions of the captured face from the multiple captured images based on image data of the second resolution.

[0172] Therefore, the electronic device 1 changes the resolution for detection based on the detection status of the face, thereby improving the detection rate and enabling high-precision detection of the person (user) using the electronic device 1.

[0173] For example, based on the fact that among multiple images captured at the specified time intervals during a specified period, there are images containing detected facial regions and images not containing detected facial regions in a specified proportion, the person detection unit 210 determines that the state is one in which facial regions are not detected continuously.

[0174] Therefore, even when faces, such as side profiles, are difficult to detect and face detection becomes unstable, electronic device 1 can improve the detection rate by changing the resolution, thus enabling high-precision detection of the person (user) using electronic device 1.

[0175] Furthermore, the aforementioned first resolution is a lower resolution than the aforementioned second resolution. When processing is being performed in a low-resolution mode for detecting facial regions based on image data of the first resolution, and it is determined that the state has changed from a state of continuous detection of facial regions to a state of discontinuous detection of facial regions, the person detection unit 210 performs processing in a high-resolution mode. In this high-resolution mode, facial regions are detected based on image data of the second resolution in a specific region (e.g., image region GR2), which corresponds to the location of the facial regions detected in the low-resolution mode processing.

[0176] Therefore, in situations where facial features, such as profile views, are difficult to detect and facial detection becomes unstable, the electronic device 1 can improve the detection rate by performing detection in a high-resolution mode, thus enabling high-precision detection of the person (user) using the electronic device 1. Furthermore, since the electronic device 1 only performs detection in high-resolution mode when facial detection becomes unstable, it can suppress increased power consumption and the load on other processing methods.

[0177] In addition, if the person detection unit 210 fails to detect the face region from a specific region (e.g., image region GR2) in high-resolution mode, it performs face region detection in low-resolution mode.

[0178] Therefore, when a facial region is not detected in a specific area (e.g., image region GR2) in high-resolution mode, the electronic device 1 assumes that the user has moved and returns to processing in low-resolution mode, thus suppressing wasteful power consumption and the load on other processing.

[0179] <Third Implementation Method>

[0180] Next, the third embodiment of the present invention will be described.

[0181] From the perspective of suppressing power consumption, it is preferable to set the detection range (DR) as small as possible when detecting facial regions from captured images. For example, Figure 3 As shown, the detection range DR for face detection is set to be smaller than the area of ​​the image region GR1 of the captured image. However, if the detection range DR is set to a small range, it may fail to detect the face depending on the position of the user present in front of the electronic device 1.

[0182] Figure 13 This is a diagram illustrating an example of the relationship between the detection range DR and the user's position. As shown, when the user U's position moves towards the end (e.g., the right end) of the image region GR3 where the image is captured, sometimes only a portion of the user U's face enters the detection range DR. In such cases, the facial region is not detected, and the user's presence cannot be correctly detected. Therefore, in this embodiment, the electronic device 1 moves the detection range DR according to the position of the detected facial region.

[0183] Figure 14 This diagram illustrates the movement of the detection range DR according to this embodiment. As shown, when the user U moves to the right of the image region GR3, the detection range DR also moves to the right according to the movement of the user U. Reference numeral C1 indicates that the detection range DR is set in the center of the image region GR3 (see Figure 1). Figure 13 The center position of the detection range DR is shown in the attached figure. Reference numeral C2 indicates the center position of the detection range DR after movement. Thus, the electronic device 1 tracks the detection range DR according to the position (movement) of the facial region, thereby accurately detecting the presence of the user.

[0184] Furthermore, the movement of the detection range DR is limited to either reaching the left side of image region GR3 on the left or reaching the right side of image region GR3 on the right. Additionally, it is shown... Figure 13 and Figure 14 The example shown illustrates that the detection range DR can only move in the left-right direction. However, the vertical range of the detection range DR can also be set to be shorter than the vertical range of the image region GR3, thus allowing it to move in the vertical direction as well. When the detection range DR moves in the vertical direction, it is also limited to either reaching the top of the image region GR3 at the top or reaching the bottom of the image region GR3 at the bottom.

[0185] The basic structure of the electronic device 1 involved in this embodiment is the same as... Figures 4-5 The structure involved in the first embodiment shown is the same, and its description is omitted.

[0186] Figure 15This diagram illustrates an example of the structure of the person detection unit 210A according to this embodiment. The person detection unit 210A shown in the diagram is related to... Figure 5 The figure shows the structure of the person detection unit 210 corresponding to this embodiment. In this figure, for the person detection unit 210... Figure 6 The corresponding structural elements of each part are labeled with the same reference numerals, and their descriptions are omitted. The person detection unit 210A includes: a face detection unit 211A, a detection state determination unit 212, and a person determination unit 213. The person detection unit 210A and... Figure 6 The difference in the structure of the person detection unit 210 shown is that the face detection unit 211A includes a detection range setting unit 21A. (See reference...) Figure 14 As explained, the detection range setting unit 21A moves the detection range DR according to the position of the detected facial region. The face detection unit 211A detects the facial region within the detection range DR set by the detection range setting unit 21A.

[0187] For example, in the initial state, the detection range setting unit 21A is set such that the center position of the detection range DR corresponds to the center position of the image region GR3 (initial position). Furthermore, if no facial region is detected in the initial state, the detection range setting unit 21A can also move the detection range DR to search the entire area of ​​the image region GR3. Additionally, the detection range setting unit 21A can also move the detection range DR according to the position of the detected facial region, and then return to the initial position if no facial region is detected.

[0188] Next, refer to Figure 16 The operation of the detection range setting unit 21A in controlling the setting of the detection range DR and the operation of the moving detection range control processing is explained.

[0189] Figure 16 This is a flowchart illustrating an example of the detection range control process involved in this embodiment. (Step S401) The detection range setting unit 21A first sets the detection range DR to an initial position. For example, as the initial position, the detection range setting unit 21A sets the center of the detection range DR to be located at the center of the image region GR3. Then, the process proceeds to step S403.

[0190] (Step S403) The detection range setting unit 21A determines whether the face region has been detected by the face detection unit 211A. If the detection range setting unit 21A determines that the face region has not been detected by the face detection unit 211A (No), the process proceeds to step S405.

[0191] (Step S405) The detection range setting unit 21A moves the detection range DR to search the entire range of the image region GR3. Then, the process proceeds to step S409.

[0192] On the other hand, if the detection range setting unit 21A determines in step S403 that the face detection unit 211A has detected a face region (yes), it proceeds to step S407.

[0193] (Step S407) The detection range setting unit 21A moves the detection range DR according to the position of the detected facial area (see reference). Figure 14 Then, proceed to step S409.

[0194] (Step S409) The detection range setting unit 21A determines whether a facial region has been detected by the face detection unit 211A. If the detection range setting unit 21A determines that a facial region has been detected by the face detection unit 211A (yes), it returns to the process in step S407 and moves the detection range DR according to the position of the detected facial region. On the other hand, if the detection range setting unit 21A determines that a facial region has not been detected by the face detection unit 211A (no), it returns to the process in step S401 and sets the detection range DR to the initial position.

[0195] [Summary of the Third Implementation]

[0196] As explained above, the detection range DR when detecting a facial region from an captured image is set to a range smaller than the image region GR3 of the captured image. Furthermore, the electronic device 1 moves the detection range DR according to the position of the detected facial region.

[0197] Therefore, electronic device 1 can suppress power consumption and detect the person (user) using electronic device 1 with high accuracy.

[0198] In addition, in the initial state, the electronic device 1 is set so that the center position of the detection range DR corresponds to the center position of the image area GR3 of the captured image.

[0199] Therefore, since the electronic device 1 sets the detection range DR so that the center of the image area GR3, where the user is more likely to be present, is included in the detection range in the initial state, it is possible to detect the person (user) using the electronic device with high accuracy.

[0200] <Fourth Implementation>

[0201] Next, the fourth embodiment of the present invention will be described.

[0202] There is a possibility that someone other than the user may approach or pass in front of the electronic device 1. For example, even if someone other than the user approaches, the electronic device 1 does not need to be activated (or preferably should not be activated). Therefore, in this embodiment, even if the facial region of someone other than the user is detected, the detection of that facial region is ignored.

[0203] Figure 17 This diagram illustrates an example of face detection for a person other than the user. In the illustrated example, images captured by electronic device 1 at predetermined time intervals are shown in sequence from time t(1) to t(3). Here, examples are captured images at various times when a person other than the user, P, passes in front of electronic device 1 from right to left. The detected facial region in each captured image shows a greater amount of movement compared to the case where the user is sitting in front of electronic device 1. Therefore, in this embodiment, electronic device 1 ignores the detection of a facial region if the amount of movement detected is above a predetermined threshold. Furthermore, a person other than the user refers to a person who is not using electronic device 1 at that time, and does not mean a person other than the owner of electronic device 1.

[0204] The basic structure of the electronic device 1 involved in this embodiment is the same as... Figures 4-5 The structure involved in the first embodiment shown is the same, and its description is omitted.

[0205] Figure 18 This diagram illustrates an example of the structure of the person detection unit 210B according to this embodiment. The person detection unit 210B shown is related to... Figure 5 The figure shows the structure of the person detection unit 210 corresponding to this embodiment. In this figure, the person detection unit 210 is shown... Figure 6 The corresponding structural elements of each part are labeled with the same reference numerals in the attached drawings, and their descriptions are omitted. The person detection unit 210B includes: a face detection unit 211B, a detection state determination unit 212, and a person determination unit 213B (an example of a determination unit). The face detection unit 211B includes a movement amount determination unit 21B.

[0206] The movement determination unit 21B determines whether the movement of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit 211B in each captured image taken at predetermined time intervals. If the movement determination unit 21B determines that the movement of the facial region is above the predetermined threshold, the face detection unit 211B invalidates the detection of that facial region. Conversely, if the movement determination unit 21B determines that the movement of the facial region is below the predetermined threshold, the face detection unit 211B enables the detection of that facial region. The predetermined threshold is, for example, a threshold preset based on the assumed amount of facial region movement when the user is using the electronic device 1. As an example, the predetermined threshold may also be a movement within the size of one face in all directions.

[0207] Furthermore, even when there is little movement in the facial area but the electronic device 1 itself is active, the detected movement of the facial area increases. Therefore, the movement determination unit 21B can also consider the activity of the electronic device 1 detected by the accelerometer 130 to determine the movement of the facial area detected by the face detection unit 211B. In other words, the movement determination unit 21B can also correct the movement of the facial area detected from the captured image based on the movement direction and amount of the electronic device 1 when the electronic device 1 is active, and determine whether the corrected value is above a predetermined threshold.

[0208] Next, refer to Figure 19 The operation of face detection processing performed by the face detection unit 211B will be explained.

[0209] Figure 19 This is a flowchart illustrating an example of the face detection processing involved in this embodiment.

[0210] (Step S501) The face detection unit 211B determines whether a facial region has been detected. If the face detection unit 211B determines that a facial region has been detected (Yes), it proceeds to step S503. On the other hand, if the face detection unit 211B determines that no facial region has been detected (No), it proceeds to step S509.

[0211] (Step S503) The face detection unit 211B calculates the amount of movement of the face region based on the position of the face region detected from each captured image taken at a predetermined time interval, and proceeds to the processing in step S505.

[0212] (Step S505) The face detection unit 211B determines whether the amount of movement of the face region is above a predetermined threshold based on the position of the face region detected from each captured image taken at predetermined time intervals. If the face detection unit 211B determines that the amount of movement of the face region is less than the predetermined threshold (No), it proceeds to step S507. On the other hand, if the face detection unit 211B determines that the amount of movement of the face region is above the predetermined threshold (Yes), it proceeds to step S509.

[0213] (Step S507) The face detection unit 211B enables the detection of the face region detected from the captured image.

[0214] (Step S509) The face detection unit 211B invalidates the detection of the face region detected from the captured image.

[0215] The person determination unit 213B determines whether a person is in front of the electronic device 1 based on the detection results of the facial region in the face detection processing described above. For example, if the facial region detected by the face detection unit 211B from the captured image is valid, the person determination unit 213B determines that a person is in front of the electronic device 1. On the other hand, if the face detection unit 211B does not detect a facial region from the captured image, or if the detection of the facial region detected from the captured image is invalid, the person determination unit 213B determines that no person is in front of the electronic device 1.

[0216] Thus, even if a facial region is detected from the captured image, but the movement of that facial region exceeds a predetermined threshold, the person detection unit 210B considers it to be the face of someone other than the user, invalidates the facial detection, and determines that no user exists. Therefore, even if someone other than the user approaches or passes in front of the electronic device 1, the electronic device 1 will not detect the approach of the user, thus preventing accidental activation.

[0217] [Summary of the Fourth Implementation]

[0218] As explained above, the electronic device 1 of this embodiment determines whether the amount of movement of the facial region is above a predetermined threshold based on the position of the facial region detected from a plurality of captured images taken at predetermined time intervals. If the amount of movement of the facial region is determined to be above the predetermined threshold, the detection of the facial region is invalidated; if the amount of movement of the facial region is determined to be less than the predetermined threshold, the detection of the facial region is valid.

[0219] Therefore, when a person other than the user approaches or passes in front of the electronic device 1, the electronic device 1 can prevent false detection of the approaching person as the user. Thus, the electronic device 1 can detect the person (user) using the electronic device 1 with high accuracy.

[0220] In addition, the electronic device 1 is equipped with an acceleration sensor 130 (an example of a sensor) for detecting the activity of the electronic device 1, and the amount of movement of the detected facial region is determined by taking into account the movement of the electronic device 1 detected by the acceleration sensor 130.

[0221] Therefore, even when the electronic device 1 itself is active, the electronic device 1 can accurately detect the amount of movement in the facial area, thus enabling high-precision detection of the person (user) using the electronic device 1.

[0222] Furthermore, the electronic device 1 includes a system processing unit 300 (an example of a processing unit) that performs system-based system processing. Additionally, if the detection of a facial region detected from the captured image is valid, the electronic device 1 determines that a user is present; if no facial region is detected from the captured image or if the detection of a facial region detected from the captured image is invalid, it determines that no user is present. Moreover, when transitioning from a state where the user is absent to a state where the user is present, the electronic device 1 transitions the system's operating state from a standby state (an example of a first operating state) to a normal operating state (an example of a second operating state).

[0223] Therefore, when a person (user) using electronic device 1 approaches the front of electronic device 1, electronic device 1 can be started from standby mode, and even if a person other than the user approaches or passes in front of electronic device 1, accidental start-up can be prevented.

[0224] Furthermore, the control method of the electronic device 1 according to this embodiment includes: a step of detecting a facial region from each captured image taken at a predetermined time interval; a step of determining whether the amount of movement of the facial region is above a predetermined threshold based on the position of the facial region detected from each captured image; a step of invalidating the detection of the facial region if it is determined that the amount of movement of the face is above the predetermined threshold, and validating the detection of the facial region if it is determined that the amount of movement of the face is less than the predetermined threshold.

[0225] Therefore, when a person other than the user approaches or passes in front of the electronic device 1, the electronic device 1 can prevent false detection of the approaching person as the user. Thus, the electronic device 1 can detect the person (user) using the electronic device 1 with high accuracy.

[0226] The various embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the specific structure is not limited to the embodiments described above, and design changes are also included without departing from the spirit of the invention. For example, the structures described in the above embodiments can be combined arbitrarily.

[0227] Furthermore, in the above embodiments, a structural example in which the camera unit 120 is built into the electronic device 1 has been described, but it is not limited to this. For example, the camera unit 120 may not be built into the electronic device 1, or it may be configured to be able to be installed on the electronic device 1 as an external accessory (e.g., any side of side 10a, 10b, 10c, etc.) and communicate with the electronic device 1 wirelessly or via wired connection.

[0228] Furthermore, in the above embodiments, the electronic device 1 detects the presence of a user by detecting the facial region captured in the captured image, but is not limited to the face; it can also detect the presence of a user by detecting at least a part of the body. Additionally, the electronic device 1 can also use a distance sensor (e.g., a proximity sensor) that detects the distance to an object. For example, a distance sensor is provided on the inner surface of the first frame 10 to detect objects (e.g., people) present in a detection range facing (forward) to the inner surface of the first frame 10. As an example, the distance sensor can be an infrared distance sensor configured to include a light-emitting part that emits infrared light and a light-receiving part that receives reflected light from the surface of the object. Furthermore, the distance sensor can also be a sensor that uses infrared light emitted by a light-emitting diode, or a sensor that uses an infrared laser, the infrared laser emitting light with a narrower wavelength than the infrared light emitted by the light-emitting diode. Furthermore, the distance sensor is not limited to an infrared distance sensor; any sensor that detects the distance to an object can be an ultrasonic sensor or a sensor using UWB (Ultra Wide Band) radar, or other types of sensors. Alternatively, the distance sensor may not be built into the electronic device 1, or it may be configured to be mounted on the electronic device 1 as an external accessory (e.g., any side of side 10a, 10b, 10c, etc.), and communicate with the electronic device 1 wirelessly or via wired connection. Furthermore, the imaging unit 120 and the distance sensor may also be integrated into one unit.

[0229] Furthermore, in the above embodiment, an example is shown where the person detection unit 210 and EC200 are provided separately, but part or all of the person detection unit 210 may also be provided in EC200. Additionally, in the above embodiment, an example is shown where EC200 includes a motion control unit 220, but part or all of the motion control unit 220 may also be provided in a processing unit other than EC200 (e.g., system processing unit 300).

[0230] Furthermore, in the above embodiment, the EC200, which operates independently of the system processing unit 300, can be any processing unit such as a sensor hub or chipset, or the above processing can be performed by a processing unit other than the EC200 instead of the EC200.

[0231] Additionally, the standby state described above may include a hibernation state, a power-off state, etc. The hibernation state is, for example, equivalent to the S4 state defined by ACPI. The power-off state is, for example, equivalent to the S5 state (power off state) defined by ACPI. Furthermore, the standby state may include at least a state where the display is off (screen off), or a screen-locked state. A screen-locked state refers to a state where a preset image (e.g., a screen-locking image) is displayed on the display, making it impossible to visually confirm the content being processed, and the device cannot be used until the lock is released (e.g., user authentication).

[0232] Furthermore, the aforementioned electronic device 1 has an internal computer system. Moreover, programs for implementing the functions of each structure of the electronic device 1 can be recorded on a computer-readable recording medium, and the processing within each structure of the electronic device 1 is performed by having the computer system read and execute the program recorded on the recording medium. Here, "having the computer system read and execute the program recorded on the recording medium" includes the computer system installation program. The term "computer system" here includes hardware such as the operating system and peripheral devices. Additionally, the "computer system" may also include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, "computer-readable recording medium" refers to portable media such as floppy disks, optical disks, ROMs, and CD-ROMs, and storage media such as hard disks built into the computer system. Thus, the recording medium containing the program can also be a non-transitory recording medium such as a CD-ROM.

[0233] Furthermore, the recording medium also includes internal or external recording media that can be accessed from a distribution server for distributing the program. Additionally, the program can be divided into multiple parts, each downloaded at different time intervals, and then assembled from various components of electronic device 1, with each part distributed via a different distribution server. Moreover, the term "computer-readable recording medium" also includes structures that retain the program for a certain period, such as a server in the case of sending the program via a network, or volatile memory (RAM) within a computer system acting as a client. Furthermore, the program described above can also be a structure used to implement the aforementioned functions. Further, it can also be a so-called differential file (differential program) that can be implemented by combining the aforementioned functions with a program already recorded in the computer system.

[0234] Alternatively, some or all of the functions of the electronic device 1 in the above embodiments can be implemented as integrated circuits such as LSI (Large Scale Integration). Each function can be processed individually, or some or all can be integrated for processing. Furthermore, the method of integrated circuit implementation is not limited to LSI; it can also be implemented using dedicated circuits or general-purpose processors. Additionally, if advancements in semiconductor technology lead to integrated circuit technologies that replace LSI, integrated circuits based on such technologies can also be used.

[0235] Furthermore, the electronic device 1 described in the above embodiments is not limited to PCs, tablet terminals, smartphones, etc., and can also be applied to home appliances and commercial appliances. As a home appliance, it can be applied to televisions, refrigerators with displays, microwave ovens, etc. For example, it can control the opening / closing of the television screen, or the opening / closing of the display screen of a refrigerator, microwave oven, etc., based on the approach or departure of a person. As a commercial appliance, it can be applied to vending machines, multimedia terminals, etc. For example, it can control the operation state, such as turning the lights on / off in a vending machine, or turning the display screen on / off in a multimedia terminal, based on the approach or departure of a person.

Claims

1. An electronic device comprising: Memory, which temporarily stores image data of images captured by the imaging device; and The processor processes the image data stored in the aforementioned memory. The aforementioned processor has: The face detection unit processes image data from multiple images captured by the aforementioned imaging device at predetermined time intervals and stored in the aforementioned memory, and detects facial regions of the captured face from the multiple images based on image data at a first resolution and image data at a second resolution; and The detection state determination unit determines whether the facial region is continuously detected in the multiple images. When the process of detecting the facial region based on the image data at the first resolution is in progress, if the detection state determination unit determines that the state changes between a state in which the facial region is continuously detected and a state in which the facial region is not continuously detected, the facial detection unit detects the facial region from the plurality of images based on the image data at the second resolution. Based on the fact that among the multiple images taken at the specified time intervals during the specified period, there are images containing both the face region detected by the face detection unit and the face region not detected, in a specified proportion, the detection state determination unit determines that the face region is not detected continuously.

2. The electronic device according to claim 1, wherein, The first resolution mentioned above is a lower resolution than the second resolution mentioned above. When the face detection unit is performing low-resolution mode processing based on the image data of the first resolution, and the detection state determination unit determines that the state has changed from continuously detecting the face region to not continuously detecting the face region, the face detection unit performs processing in high-resolution mode. In the high-resolution mode, the face region is detected based on the image data of the second resolution in a specific region, and the specific region corresponds to the position of the face region detected in the low-resolution mode processing.

3. The electronic device according to claim 2, wherein, If the face detection unit fails to detect the face region from the specific region in the high-resolution mode, it performs face region detection in the low-resolution mode.

4. The electronic device according to any one of claims 1 to 3, wherein, When the face detection unit detects the face region from the multiple images, the detection range is set to be smaller than the range of the image region of the captured image. The aforementioned electronic device also includes a detection range setting unit, which moves the detection range according to the position of the facial region detected by the facial detection unit.

5. The electronic device according to claim 4, wherein, In the initial state, the detection range setting unit is set such that the center position of the detection range corresponds to the center position of the image area of ​​the captured image.

6. The electronic device according to any one of claims 1 to 3, wherein, It also includes a motion determination unit, which determines whether the motion of the facial region is above a predetermined threshold based on the position of the facial region detected by the facial detection unit from the multiple images. If the movement amount determination unit determines that the movement amount of the facial region is above a predetermined threshold, the facial detection unit invalidates the detection of the facial region; if the movement amount determination unit determines that the movement amount of the facial region is less than the predetermined threshold, the facial detection unit makes the detection of the facial region valid.

7. The electronic device according to claim 6, wherein, It also has sensors to detect the activity of the aforementioned electronic devices. The movement determination unit considers the activity of the electronic device detected by the sensor to determine the movement of the facial region detected by the face detection unit.

8. The electronic device according to claim 6, wherein, It also has: The processing department performs system-based processing. The person determination unit determines that a user exists if the facial area detected by the facial detection unit is valid, and determines that a user does not exist if the facial area is not detected by the facial detection unit or if the facial area detected by the facial detection unit is invalid. as well as When the determination result of the aforementioned character determination unit shifts from a state where there is no user to a state where there is a user, the motion control unit shifts the motion state of the aforementioned system from a first motion state where at least a portion of the actions processed by the aforementioned system is restricted to a second motion state where the actions processed by the aforementioned system are activated compared to the first motion state.

9. An electronic device comprising: Memory, which temporarily stores image data of images captured by the imaging device; The processor processes the image data stored in the aforementioned memory; and Sensors detect the activity of the aforementioned electronic devices; The aforementioned processor has: The face detection unit processes image data from multiple images captured by the aforementioned imaging device at predetermined time intervals and stored in the aforementioned memory, and detects facial regions from the multiple images in which a face has been captured; and The movement determination unit determines whether the movement of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit in the multiple images. If the movement amount determination unit determines that the movement amount of the facial region is above a predetermined threshold, the face detection unit invalidates the detection of the facial region; if the movement amount determination unit determines that the movement amount of the facial region is below the predetermined threshold, the face detection unit enables the detection of the facial region. The movement determination unit considers the activity of the electronic device detected by the sensor to determine the movement of the facial region detected by the face detection unit.

10. The electronic device according to claim 9, wherein, It also has: The processing department performs system-based processing. The person determination unit determines that a user exists if the facial area detected by the facial detection unit is valid, and determines that a user does not exist if the facial area is not detected by the facial detection unit or if the facial area detected by the facial detection unit is invalid. as well as When the determination result of the aforementioned character determination unit shifts from a state where there is no user to a state where there is a user, the motion control unit shifts the motion state of the aforementioned system from a first motion state where at least a portion of the actions processed by the aforementioned system is restricted to a second motion state where the actions processed by the aforementioned system are activated compared to the first motion state.

11. A control method for an electronic device, the electronic device comprising: a memory temporarily storing image data of an image captured by an imaging device, and a processor processing the image data stored in the memory, the control method comprising: The face detection unit processes image data of multiple images captured by the aforementioned imaging device at predetermined time intervals and stored in the aforementioned memory, and detects facial regions of the captured face from the multiple images based on image data of a first resolution and image data of a second resolution. as well as The detection state determination unit determines whether the state of the facial region is continuously detected in the above multiple images. In the step of detecting the facial region by the face detection unit, when processing is being performed to detect the facial region based on the image data of the first resolution, if the detection state determination unit determines that the state changes between a state in which the facial region is continuously detected and a state in which the facial region is not continuously detected, the facial region is detected from the plurality of images based on the image data of the second resolution. In the determination step of the detection state determination unit, based on the fact that among the plurality of images taken at the specified time intervals during the specified period, there are images that contain the facial region detected by the face detection unit and images that do not detect the facial region, the state is determined to be one in which the facial region is not detected continuously.

12. A control method for an electronic device, the electronic device comprising: a memory temporarily storing image data of an image captured by an imaging device, a processor processing the image data stored in the memory, and a sensor detecting the activity of the electronic device, the control method comprising: The face detection unit processes image data of multiple images captured by the aforementioned imaging device at predetermined time intervals and stored in the aforementioned memory, and detects facial regions from the multiple images in which a face has been captured. as well as The motion determination unit determines whether the motion of the facial region is above a predetermined threshold based on the position of the facial region detected by the face detection unit from the multiple images. In the step of detecting the facial region by the facial detection unit, if the movement amount determination unit determines that the movement amount of the facial region is above a predetermined threshold, the detection of the facial region is invalidated; if the movement amount determination unit determines that the movement amount of the facial region is less than the predetermined threshold, the detection of the facial region is valid. In the step of determining the amount of movement, the movement of the electronic device detected by the sensor is taken into account to determine the amount of movement of the facial region detected by the face detection unit.

Images