Electronic device and control method

Abstract

The present application relates to an electronic device and a control method, which appropriately controls an operation state of the electronic device according to a usage situation. The electronic device includes a memory that temporarily stores a program of a system, a first processor that realizes a function of the system by executing the program, a second processor that detects a face region in which a face is photographed from an image photographed by a photographed portion, and a third processor that switches a first process in which first information is output when the face region is detected by the second processor and second information is output when the face region is not detected, and a second process in which the first information is output regardless of the detection of the face region by the second processor, and when a certain time has not passed based on an operation input by a user, the first processor restricts use of a function of at least a part of the system, and when the second information is acquired from the third processor, the use of the function of at least the part of the system is restricted without waiting for the certain time.

Description

Technical Field

[0001] This invention relates to electronic devices and control methods. Background Technology

[0002] There exists an electronic device that transitions to an active state when a person approaches, and transitions to a standby state where all functions are disabled when the person moves away. For example, Patent Document 1 discloses a technique that uses an infrared sensor to detect the intensity of infrared light, thereby detecting whether a person is approaching or moving away to control the electronic device's operational state.

[0003] In recent years, advancements in computer vision and other technologies have led to increased accuracy in facial detection from images. Consequently, facial detection has begun to replace infrared sensor-based person detection. While infrared sensors reflect infrared light back from both people and objects, facial detection prevents the misidentification of simple objects as people. For example, in electronic devices such as personal computers, cameras used for capturing images for facial detection are positioned to capture images of the person using the device.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2016-148895

[0005] However, even when someone is using an electronic device, their face may not be detected in the camera's image, depending on how the device is being used. For example, cameras equipped with electronic devices may have a shutter (camera cover) that physically blocks the camera's shooting direction from a privacy perspective. If the shutter is closed to block the camera's shooting direction, the face may not be detected even if someone is using the device. Similarly, covering the camera with a hand also blocks the camera's shooting direction, thus preventing face detection. Furthermore, the face may not be detected depending on the relationship between the person using the device and the camera's shooting direction. There is a problem where the electronic device switches to standby mode when no face is detected despite the presence of someone using the device. Summary of the Invention

[0006] The present invention was made in view of the above circumstances, and one of its objectives is to provide an electronic device and a control method for appropriately controlling the operating state according to the usage conditions.

[0007] The present invention was made to solve the aforementioned problems. The electronic device according to a first aspect of the present invention includes: a memory that temporarily stores a program of the system; a first processor that executes the program to implement the functions of the system; a second processor that detects a facial region containing a face from an image captured by a camera; and a third processor that switches between executing a first process and a second process. In the first process, if the facial region is detected by the second processor, first information is output; if the facial region is not detected, second information is output. In the second process, the first information is output regardless of the facial region detection performed by the second processor. If there is no user input for a certain period of time, the first processor restricts the use of at least a portion of the functions of the system. And if the second information is obtained from the third processor, the use of at least a portion of the functions of the system is restricted without waiting for the specified period of time.

[0008] Furthermore, the electronic device according to the second aspect of the present invention includes: a memory that temporarily stores a program of the system; a first processor that executes the program to realize the function of the system; a second processor that detects a facial region containing a face from an image captured by a camera, and, if the facial region is detected, detects the direction of the gaze of the eyes contained in the face within the facial region; and a third processor that switches between executing a first process and a second process. In the first process, if the facial region is detected by the second processor and the direction of the gaze is within a predetermined range, first information is output; if the facial region is detected and the direction of the gaze is outside the predetermined range or if the facial region is not detected, second information is output. In the second process, the first information is output regardless of the detection of the facial region and the direction of the gaze by the second processor. If there is no user input for a certain period of time, the first processor restricts the use of at least a portion of the functions of the system, and if the second information is obtained from the third processor, the use of at least a portion of the functions of the system is restricted without waiting for the certain period of time.

[0009] In the aforementioned electronic device, the second processor may also determine whether the shooting direction of the shooting unit is blocked based on the image captured by the shooting unit. If the second processor determines that the shooting direction of the shooting unit is not blocked, the third processor performs the first process. If the second processor determines that the shooting direction of the shooting unit is blocked, the third processor performs the second process.

[0010] The aforementioned electronic device may also include: a first frame having at least the aforementioned shooting unit, a second frame having at least the aforementioned input unit, and a rotation mechanism connecting the aforementioned first frame and the second frame to be able to rotate relative to each other. When the rotation angle of the aforementioned first frame and the second frame being rotated relative to each other is within a predetermined range, the aforementioned third processor executes the aforementioned first process, and when the aforementioned rotation angle is outside the predetermined range, the aforementioned second process is executed.

[0011] The aforementioned electronic device may also have an output interface that outputs image data for displaying images based on the functions of the aforementioned system on an external display. When the images are not displayed on the external display, the third processor performs the first processing described above, and when the images are displayed on the external display, it performs the second processing described above.

[0012] In the aforementioned electronic device, the third processor may also repeatedly execute the first process or the second process, and the cycle of repeatedly executing the second process is longer than the cycle of repeatedly executing the first process.

[0013] Furthermore, the control method of an electronic device according to the third aspect of the present invention, which includes a memory for a program with a temporary storage system, a first processor, a second processor, and a third processor for implementing the functions of the system by executing the program, comprises: a step in which the second processor detects a facial region containing a face in an image captured by a camera; a step in which the third processor switches between executing a first process and a second process, wherein in the first process, first information is output when the facial region is detected by the second processor, and second information is output when the facial region is not detected; and in the second process, the first information is output regardless of how the facial region is detected by the second processor; a step in which the first processor restricts the use of at least a portion of the functions of the system when there is no user input for a certain period of time; and a step in which, when the second information is obtained from the third processor, the use of at least a portion of the functions of the system is restricted without waiting for the certain period of time.

[0014] Furthermore, the control method of an electronic device according to the fourth aspect of the present invention, which includes a memory for a program with a temporary storage system, a first processor, a second processor, and a third processor for implementing the functions of the system by executing the program, includes: the second processor detecting a facial region containing a face in an image captured by a camera, and, if the facial region is detected, detecting the direction of the gaze of the eyes contained in the face within the facial region; the third processor switching between executing a first process and a second process, wherein, in the first process, if the second processor detects the facial region and the direction of the gaze is within a predetermined range, first information is output; if the facial region is detected and the direction of the gaze is outside the predetermined range or if the facial region is not detected, second information is output; in the second process, the first information is output regardless of how the second processor detects the facial region and the direction of the gaze; the first processor restricting the use of at least a portion of the functions of the system when there is no user input for a certain period of time; and, if the second information is obtained from the third processor, restricting the use of at least a portion of the functions of the system without waiting for the certain period of time.

[0015] According to the above-described method of the present invention, the operating state of the electronic device can be appropriately controlled according to the usage conditions. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the outline of HPD processing of the electronic device according to the first embodiment.

[0017] Figure 2 This is a perspective view showing a structural example of the appearance of the electronic device according to the first embodiment.

[0018] Figure 3 This is a diagram showing an example of the open state of the camera shutter according to the first embodiment.

[0019] Figure 4 This is a diagram illustrating an example of the closed state of the camera shutter according to the first embodiment.

[0020] Figure 5 This diagram illustrates the state in which the shooting direction of the camera according to the first embodiment is obstructed by the user's hand.

[0021] Figure 6 This is a block diagram illustrating an example of the hardware structure of the electronic device according to the first embodiment.

[0022] Figure 7This is a block diagram illustrating an example of the functional structure of the electronic device according to the first embodiment.

[0023] Figure 8 This is a flowchart illustrating an example of HPD processing according to the first embodiment.

[0024] Figure 9 This is a flowchart illustrating an example of the sleep processing involved in the first embodiment.

[0025] Figure 10 This is a block diagram illustrating an example of the functional structure of the electronic device according to the second embodiment.

[0026] Figure 11 This is a flowchart illustrating an example of HPD processing according to the second embodiment.

[0027] Figure 12 This is a block diagram illustrating an example of the functional structure of the electronic device according to the third embodiment.

[0028] Figure 13 This is a flowchart illustrating an example of HPD processing according to the third embodiment.

[0029] Explanation of reference numerals in the attached figures

[0030] 1, 1A, 1B… Electronic device; 10… First frame; 20… Second frame; 15… Hinge mechanism; 110… Display unit; 120… Camera (capturing unit); 130… Accelerometer sensor; 140… Power button; 150… Input device; 151… Keyboard; 153… Touchpad; 160… Image output terminal; 200… EC; 210, 210A, 210B… Face detection unit; 211… Face detection processing unit; 212… HPD processing unit; 213… Camera status determination unit; 220, 220A… 220B…Motion control unit; 221A…Frame status determination unit; 221B…External display determination unit; 222, 222A, 222B…HPD information output unit; 300…System processing unit; 302…CPU; 304…GPU; 306…Memory controller; 308…I / O controller; 310…System memory; 320…Sleep processing unit; 321…Timing unit; 322…HPD information acquisition unit; 323…Sleep control unit; 350…Communication unit; 360…Storage unit; 400…Power supply unit. Detailed Implementation

[0031] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0032] <First Implementation>

[0033] [summary]

[0034] First, an overview of the electronic device 1 according to the first embodiment will be described. The electronic device 1 according to this embodiment is, for example, a notebook PC (Personal Computer).

[0035] Electronic device 1 has at least a "normal operating state" and a "standby state" as its operating states. The normal operating state refers to the operating state in which processing can be performed without special restrictions, for example, equivalent to the S0 state specified in ACPI (Advanced Configuration and Power Interface).

[0036] Standby state refers to a state in which the use of at least some functions of a system is restricted. For example, standby state can be stand-by mode or sleep mode, or it can be the modern standby mode in Windows (registered trademark), equivalent to the S3 state (sleep state) defined in ACPI. Furthermore, standby state can also be a state in which at least the display unit is turned off (screen off), or a screen-locked state. Screen lock refers to a state in which a preset image (e.g., a screen lock image) is displayed on the display unit, making it impossible to visually confirm the content being processed, and the system cannot be used until the lock is unlocked through user authentication, etc. In other words, standby state is equivalent to any of the following: a state in which power consumption is lower than normal operation, a state in which the user cannot visually confirm the operation content of electronic device 1, or a state in which the user cannot use electronic device 1.

[0037] In addition, the system's operating states include a "stop state," which consumes less power than the standby state. A stop state includes, for example, a rest state or a power-off state. A rest state is, for example, equivalent to state S4 as defined in ACPI. A power-off state is, for example, equivalent to state S5 (power off) as defined in ACPI.

[0038] Hereinafter, the process of transitioning the operating state of a system from a standby state or a stopped state to a normal operating state is sometimes referred to as startup. The standby state and the stopped state, for example, have a lower degree of operational activity compared to the normal operating state; therefore, startup of electronic device 1 is the action of activating the system within electronic device 1.

[0039] Figure 1This diagram illustrates the general outline of the HPD (Human Presence Detection) processing of the electronic device 1 according to this embodiment. The electronic device 1 detects a person (i.e., a user) present in its vicinity. The process of detecting the presence of this person is called HPD (Human Presence Detection) processing. The electronic device 1 detects the presence or absence of a person through HPD processing and controls the system's operational state based on the detection result. For example, as... Figure 1 As shown in (A), when electronic device 1 detects a change from a state where no person is in front of it (Absence) to a state where a person is present (Presence), i.e., when a person approaches electronic device 1 (Approach), it determines that a user is approaching, automatically activates the system, and transitions it to its normal operating state. Furthermore, as... Figure 1 As shown in (B), when a person is present in front of electronic device 1, electronic device 1 determines that a user is present and continues its normal operation. Furthermore, as... Figure 1 As shown in (C), when electronic device 1 detects a change from a state where a person is present in front of electronic device 1 to a state where no person is present, i.e., when the person leaves electronic device 1, it determines that the user has left and causes the system to switch to standby mode.

[0040] For example, electronic device 1 has a face detection function. It determines whether a user is present in front of (front of) electronic device 1 by detecting facial regions in an image captured from the front (front side) of the camera. If electronic device 1 detects a facial region in the captured image, it determines that a user is present. Conversely, if electronic device 1 does not detect a facial region in the captured image, it determines that no user is present. That is, if electronic device 1 detects that a user is approaching electronic device 1 when no facial region is detected in the captured image, it transitions the system to a normal operating state. Conversely, if electronic device 1 does not detect that a facial region is detected in the captured image, it detects that a user has left electronic device 1 when no facial region is detected, and it transitions the system to a standby state.

[0041] [External structure of electronic devices]

[0042] Figure 2 This is a perspective view showing a structural example of the appearance of the electronic device 1 according to this embodiment.

[0043] Electronic device 1 includes a first frame 10, a second frame 20, and a hinge mechanism 15. The first frame 10 and the second frame 20 are connected by the hinge mechanism 15. The first frame 10 is capable of rotating relative to the second frame 20 about a rotation axis formed by the hinge mechanism 15. The opening angle formed by the rotation of the first frame 10 and the second frame 20 is illustrated as "θ".

[0044] The first frame 10 is also referred to as the A cover or the display frame. The second frame 20 is also referred to as the C cover or the system frame. In the following description, the sides of the first frame 10 and the second frame 20 that have the hinge mechanism 15 are referred to as sides 10c and 20c, respectively. The sides of the first frame 10 and the second frame 20 opposite to sides 10c and 20c are referred to as sides 10a and 20a, respectively. In the illustration, the direction from side 20a to side 20c is referred to as "rear", and the direction from side 20c to side 20a is referred to as "front". The right and left sides relative to the rear are referred to as "right" and "left", respectively. The left side of the first frame 10 and the second frame 20 are referred to as sides 10b and 20b, respectively, and the right side is referred to as sides 10d and 20d, respectively. In addition, the state in which the first frame 10 and the second frame 20 overlap and are completely closed (the state with an opening angle θ = 0°) is referred to as the "closed state". The faces of the first frame 10 and the second frame 20 facing each other in the closed state are called their "inner faces," and the faces opposite to their inner faces are called their "outer faces." Furthermore, the state in which the first frame 10 and the second frame 20 are open relative to the closed state is called the "open state."

[0045] Figure 2 The electronic device 1 shown is an example of an open state. The open state is the state where the side 10a of the first frame 10 is separated from the side 20a of the second frame 20. In the open state, the inner surfaces of both the first frame 10 and the second frame 20 are visible. The open state is one of the states in which the user uses the electronic device 1, typically mostly used with an opening angle θ of approximately 100 to 130°. Furthermore, the range of the opening angle θ for the open state can be arbitrarily determined based on factors such as the range of angles through which rotation is possible via the hinge mechanism 15.

[0046] A display unit 110 is provided on the inner surface of the first frame 10. The display unit 110 is configured to include a liquid crystal display (LCD), an organic EL (electroluminescence) display, or the like. Furthermore, a camera 120 is provided in the area surrounding the display unit 110 on the inner surface of the first frame 10. For example, the camera 120 is positioned on the side 10a side of the area surrounding the display unit 110. Moreover, the position of the camera 120 is just one example; it can be placed in any other location as long as its direction is facing directly towards the inner surface of the first frame 10 (front).

[0047] In its open state, camera 120 captures a defined shooting range facing (front) the inner surface of the first frame 10. The defined shooting range refers to the range of the viewing angle determined by the shooting element of camera 120 and the optical lens positioned in front of the shooting surface of the shooting element. For example, camera 120 can capture an image including a person (user) present in front of (front of) electronic device 1. Electronic device 1 performs HPD processing by detecting the facial region from the captured image, as shown in reference... Figure 1 The operating state of the control system is explained in that way.

[0048] Additionally, a power button 140 is provided on the side 20b of the second frame 20. The power button 140 is an operating element used by the user to indicate power on (transition from the stop state to the normal operating state) and power off (transition from the normal operating state to the stop state). Furthermore, a keyboard 151 and a touchpad 153 are provided on the inner surface of the second frame 20 as input devices. Moreover, as input devices, the keyboard 151 and touchpad 153 can be replaced, or a touch sensor can be included in addition to the keyboard 151 and touchpad 153; a mouse or an external keyboard can also be connected. In the case of a structure with a touch sensor, it can also be configured as a touch panel that receives operation in an area corresponding to the display surface of the display unit 110. Additionally, a microphone for inputting voice can also be included in the input device.

[0049] Furthermore, in the closed state where the first frame 10 and the second frame 20 are closed, the display unit 110 and camera 120 located on the inner surface of the first frame 10 and the keyboard 151 and touchpad 153 located on the inner surface of the second frame 20 are covered by the other frame surface, thus rendering them unable to function.

[0050] [Camera shutter mechanism]

[0051] Next, the structure of the camera shutter installed in the camera 120 will be explained. The camera 120 is equipped with a camera shutter that can physically block the shooting direction through user operation. Figure 3 as well as Figure 4 This diagram illustrates an example of the camera shutter according to this embodiment. The camera shutter 120S is configured to move towards a position that obscures the shooting direction of the camera 120 disposed on the first frame 10 (towards...). Figure 3 (Sliding movement in the direction indicated by arrow Y1). Figure 3 This indicates that the camera shutter speed is open at 120 seconds, meaning the shooting direction of the camera at 120 seconds is not obstructed. On the other hand, Figure 4 At camera shutter speed 120s from Figure 3 The state shown is the state where the camera moves in the direction indicated by arrow Y1 and is closed, indicating that the shooting direction of camera 120 is blocked.

[0052] For example, as referenced Figure 1 As explained, if the electronic device 1 fails to detect the face area in a selfie image during normal operation, it detects that the user has left the electronic device 1 (Leave) and transitions the system to standby mode. However, as... Figure 4 As shown, when the shooting direction of the camera 120 is blocked by the camera shutter 120S, even if there is a user in front of the electronic device 1, the user will not be reflected in the captured image by the camera 120, so the facial area cannot be detected from the captured image.

[0053] in addition, Figure 5 This diagram illustrates a state where the shooting direction of camera 120 is obstructed by the user's hand. Similarly, as shown in this diagram, even if a user is present in front of electronic device 1 (front view), the user is not reflected in the captured image by camera 120, and therefore the facial area cannot be detected from the captured image.

[0054] Therefore, in this embodiment, it is determined whether the shooting direction is obstructed, and if it is obstructed, the system is controlled to not transition to a standby state even if no facial region is detected in the captured image. The structure and processing of this embodiment will be described in detail below.

[0055] [Hardware structure of electronic devices]

[0056] Figure 6 This is a block diagram illustrating an example of the hardware structure of the electronic device 1 according to this embodiment. Figure 6 In the middle, to and Figure 2The corresponding structures of each part are given the same reference numerals. The electronic device 1 comprises: a display unit 110, a camera 120, an accelerometer 130, a power button 140, an input device 150, an image output terminal 160, an EC (Embedded Controller) 200, a face detection unit 210, a system processing unit 300, a communication unit 350, a storage unit 360, and a power supply unit 400. The display unit 110 displays display data (images) generated based on system processing executed by the system processing unit 300 and the processing of application programs acting on the system processing.

[0057] Camera 120 captures an image of an object within a predetermined viewing angle facing (front) the inner surface of the first frame 10, and outputs the captured image to the system processing unit 300 and the face detection unit 210. Camera 120 can be a conventional camera or an infrared camera. A conventional camera is one that uses a visible light sensor (e.g., an RGB camera) as its imaging element. An infrared camera is one that uses an infrared sensor (i.e., an infrared sensor) as its imaging element. Camera 120 temporarily stores the image data of the captured image in the system memory 310.

[0058] Accelerometer 130 detects the orientation of electronic device 1 relative to the direction of gravity and outputs a detection signal indicating the detection result to EC200. For example, accelerometer 130 is respectively disposed in the first frame 10 and the second frame 20, detects the orientation of the first frame 10 and the second frame 20, and outputs a detection signal indicating the detection result to EC200. Based on the detection results of the orientation of the first frame 10 and the second frame 20, the opening and closing state of electronic device 1 and the opening angle θ between the first frame 10 and the second frame 20 can be detected. In addition, gyroscope sensors, tilt sensors, geomagnetic sensors, etc., can be added instead of accelerometer 130 or based on accelerometer 130.

[0059] The power button 140 outputs an operation signal to the EC200 based on the user's operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to receiving operations on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC200.

[0060] The image output terminal 160 is a connection terminal for connecting to an external display (display device). For example, the image output terminal 160 is an HDMI (registered trademark) terminal, a USB Type-C terminal, a monitor port, etc.

[0061] The power supply unit 400 supplies power to each component of the electronic device 1 via a power supply system for supplying power to each component, based on the operating status of each component. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of the DC power supplied from the AC (Alternate Current) / DC adapter or battery pack to the voltage required by each component. The power, after voltage conversion by the DC / DC converter, is supplied to each component via the respective power supply system. For example, the power supply unit 400 supplies power to each component via the respective power supply system based on control signals corresponding to the operating status of each component input from the EC200.

[0062] The EC200 is a microcomputer configured to include a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), flash memory, multiple channels of A / D input terminals, D / A output terminals, and digital input / output terminals. For example, the EC200's CPU reads a control program (firmware) pre-stored in the ROM, either internally or externally, and executes the read program to perform its functions. The EC200 is connected to an accelerometer 130, a power button 140, an input device 150, a face detection unit 210, a system processing unit 300, and a power supply unit 400.

[0063] For example, if EC200 receives an operation signal corresponding to the user's operation of the power button 140, it instructs the system processing unit 300 to start the system. Furthermore, based on the detection results from the face detection unit 210, EC200 instructs the system to start and to transition between operating states. Additionally, EC200 communicates with the power supply unit 400 to obtain information about the battery status (remaining capacity, etc.) and outputs control signals to the power supply unit 400 to control the power supply corresponding to the operating states of each part of the electronic device 1.

[0064] In addition, EC200 acquires operation signals from input devices such as input device 150, and outputs the operation signals required for processing by system processing unit 300 from the acquired operation signals to system processing unit 300. Furthermore, EC200 acquires detection signals from acceleration sensor 130, and based on the acquired detection signals, detects the direction of electronic device 1 (the direction of the first frame 10 and the second frame 20), the opening angle θ of the first frame 10 and the second frame 20, etc.

[0065] In addition, part of the EC200's functionality can also be configured as a sensor hub or chipset.

[0066] The face detection unit 210 is a processor that processes image data from images captured by the camera 120. For example, the face detection unit 210 performs face detection processing, which acquires images captured by the camera 120 and detects facial regions of the captured face from the acquired images. Additionally, the face detection unit 210 performs HPD processing, which detects the presence of a user (person) in front of the electronic device 1 based on the detection results of the face detection processing and outputs the detection results.

[0067] The system processing unit 300 is configured to include: a CPU (Central Processing Unit) 302, a GPU (Graphics Processing Unit) 304, a memory controller 306, an I / O (Input-Output) controller 308, and system memory 310. It is capable of executing various applications on the OS through OS-based system processing. Sometimes, the CPU 302 and GPU 304 are collectively referred to as the processor.

[0068] CPU 302 performs OS-based processing and processing based on applications operating on the OS. Furthermore, CPU 302 transitions the system's operating state via instructions from EC 200. For example, if the operating state is a stopped or standby state, and a start instruction is received from EC 200, CPU 302 performs start-up processing to transition from the stopped or standby state to the normal operating state. Conversely, if CPU 302 receives an instruction to switch to standby while in the normal operating state, it transitions to standby. Finally, if CPU 302 receives a shutdown instruction while in the normal operating state, it performs shutdown processing to transition from the normal operating state to the stopped state.

[0069] Additionally, during the startup process, CPU 302 performs a login process to determine whether OS access is permitted. If CPU 302 initiates OS-based startup, it performs the login process before granting OS access, and temporarily suspends the transition to the normal operating state until login is granted during the login process. During the login process, user authentication is performed to determine whether the person using electronic device 1 is a pre-registered legitimate user. Authentication methods include password authentication, facial recognition, and fingerprint authentication.

[0070] If authentication is successful, CPU302 allows login and resumes the temporarily suspended system processing. Conversely, if authentication fails, login is not allowed, and the suspended system processing remains in place.

[0071] GPU 304 is connected to display unit 110. GPU 304 performs image processing to generate display data based on the control of CPU 302. GPU 304 outputs the generated display data to display unit 110. Furthermore, CPU 302 and GPU 304 can be integrated into a single core, or the load can be shared among the individual CPU 302 and GPU 304 cores. The number of processors is not limited to one; multiple processors are also possible.

[0072] The memory controller 306 controls the CPU 302 and GPU 304 to read and write data from the system memory 310, storage unit 360, etc.

[0073] I / O controller 308 controls the input and output of data from communication unit 350, display unit 110 and EC200.

[0074] System memory 310 serves as the area for reading the processor's executable program and the area for writing processing data. Additionally, system memory 310 temporarily stores image data of images captured by camera 120.

[0075] The communication unit 350 connects to other devices via a wireless or wired communication network to transmit and receive various types of data. For example, the communication unit 350 is configured to include a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark).

[0076] The storage unit 360 is configured to include storage media such as HDD (Hard Disk Drive), SDD (Solid State Drive), ROM, and flash memory. In addition to various programs such as the OS, device drivers, and applications, the storage unit 360 also stores various data acquired through program actions.

[0077] In addition, the system processing unit 300 can be packaged as a SOC (System on a chip), or some of its functions can be configured as chipsets or other components such as sensor hubs.

[0078] [Functional Structure]

[0079] Next, the functional structure of the electronic device 1 controlling the operating state of the system via HPD processing will be explained. Furthermore, in the following explanation, the functional structure of the process that performs the transition from the normal operating state to the standby state (hereinafter referred to as "sleep processing") will be described in detail.

[0080] Figure 7This is a block diagram illustrating an example of the functional structure of the electronic device 1 according to this embodiment. The electronic device 1 includes a face detection unit 210, a motion control unit 220, and a sleep processing unit 320. The face detection unit 210 and... Figure 6 This corresponds to the face detection unit 210 shown. The motion control unit 220 is via... Figure 6 The EC200 shown is functionally structured through an execution control program. The sleep processing unit 320 is... Figure 6 The system processing unit 300 shown is a functional structure implemented by executing OS programs.

[0081] The face detection unit 210 includes a face detection processing unit 211, an HPD processing unit 212, and a camera state determination unit 213. The face detection processing unit 211 reads image data of images captured by the camera 120 at predetermined time intervals from the system memory 310, and performs image processing and image analysis on each captured image at predetermined time intervals.

[0082] For example, the face detection processing unit 211 detects facial regions from each captured image at predetermined time intervals. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects faces based on facial feature information, learning data (learned model) based on facial feature information obtained through machine learning, or a face detection library. Furthermore, the predetermined time interval can be set to, for example, a 15-second interval or a 10-second interval, but can be any arbitrary time interval. Moreover, in the case of the shortest time interval, detection is performed on a unit of all consecutive frames. The face detection processing unit 211 detects facial regions from each captured image and outputs coordinate information of the detected facial regions.

[0083] HPD processing unit 212 determines whether a user is present in front of electronic device 1 based on whether face detection processing unit 211 detects a facial region in the captured image. For example, if face detection processing unit 211 detects a facial region in the captured image, HPD processing unit 212 determines that a user is present in front of electronic device 1. On the other hand, if face detection processing unit 211 does not detect a facial region in the captured image, HPD processing unit 212 determines that no user is present in front of electronic device 1. Furthermore, HPD processing unit 212 outputs HPD information based on the determination result of whether a user is present in front of electronic device 1.

[0084] For example, when the determination result changes from a state where there is no user in front of the electronic device 1 to a state where there is a user, the HPD processing unit 212 outputs HPD information indicating that the user is approaching the electronic device 1 (hereinafter referred to as "Approach information"). Additionally, during the period when it is determined that there is a user in front of the electronic device 1, the HPD processing unit 212 outputs HPD information indicating that there is a user in front of the electronic device 1 (hereinafter referred to as "Presence information"). Furthermore, when the detection state changes from a state where there is a user in front of the electronic device 1 to a state where there is no user, the HPD processing unit 212 outputs HPD information indicating that the user is leaving the electronic device 1 (hereinafter referred to as "Leave information"). Based on the detection results of the face region by the face detection processing unit 211, the HPD processing unit 212 outputs approach information, presence information, or leave information to the motion control unit 220.

[0085] The camera state determination unit 213 determines whether the shooting direction of the camera 120 is obstructed based on each captured image taken by the camera 120 at predetermined time intervals. The state where the shooting direction of the camera 120 is obstructed refers to, for example... Figure 4 As shown, the camera shutter is closed at 120 seconds, or as... Figure 5 As shown, the shooting direction of camera 120 is obstructed by the user's hand, etc.

[0086] For example, the camera state determination unit 213 determines whether an image area with a predetermined proportion (e.g., 90%) or more is a black area for each captured image by the camera 120. It reads image data of captured images taken by the camera 120 at predetermined time intervals from the system memory 310 and determines whether a predetermined proportion (e.g., 90%) or more of the read image data is black. For example, the camera state determination unit 213 determines pixels in the captured image whose brightness value is less than a predetermined value as black. As an example, in the 256 grayscale levels of brightness value 0 (black) to brightness value 255 (white), if the brightness value is less than 10, the camera state determination unit 213 determines it to be black.

[0087] Furthermore, if the camera state determination unit 213 determines that a predetermined proportion or more of the captured image is black, it determines whether the state of blackness in the captured image continues for a predetermined time (e.g., 10 seconds). Moreover, if the camera state determination unit 213 determines that the state of blackness in the captured image continues for the predetermined time, it determines that the shooting direction of the camera 120 is obstructed. On the other hand, if the camera state determination unit 213 determines that a predetermined proportion or more of the captured image is not black, or if the state of blackness in the captured image does not continue for the predetermined time, it determines that the shooting direction of the camera 120 is not obstructed. The camera state determination unit 213 outputs information indicating whether the shooting direction of the camera 120 is obstructed (hereinafter referred to as "camera state information") to the motion control unit 220.

[0088] Based on the detection results from the face detection unit 210, the motion control unit 220 instructs the system processing unit 300 to control the motion state of the system. For example, the motion control unit 220 acquires proximity information, presence information, or departure information output from the face detection unit 210 (HPD processing unit 212). In addition, the motion control unit 220 acquires camera status information output from the face detection unit 210 (camera status determination unit 213).

[0089] For example, when the motion control unit 220 receives proximity information in standby mode, it instructs the system to transition from standby mode to normal operation mode. For example, the motion control unit 220 instructs the system processing unit 300 to start the system. More specifically, when starting the system, the motion control unit 220 outputs a control signal to the power supply unit 400 to supply the power required for the operation of each component of the electronic device 1. Then, the motion control unit 220 outputs a start signal to the system processing unit 300 to instruct the system to start. If the system processing unit 300 receives the start signal, it starts the system and transitions it from standby mode to normal operation mode.

[0090] Additionally, the motion control unit 220 includes an HPD information output unit 222. In normal operation, the HPD information output unit 222 switches between a face detection enabled mode and a face detection disabled mode based on camera state information obtained from the face detection unit 210 (camera state determination unit 213). For example, based on the camera state information, the HPD information output unit 222 executes the face detection enabled mode when the shooting direction of the camera 120 is not obstructed. On the other hand, based on the camera state information, the HPD information output unit 222 executes the face detection disabled mode when the shooting direction of the camera 120 is obstructed.

[0091] In the face detection effective mode, when the face detection unit 222 obtains presence information from the face detection unit 210, it outputs the presence information as HPD information to the system processing unit 300; when it obtains departure information from the face detection unit 210, it outputs the departure information as HPD information to the system processing unit 300. That is, in the face detection effective mode, when the face detection unit 210 detects a face region from the captured image, the HPD information output unit 222 outputs presence information; when no face region is detected, it outputs departure information.

[0092] On the other hand, in the face detection invalid mode, the HPD information output unit 222 will output presence information to the system processing unit 300 regardless of the HPD information output from the face detection unit 210. That is, in the face detection invalid mode, the HPD information output unit 222 will output presence information regardless of the detection of the face region in the captured image.

[0093] The sleep processing unit 320 performs sleep processing to transition the system from a normal operating state to a standby state. The sleep processing unit 320 includes a timing unit 321, an HPD information acquisition unit 322, and a sleep control unit 323.

[0094] The timing unit 321 is configured to include a timer that measures the elapsed time since the last operation input during normal operation. The timer in the timing unit 321 is reset each time a user operation input is detected. The user operation input is, for example, an operation input based on the user's operation on the input device 150.

[0095] HPD information acquisition unit 322 acquires HPD information output from motion control unit 220 (HPD information output unit 222). For example, in normal operation, HPD information acquisition unit 322 acquires presence information or departure information from motion control unit 220.

[0096] In normal operation, if there is no user input for a certain period of time, the sleep control unit 323 transitions the system from normal operation to standby mode, as a function of the OS. For example, the sleep control unit 323 obtains information from the EC200 regarding the presence or absence of operation on the input device 150. If there is operation on the input device 150, the timer of the timing unit 321 is reset. Furthermore, the sleep control unit 323 determines whether the elapsed time measured by the timing unit 321 has reached a preset sleep time. If the sleep time has been reached, it is determined that there has been no user input for a certain period of time, and the system transitions from normal operation to standby mode. The sleep time is set to, for example, 5 minutes. In addition, the sleep time can also be set by the user to any time.

[0097] Furthermore, when the HPD information acquisition unit 322 acquires departure information, the sleep control unit 323, in response to acquiring the departure information, transitions the system from the normal operation state to the standby state. That is, when the HPD information acquisition unit 322 acquires departure information, the sleep control unit 323 does not wait for a certain period of time in a state without user input, but transitions the system from the normal operation state to the standby state. On the other hand, during the period when the HPD information acquisition unit 322 acquires presence information, the sleep control unit 323 continues the normal operation state.

[0098] Therefore, when the shooting direction of the camera 120 is not obstructed, if no facial area is detected in the captured image, the electronic device 1 will switch the system from the normal operation state to the standby state. When the shooting direction of the camera 120 is obstructed and no facial area is detected in the captured image, the system can continue to operate normally without switching to the standby state.

[0099] Furthermore, the sleep control unit 323 can also transition the system from normal operation to standby mode if the HPD information acquisition unit 322 acquires the departure information and the HPD information acquisition unit 322 continues to acquire the departure information for a predetermined time (e.g., 30 seconds). In other words, even if the HPD information acquisition unit 322 acquires the departure information, the sleep control unit 323 can continue normal operation if the HPD information acquisition unit 322 acquires the presence information until the predetermined time has elapsed. Therefore, the electronic device 1 continues normal operation even when the user briefly leaves and immediately returns, thus preventing the system from transitioning to standby mode without the user's intention to interrupt use, providing excellent convenience.

[0100] [HPD processing actions]

[0101] Next, refer to Figure 8 The operation of HPD processing, in which the EC200 (motion control unit 220) switches between face detection effective mode and face detection ineffective mode and outputs HPD information in normal operation state, is explained.

[0102] Figure 8 This is a flowchart illustrating an example of HPD processing according to this embodiment.

[0103] (Step S101) EC200 determines whether the shooting direction of camera 120 is obstructed based on the camera state information obtained from face detection unit 210. If EC200 determines that the shooting direction of camera 120 is not obstructed (No), it proceeds to step S103. On the other hand, EC200 determines whether the shooting direction of camera 120 is obstructed based on the camera state information obtained from face detection unit 210. If EC200 determines that the shooting direction of camera 120 is obstructed (Yes), it proceeds to step S111.

[0104] (Step S103) EC200 is set to face detection effective mode, and proceed to step S105.

[0105] (Step S105) In the face detection active mode, the EC200 outputs presence information or departure information to the system processing unit 300 based on the face region detection results performed by the face detection unit 210. Specifically, if the EC200 obtains presence information from the face detection unit 210, it outputs presence information to the system processing unit 300; if it obtains departure information, it outputs departure information to the system processing unit 300. Then, it returns to the processing in step S101 and repeats the HPD processing.

[0106] (Step S111) EC200 is set to face detection invalid mode, and proceed to step S113.

[0107] (Step S113) In the face detection invalid mode, EC200 outputs presence information to the system processing unit 300. Then, the process proceeds to step S115.

[0108] (Step S115) After the EC200 idles for a specified time (e.g., 1 second), it returns to the processing in step S101 and repeats the HPD processing. In the face detection invalid mode, since the results of face detection processing are not reflected in the HPD processing, the processing cycle is extended compared to the face detection valid mode. This reduces power consumption. Furthermore, in the face detection invalid mode, the detection frame rate can be set lower compared to the face detection valid mode.

[0109] [Sleep processing actions]

[0110] Next, refer to Figure 9 The sleep processing operations performed by CPU 302 (sleep control unit 323) in its normal operating state will be explained. Figure 9 This is a flowchart illustrating an example of the sleep processing involved in this embodiment.

[0111] (Step S151) CPU 302 determines whether it has obtained presence information from EC200. If it determines that presence information has been obtained (Yes), CPU 302 proceeds to step S153. On the other hand, if it determines that presence information has not been obtained (No), CPU 302 proceeds to step S155.

[0112] (Step S153) During the period when the presence information is obtained, the CPU 302 determines whether a certain amount of time has elapsed since the user's last operation input. For example, the CPU 302 determines whether the elapsed time since the user's last operation input has reached a preset sleep time (e.g., 5 minutes), thereby determining whether a certain amount of time has elapsed since the user's last operation input. If the CPU 302 determines that a certain amount of time has elapsed since the user's last operation input (No), it returns to the processing in step S151. On the other hand, if the CPU 302 determines that a certain amount of time has elapsed since the user's last operation input (Yes), it determines that there has been no user operation input for a certain period of time, and causes the system to transition from the normal operation state to the standby state (Step S157).

[0113] (Step S155) CPU 302 determines whether it has obtained departure information from EC200. If CPU 302 determines that it has not obtained departure information (No), it returns to the processing in step S151. On the other hand, if CPU 302 determines that it has obtained departure information (Yes), it causes the system to transition from the normal operation state to the standby state (Step S157).

[0114] Furthermore, in step S155, if the CPU302 obtains departure information from the EC200 for a specified period of time (e.g., 30 seconds), it can also cause the system to transition from the normal operating state to the standby state.

[0115] [Summary of the First Implementation]

[0116] As described above, the electronic device 1 according to this embodiment includes: a system memory 310 (an example of a memory), a CPU 302 (an example of a first processor), a face detection unit 210 (an example of a second processor), and an EC 200 (an example of a third processor). The system memory 310 temporarily stores the program of an OS (an example of a system). The CPU 302 implements the functions of the system by executing the program of the OS stored in the system memory 310. The face detection unit 210 detects facial regions with faces captured from images (captured images) captured by the camera 120 (an example of a capturing unit). The EC 200 switches between executing a face detection valid mode (an example of a first process) that outputs presence information (an example of a first information) when a face region is detected by the face detection unit 210 from the captured image, and outputs departure information (an example of a second information) when no face region is detected, and a face detection invalid mode that outputs presence information regardless of the detection of the face region (an example of a second process). Furthermore, if there is no user input for a certain period of time, the CPU 302 restricts the use of at least a portion of the system's functions (e.g., transitions to a standby state), and if departure information is obtained from the EC 200, it restricts the use of at least a portion of the system's functions (e.g., transitions to a standby state) without waiting for the aforementioned period of time.

[0117] Thus, electronic device 1 can switch between a face detection active mode that transitions the system to a standby state based on face detection and a face detection inactive mode that transitions the system to a standby state regardless of face detection if there is no user input for a certain period of time. Therefore, it can appropriately control the action state according to the usage situation.

[0118] For example, the face detection unit 210 determines whether the shooting direction of the camera 120 is obstructed based on the captured image taken by the camera 120. If the face detection unit 210 determines that the shooting direction of the camera 120 is not obstructed, the EC200 executes the face detection valid mode; if it determines that the shooting direction of the camera 120 is obstructed, it executes the face detection invalid mode.

[0119] Therefore, when the camera shutter 120S is open, if the electronic device 1 does not detect a facial area in the captured image by the camera 120S, it can switch the system from the normal operating state to the standby state. Conversely, if the camera shutter 120S is closed and no facial area is detected in the captured image, the system can continue in the normal operating state without switching to the standby state. Thus, the electronic device 1 can appropriately control the operating state according to the usage situation.

[0120] Furthermore, there are methods that add a component to detect whether the camera shutter 120S is open or closed, but this increases cost and cannot detect whether a hand is covering the camera 120. According to this embodiment, since it is not necessary to add a component to detect the state of the camera shutter 120S, cost can be suppressed. In addition, according to this embodiment, the operation state can be appropriately controlled not only based on the state of the camera shutter 120S, but also based on whether a hand is covering the camera 120.

[0121] In addition, the EC200 repeatedly executes the effective face detection mode or the ineffective face detection mode, making the cycle of repeatedly executing the ineffective face detection mode longer than the cycle of repeatedly executing the effective face detection mode.

[0122] Therefore, electronic device 1 can reduce power consumption in face detection invalid mode compared to face detection valid mode.

[0123] Furthermore, the control method of the electronic device 1 according to this embodiment includes: a step in which the face detection unit 210 (an example of a second processor) detects a facial region containing a face in an image (captured image) captured by the camera 120 (an example of a capturing unit); a step in which the EC200 (an example of a third processor) switches between executing: a face detection valid mode (an example of a first process) in which the face detection unit 210 outputs presence information (an example of a first information) when a facial region is detected by the face detection unit 210 from the captured image, and outputs departure information (an example of a second information) when no facial region is detected; and a face detection invalid mode (an example of a second process) in which presence information is output regardless of the face region detected by the face detection unit 210; a step in which the system memory 310 (an example of a memory) restricts the use of at least a portion of the functions of the system (e.g., transitions to a standby state) when there is no user operation input for a certain period of time; and a step in which the system restricts the use of at least a portion of the functions of the system without waiting for the aforementioned certain period of time when departure information is obtained from the EC200 (e.g., transitions to a standby state).

[0124] Thus, electronic device 1 can switch between a face detection active mode that causes the system to switch to standby mode based on face detection and a face detection inactive mode that causes the system to switch to standby mode regardless of face detection if there is no user input for a certain period of time. Therefore, it can appropriately control the action state according to the usage situation.

[0125] <Second Implementation>

[0126] Next, the second embodiment of the present invention will be described.

[0127] In the first embodiment, an example of switching between a face detection effective mode and a face detection ineffective mode based on whether the shooting direction of the camera 120 is obstructed was described in the normal operating state. In contrast, in this embodiment, an example of switching between the face detection effective mode and the face detection ineffective mode based on the opening angle θ between the first frame 10 and the second frame 20 will be described in the normal operating state.

[0128] Depending on the opening angle θ between the first frame 10 and the second frame 20, sometimes the user's face cannot enter the field of view of the camera 120. Therefore, in this embodiment, when the opening angle θ between the first frame 10 and the second frame 20 is within a specified range (e.g., 70° to 135°), it is set to a face detection effective mode, and when the opening angle θ is outside the specified range, it is set to a face detection invalid mode.

[0129] Figure 10 This is a block diagram illustrating an example of the functional structure of the electronic device 1A according to this embodiment. Furthermore, the external structure and hardware structure of the electronic device 1A are similar to... Figure 2 as well as Figure 6 The electronic device 1 shown in the first embodiment has the same appearance and hardware structure.

[0130] Figure 10 The electronic device 1A shown includes a face detection unit 210A, a motion control unit 220A, and a sleep processing unit 320. The face detection unit 210A and... Figure 6 Corresponding to the face detection unit 210 shown, it includes a face detection processing unit 211 and an HPD processing unit 212. The face detection processing unit 211 and the HPD processing unit 212 are... Figure 7 The face detection processing unit 211 and HPD processing unit 212 shown in the first embodiment have the same structure.

[0131] Motion control unit 220A is through Figure 6 The EC200 shown has a functional structure implemented by executing control programs, including a frame status determination unit 221A and an HPD information output unit 222A.

[0132] The frame state determination unit 221A detects the opening angle θ (rotation angle) when the first frame 10 and the second frame 20 are rotated relative to each other based on the detection result of the acceleration sensor 130, and determines whether the detected opening angle θ is within a specified range (e.g., 70° to 135°).

[0133] In normal operation, the HPD information output unit 222A switches between a face detection effective mode and a face detection ineffective mode based on the determination result of the frame state determination unit 221A. For example, if the frame state determination unit 221A determines that the opening angle θ of the first frame 10 and the second frame 20 is within a specified range, the HPD information output unit 222A executes the face detection effective mode. On the other hand, if the frame state determination unit 221A determines that the opening angle θ of the first frame 10 and the second frame 20 is outside the specified range, the HPD information output unit 222A executes the face detection ineffective mode.

[0134] In the face detection effective mode, when the face detection unit 210A obtains presence information, it outputs the presence information as HPD information to the system processing unit 300; when the face detection unit 210A obtains departure information, it outputs the departure information as HPD information to the system processing unit 300. That is, in the face detection effective mode, when the face detection unit 210A detects a face region from the captured image, the HPD information output unit 222A outputs presence information; when no face region is detected, it outputs departure information.

[0135] On the other hand, in the face detection invalid mode, the HPD information output unit 222A will output presence information to the system processing unit 300 regardless of the HPD information output from the face detection unit 210A. That is, in the face detection invalid mode, the HPD information output unit 222A will output presence information regardless of the detection of the face region in the captured image.

[0136] Sleep processing unit 320 is through Figure 6 The system processing unit 300 shown implements a functional structure by executing OS programs, and... Figure 7 The sleep processing unit 320 shown in the first embodiment is the same.

[0137] Therefore, when the opening angle θ between the first frame 10 and the second frame 20 is within the specified range, if the electronic device 1A does not detect a facial area from the captured image, it will switch the system from the normal operating state to the standby state. If the opening angle θ between the first frame 10 and the second frame 20 is outside the specified range and no facial area is detected from the captured image, the system can continue in the normal operating state without switching to the standby state.

[0138] Next, refer to Figure 11 The operation of HPD processing, in which the EC200 (motion control unit 220A) according to this embodiment switches between face detection effective mode and face detection ineffective mode and outputs HPD information in normal operation state, will be described.

[0139] Figure 11 This is a flowchart illustrating an example of HPD processing according to this embodiment. Figure 11 In the middle, to and Figure 8 The processes shown are labeled with the same reference numerals, and their descriptions are omitted. Figure 11 The HPD processing shown is relative to Figure 8 The HPD processing shown differs only in step S101A.

[0140] (Step S101A) Based on the detection result of the accelerometer 130, the EC200 detects the opening angle θ (rotation angle) when the first frame 10 and the second frame 20 are rotated relative to each other, and determines whether the detected opening angle θ is within a specified range (e.g., 70° to 135°). If the EC200 determines that the opening angle θ is within the specified range (yes), it sets the mode to face detection valid mode (step S103). On the other hand, if the EC200 determines that the opening angle θ is outside the specified range (no), it sets the mode to face detection invalid mode (step S111). The following processing is the same as... Figure 8 The processing shown is the same.

[0141] Furthermore, the sleep processing actions and references involved in this embodiment Figure 9 The sleep processing operation involved in the first embodiment described herein is the same, and its description is omitted.

[0142] [Summary of the Second Implementation]

[0143] As described above, the electronic device 1A according to this embodiment includes: a first frame 10 having at least a camera 120 (an example of an imaging unit), a second frame 20 having at least an input device 150 (an example of an input unit), and a hinge mechanism 15 (an example of a rotation mechanism) connecting the first frame 10 and the second frame 20 so that they can rotate relative to each other. When the opening angle θ (rotation angle) when the first frame 10 and the second frame 20 are rotated relative to each other is within a predetermined range, the EC200 (an example of a third processor) executes a face detection valid mode (an example of a first process); when the opening angle θ is outside the predetermined range, it executes a face detection invalid mode (an example of a second process).

[0144] Therefore, when the opening angle θ between the first frame 10 and the second frame 20 is within a specified range, if the electronic device 1A does not detect a facial region in the captured image by the camera 120, it can switch the system from the normal operating state to the standby state. Conversely, if the opening angle θ between the first frame 10 and the second frame 20 is outside the specified range and no facial region is detected in the captured image, the system can continue in the normal operating state without switching to the standby state. Thus, the electronic device 1A can appropriately control the operating state according to the usage situation.

[0145] <Third Implementation Method>

[0146] Next, the third embodiment of the present invention will be described.

[0147] In this embodiment, an example of switching between a face detection effective mode and a face detection ineffective mode based on whether an external display is used will be described under normal operating conditions.

[0148] Figure 12 This is a block diagram illustrating an example of the functional structure of the electronic device 1B according to this embodiment. Furthermore, the external structure and hardware structure of the electronic device 1B are similar to... Figure 2 as well as Figure 6 The electronic device 1 shown in the first embodiment has the same appearance and hardware structure.

[0149] Electronic device 1B has an image output terminal 160 (see reference). Figure 6 The electronic device 1B displays images by outputting image data to an external display (an external display device) connected to the image output terminal 160. Alternatively, the electronic device 1B may also use the communication unit 350 (see reference 160). Figure 6 As an output interface, it outputs image data to an external display via wireless communication such as Wi-Fi (registered trademark), thereby displaying the image.

[0150] Since it is highly likely that the user using the electronic device 1B, whose image is displayed on an external display, is not face-to-face with the electronic device 1B, the user's face may not be reflected in the captured image by the camera 120, or even if it is reflected, the facial area may be difficult to detect due to factors such as a side profile. Therefore, in this embodiment, a face detection enabled mode is set when the image of the electronic device 1B is not displayed on the external display, and a face detection disabled mode is set when the image is displayed on the external display.

[0151] Figure 12The electronic device 1B shown includes a face detection unit 210B, a motion control unit 220B, and a sleep processing unit 320. The face detection unit 210B and... Figure 6 Corresponding to the face detection unit 210 shown, it includes a face detection processing unit 211 and an HPD processing unit 212. The face detection processing unit 211 and the HPD processing unit 212 are... Figure 7 The face detection processing unit 211 and HPD processing unit 212 shown in the first embodiment have the same structure.

[0152] Motion control unit 220B is through Figure 6 The EC200 shown has a functional structure implemented by executing control programs, including an external display determination unit 221B and an HPD information output unit 222B.

[0153] The external display determination unit 221B determines whether to display the image of the electronic device 1B on an external display. The image of the electronic device 1B is an image of the system function executed by the CPU 302, and corresponds to the image displayed on the display unit 110. For example, through the display settings of the OS, when an external display is connected, it is possible to set whether to display on the display unit 110 (internal display) and the external display (or both). For example, the external display determination unit 221B obtains the display setting information of the OS to determine whether to display the image of the electronic device 1B on the external display.

[0154] In normal operation, the HPD information output unit 222B switches between a face detection valid mode and a face detection invalid mode based on the determination result of the external display determination unit 221B. For example, if the external display determination unit 221B determines that the image of the electronic device 1B should not be displayed on the external display, the HPD information output unit 222B executes the face detection valid mode. On the other hand, if it is determined that the image of the electronic device 1B should be displayed on the external display, the HPD information output unit 222B executes the face detection invalid mode.

[0155] In the face detection effective mode, when the face detection unit 222B obtains presence information from the face detection unit 210B, it outputs the presence information as HPD information to the system processing unit 300; when it obtains departure information from the face detection unit 210B, it outputs the departure information as HPD information to the system processing unit 300. That is, in the face detection effective mode, when the face detection unit 210B detects a face region from the captured image, the HPD information output unit 222B outputs presence information; when no face region is detected, it outputs departure information.

[0156] On the other hand, in the face detection invalid mode, the HPD information output unit 222B will output presence information to the system processing unit 300 regardless of the HPD information output from the face detection unit 210B. That is, in the face detection invalid mode, the HPD information output unit 222B will output presence information regardless of the detection of the face region in the captured image.

[0157] Sleep processing unit 320 is through Figure 6 The system processing unit 300 shown implements a functional structure by executing OS programs, and... Figure 7 The sleep processing unit 320 shown in the first embodiment is the same.

[0158] Therefore, if the image of the electronic device 1B is not displayed on an external display (i.e., displayed on the display unit 110), and no facial area is detected from the captured image, the system will switch from the normal operation state to the standby state. If the image of the electronic device 1B is displayed on an external display and no facial area is detected from the captured image, the normal operation state can continue without switching to the standby state.

[0159] Next, refer to Figure 13 The operation of HPD processing, in which the EC200 (motion control unit 220B) according to this embodiment switches between face detection effective mode and face detection ineffective mode and outputs HPD information in normal operation state, will be described.

[0160] Figure 13 This is a flowchart illustrating an example of HPD processing according to this embodiment. Figure 13 In the middle, to and Figure 8 The processes shown are labeled with the same reference numerals, and their descriptions are omitted. Figure 13 The HPD processing shown is relative to Figure 8 The HPD processing shown differs only in step S101B.

[0161] (Step S101B) EC200 determines whether to display the image of electronic device 1B on an external display based on the display settings information of the OS. If EC200 determines that the image is not displayed on an external display (No), it sets the mode to face detection enabled (Step S103). On the other hand, if EC200 determines that the image is displayed on an external display (Yes), it sets the mode to face detection disabled (Step S111). The following processing is the same as... Figure 8 The processing shown is the same.

[0162] Furthermore, the sleep processing actions and references involved in this embodiment Figure 9The sleep processing operation involved in the first embodiment described herein is the same, and its description is omitted.

[0163] [Summary of the Third Implementation]

[0164] As described above, the electronic device 1B according to this embodiment includes an image output terminal 160 (an example of an output interface) that outputs image data for displaying images based on system functions on an external display. When the image is not displayed on the external display, the EC200 (an example of a third processor) executes a face detection enabled mode (an example of a first process), and when the image is displayed on the external display, it executes a face detection disabled mode (an example of a second process).

[0165] Therefore, when the image of the electronic device 1B is not displayed on an external display (i.e., displayed on the display unit 110), if no facial area is detected in the image captured by the camera 120, the system can transition from the normal operating state to the standby state. Conversely, when the image of the electronic device 1B is displayed on an external display and no facial area is detected in the captured image, the system can continue in the normal operating state without transitioning to the standby state. Thus, the electronic device 1B can appropriately control its operating state according to the usage situation.

[0166] The various embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the specific structure is not limited to the embodiments described above, and also includes designs that do not depart from the spirit of the present invention. For example, the structures described in the above embodiments can be arbitrarily combined.

[0167] Furthermore, in the above embodiment, the face detection units 210, 210A, and 210B detect facial regions containing faces from the captured images taken by the camera 120. However, even when a facial region is detected, the gaze of the eyes contained within the face region can be further detected. Moreover, the EC200 (motion control units 220, 220A, and 220B) can, in the face detection active mode (an example of the first processing), output presence information (an example of the first information) when a facial region is detected from the captured images taken by the camera 120 and the direction of the gaze is within a predetermined range, and output departure information (an example of the second information) when a facial region is detected and the direction of the gaze is outside the predetermined range, or when no facial region is detected. Here, the predetermined range in the direction of the gaze is preset to a range that allows it to be determined that the user is looking at the electronic devices 1, 1A, and 1B (i.e., using the electronic devices 1, 1A, and 1B). That is, in the face detection valid mode, electronic devices 1, 1A, and 1B can control the system's operation state not only based on the presence or absence of the user (the presence or absence of face region detection) but also based on whether the user's gaze direction is within a specified range. On the other hand, in the face detection invalid mode (an example of the second process), electronic devices 1, 1A, and 1B output presence information (an example of the first information) regardless of whether the face region or the gaze direction is detected. Thus, when the system's operation state is controlled not only based on the presence or absence of the user (the presence or absence of face region detection) but also based on whether the user's gaze direction is within a specified range, the structure of switching between the face detection valid mode and the face detection invalid mode in the above embodiments can also be applied.

[0168] Furthermore, in the above embodiments, a structural example of having a camera 120 built into the electronic device 1 (1A, 1B) has been described, but it is not limited to this. For example, the camera 120 may not be built into the electronic device 1 (1A, 1B), or it may be configured to be able to be installed as an external accessory of the electronic device 1 (1A, 1B) (for example, on any side of side 10a, 10b, 10c, etc.), and communicate with the electronic device 1 (1A, 1B) wirelessly or via wired connection.

[0169] Furthermore, in the above embodiments, electronic devices 1 (1A, 1B) detect the presence of a user by detecting a facial region captured in an image, but are not limited to a face; they can also detect the presence of a user by detecting an area captured of at least a part of a body. Additionally, electronic devices 1 (1A, 1B) can also use a distance sensor (e.g., a proximity sensor) that detects the distance to an object. For example, a distance sensor is provided on the inner surface of the first frame 10 to detect objects (e.g., people) present within a detection range in a direction facing (forward) to the inner surface of the first frame 10. As an example, the distance sensor can be an infrared distance sensor, configured to include a light-emitting part that emits infrared light and a light-receiving part that receives reflected light from the surface of an object. Furthermore, the distance sensor can also be a sensor that uses infrared light emitted by a light-emitting diode (LED), or a sensor that uses an infrared laser that emits light with a wavelength narrower than that emitted by an LED. Furthermore, the distance sensor is not limited to an infrared distance sensor; it can be any sensor that detects the distance to an object, such as an ultrasonic sensor or a sensor using UWB (Ultra Wide Band) radar, or other types of sensors. Additionally, the distance sensor may not be built into the electronic device 1 (1A, 1B), or it can be configured to be mounted as an external accessory to the electronic device 1 (1A, 1B) (e.g., any side of side 10a, 10b, 10c, etc.), communicating with the electronic device 1 wirelessly or via a wired connection. Furthermore, the camera 120 and the distance sensor can also be integrated into one unit.

[0170] Furthermore, in the above embodiment, an example is shown where the face detection unit 210 is separate from the EC200. However, it is also possible for the EC200 to have a portion or all of the face detection unit 210, and the face detection unit 210 and the EC200 may be housed in a single package. Alternatively, the system processing unit 300 may have a portion or all of the face detection unit 210, and the face detection unit 210 and the system processing unit 300 may be housed in a single package. Additionally, the motion control unit 220 may be configured as a functional unit other than the EC200 (e.g., the system processing unit 300).

[0171] Furthermore, the aforementioned electronic devices 1 (1A, 1B) have an internal computer system. Moreover, programs for implementing the functions of each structure of the aforementioned electronic devices 1 (1A, 1B) can be recorded on a computer-readable recording medium. The processing within each structure of the aforementioned electronic devices 1 (1A, 1B) is performed by having the computer system read and execute the program recorded on the recording medium. Here, "having the computer system read and execute the program recorded on the recording medium" includes the computer system installation program. The term "computer system" here includes hardware such as an operating system and peripheral devices. Additionally, a "computer system" may also include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, "computer-readable recording medium" refers to portable media such as floppy disks, optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into the computer system. Thus, the recording medium storing the program can also be a non-temporary recording medium such as a CD-ROM.

[0172] Furthermore, the recording medium may also include internal or external recording media accessible from a distribution server for distributing the program. Additionally, the program may be divided into multiple parts, each downloaded at different time intervals, and then assembled from various components of electronic device 1 (1A, 1B), with each part distributed via a different distribution server. Moreover, the term "computer-readable recording medium" also includes structures that retain the program for a certain period, such as a server in the case of sending the program via a network, or volatile memory (RAM) within a computer system acting as a client. Furthermore, the aforementioned program may be part of a structure used to implement the functions described above. Further, it may be a so-called differential file (differential program) that can be implemented by combining the aforementioned functions with a program already recorded in the computer system.

[0173] Alternatively, some or all of the functions of the electronic devices 1 (1A, 1B) in the above embodiments can be implemented as integrated circuits such as LSI (Large Scale Integration). Each function can be processed individually, or some or all can be integrated for processing. Furthermore, the method of integrated circuit implementation is not limited to LSI; it can also be implemented using dedicated circuits or general-purpose processors. Additionally, if advancements in semiconductor technology lead to integrated circuit technologies that replace LSI, integrated circuits based on such technologies can also be used.

[0174] Furthermore, electronic device 1 is not limited to a laptop PC, but can also be a desktop PC, tablet computer, smartphone, etc. Moreover, electronic device 1 is not limited to PCs, tablet computers, smartphones, etc., and can also be applied to home appliances and commercial appliances. As a home appliance, it can be applied to televisions, refrigerators with displays, microwave ovens, etc. For example, it can control the opening / closing of the television screen, or the opening / closing of the display screen of a refrigerator, microwave oven, etc., based on the approach or departure of a person. Additionally, as a commercial appliance, it can be applied to vending machines, multimedia terminals, etc. For example, it can control the operation state based on the approach or departure of a person, such as turning the lights on / off of a vending machine, or turning the display screen on / off of a multimedia terminal.

Claims

1. An electronic device comprising: Memory, which temporarily stores the system's programs; The first processor executes the above program to realize the system's functions; The second processor detects facial regions containing faces from images captured by the camera; and The third processor switches between executing the first and second processes, where... In the first process described above, first information is output if the facial region is detected by the second processor, and second information is output if the facial region is not detected. In the second process described above, the first information is output regardless of how the facial region is detected by the second processor. If there is no user input for a certain period of time, the first processor restricts the use of at least a portion of the system's functions; and if the second information is obtained from the third processor, the processor restricts the use of at least a portion of the system's functions without waiting for the specified time. The second processor determines whether the shooting direction of the camera unit is obstructed based on the image captured by the camera unit. If the second processor determines that the shooting direction of the camera unit is not obstructed, the third processor performs the first process; if the second processor determines that the shooting direction of the camera unit is obstructed, the third processor performs the second process.

2. The electronic device according to claim 1, wherein, have: The first frame has at least the aforementioned camera unit; The second frame has at least an input section; and A rotating mechanism connects the first frame and the second frame so that they can rotate relative to each other. When the rotation angle of the first frame and the second frame is rotated relative to each other is within a specified range, the third processor performs the first process; when the rotation angle is outside the specified range, it performs the second process.

3. The electronic device according to claim 1, wherein, It has an output interface that outputs image data for displaying images based on the functions of the above system on an external display. When the image is not displayed on the external display, the third processor performs the first process; when the image is displayed on the external display, it performs the second process.

4. The electronic device according to any one of claims 1 to 3, wherein, The third processor repeatedly executes the first process or the second process, and the cycle of repeatedly executing the second process is longer than the cycle of repeatedly executing the first process.

5. An electronic device comprising: The first frame must have at least a camera module; The second frame has at least an input section; A rotating mechanism connects the first frame and the second frame so that they can rotate relative to each other; Memory, which temporarily stores the system's programs; The first processor executes the above program to realize the system's functions; The second processor detects a facial region containing a face from an image captured by the aforementioned imaging unit, and, if the facial region is detected, detects the direction of the gaze of the eyes contained within the face in the aforementioned facial region. as well as The third processor switches between executing the first process and the second process. In the first process, if the second processor detects the facial region and the direction of the gaze is within a predetermined range, first information is output. If the second processor detects the facial region and the direction of the gaze is outside the predetermined range, or if the facial region is not detected, second information is output. In the second process, the first information is output regardless of the detection of the facial region and the direction of the gaze performed by the second processor. If there is no user input for a certain period of time, the first processor restricts the use of at least a portion of the system's functions; and if the second information is obtained from the third processor, the processor restricts the use of at least a portion of the system's functions without waiting for the specified time. When the rotation angle of the first frame and the second frame is rotated relative to each other is within a specified range, the third processor executes the first process; when the rotation angle is outside the specified range, the second process is executed.

6. The electronic device according to claim 5, wherein, The second processor determines whether the shooting direction of the camera unit is obstructed based on the image captured by the camera unit. If the second processor determines that the shooting direction of the camera unit is not obstructed, the third processor performs the first process; if the second processor determines that the shooting direction of the camera unit is obstructed, the third processor performs the second process.

7. The electronic device according to claim 5, wherein, It has an output interface that outputs image data for displaying images based on the functions of the above system on an external display. When the image is not displayed on the external display, the third processor performs the first process; when the image is displayed on the external display, it performs the second process.

8. The electronic device according to any one of claims 5 to 7, wherein, The third processor repeatedly executes the first process or the second process, and the cycle of repeatedly executing the second process is longer than the cycle of repeatedly executing the first process.

9. A control method for an electronic device, the electronic device comprising: a memory for temporarily storing a system program; a first processor, a second processor, and a third processor for executing the program to implement the system's functions. The above control methods include: The step described above, where the second processor detects a facial region containing a face from an image captured by the camera; The third processor switches between executing the first processing and the second processing steps. In the first processing, first information is output when the second processor detects the facial region, and second information is output when the facial region is not detected. In the second processing, the first information is output regardless of how the second processor detects the facial region. In the absence of user input for a certain period of time, the first processor restricts the use of at least a portion of the functions of the system. When the second information is obtained from the third processor, the first processor does not wait for the specified time and restricts the use of at least a portion of the system's functions. The second processor determines whether the shooting direction of the camera unit is blocked based on the image captured by the camera unit. as well as If the second processor determines that the shooting direction of the camera unit is not obstructed, the third processor performs the first process; if it determines that the shooting direction of the camera unit is obstructed, it performs the second process.

10. A control method for an electronic device, the electronic device comprising: a first frame having at least a camera unit, a second frame having at least an input unit, a rotation mechanism connecting the first frame and the second frame to be rotatable relative to each other, a memory temporarily storing a program of the system, a first processor, a second processor, and a third processor for implementing the functions of the system by executing the program. The above control methods include: The second processor detects a facial region containing a face from an image captured by the imaging unit, and, if the facial region is detected, detects the direction of the gaze of the eyes contained within the face region. The third processor switches between executing the first processing and the second processing steps. In the first processing, if the second processor detects the facial region and the direction of the gaze is within a specified range, first information is output. If the second processor detects the facial region and the direction of the gaze is outside the specified range, or if the facial region is not detected, second information is output. In the second processing, the first information is output regardless of how the second processor detects the facial region and the direction of the gaze. In the absence of user input for a certain period of time, the first processor restricts the use of at least a portion of the functions of the system. Upon receiving the second information from the third processor, the first processor does not wait for the specified time before restricting the use of at least a portion of the system's functionality; and When the rotation angle of the first frame and the second frame is rotated relative to each other is within a specified range, the third processor performs the first process; when the rotation angle is outside the specified range, it performs the second process.

Images