Information processing apparatus and control method

Abstract

To improve responsiveness when detecting the direction of a face and controlling the screen luminance of a display part.SOLUTION: The information processing device includes a memory that temporarily stores image data of an image captured by the imaging unit, and a processor that processes the image data stored in the memory. The processor processes image data of an image stored in a memory to perform face detection processing for detecting a face area where a face is imaged and a direction of the face from the image, and luminance control processing for controlling screen luminance of a display unit on the basis of the direction of the face detected by the face detection processing. Further, in the face detection process, the processor determines that the detected face is oriented in the first direction when the detected face is oriented in a preset first angle range, and determines that the detected face is oriented in the first direction before the detected face is oriented in the first angle range, based on an amount of change of the detected face toward the first direction and a moving distance of the face, even if the detected face is oriented in a direction outside the first angle range.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an information processing device and a control method. [Background technology]

[0002] There is a device that transitions to a usable state when a person approaches, and transitions to a standby state in which all functions are stopped except for some functions when the person leaves. For example, Patent Document 1 uses an infrared sensor to detect whether a person is approaching or whether the person has moved away.

[0003] In recent years, advances in computer vision and other technologies have led to improved accuracy in detecting faces from images. As a result, instead of detecting people using infrared sensors, face detection is now being used to detect people. Face detection not only detects people but also their facial orientation, making it possible to control the device based on the orientation of the face (whether the person is facing forward, facing sideways, etc.). For example, when the person's face is not facing forward (toward the device), the brightness of the display screen is reduced to save power. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-148895 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, if the screen brightness is lowered when the user's face is not facing forward (e.g., when the user is facing sideways), it is desirable to restore the screen brightness when the user faces forward again. However, when the user's face changes from sideways to facing forward, it takes time (e.g., about one second) to detect that the user is now facing forward, and the responsiveness of restoring the screen brightness after the user faces forward is poor. For this reason, it is difficult to proactively implement control to lower the screen brightness according to the user's face direction, and in some cases this does not contribute sufficiently to power saving.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objects is to provide an information processing device and a control method that improve responsiveness when detecting the direction of a face and controlling the screen brightness of a display unit. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and an information processing device according to a first aspect of the present invention comprises a memory that temporarily stores image data of an image captured by an imaging unit, and a processor that processes the image data stored in the memory, and the processor processes the image data of the image stored in the memory to perform a face detection process that detects a face area in the image where a face is captured and the direction of the face, and a brightness control process that controls the screen brightness of a display unit based on the direction of the face detected by the face detection process, and the processor determines that the direction of the detected face is a first direction when it is within a predetermined first angle range in the face detection process, and even if the direction of the detected face is outside the first angle range, it determines that the direction of the detected face is the first direction before it enters the first angle range based on the amount of change in the direction of the first direction and the distance the face moves.

[0008] In the above information processing device, in the face detection process, even if the orientation of the detected face is outside the first angle range, if the orientation of the detected face changes by more than a predetermined angle from a second orientation outside the first angle range toward the first orientation, the processor may determine whether the distance the face moved when the orientation of the face changed by more than the predetermined angle is less than a predetermined value, and if the distance the face moved is less than the predetermined value, determine that the orientation is the first orientation.

[0009] In the above information processing device, the processor may, in the face detection process, determine whether the distance the face has moved left and right is less than a predetermined value if the change in the direction of the face is a change in yaw angle, and determine whether the distance the face has moved up and down is less than a predetermined value if the change in the direction of the face is a change in pitch angle.

[0010] In the above information processing device, the processor may determine that the detected orientation of the face is the first orientation in the face detection process, even if the orientation of the detected face is outside the first angle range, if the orientation of the detected face changes from the second orientation toward the first orientation by more than the specified angle within a specified time and the movement distance of the face is less than a specified value.

[0011] In the above information processing device, the processor may perform, in the face detection process, a process of determining whether or not the face is in the first orientation within a second angle range wider than the first angle range, and may perform a process of determining whether or not the face is in the first orientation outside the first angle range within the second angle range based on the amount of change in the orientation of the face toward the first orientation and the distance the face has moved.

[0012] In the information processing device, the processor may, in the face detection process, reduce the detection cycle when the orientation of the detected face is within the first angle range compared to the detection cycle when the orientation is outside the first angle range.

[0013] In the above information processing device, the processor may, in the face detection process, reduce the detection period when the orientation of the detected face is within the first angle range compared to the detection period when the orientation of the detected face is outside the first angle range within the second angle range.

[0014] In the above information processing device, the processor may reduce the screen brightness when the orientation of the face detected by the face detection process changes from within the first angle range to outside the first angle range during the brightness control process.

[0015] In the above information processing device, the processor may, in the brightness control process, when the screen brightness is reduced and it is determined that the orientation of the face detected by the face detection process is the first orientation, return the screen brightness to the level before it was reduced.

[0016] In the information processing device, the imaging unit may output image data of an image captured at a predetermined angle of view in a direction facing the screen of the display unit.

[0017] Furthermore, according to a second aspect of the present invention, a control method for an information processing device having a memory for temporarily storing image data of an image captured by an imaging unit and a processor for processing the image data stored in the memory includes a face detection step in which the processor processes the image data of the image stored in the memory to detect a face area in the image where a face is captured and the direction of the face, and a brightness control step in which the screen brightness of the display unit is controlled based on the direction of the face detected by the face detection step, wherein in the face detection step, the detected direction of the face is determined to be a first direction if it is within a predetermined first angle range, and even if the detected direction of the face is outside the first angle range, the detected direction of the face is determined to be the first direction before it enters the first angle range based on the amount of change in the direction of the first direction and the distance the face moves. [Effects of the Invention]

[0018] According to the above aspects of the present invention, it is possible to improve the responsiveness when detecting the direction of the face and controlling the screen brightness of the display unit. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a perspective view showing an example of the external configuration of an information processing apparatus according to an embodiment. [Figure 2] FIG. 4 is a diagram showing an example of a person detection range of the information processing apparatus according to the embodiment. [Figure 3] 5A and 5B are schematic diagrams illustrating control of screen brightness depending on the direction of a face according to an embodiment. [Figure 4] FIG. 1 is a schematic block diagram showing an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 5] FIG. 1 is a schematic block diagram showing an example of the functional configuration of an information processing apparatus according to an embodiment. [Figure 6] 10A and 10B are diagrams showing an example of a face angle used in face direction determination processing according to the embodiment. [Figure 7] 5A and 5B are schematic diagrams showing an example of a detection state by face detection processing and a control state by screen brightness processing according to the embodiment. [Figure 8] 5A and 5B are schematic diagrams showing examples of face angle detection from a camera viewpoint according to the embodiment. [Figure 9] 10A and 10B are diagrams showing an example of erroneous detection of a face angle according to the embodiment. [Figure 10] 10A and 10B are diagrams showing an example of determination based on a moving distance of a face according to the embodiment; [Figure 11] 10 is a flowchart showing an example of basic face direction determination processing according to the embodiment. [Figure 12] 10 is a flowchart showing an example of an overall face direction determination process according to the embodiment. [Figure 13] 10 is a flowchart showing an example of a screen brightness control process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [overview] First, an overview of the information processing apparatus according to the embodiment will be described. FIG. 1 is a perspective view showing an example of the external configuration of an information processing device 1 according to this embodiment.

[0021] The information processing device 1 is, for example, a notebook (clamshell) type PC (Personal Computer). The information processing device 1 includes a first housing 10, a second housing 20, and a hinge mechanism 15. The first housing 10 and the second housing 20 are connected using the hinge mechanism 15. The first housing 10 is rotatable relative to the second housing 20 around a rotation axis defined by the hinge mechanism 15. The opening angle resulting from the rotation of the first housing 10 and the second housing 20 is shown as "θ".

[0022] The first housing 10 is also referred to as the A cover or display housing. The second housing 20 is also referred to as the C cover or system housing. In the following description, the sides of the first housing 10 and the second housing 20 that are provided with the hinge mechanism 15 are referred to as side 10c and 20c, respectively. The sides of the first housing 10 and the second housing 20 opposite side 10c and 20c are referred to as side 10a and 20a, respectively. In the illustration, the direction from side 20a toward side 20c is referred to as the "rear," and the direction from side 20c toward side 20a is referred to as the "front." The right and left sides of the rear are referred to as the "right" and "left," respectively. The left sides of the first housing 10 and the second housing 20 are referred to as side 10b and 20b, respectively, and the right sides are referred to as side 10d and 20d, respectively. Furthermore, the state in which the first housing 10 and the second housing 20 overlap and are completely closed (a state in which the opening angle θ=0°) is referred to as the "closed state." In the closed state, the surfaces of the first housing 10 and the second housing 20 that face each other are referred to as the "inner surfaces," and the surfaces opposite the inner surfaces are referred to as the "outer surfaces." Furthermore, the state in which the first housing 10 and the second housing 20 are open relative to the closed state is referred to as the "open state."

[0023] The external appearance of the information processing device 1 shown in FIG. 1 is an example of an open state. In the open state, the side surface 10a of the first housing 10 and the side surface 20a of the second housing 20 are separated. In the open state, the inner surfaces of the first housing 10 and the second housing 20 are exposed. The open state is one of the states in which a user uses the information processing device 1, and it is typically used with an opening angle θ of approximately 100 to 130°. The range of the opening angle θ that results in the open state can be determined arbitrarily depending on the range of angles that can be rotated by the hinge mechanism 15, etc.

[0024] A display unit 110 is provided on the inner surface of the first housing 10. The display unit 110 is configured to include a liquid crystal display (LCD), an organic electroluminescence (EL) display, or the like. An imaging unit 120 is provided in a peripheral region of the display unit 110 on the inner surface of the first housing 10. For example, the imaging unit 120 is disposed on the side surface 20a side of the peripheral region of the display unit 110. Note that the position where the imaging unit 120 is disposed is an example, and the imaging unit 120 may be disposed in another location as long as it can face in a direction facing the display screen of the display unit 110.

[0025] In the open state, the imaging unit 120 captures an image of a predetermined imaging range in the direction facing the display screen of the display unit 110 (i.e., in front of the information processing device 1). The predetermined imaging range is the range of the angle of view determined by the imaging element of the imaging unit 120 and an optical lens provided in front of the imaging surface of the imaging element. For example, the imaging unit 120 can capture an image including a person (user) present in front of (on the front side of) the information processing device 1.

[0026] A power button 140 is provided on the side surface 20b of the second housing 20. The power button 140 is an operator that allows the user to instruct turning the power on or off, transitioning from a standby state to a normal operating state, transitioning from the normal operating state to a standby state, etc. The normal operating state is an operating state of the system in which processing can be executed without any particular restrictions, and corresponds to, for example, the S0 state defined by ACPI (Advanced Configuration and Power Interface).

[0027] The standby state is a state in which at least a portion of system processing is restricted and in which power consumption is lower than in the normal operating state. For example, the standby state may be a standby state, a sleep state, or the like, and may be a state equivalent to Modern Standby in Windows (registered trademark) or the S3 state (sleep state) defined by ACPI. The standby state may also include a state in which at least the display of the display unit is turned off (screen off) or a state in which the screen is locked. The screen lock is a state in which a preset image (e.g., a screen lock image) is displayed on the display unit so that the contents of processing cannot be seen, and the device cannot be used until the lock is released (e.g., user authentication is performed).

[0028] Furthermore, a keyboard 151 and a touchpad 153 are provided on the inner surface of the second housing 20 as input devices that accept user operation inputs. Note that, instead of or in addition to the keyboard 151 and the touchpad 153, a touch sensor may be provided as the input device, or a mouse or an external keyboard may be connected. In the case where a touch sensor is provided, an area corresponding to the display screen of the display unit 110 may be configured as a touch panel that accepts operations. The input device may also include a microphone that inputs voice.

[0029] In addition, when the first housing 10 and the second housing 20 are in a closed state, the display unit 110 and the imaging unit 120 provided on the inner surface of the first housing 10, and the keyboard 151 and the touchpad 153 provided on the inner surface of the second housing 20 are covered by the surface of the other housing, and are unable to function.

[0030] The information processing device 1 executes HPD (Human Presence Detection) processing to detect a person present in front of the information processing device 1 based on the captured image captured by the imaging unit 120.

[0031] 2 is a diagram showing an example of a person detection range of the information processing device 1 according to this embodiment. In the example shown, a detection range FoV (Field of View: detection field of view angle) in front of the information processing device 1 is a range in which a person can be detected.

[0032] For example, the information processing device 1 determines whether or not a person (user) is present in front of the information processing device 1 by detecting a face area in which a face is captured from a captured image captured by the imaging unit 120. The detection range FoV corresponds to the imaging field angle at which the information processing device 1 captures an image. If a face area is detected from the captured image, the information processing device 1 determines that a person (user) is present. On the other hand, if a face area is not detected from the captured image, the information processing device 1 determines that a person (user) is not present.

[0033] The information processing device 1 uses HPD processing to control the system operation state of the information processing device 1 depending on whether or not a person (user) is present. For example, the information processing device 1 is controlled to a normal operation state when a person (user) is present in front of the information processing device 1, and is controlled to a standby state when no person (user) is present in front of the information processing device 1.

[0034] Furthermore, when a person (user) is present in front of the information processing device 1, the information processing device 1 detects the face orientation of the person (user). For example, the information processing device 1 determines whether the face of the person (user) is facing the direction of the information processing device 1 (the direction of the display unit 110 and the imaging unit 120). The face orientation here refers to the orientation corresponding to the rotation angle of the face in the left-right direction. Hereinafter, a state in which the face is facing the direction of the information processing device 1 (the direction of the display unit 110 and the imaging unit 120) is referred to as a state in which the face is facing forward. Furthermore, a state in which the face is facing right or left with respect to the front is referred to as a state in which the face is facing sideways.

[0035] For example, the information processing device 1 controls (dimming control) the screen brightness of the display unit 110 depending on whether the face of the person (user) is facing forward or not. Specifically, when the face is not facing forward (for example, when the face is turned sideways), the information processing device 1 saves power by lowering the screen brightness of the display unit 110. Furthermore, when the face faces forward again, the information processing device 1 returns the screen brightness to its original level before lowering it.

[0036] Hereinafter, the original screen brightness before being reduced will be referred to as "standard brightness." Also, the screen brightness reduced from the standard brightness when the face is not facing forward (for example, when the face is turned sideways) will be referred to as "low brightness." Low brightness is at least brightness lower than the standard brightness, and the lower the brightness, the greater the power saving effect. For example, low brightness may be a brightness of about 0 to 10% of the standard brightness.

[0037] Here, when the face direction changes from sideways to facing forward, it takes time (for example, about one second) for the face detection process to detect that the face is facing forward. Therefore, if the information processing device 1 performs control to return to standard brightness after detecting that the face is facing forward, there will be a delay between the face facing forward and the return to standard brightness, resulting in poor responsiveness. Therefore, the information processing device 1 predicts that the face will face forward by detecting a change in the face direction from sideways to facing forward, and performs control to return to standard brightness before the face is completely facing forward.

[0038] 3 is a schematic diagram illustrating the control of screen brightness depending on the face orientation according to this embodiment. This diagram shows the control of screen brightness when the face of a person (user) in front of information processing device 1 changes from a sideways orientation to a forward orientation. (A) shows a state in which the face is facing sideways, and (B), (C), and (D) show a state in which the face orientation is gradually changed to a forward orientation, and (E) shows a state in which the face orientation is facing completely forward.

[0039] When the face is facing sideways as shown in (A), the screen brightness is controlled to low. The information processing device 1 determines that the face is facing forward based on the change in the face orientation from the sideways state shown in (A) to the front as shown in (B), and performs control to switch the screen brightness to standard brightness from (C) onwards before (E). As a result, although the screen brightness is controlled to low in (A) and (B), as the face orientation changes from sideways to frontwards, it is switched to standard brightness before the face orientation completely faces forward.

[0040] For example, the information processing device 1 determines whether the face is facing forward based on the amount of change in the face direction toward the front. Specifically, the information processing device 1 determines that the face is facing forward when the face has changed toward the front by a predetermined angle or more (e.g., 10° or more) within a predetermined time (e.g., within one second). This allows the information processing device 1 to perform control to switch to standard brightness before the face turns completely forward, thereby improving responsiveness.

[0041] The configuration of the information processing device 1 according to this embodiment will be described in detail below. [Hardware configuration of information processing device] Fig. 4 is a schematic block diagram showing an example of the hardware configuration of an information processing device 1 according to this embodiment. In Fig. 4, components corresponding to those in Fig. 1 are assigned the same reference numerals. The information processing device 1 includes a display unit 110, an imaging unit 120, a power button 140, an input device 150, a communication unit 160, a storage unit 170, an EC (Embedded Controller) 200, a face detection unit 210, a main processing unit 300, and a power supply unit 400.

[0042] The display unit 110 displays display data (images) generated based on system processing executed by the main processing unit 300 and processing of application programs running on the system processing.

[0043] The imaging unit 120 captures an image of an object within a predetermined imaging range (angle of view) in the direction facing the inner surface of the first housing 10 (forward), and outputs the captured image to the main processing unit 300 and the face detection unit 210. For example, the imaging unit 120 includes a visible light camera (RGB camera) that captures images using visible light, and an infrared camera (IR camera) that captures images using infrared light.

[0044] The imaging unit 120 may be configured to include either a visible light camera or an infrared camera, or may be configured to include both.

[0045] The power button 140 outputs an operation signal to the EC 200 in response to a user operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to accepting an operation on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC 200.

[0046] The communication unit 160 is communicably connected to other devices via a wireless or wired communication network, and transmits and receives various types of data. For example, the communication unit 160 is configured to include a wired LAN interface such as Ethernet (registered trademark), a wireless LAN interface such as Wi-Fi (registered trademark), etc.

[0047] The storage unit 170 includes storage media such as a hard disk drive (HDD), a solid state drive (SSD), RAM, and ROM. The storage unit 170 stores various programs such as an OS, device drivers, and applications, as well as various data acquired by the operation of the programs.

[0048] The power supply unit 400 supplies power to each unit of the information processing device 1 according to the operating state of each unit. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of direct current power supplied from an AC (Alternate Current) / DC adapter or a battery (battery pack) into the voltage required by each unit. The power whose voltage is converted by the DC / DC converter is supplied to each unit via each power supply system. For example, the power supply unit 400 supplies power to each unit via each power supply system based on a control signal input from the EC 200.

[0049] The EC200 is a microcomputer including a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and I / O (Input / Output) logic circuits. The CPU of the EC200 reads a control program (firmware) pre-stored in its own ROM, executes the read control program, and performs its functions. The EC200 operates independently of the main processing unit 300, controls the operation of the main processing unit 300, and manages its operating state. The EC200 is also connected to the power button 140, the input device 150, the power supply unit 400, and the like.

[0050] For example, the EC 200 communicates with the power supply unit 400 to acquire information on the battery status (such as remaining capacity) from the power supply unit 400, and outputs to the power supply unit 400 control signals and the like for controlling the supply of power according to the operating state of each unit of the information processing device 1. The EC 200 also acquires operation signals from the power button 140 and the input device 150, and outputs to the main processing unit 300 operation signals related to processing by the main processing unit 300 among the acquired operation signals.

[0051] The face detection unit 210 is configured to include a processor that processes image data of the captured image captured by the imaging unit 120. The face detection unit 210 acquires image data of the captured image captured by the imaging unit 120 and temporarily stores the acquired image data in memory. The memory that stores the image data may be the system memory 304 or a memory (not shown) within the face detection unit 210.

[0052] For example, the face detection unit 210 performs face detection processing to detect a face area from the captured image and detect the face orientation of the face image in the detected face area by processing image data of the captured image acquired from the imaging unit 120. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects a face based on facial feature information, or face detection using learning data (trained model) or a face detection library that has been machine-learned based on facial feature information.

[0053] In addition, in the face detection process, the face detection unit 210 performs face direction determination processing to determine whether or not the face is facing forward, as described with reference to Fig. 3. The face detection unit 210 determines whether or not the face is facing forward based on the face direction detected from the captured image, and transmits the determination result to the main processing unit 300. In addition, the face detection unit 210 controls the detection frame rate when performing the face detection process.

[0054] The main processing unit 300 is composed of a CPU (Central Processing Unit) 301, a GPU (Graphics Processing Unit) 302, a chipset 303, and a system memory 304, and is capable of executing processing of various application programs on an OS (Operating System) through system processing based on the OS.

[0055] The CPU 301 is a processor that executes processes based on BIOS programs, processes based on OS programs, and processes based on application programs that run on the OS. For example, the CPU 301 executes a startup process that starts the system from a standby state and transitions it to a normal operating state, and a sleep process that transitions it from the normal operating state to a standby state. The CPU 301 also executes a screen brightness control process that controls the screen brightness of the display unit 110 based on the results of the face detection process by the face detection unit 210 described above.

[0056] The GPU 302 is connected to the display unit 110. The GPU 302 generates display data by executing image processing under the control of the CPU 301. The GPU 302 outputs the generated display data to the display unit 110.

[0057] The chipset 303 has a function as a memory controller and a function as an I / O controller. For example, the chipset 303 controls the reading and writing of data from the system memory 304, the storage unit 170, etc. by the CPU 301 and the GPU 302. The chipset 303 also controls the input and output of data from the communication unit 160, the display unit 110, and the EC 200. The chipset 303 also has a function as a sensor hub. For example, the chipset 303 acquires detection results from the face detection process obtained from the face detection unit 210 and outputs the results to the CPU 301.

[0058] The system memory 304 is used as a reading area for programs executed by the CPU 301 and a working area for writing processing data, etc. The system memory 304 also temporarily stores image data of images captured by the imaging unit 120.

[0059] The CPU 301, GPU 302, and chipset 303 may be configured as a single integrated processor, or some or each may be configured as individual processors. For example, in a normal operating state, the CPU 301, GPU 302, and chipset 303 are all in operation, but in a standby state, only at least a part of the chipset 303 is in operation.

[0060] [Functional configuration of information processing device] Next, the functional configuration of the information processing device 1 that controls the screen brightness according to the direction of the face will be described in detail.

[0061] 5 is a schematic block diagram showing an example of the functional configuration of the information processing device 1 according to this embodiment. The information processing device 1 includes a face detection unit 210 and a system processing unit 310. The face detection unit 210 corresponds to the face detection unit 210 in FIG. 4, and includes a face area detection unit 211, a face angle detection unit 212, a face direction determination unit 213, and a detection cycle control unit 214 as functional components for performing face detection processing.

[0062] The face area detection unit 211 processes image data of the captured image acquired from the imaging unit 120 to detect a face area in which a face is captured from the captured image.

[0063] The face angle detection unit 212 detects the direction (face angle) of the face captured in the face area detected from the captured image by the face area detection unit 211. For example, the face angle detection unit 212 detects the face angle in a range of ±90°, with the face angle being 0° when the face is facing forward. The face direction determination unit 213 determines whether the face is facing forward based on the face angle detected by the face angle detection unit 212.

[0064] 6 is a diagram showing an example of a face angle used in the face direction determination process according to this embodiment. In this diagram, a face angle of 0° indicates that the face is facing completely forward, and a face angle of +90° or −90° indicates that the face is facing completely to the side. The face angle detection unit 212 detects the face angle within a range of ±90°, for example. This is because when the face angle is outside the range of ±90°, it becomes difficult to capture an image of facial feature points, making it impossible to correctly detect the face area.

[0065] The face direction determination unit 213 performs face direction determination processing to determine whether the face is facing forward based on the face angle detected by the face angle detection unit 212 within a second angle range (e.g., a range of ±70°), but does not perform the processing outside the second angle range (e.g., a range of ±70°). For example, when the face angle detected by the face angle detection unit 212 is within a first angle range (e.g., a range of ±50°) of the second angle range, the face direction determination unit 213 determines that the face is facing forward. Therefore, when the face angle detected by the face angle detection unit 212 is outside the first angle range (e.g., a range of ±50°) of the second angle range, the face direction determination unit 213 determines that the face is not facing forward (i.e., facing sideways).

[0066] Furthermore, even if the face angle detected by the face angle detection unit 212 is outside the first angle range (for example, a range of ±50°) (i.e., facing sideways), the face direction determination unit 213 determines that the face is facing forward (faced forward) before the detected face direction falls within the first angle range, based on the amount of change from sideways to forward. Specifically, the information processing device 1 predicts that the face will face forward if the face angle changes toward the front by a predetermined angle or more (for example, 10° or more) within a predetermined time (for example, within one second), determines that the face is facing forward before falling within the first angle range (for example, a range of ±50°), and sets the face angle to 0°.

[0067] In this way, the face direction determination unit 213 performs the face direction determination process described above in a second angle range (for example, a range of ±70°) that is wider than a first angle range (for example, a range of ±50°) in which it is determined that the face direction is facing forward based on the face angle. Outside the first angle range of the second angle range (for example, a range of "+50° to +70°" and a range of "-50° to -70°"), the face direction determination unit 213 performs face direction determination process to determine whether the face is facing forward or not based on the amount of change in the face direction from sideways to forward.

[0068] Furthermore, when the face direction determination unit 213 determines that the face is facing forward (facing forward), it outputs "Attention" information to the system processing unit 310 as information indicating that the user is looking at (paying attention to) the information processing device 1. On the other hand, when the face direction determination unit 213 determines that the face is not facing forward, it outputs "No Attention" information to the system processing unit 310 as information indicating that the user is not looking at (paying attention to) the information processing device 1.

[0069] Returning to FIG. 5, the detection cycle control unit 214 controls the detection frame rate (detection cycle) in the face detection process based on the face angle detected by the face angle detection unit 212. For example, the detection cycle control unit 214 reduces the detection frame rate when the face angle is within a first angle range (e.g., a range of ±50°) to a rate lower than the detection frame rate when the face angle is outside the first angle range (e.g., a range of ±50°). More specifically, the detection cycle control unit 214 reduces the detection frame rate when the face angle is within the first angle range (e.g., a range of ±50°) to a rate lower than the detection frame rate when the face angle is outside the first angle range (e.g., a range of ±50°) within a second angle range (e.g., a range of ±70°). This makes it possible to suppress an increase in power consumption when the user is using the information processing device 1 when the face angle is within the first angle range (e.g., a range of ±50°) (i.e., when the face is facing forward).

[0070] On the other hand, when the screen brightness is set to low, the detection frame rate is increased when the face angle is outside the first angle range (for example, a range of ±50°) and within a second angle range (for example, a range of ±70°) so that the brightness can be immediately switched to standard when the face faces forward. In this case, although increasing the detection frame rate increases power consumption, the screen brightness is controlled to low, so that power consumption can be reduced overall.

[0071] As an example, the detection cycle control unit 214 lowers the detection frame rate to "1 fps" when the face angle is within the range of ±50°, and raises the detection frame rate to "15 fps" when the face angle is within the range of ±50° to ±70°. Note that when the face angle is within the range of ±70° or more, the face direction determination process is not performed, so there is no need to increase the detection frame rate, and the detection frame rate is controlled to "1 fps".

[0072] In the following, the mode in which face detection processing is performed at a high detection frame rate (e.g., 15 fps) will be referred to as "HP (High Power) mode," and the mode in which face detection processing is performed at a low detection frame rate (e.g., 1 fps) will be referred to as "LP (Low Power) mode."

[0073] The system processing unit 310 is a functional configuration realized by the CPU 11 executing the BIOS and OS programs. For example, the system processing unit 310 includes a screen brightness control unit 311 and a timer 312 as functional configurations realized by the CPU 11 executing the OS programs.

[0074] The screen brightness control unit 311 controls the screen brightness of the display unit 110 based on the orientation of the face detected by the face detection unit 210. For example, when the screen brightness control unit 311 acquires "No Attention" information from the face detection unit 210 in a normal operating state, it controls the screen brightness to low. That is, when the orientation of the face detected by the face detection unit 210 changes from within a first angle range (for example, a range of ±50°) to outside the first angle range, the screen brightness control unit 311 reduces the screen brightness.

[0075] Furthermore, when the screen brightness control unit 311 acquires "Attention" information from the face detection unit 210 while the screen brightness is controlled to low brightness, the screen brightness control unit 311 returns the screen brightness to standard brightness. That is, when the screen brightness control unit 311 determines that the orientation of the face detected by the face detection unit 210 is facing forward while the screen brightness is reduced, the screen brightness control unit 311 returns the screen brightness to the standard brightness before it was reduced.

[0076] The timer 312 is a timer that measures the waiting time from when "No Attention" information is acquired from the face detection unit 210 in a normal operating state until the screen brightness is controlled to low brightness. If the screen brightness control unit 311 acquires "Attention" information before a predetermined waiting time has elapsed even after acquiring "No Attention" information, the screen brightness control unit 311 does not control the screen brightness to low brightness but keeps it at standard brightness. If the screen brightness control unit 311 does not acquire "Attention" information within a predetermined waiting time after acquiring "No Attention" information, the screen brightness control unit 311 controls the screen brightness to low brightness. This prevents the screen brightness from being reduced even if the user looks away for a moment while using the information processing device 1. The predetermined waiting time is preset to, for example, 10 seconds. Note that this predetermined waiting time may be configured to be user-settable.

[0077] With reference to FIG. 7, the detection state by the face detection process and the control state by the screen brightness process when the face orientation changes from sideways to frontally will be described. 7 is a schematic diagram showing an example of the detection state by the face detection process and the control state by the screen brightness process according to this embodiment. Note that, in the following, the face angle will be described as an absolute value because the same processing is performed for positive and negative values. For example, the absolute value of "±50°" will be described as "50°".

[0078] When the detected face angle is between 90° and 70°, for example, the detection frame rate is controlled to LP mode ("LP mode (1)" shown in FIG. 7)). It is also determined that the face direction is not facing forward ("No Attention"), and the screen brightness is controlled to low brightness. When the face angle becomes 70° or less, the detection frame rate switches from LP mode ("LP mode (1)" shown in FIG. 7) to HP mode, and the face direction determination process starts. For example, if the face direction changes from 70° to 60° (a change of 10° or more) within one second, the face angle is set to 0°, the face is determined to be facing forward ("Attention"), and the screen brightness is controlled from low brightness to standard brightness. Thereafter, until the detected face angle reaches 50°, it is determined that the face direction is not facing forward ("No Attention") because the face angle is 50° or more. However, because a waiting time (e.g., 10 seconds) is provided for switching from standard brightness to low brightness, standard brightness continues. When the detected face angle is less than 50°, the face is determined to be facing forward ("Attention"), and standard brightness continues. In other words, as the face orientation changes from sideways to forward, the brightness switches to standard brightness before the face orientation completely turns forward.

[0079] Furthermore, when the detected face angle becomes less than 50°, the detection frame rate switches from HP mode to LP mode ("LP mode (2)" shown in Figure 7). In this way, when the screen brightness is standard and the face is facing forward ("Attention"), power consumption is reduced by switching the detection frame rate to LP mode, but when the face direction determination process is performed while the screen brightness is controlled to low brightness, switching the detection frame rate to HP mode allows the brightness to quickly return to standard when the face is facing forward ("Attention").

[0080] In this way, when detecting the direction of the user's face to control the screen brightness of the display unit 110, the information processing device 1 detects that the user is facing forward before facing completely forward based on a change in the direction of the user's face (face angle) toward the front. However, because the user's face (face angle) is detected from the viewpoint (camera viewpoint) of the imaging unit 120, there is a concern that the detected face angle will change if the user moves their face, for example, horizontally or vertically, and a change in the direction of the face will be detected even if the user has not actually changed the direction of their face.

[0081] 8 is a schematic diagram showing an example of face angle detection from a camera viewpoint according to this embodiment. This diagram shows the relationship between the orientation and position (horizontal position) of the user's face relative to the information processing device 1, the image captured by the imaging unit 120, and the face angle detected from the captured image (i.e., the face angle from the camera viewpoint).

[0082] (A) is an example in which both the orientation and position of the user's face are in front of the information processing device 1. In this case, the user's face facing forward appears in the center of the image captured by the imaging unit 120, and the face angle detected from the captured image (i.e., the face angle from the camera's viewpoint) is 0°, which indicates the front direction (see FIG. 6).

[0083] (B) is an example in which the orientation of the user's face remains the same as in (A), but the position of the face moves to the right toward the information processing device 1. In this case, the user's face facing the left edge of the information processing device 1 appears at the left edge of the image captured by the imaging unit 120, so the face angle detected from the captured image (i.e., the face angle from the camera's viewpoint) changes from the direction of the imaging unit 120 (front direction) to the left, and becomes, for example, 30° (+30°).

[0084] (C) is an example in which the position of the user's face remains the same as in (B), but the orientation of the face has changed toward the imaging unit 120. In this case, the orientation of the user's face shown on the left edge of the image captured by the imaging unit 120 is facing toward the imaging unit 120, and therefore the face angle detected from the captured image (i.e., the face angle from the camera's viewpoint) is, for example, 0°, which indicates a forward direction.

[0085] In this way, even if the user's facial orientation does not change, the facial angle from the camera viewpoint may change due to a change in the position of the face. For this reason, for example, even if the user does not change the orientation of their face, but their body sways slightly from side to side and their face moves, it may be erroneously detected that the facial orientation (facial angle) has changed to the front direction, even though they did not look at the screen of the information processing device 1.

[0086] 9 is a diagram showing an example of erroneous detection of a face angle according to this embodiment. This diagram shows an example in which, when a user is facing sideways with respect to information processing device 1, the actual face orientation remains unchanged but the face moves sideways, causing the face angle to change to a forward direction. In the illustrated example, at the user's position shown in (A), the face angle detected from the image captured by imaging unit 120 is 70°. However, as shown in (B), the actual face orientation remains unchanged but the face moves sideways, causing the detected value of the face angle to become 50°, and from the camera's viewpoint, it appears as if the face has changed to a forward direction.

[0087] As described with reference to FIG. 6 , when the face angle changes toward the front by a predetermined angle or more (e.g., 10° or more) within a predetermined time (e.g., within one second) outside the first angle range (e.g., a range of 70° to 50°) of the second angle range, the face direction determination unit 213 determines that the face is facing forward and sets the face angle to 0°. Therefore, as shown in FIG. 9 , the face direction determination unit 213 may erroneously determine that the face is facing forward because the face has moved sideways, even though the actual face direction remains sideways. For example, if the face direction determination unit 213 makes an erroneous determination when the screen brightness is low, the screen brightness may be set to standard brightness even though the user's face is not facing forward, which is undesirable. Furthermore, if the determination by the face direction determination unit 213 as to whether the face is facing forward changes every time the user's face moves, the screen brightness may flicker, changing from low brightness to standard brightness, or the like, which is undesirable.

[0088] Therefore, when the face angle changes by a predetermined angle or more (e.g., 10° or more) in the front direction within a predetermined time (e.g., within 1 second), the face direction determination unit 213 determines whether the face is facing forward, taking into account not only the amount of change in the face angle but also the movement distance of the face.

[0089] FIG. 10 is a diagram showing an example of determination based on the movement distance of the face according to the present embodiment. For example, as shown in (A), even if the face angle changes by a predetermined angle or more (e.g., 10° or more) in the front direction within a predetermined time (e.g., within 1 second), if the movement distance of the face at that time is equal to or greater than a predetermined value (d ≧ dth), it is assumed that the change in the face angle is detected due to the change in the position of the face rather than the actual change in the face orientation, and it is predicted that the face is not facing forward, and it is determined that the face is not facing forward (i.e., sideways). On the other hand, as shown in (B), when the movement distance of the face when the face angle changes by a predetermined angle or more (e.g., 10° or more) in the front direction within a predetermined time (e.g., within 1 second) is less than the predetermined value (d < dth), it is predicted that the face is facing forward, and it is determined that the face has faced forward before falling within the first angle range (e.g., the range of ±50°), and the face angle is set to 0°.

[0090] Here, the threshold value (the above-mentioned predetermined value dth) for determining the movement distance (d) of the face may be, for example, 50% of the width of the face region (bounding box) of the detected face. The threshold value (the above-mentioned predetermined value dth) may be a preset value (pixel value), or may be a preset value (pixel value) according to the distance from the detected face.

[0091] That is, even if the face angle detected by the face angle detection unit 212 is outside the first angle range (for example, a range of ±50°) (i.e., facing sideways), the face direction determination unit 213 determines that the face was facing forward (faced forward) before the detected face direction entered the first angle range, based on the amount of change from sideways to forward and the distance the face moved. Specifically, when the face angle changes toward the forward direction by a predetermined angle or more (for example, 10° or more) within a predetermined time (for example, within one second), the information processing device 1 determines whether the distance the face moved when the face direction changed by the predetermined angle or more is less than a predetermined value, and if the distance the face moved is less than the predetermined value, determines that the face is facing forward and sets the face angle to 0°.

[0092] Here, facial movement is not limited to horizontal movement but can also be vertical movement. When the face moves horizontally (left and right), even if the face direction does not change, it may be detected as a change in the face angle in the horizontal direction (yaw) from the camera's viewpoint. Also, when the face moves vertically (up and down), even if the face direction does not change, it may be detected as a change in the face angle in the vertical direction (pitch) from the camera's viewpoint. Therefore, when the change in face direction (face angle) is a yaw angle change, the face direction determination unit 213 determines whether the horizontal movement distance of the face is less than a predetermined value. Also, when the change in face direction (face angle) is a pitch angle change, the face direction determination unit 213 determines whether the vertical movement distance of the face is less than a predetermined value.

[0093] [Processing behavior] Next, the operations of the face direction determination process executed by the face detection unit 210 and the screen brightness control process executed by the system processing unit 310 will be described.

[0094] First, the basic operation of the face direction determination process by the face detection unit 210, which determines whether the face has turned forward based on the amount of change in the face direction and the distance the face has moved outside the first angle range, will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the basic face direction determination process according to this embodiment.

[0095] (Step S11) The face detection unit 210 detects the face angle based on the captured image acquired from the imaging unit 120, and determines whether the detected face angle is equal to or greater than 50° and less than 70° (50°<face angle≦70°). If the face detection unit 210 determines that the detected face angle is not equal to or greater than 50° and less than 70° (NO), it performs the processing of step S11 again. On the other hand, if the face detection unit 210 determines that the detected face angle is equal to or greater than 50° and less than 70° (YES), it proceeds to the processing of step S13.

[0096] (Step S13) The face detection unit 210 determines whether the amount of change in the detected face angle is a change of 10° or more in the forward direction (0° direction) within one second. If the face detection unit 210 determines that the face angle has not changed by 10° or more in the forward direction within one second (NO), the process returns to step S11. On the other hand, if the face detection unit 210 determines that the face angle has changed by 10° or more in the forward direction within one second (YES), the process proceeds to step S15.

[0097] (Step S15) The face detection unit 210 determines whether the distance the face moved when the face angle changed by 10° or more in step S13 is less than a predetermined value (for example, 50% of the width of the face area (bounding box)). If the face detection unit 210 determines that the distance the face moved is greater than or equal to the predetermined value (NO), the process returns to step S11. On the other hand, if the face detection unit 210 determines that the distance the face moved is less than the predetermined value (YES), the process proceeds to step S17.

[0098] (Step S17) The face detection unit 210 determines that the face is facing forward, sets the face angle to 0°, and outputs “Attention” information to the system processing unit 310.

[0099] Next, the overall operation of the face direction determination process by the face detection unit 210 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the overall face direction determination process according to this embodiment. Here, it is assumed that the information processing device 1 is in a normal operating state, the user's face is facing forward, and the face detection process is being performed in LP mode.

[0100] (Step S101) In LP mode, face detection unit 210 detects a face angle based on a captured image acquired from imaging unit 120, and determines whether the detected face angle is less than 50° (within a range of ±50°). If face detection unit 210 determines that the detected face angle is less than 50° (YES), it determines that the face is facing forward, and performs the processing of step S101 again. On the other hand, if face detection unit 210 determines that the detected face angle is 50° or more (outside a range of ±50°) (NO), the processing proceeds to step S103.

[0101] (Step S103) The face detection unit 210 determines that the face is not facing forward, outputs "No Attention" information to the system processing unit 310, and proceeds to the processing of step S105.

[0102] (Step S105) Face detection unit 210 switches from LP mode to HP mode under control of system control unit 310, and in HP mode detects the face angle based on the captured image acquired from imaging unit 120. For example, after outputting "No Attention" information to system processing unit 310 in step S103, face detection unit 210 waits for an instruction to switch to HP mode from system control unit 310 before switching to HP mode. Then, the process proceeds to step S107.

[0103] (Step S107) The face detection unit 210 determines the face angle detected in HP mode. If the face detection unit 210 determines that the detected face angle is 70° or greater, the process proceeds to step S109. On the other hand, if the face detection unit 210 determines that the detected face angle is less than 50°, the process proceeds to step S113. Furthermore, if the face detection unit 210 determines that the detected face angle is 50° or greater and less than 70°, the process proceeds to step S117.

[0104] (Step S109) If the face angle is 70° or more, face detection unit 210 does not perform face direction determination processing, so it switches from HP mode to LP mode, and in LP mode, detects the face angle based on the captured image acquired from imaging unit 120. Then, the process proceeds to step S111.

[0105] (Step S111) The face detection unit 210 determines whether the face angle detected in LP mode is 70° or greater (outside the range of ±70°). If the face detection unit 210 determines that the detected face angle is 70° or greater (YES), it performs the process of step S111 again. On the other hand, if the face detection unit 210 determines that the detected face angle is less than 70° (within the range of ±70°) (NO), it returns to the process of step S105 and switches from LP mode to HP mode.

[0106] (Step S113) If face detection unit 210 determines that the detected face angle is less than 50° (within a range of ±50°) (YES), it determines that the face is facing forward and outputs "Attention" information to system processing unit 310. Then, the process proceeds to step S115.

[0107] (Step S115) Face detection unit 210 switches from HP mode to LP mode under control of system control unit 310, and in LP mode detects the face angle based on the captured image acquired from imaging unit 120. For example, after outputting "Attention" information to system processing unit 310 in step S113, face detection unit 210 waits for an instruction to switch to LP mode from system control unit 310 before switching to LP mode. Then, the process returns to step S101.

[0108] (Step S117) The face detection unit 210 determines whether the amount of change in the face angle detected in HP mode is a change of 10° or more in the forward direction (0° direction) within one second. If the face detection unit 210 determines that the face angle has not changed by 10° or more in the forward direction within one second (NO), the process returns to step S107. On the other hand, if the face detection unit 210 determines that the face angle has changed by 10° or more in the forward direction within one second (YES), the process proceeds to step S119.

[0109] (Step S119) The face detection unit 210 determines whether the distance the face moved when the face angle changed by 10° or more in step S117 is less than a predetermined value (for example, 50% of the width of the face area (bounding box)). If the face detection unit 210 determines that the distance the face moved is greater than or equal to the predetermined value (NO), the process returns to step S107. On the other hand, if the face detection unit 210 determines that the distance the face moved is less than the predetermined value (YES), the process proceeds to step S121.

[0110] Next, the operation of the screen brightness control process executed by the system processing unit 310 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the screen brightness control process according to this embodiment. Here, it is assumed that the information processing device 1 is in a normal operating state, the user's face is facing forward, and the screen brightness is set to standard brightness. At this time, the detection mode of the face detection unit 210 is the LP mode.

[0111] (Step S201) The screen brightness control unit 311 determines whether or not "No Attention" information has been acquired from the face detection unit 210. If the screen brightness control unit 311 determines that "No Attention" information has not been acquired (NO), it performs the process of step S201 again. On the other hand, if the screen brightness control unit 311 determines that "No Attention" information has been acquired (YES), it starts counting the standby time using the timer 312 (step S203). Then, the process proceeds to step S205.

[0112] (Step S205) The screen brightness control unit 311 determines whether or not "Attention" information has been acquired from the face detection unit 210. When the screen brightness control unit 311 determines that "Attention" information has not been acquired (NO), the process proceeds to step S207.

[0113] (Step S207) The screen brightness control unit 311 determines whether a predetermined waiting time (e.g., 10 seconds) has elapsed (i.e., whether the timer has finished) based on the value of the timer 312. If the screen brightness control unit 311 determines that the predetermined waiting time (e.g., 10 seconds) has not elapsed (i.e., the timer has not finished) (step S207: NO), the process returns to step S205. If the screen brightness control unit 311 determines that "Attention" information has been acquired before the predetermined waiting time (e.g., 10 seconds) has elapsed (step S205: YES), the process returns to step S201. At this time, the timer 312 is reset.

[0114] On the other hand, if it is determined in step S207 that the predetermined waiting time (for example, 10 seconds) has elapsed (step S207: YES), screen brightness control unit 311 changes the screen brightness to low brightness (step S209), and then proceeds to the processing of step S211.

[0115] (Step S211) The screen brightness control unit 311 transmits an instruction to switch to the HP mode to the face detection unit 210, and the process proceeds to step S213. (Step S213) The screen brightness control unit 311 determines whether or not "Attention" information has been acquired from the face detection unit 210. If the screen brightness control unit 311 determines that "Attention" information has not been acquired (NO), it performs the process of step S211 again. On the other hand, if the screen brightness control unit 311 determines that "Attention" information has been acquired (YES), it returns the screen brightness to standard brightness and transmits an instruction to switch to LP mode to the face detection unit 210 (step S215).

[0116] [Summary of the embodiment] As described above, the information processing device 1 according to this embodiment includes a memory (for example, the system memory 304) that temporarily stores image data of a captured image captured by the imaging unit 120. The information processing device 1 also includes a face detection unit 210 and a CPU 301 as examples of processors. The face detection unit 210 processes the image data of the captured image stored in the memory to execute face detection processing to detect a face area in which a face is captured from the captured image and the orientation of the face. The CPU 301 also executes brightness control processing to control the screen brightness of the display unit 110 based on the orientation of the face detected by the face detection processing. In the face detection processing, the face detection unit 210 determines that the orientation of the face is forward (first orientation) when the orientation of the detected face is within a preset first angle range (for example, a range of ±50°). Furthermore, even if the orientation of the detected face is outside the first angle range (for example, a range of ±50°), the face detection unit 210 determines that the orientation of the detected face is in the forward direction (the direction of the first orientation) before it enters the first angle range, based on the amount of change in the orientation of the detected face toward the forward direction (the direction of the first orientation) and the movement distance of the face.

[0117] As a result, when detecting the direction of the user's face and controlling the screen brightness of the display unit 110, the information processing device 1 determines that the user has turned to the front before facing completely forward based on a change in the direction of the user's face toward the front, thereby improving responsiveness when detecting the direction of the face and controlling the screen brightness of the display unit 110. This improved responsiveness makes it possible to actively control the screen brightness (for example, lower the screen brightness) according to the direction of the face, thereby contributing to power saving. Furthermore, when determining that the user has turned to the front before facing completely forward based on a change in the direction of the user's face toward the front, the information processing device 1 takes into account not only a change in the face angle from the camera's viewpoint but also the movement distance of the face, thereby reducing erroneous determinations caused by detecting a change in the face angle due to a movement in the position of the face even though the direction of the face has not actually changed.

[0118] For example, in the face detection process, even if the orientation of the detected face is outside a first angle range (for example, a range of ±50°), if the orientation of the detected face changes by a predetermined angle or more (for example, 10° or more) from a sideways orientation (an example of a second orientation) outside the first angle range (for example, a range of ±50°) to a forward orientation within a predetermined time (for example, within one second), the face detection unit 210 determines whether the movement distance of the face when the orientation of the face changes by the predetermined angle or more is less than a predetermined value (for example, 50% of the width of the face area).Then, if the movement distance of the face is less than the predetermined value, the face detection unit 210 determines that the orientation of the face is forward (determines that the face is facing forward).

[0119] As a result, the information processing device 1 determines that the face has turned forward based on a change in the face angle when the face has moved a small distance, and can therefore accurately detect that the user's face is turning forward before it turns completely forward in response to a change in the face direction from sideways to forward.

[0120] Note that if there is a change of a predetermined angle or more (e.g., 10° or more) toward the front, it is not necessary to determine whether the change occurred within a predetermined time period (e.g., within one second). For example, in the face detection process, even if the direction of the detected face is outside a first angle range (e.g., a range of ±50°), if the direction of the detected face changes from a sideways direction (an example of a second direction) outside the first angle range (e.g., a range of ±50°) toward the front by a predetermined angle or more (e.g., 10° or more) and the moving distance of the face is less than a predetermined value, the face detection unit 210 determines that the direction of the face is forward (determines that the face is facing forward).

[0121] As a result, the information processing device 1 determines that the face has turned forward based on a change in the face angle when the face has moved a small distance, and can therefore accurately detect that the user's face is turning forward before it turns completely forward in response to a change in the face direction from sideways to forward.

[0122] Furthermore, in the face detection process, the face detection unit 210 performs a face direction determination process for determining whether or not the face is facing forward (whether or not the face is in a first direction) within a second angle range (for example, a range of ±70°) that is wider than the first angle range (for example, a range of ±50°).The face detection unit 210 then performs a face direction determination process for determining whether or not the face is facing forward based on the amount of change in the detected face direction toward the front and the movement distance of the face, outside the first angle range (for example, a range of ±50°) within the second angle range (for example, a range of ±70°).

[0123] As a result, even for a face facing sideways, the information processing device 1 can detect that the face is facing forward before the face turns completely forward, based on the amount of change in the forward direction and the movement distance of the face.

[0124] In addition, in the face detection process, the face detection unit 210 reduces the detection frame rate (detection period) when the detected face orientation (e.g., face angle) is within a first angle range (e.g., a range of ±50°) to be lower than the detection frame rate when it is outside the first angle range.

[0125] This allows the information processing device 1 to suppress an increase in power consumption when the user is using the information processing device 1 (when the user's face is facing forward), and when the screen brightness is set to low (when the user's face is facing sideways), the speed at which the face orientation is detected can be increased to switch to standard brightness more quickly when the face faces forward.

[0126] For example, in face detection processing, the face detection unit 210 reduces the detection frame rate (detection period) when the detected face orientation (e.g., face angle) is within a first angle range (e.g., a range of ±50°) to be lower than the detection frame rate when the detected face orientation is outside the first angle range within a second angle range (e.g., a range of ±70°).

[0127] This allows the information processing device 1 to suppress an increase in power consumption when the user is using the information processing device 1 (when the user's face is facing forward), or conversely when the user is unlikely to face forward, such as when the user is facing completely to the side or backward, and when the face angle is such that the user is likely to face forward (a face angle close to the front), the detection speed of the face direction can be increased to speed up the switch to standard brightness when the face faces forward.

[0128] In addition, during brightness control processing, if the face orientation (e.g., face angle) detected by face detection unit 210 changes from within a first angle range (e.g., a range of ±50°) to outside the first angle range, CPU 301 reduces the screen brightness to low brightness.

[0129] As a result, when the user is not using the information processing device 1 (when the user's face is turned sideways), the information processing device 1 can reduce power consumption by lowering the screen brightness of the display unit 110 that the user is not looking at.

[0130] Furthermore, in the brightness control process, when the CPU 301 determines that the orientation of the face detected by the face detection unit 210 is forward (first orientation) while the screen brightness has been reduced to low brightness, it returns the screen brightness to the level before reduction (standard brightness).

[0131] This allows the information processing device 1 to make the display on the display unit 110 visible when the user uses the information processing device 1 (when the user faces forward).

[0132] Furthermore, the imaging unit 120 outputs image data of an image captured within a detection range FoV (predetermined angle of view) in a direction facing the screen of the display unit 110.

[0133] This allows the information processing device 1 to appropriately detect the direction of the face of the user who is using the information processing device 1.

[0134] Furthermore, the control method in the information processing device 1 according to this embodiment includes a face detection step in which the face detection unit 210 detects a face area in the captured image and the orientation of the face by processing image data of the captured image captured by the imaging unit 120 and temporarily stored in a memory (for example, system memory 304), and a brightness control step in which the CPU 301 controls the screen brightness of the display unit 110 based on the orientation of the face detected in the face detection step. Furthermore, in the face detection step, the face detection unit 210 determines that the orientation of the face is forward (first orientation) when the orientation of the detected face is within a preset first angle range (for example, a range of ±50°), and even if the orientation of the detected face is outside the first angle range (for example, a range of ±50°), determines that the orientation of the detected face is forward before it falls within the first angle range based on an amount of change in the orientation of the face toward the forward direction (direction of the first orientation) and a moving distance of the face.

[0135] As a result, when detecting the direction of the user's face and controlling the screen brightness of the display unit 110, the information processing device 1 determines that the user has faced forward before facing completely forward based on a change in the direction of the user's face toward the front, thereby improving responsiveness when detecting the direction of the face and controlling the screen brightness of the display unit 110. This improved responsiveness makes it possible to actively control the screen brightness (for example, lower the screen brightness) according to the direction of the face, thereby contributing to power saving. Furthermore, when determining that the user has faced forward before facing completely forward based on a change in the direction of the user's face toward the front, the information processing device 1 takes into account not only a change in the face angle from the camera's viewpoint but also the movement distance of the face, thereby reducing erroneous determinations caused by detecting a change in the face angle due to a movement in the position of the face even though the direction of the face has not actually changed.

[0136] In the above embodiment, the change in face direction has been mainly described in terms of a change in the horizontal direction (left-right direction), but the same applies to a change in the vertical direction (up-down direction). That is, the face direction determination process described with reference to FIG. 6 can be applied not only to the horizontal (left-right direction) face angle but also to the vertical (up-down direction) face angle. Furthermore, face movement is not limited to horizontal (left-right) movement but also includes vertical (up-down) movement. When the face moves in the horizontal direction (left-right direction), even if the face direction does not change, this may be detected as a change in the horizontal (yaw) face angle from the camera's viewpoint. Furthermore, when the face moves in the vertical direction (up-down direction), even if the face direction does not change, this may be detected as a change in the vertical (pitch) face angle from the camera's viewpoint. Therefore, when the change in face direction (face angle) is a yaw angle change, the face detection unit 210 determines whether the horizontal movement distance of the face is less than a predetermined value. Furthermore, when the change in the face direction (face angle) is a change in the angle of pitch, the face direction determining unit 213 determines whether or not the moving distance of the face in the vertical direction is less than a predetermined value.

[0137] This allows the information processing device 1 to suppress erroneous determinations caused by detecting a change in the face angle due to a movement in the face position, even when the face direction has not actually changed, regardless of whether the user's face direction has moved horizontally or vertically.

[0138] Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configurations are not limited to the above-described embodiments, and the present invention also includes designs that do not deviate from the gist of the present invention. For example, the configurations described in the above-described embodiments can be combined in any manner.

[0139] Furthermore, although the configuration example in which the imaging unit 120 is built into the information processing device 1 has been described, the present invention is not limited to this. For example, the imaging unit 120 does not have to be built into the information processing device 1, but may be configured to be attachable to the information processing device 1 (for example, any one of the side surfaces 10a, 10b, 10c, etc.) as an external accessory of the information processing device 1, and may be communicatively connected to the information processing device 1 wirelessly or via a wire.

[0140] In addition, in the above embodiment, an example has been shown in which the face detection unit 210 is provided separately from the CPU 301 and the chipset 303, but part or all of the face detection unit 210 may be provided in the chipset 303, or may be provided in a processor integrated with the CPU 301 or the chipset 303. For example, the CPU 301, the chipset 303, and the face detection unit 210 may be configured as separate processors, or may be integrated into a single processor. Also, part or all of the face detection unit 210 may be provided in the EC 200.

[0141] In addition, in the above embodiment, an example has been shown in which the face detection unit 210 is provided separately from the CPU 301 and the chipset 303, but part or all of the face detection unit 210 may be provided in the chipset 303, or may be provided in a processor integrated with the CPU 301 or the chipset 303. For example, the CPU 301, the chipset 303, and the face detection unit 210 may be configured as separate processors, or may be integrated into a single processor. Also, part or all of the face detection unit 210 may be provided in the EC 200.

[0142] The above-mentioned standby state may include a hibernation state, a power-off state, etc. The hibernation state corresponds to, for example, the S4 state defined by ACPI. The power-off state corresponds to, for example, the S5 state (shutdown state) defined by ACPI. Among the standby states, the standby state, sleep state, hibernation state, and power-off state are states in which power consumption is lower (states in which power consumption is reduced) than in the normal operating state.

[0143] The information processing device 1 described above includes an internal computer system. A program for implementing the functions of each component of the information processing device 1 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing in each component of the information processing device 1. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including the Internet, a WAN, a LAN, a dedicated line, or other communication lines. The term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. The recording medium storing the program may also be a non-transitory recording medium such as a CD-ROM.

[0144] The recording medium also includes internal or external recording media accessible from a distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined by the components of the information processing device 1, or each divided program may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a medium that realizes part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-described functions in combination with a program already stored in the computer system.

[0145] Furthermore, some or all of the functions of the information processing device 1 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually implemented as a processor, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

[0146] Furthermore, the information processing device 1 of the above embodiment is not limited to a notebook PC, but may be, for example, a desktop PC. [Explanation of symbols]

[0147] 1 Information processing device, 10 First housing, 20 Second housing, 15 Hinge mechanism, 110 Display unit, 120 Imaging unit, 140 Power button, 150 Input device, 151 Keyboard, 153 Touchpad, 160 Communication unit, 170 Memory unit, 200 EC, 210 Face detection unit, 211 Face area detection unit, 212 Face angle detection unit, 213 Face direction determination unit, 214 Detection cycle control unit, 300 Main processing unit, 301 CPU, 302 GPU, 303 Chipset, 304 System memory, 310 System processing unit, 311 Screen brightness control unit, 312 Timer, 400 Power supply unit

Claims

1. a memory for temporarily storing image data of an image captured by the imaging unit; a processor for processing image data stored in said memory; Equipped with The processor: a face detection process for detecting a face area in the image and a direction of the face by processing image data of the image stored in the memory; a brightness control process for controlling the screen brightness of a display unit based on the orientation of the face detected by the face detection process; and The processor: In the face detection process, when the orientation of the detected face is within a preset first angle range, it is determined that the orientation is a first orientation, and even if the orientation of the detected face is outside the first angle range, it is determined that the orientation is the first orientation before the orientation of the detected face falls within the first angle range based on an amount of change in the direction of the first orientation and a moving distance of the face. Information processing device.

2. The processor: In the face detection process, even if the orientation of the detected face is outside the first angle range, if the orientation of the detected face changes by a predetermined angle or more from a second orientation outside the first angle range toward the first orientation, it is determined whether or not a movement distance of the face when the orientation of the face changes by the predetermined angle or more is less than a predetermined value, and if the movement distance of the face is less than the predetermined value, it is determined that the orientation is the first orientation. The information processing device according to claim 1 .

3. The processor: In the face detection process, if the change in the face direction is a change in yaw angle, it is determined whether or not a distance moved in the left-right direction of the face is less than a predetermined value, and if the change in the face direction is a change in pitch angle, it is determined whether or not a distance moved in the up-down direction of the face is less than a predetermined value. The information processing device according to claim 2 .

4. The processor: In the face detection process, even if the orientation of the detected face is outside the first angle range, if the orientation of the detected face changes from the second orientation toward the first orientation by at least the predetermined angle within a predetermined time and the moving distance of the face is less than a predetermined value, the orientation is determined to be the first orientation. The information processing device according to claim 2 .

5. The processor: In the face detection process, a process of determining whether or not the face is in the first orientation is performed within a second angle range wider than the first angle range, and a process of determining whether or not the face is in the first orientation is performed outside the first angle range within the second angle range based on an amount of change in the orientation of the face toward the first orientation and a moving distance of the face. The information processing device according to claim 1 .

6. The processor: In the face detection process, a detection cycle when the orientation of the detected face is within the first angle range is set shorter than a detection cycle when the orientation is outside the first angle range. The information processing device according to claim 1 .

7. The processor: In the face detection process, a detection cycle when the orientation of the detected face is within the first angle range is set shorter than a detection cycle when the orientation of the detected face is outside the first angle range within the second angle range. The information processing device according to claim 4 .

8. The processor: In the brightness control process, when the orientation of the face detected by the face detection process changes from within the first angle range to outside the first angle range, the screen brightness is reduced. The information processing device according to claim 1 .

9. The processor: In the brightness control process, when it is determined that the orientation of the face detected by the face detection process is the first orientation in a state in which the screen brightness has been reduced, the screen brightness is restored to the level before the reduction. The information processing device according to claim 7 .

10. the imaging unit outputs image data of an image captured at a predetermined angle of view in a direction facing the screen of the display unit. The information processing device according to claim 1 .

11. 1. A control method for an information processing device including a memory that temporarily stores image data of an image captured by an imaging unit, and a processor that processes the image data stored in the memory, the processor: a face detection step of detecting a face area in the image and a direction of the face by processing image data of the image stored in the memory; a brightness control step of controlling screen brightness of a display unit based on the orientation of the face detected in the face detection step; Including, In the face detection step, when the detected orientation of the face is within a preset first angle range, it is determined that the orientation of the face is a first orientation, and even if the detected orientation of the face is outside the first angle range, it is determined that the orientation of the face is the first orientation before it falls within the first angle range based on an amount of change in the direction of the first orientation and a moving distance of the face. Control method.

Images