Information processing device and control method

Abstract

To detect contact or departure of a person regardless of the ambient brightness.SOLUTION: An information processing device comprises: a first imaging part that performs imaging by using visible light; a second imaging part that performs imaging by using infrared light; a memory that acquires imaging data imaged by using the first imaging part or the second imaging part and temporarily stores the data; and a processor that detects the existence of a person on the basis of the imaging data imaged by using the first imaging part. The processor detects, on the basis of a result of detection of the ambient brightness, the existence of the person on the basis of the imaging data imaged by using the second imaging part in the case where the ambient brightness is a first threshold value or less.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device and a control method. [Background technology]

[0002] There is a device that transitions to a usable operating state when a person approaches and transitions to a standby state when the person leaves. For example, Patent Document 1 discloses a technology that uses an infrared sensor to detect the strength of infrared rays to detect whether a person is approaching or has left, and controls the operating state of the device accordingly.

[0003] In recent years, with the development of computer vision and the like, technology for detecting faces from images has become common, and for example, in information processing devices such as personal computers, the approach or departure of a person is detected depending on whether a face is detected from an image captured in front of the device by a camera, and the system is started up or transitioned to a waiting state (for example, system lock, standby, etc.) is controlled. When an infrared sensor is used, infrared light is reflected back from both people and objects other than people, but by using face detection, people can be detected more accurately. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-148895 A Summary of the Invention [Problem to be solved by the invention]

[0005] In information processing devices such as the above-mentioned personal computers, the approach and departure of people are generally detected using an RGB camera that captures images using visible light. However, while the approach and departure of people can be detected using an RGB camera in the ambient light of a normal office, in a dark environment, the captured image may be too dark to be detected correctly.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objects is to provide an information processing device and a control method that can detect the approach and departure of a person regardless of the surrounding brightness. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and an information processing device according to a first aspect of the present invention includes a first imaging unit that captures images using visible light, a second imaging unit that captures images using infrared light, a memory that acquires and temporarily stores imaging data captured using the first imaging unit or the second imaging unit, and a processor that detects the presence of a person based on the imaging data captured using the first imaging unit, and when the surrounding brightness is equal to or lower than a first threshold value based on a detection result of the surrounding brightness, the processor detects the presence of a person based on the imaging data captured using the second imaging unit.

[0008] In the above information processing device, a display unit is provided that displays an image based on a set value of a screen brightness, and the processor may detect the presence of a person based on the imaging data captured using the first imaging unit when the set value of the screen brightness is higher than a second threshold even when the surrounding brightness is below the first threshold, and detect the presence of a person based on the imaging data captured using the second imaging unit when the set value of the screen brightness is below the second threshold.

[0009] In the above information processing device, the processor may detect the presence of a person based on the imaging data captured using the second imaging unit when the presence of a person is not detected based on the imaging data captured using the first imaging unit when the ambient brightness is below the first threshold and the setting value of the screen brightness is higher than a second threshold.

[0010] In the above information processing device, the processor may detect the presence of a person based on the imaging data captured using the second imaging unit when the ambient brightness is below the first threshold and the presence of a person is not detected based on the imaging data captured using the first imaging unit.

[0011] The information processing device may further include an ambient light sensor, and the processor may detect the brightness of the surroundings using the ambient light sensor.

[0012] In the information processing device, the processor may detect the brightness of an environment using the first imaging unit.

[0013] In the information processing device, the processor may output a result of the detection of the presence of a person as information for controlling an operating state of a system.

[0014] Furthermore, according to a second aspect of the present invention, a control method for an information processing device including a first imaging unit that captures images using visible light, a second imaging unit that captures images using infrared light, a memory that acquires and temporarily stores imaging data captured using the first imaging unit or the second imaging unit, and a processor that detects the presence of a person based on the imaging data captured using the first imaging unit includes the steps of: detecting ambient brightness by the processor; and, when the detected ambient brightness is equal to or less than a first threshold, detecting the presence of a person based on the imaging data captured using the second imaging unit. Effect of the Invention

[0015] According to the above-described aspects of the present invention, the approach and departure of a person can be detected regardless of the brightness of the surroundings. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a perspective view showing an example of the external configuration of an information processing apparatus according to a first embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of a person detection range of the information processing device according to the first embodiment. [Diagram 3] FIG. 2 is a view for explaining an overview of HPD processing of the information processing device according to the first embodiment. [Figure 4] 4A and 4B are explanatory views of HPD processing using an RGB camera and an IR camera according to the first embodiment. [Diagram 5] FIG. 1 is a schematic block diagram showing an example of a hardware configuration of an information processing apparatus according to a first embodiment. [Figure 6] FIG. 1 is a schematic block diagram showing an example of a functional configuration of an information processing device according to a first embodiment. [Figure 7] 6 is a flowchart showing an example of HPD processing according to the first embodiment. [Figure 8] 10 is a flowchart showing an example of HPD processing according to the second embodiment. [Figure 9] 13 is a flowchart showing an example of HPD processing according to the third embodiment. [Figure 10] 13 is a flowchart showing an example of HPD processing according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <First embodiment> [overview] First, an overview of the information processing device according to this embodiment will be described. FIG. 1 is a perspective view showing an example of the external configuration of an information processing device 1 according to the present embodiment.

[0018] The information processing device 1 is, for example, a notebook (clamshell) type PC (Personal Computer). The information processing device 1 includes a first housing 10, a second housing 20, and a hinge mechanism 15. The first housing 10 and the second housing 20 are connected using the hinge mechanism 15. The first housing 10 is rotatable relative to the second housing 20 around a rotation axis formed by the hinge mechanism 15. The opening angle resulting from the rotation of the first housing 10 and the second housing 20 is illustrated as "θ".

[0019] The first housing 10 is also called the A cover or the display housing. The second housing 20 is also called the C cover or the system housing. In the following description, the sides of the first housing 10 and the second housing 20 on which the hinge mechanism 15 is provided are called the sides 10c and 20c, respectively. The sides of the first housing 10 and the second housing 20 opposite the sides 10c and 20c are called the sides 10a and 20a, respectively. In the drawings, the direction from the side 20a to the side 20c is called the "rear", and the direction from the side 20c to the side 20a is called the "front". The right and left sides of the rear are called the "right" and "left", respectively. The left sides of the first housing 10 and the second housing 20 are called the sides 10b and 20b, respectively, and the right sides are called the sides 10d and 20d, respectively. Moreover, a state in which the first housing 10 and the second housing 20 overlap and are completely closed (a state in which the opening angle θ=0°) is referred to as a "closed state." In the closed state, the surfaces of the first housing 10 and the second housing 20 that face each other are referred to as the "inner surfaces," and the surfaces opposite the inner surfaces are referred to as the "outer surfaces." Moreover, a state in which the first housing 10 and the second housing 20 are open relative to the closed state is referred to as an "open state."

[0020] The external appearance of the information processing device 1 shown in FIG. 1 shows an example of an open state. In the open state, the side surface 10a of the first housing 10 and the side surface 20a of the second housing 20 are separated. In the open state, the inner surfaces of the first housing 10 and the second housing 20 are exposed. The open state is one of the states when the user uses the information processing device 1, and is typically used in a state where the opening angle θ is about 100 to 130°. The range of the opening angle θ in the open state can be arbitrarily determined depending on the range of angles that can be rotated by the hinge mechanism 15, etc.

[0021] A display unit 110 is provided on the inner surface of the first housing 10. The display unit 110 is configured to include a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display. An imaging unit 120 and an ambient light sensor 130 are provided in a peripheral region of the display unit 110 on the inner surface of the first housing 10. The position of the imaging unit 120 shown in the figure is an example, and the imaging unit 120 may be located at another position as long as it can face the direction facing the display screen of the display unit 110. The position of the ambient light sensor 130 shown in the figure is an example, and the ambient light sensor 130 may be located at another position as long as it can detect the brightness around the information processing device 1. For example, the ambient light sensor 130 may be located at any position on the inner surface of the first housing 10 or the inner surface of the second housing 20.

[0022] In the open state, the imaging unit 120 captures an image of a predetermined imaging range in a direction facing the display screen of the display unit 110 (i.e., in front of the information processing device 1). The predetermined imaging range is a range of an angle of view determined by an imaging element of the imaging unit 120 and an optical lens provided in front of the imaging surface of the imaging element. For example, the imaging unit 120 can capture an image including a person (user) present in front of the information processing device 1 (on the front side).

[0023] Here, the imaging unit 120 is configured to include an RGB camera and an IR (InfraRed) camera. The RGB camera is a camera that captures images using visible light and includes an RGB image sensor. The RGB image sensor is an imaging element in which R pixels provided with color filters that transmit the R (red) wavelength band, G pixels provided with color filters that transmit the G (green) wavelength band, and B pixels provided with color filters that transmit the B (blue) wavelength band are arranged. On the other hand, the IR camera is a camera that captures images using infrared light and includes a light-emitting unit that emits infrared light and an IR image sensor. The IR image sensor is an imaging element in which IR (InfraRed) pixels are arranged, into which reflected light of infrared light emitted from the light-emitting unit can be incident.

[0024] The ambient light sensor 130 (ALS: Ambient Light Sensor) is a sensor for detecting the brightness (e.g., illuminance) of the surroundings of the information processing device 1. The ambient light sensor 130 detects the amount of surrounding light, performs photoelectric conversion, and outputs the result as an output value. For example, the ambient light sensor 130 is configured to include a phototransistor. The ambient light sensor 130 converts a current flowing through the phototransistor, which changes depending on the amount of surrounding light, into a voltage using a resistor, and outputs the voltage value.

[0025] A power button 140 is provided on the side surface 20b of the second housing 20. The power button 140 is an operator for the user to instruct power on or off, transition from a standby state to a normal operation state, transition from the normal operation state to a standby state, etc. The normal operation state is an operation state of the system in which processing can be executed without particular restrictions, and corresponds to, for example, the S0 state defined by ACPI (Advanced Configuration and Power Interface).

[0026] The standby state is a state in which at least a part of the system processing is restricted. For example, the standby state is a state in which the system is locked. In the locked state of the system, a preset image (for example, a lock image) is displayed on the display unit so that the contents being processed cannot be seen. To release the lock, login authentication by the user is required. The standby state may be a standby state, a sleep state, etc., and may be a state equivalent to modern standby in Windows (registered trademark) or the S3 state (sleep state) defined by ACPI. For example, the standby state is a state in which power consumption is lower than that in the normal operating state.

[0027] Further, on the inner surface of the second housing 20, a keyboard 151 and a touch pad 153 are provided as input devices that accept operation inputs from the user. Note that, instead of or in addition to the keyboard 151 and the touch pad 153, a touch sensor may be provided as the input device, or a mouse or an external keyboard may be connected. In the case where a touch sensor is provided, an area corresponding to the display screen of the display unit 110 may be configured as a touch panel that accepts operations. Furthermore, the input device may include a microphone into which voice is input.

[0028] In addition, when the first housing 10 and the second housing 20 are in a closed state, the display unit 110 and the imaging unit 120 provided on the inner surface of the first housing 10, and the keyboard 151 and the touchpad 153 provided on the inner surface of the second housing 20 are covered by the surface of the other housing, and are unable to function.

[0029] The information processing device 1 executes a human presence detection (HPD) process to detect a person present in front of the information processing device 1, based on a captured image captured by the imaging unit 120.

[0030] 2 is a diagram showing an example of a person detection range of the information processing device 1 according to this embodiment. In the example shown in the figure, a detection range FoV (Field of View: detection viewing angle) in front of the information processing device 1 is a range in which a person can be detected.

[0031] For example, the information processing device 1 detects a face area in which a face is captured from a captured image captured by the imaging unit 120, thereby determining whether or not a person (user) is present in front of the information processing device 1. The detection range FoV corresponds to the imaging angle of view captured by the information processing device 1. When a face area is detected from the captured image, the information processing device 1 determines that a person is present. On the other hand, when a face area is not detected from the captured image, the information processing device 1 determines that a person is not present.

[0032] The information processing device 1 controls the operation state of the system of the information processing device 1 according to the presence or absence of a person through HPD processing. For example, the information processing device 1 is controlled to a normal operation state when a person is present in front of the information processing device 1, and is controlled to a standby state when no person is present in front of the information processing device 1.

[0033] FIG. 3 is a diagram for explaining an outline of the HPD process of the information processing device 1 according to the present embodiment. The information processing device 1 detects a person in front of the information processing device 1 by the HPD process, and controls the operation state of the system of the information processing device 1 based on the presence or absence of the person. For example, as shown in (A) of FIG. 3, when the information processing device 1 detects a change from a state in which no person is present in front of the information processing device 1 (Absence) to a state in which a person is present (Presence), that is, when the information processing device 1 detects an approach of a person to the information processing device 1 (Approach), in the standby state, the system is automatically started and transitioned to a normal operation state. Also, as shown in (B) of FIG. 3, when the information processing device 1 detects a state in which a person is present in front of the information processing device 1 (Presence), in the normal operation state, the information processing device 1 continues the normal operation state. Also, as shown in (C) of FIG. 3, when the information processing device 1 detects a change from a state in which a person is present in front of the information processing device 1 (Presence) to a state in which a person is not present (Absence), that is, when the information processing device 1 detects a departure of the person from the information processing device 1 (Leave), the system is transitioned to the standby state.

[0034] Here, the information processing device 1 normally performs HPD processing to detect the presence of a person using the RGB camera of the imaging unit 120. Furthermore, in a dark environment, the information processing device 1 performs HPD processing using an IR camera.

[0035] Fig. 4 is an explanatory diagram of HPD processing using an RGB camera and an IR camera according to this embodiment. Fig. 4(A) shows an example of an image captured using an RGB camera in a normal office environment (bright surroundings), and if a person is present in front of the information processing device 1, the captured image shows the person's face, making it possible to detect the presence of the person by face detection. Therefore, the HPD processing using the RGB camera makes it possible to detect the approach and departure of a person.

[0036] On the other hand, FIG. 4B shows an example of an image captured by using an RGB camera in a dark environment. Even if a person is present in front of the information processing device 1, the captured image is dark and it is impossible to detect the presence of the person by face detection. Therefore, in a dark environment, the HPD process using the RGB camera may not be able to detect the approach and departure of a person. Therefore, the information processing device 1 performs HPD process using the RGB camera of the imaging unit 120 normally (in a bright environment), and performs HPD process using the IR camera of the imaging unit 120 in a dark environment. Note that when an IR camera is used, infrared light is emitted, so power consumption is higher than when an RGB camera is used. However, by using the IR camera only in a dark environment, HPD process can be performed regardless of the brightness of the surroundings without increasing normal power consumption.

[0037] The configuration of the information processing device 1 according to this embodiment will be described in detail below. [Hardware configuration of information processing device] Fig. 5 is a schematic block diagram showing an example of a hardware configuration of the information processing device 1 according to this embodiment. In Fig. 4, the same reference numerals are used for components corresponding to those in Fig. 1. The information processing device 1 includes a display unit 110, an imaging unit 120, an ambient light sensor 130, a power button 140, an input device 150, a communication unit 160, a storage unit 170, an EC (Embedded Controller) 200, a face detection unit 210, a main processing unit 300, and a power supply unit 400.

[0038] The display unit 110 displays display data (images) generated based on system processing executed by the main processing unit 300 and processing of application programs running on the system processing.

[0039] The imaging unit 120 captures an image of an object within a predetermined imaging range (angle of view) in a direction (forward) facing the inner surface of the first housing 10, and outputs the captured image to the main processing unit 300 and the face detection unit 210. For example, the imaging unit 120 includes an RGB camera 121 and an IR camera 122 that captures images using infrared light.

[0040] The RGB camera 121 is a visible light camera that captures an image using visible light, and as described above, includes an RGB image sensor in which R pixels provided with color filters that transmit the R (red) wavelength band, G pixels provided with color filters that transmit the G (green) wavelength band, and B pixels provided with color filters that transmit the B (blue) wavelength band are arranged. The RGB camera 121 outputs RGB image (visible light image) data obtained by photoelectrically converting visible light incident via the RGB color filters as captured image data (captured image data).

[0041] The IR camera 122 is an infrared camera that captures images using infrared light, and as described above, includes an IR image sensor in which IR pixels that can receive infrared light are arranged. Since the IR pixels are not provided with color filters, light including infrared light is incident on the IR pixels. The IR camera 122 includes a light-emitting unit that emits infrared light, and receives light including reflected infrared light output from the light-emitting unit, and outputs IR image (infrared image) data obtained by photoelectrically converting the incident light as captured image data (imaging data).

[0042] In the following, the RGB image (visible light image) data and the IR image data will be referred to as a unit "captured image."

[0043] The ambient light sensor 130 detects the brightness (amount of light) around the information processing device 1, and outputs a voltage value according to the detected brightness as a detection result.

[0044] The power button 140 outputs an operation signal to the EC 200 in response to a user operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to accepting an operation on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC 200.

[0045] The communication unit 160 is communicably connected to other devices via a wireless or wired communication network, and transmits and receives various data. For example, the communication unit 160 includes a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark).

[0046] The storage unit 170 includes storage media such as a hard disk drive (HDD), a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), a flash ROM, etc. The storage unit 170 stores various programs such as an OS, device drivers, and applications, as well as various data acquired by the operation of the programs.

[0047] The power supply unit 400 supplies power to each unit according to the operating state of each unit of the information processing device 1. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of direct current power supplied from an AC (Alternate Current) / DC adapter or a battery (battery pack) into a voltage required by each unit. The power whose voltage has been converted by the DC / DC converter is supplied to each unit via each power supply system. For example, the power supply unit 400 supplies power to each unit via each power supply system based on a control signal input from the EC 200.

[0048] The EC200 is a microcomputer including a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and an input / output (I / O) logic circuit. The CPU of the EC200 reads a control program (firmware) prestored in its own ROM, and executes the read control program to fulfill its functions. The EC200 operates independently of the main processing unit 300, controls the operation of the main processing unit 300, and manages its operating state. The EC200 is also connected to a power button 140, an input device 150, a power supply unit 400, and the like.

[0049] For example, the EC 200 communicates with the power supply unit 400 to obtain information on the battery state (such as remaining capacity) from the power supply unit 400 and outputs control signals and the like for controlling the supply of power according to the operating state of each unit of the information processing device 1 to the power supply unit 400. The EC 200 also obtains operation signals from the power button 140 and the input device 150, and outputs to the main processing unit 300 operation signals related to the processing of the main processing unit 300 among the obtained operation signals.

[0050] The face detection unit 210 is configured to include a processor that executes HPD processing by face detection based on a captured image captured by the imaging unit 120. The face detection unit 210 acquires the captured image captured by the imaging unit 120, and temporarily stores the acquired captured image (captured image data) in a memory. The memory that stores the captured image (captured image data) may be the system memory 304, or may be a memory (not shown) in the face detection unit 210.

[0051] For example, the face detection unit 210 detects the presence of a person by detecting a face area in which a face is captured from the captured image by processing the captured image (captured image data) acquired from the imaging unit 120. As a method for detecting the face area, any detection method can be applied, such as a face detection algorithm that detects a face based on facial feature information, or face detection using learning data (trained model) or a face detection library that is machine-learned based on facial feature information.

[0052] Furthermore, the face detection unit 210 controls whether to perform HPD processing using the RGB camera 121 of the imaging unit 120 or the IR camera 122 based on the surrounding brightness. The HPD processing using the RGB camera 121 or the IR camera 122 will be described in detail later.

[0053] The main processing unit 300 is composed of a CPU (Central Processing Unit) 301, a GPU (Graphic Processing Unit) 302, a chipset 303, and a system memory 304, and is capable of executing processing of various application programs on an OS (Operating System) through system processing based on the OS.

[0054] The CPU 301 is a processor that executes processes based on BIOS programs, processes based on OS programs, processes based on application programs that run on the OS, etc. For example, the CPU 301 executes a startup process that starts the system from a standby state and transitions it to a normal operating state, a sleep process that transitions it from the normal operating state to a standby state, etc.

[0055] The GPU 302 is connected to the display unit 110. The GPU 302 executes image processing and generates display data under the control of the CPU 301. The GPU 302 outputs the generated display data to the display unit 110.

[0056] The chipset 303 has a function as a memory controller and a function as an I / O controller. For example, the chipset 303 controls the reading and writing of data from the system memory 304, the storage unit 170, etc. by the CPU 301 and the GPU 302. The chipset 303 also controls the input and output of data from the communication unit 160, the display unit 110, and the EC 200. The chipset 303 also has a function as a sensor hub.

[0057] For example, the chipset 303 acquires the result of detection of the surrounding brightness by the ambient light sensor 130 and the result of detection of the HPD processing by the face detection unit 210, and transmits them to the CPU 301. Furthermore, based on the result of detection of the surrounding brightness by the ambient light sensor 130, the chipset 303 controls whether the face detection unit 210 performs the HPD processing using the RGB camera 121 or the IR camera 122.

[0058] The system memory 304 is used as a reading area for the programs executed by the CPU 301, a working area for writing processing data, etc. The system memory 304 also temporarily stores captured images (captured image data) captured by the imaging unit 120.

[0059] The CPU 301, the GPU 302, and the chipset 303 may be configured as a single integrated processor, or some or each of them may be configured as individual processors. For example, in a normal operation state, the CPU 301, the GPU 302, and the chipset 303 are all in operation, but in a standby state, for example, only at least a part of the chipset 303 is in operation.

[0060] [Functional configuration of information processing device] Next, the functional configuration of the information processing device 1 that performs HPD processing using the RGB camera 121 or the IR camera 122 will be described in detail.

[0061] 6 is a schematic block diagram showing an example of a functional configuration of the information processing device 1 according to this embodiment. The information processing device 1 includes an imaging unit 120 including an RGB camera 121 and an IR camera 122, an ambient light sensor 130, a face detection unit 210, and a control unit 310. The imaging unit 120, the ambient light sensor 130, and the face detection unit 210 correspond to the imaging unit 120, the ambient light sensor 130, and the face detection unit 210 shown in FIG. 5, respectively.

[0062] The face detection unit 210 executes HPD processing to detect a person present in front of the information processing device 1 (within the detection range FoV) using the imaging unit 120. Specifically, the face detection unit 210 detects a face area in which a face is captured from a captured image acquired from the imaging unit 120. When the face detection unit 210 detects a face area from a captured image, it determines that a person is present in front of the information processing device 1 (Presence), and outputs "True" as the detection result of the HPD processing. On the other hand, when the face detection unit 210 does not detect a face area from a captured image, it determines that no person is present in front of the information processing device 1 (Absence), and outputs "False" as the detection result of the HPD processing. For example, the face detection unit 210 outputs the detection result of this HPD processing to the control unit 310 as information for controlling the operating state of the system.

[0063] The control unit 310 includes an HPD information acquisition unit 311, an operation control unit 312, and a detection mode control unit 313 as functional components executed by, for example, the CPU 301 or the chipset 303 (for example, a sensor hub).

[0064] The HPD information acquisition unit 311 acquires information indicating the detection result of the HPD processing from the face detection unit 210. For example, the HPD information acquisition unit 311 acquires "True" or "False" as the detection result of the HPD processing.

[0065] The operation control unit 312 controls the system operation state based on the detection result of the HPD processing acquired by the HPD information acquisition unit 311. For example, when the HPD information acquisition unit 311 acquires "True" as the detection result of the HPD processing from the face detection unit 210 in the standby state, the operation control unit 312 starts up the system. Also, when the HPD information acquisition unit 311 acquires "False" as the detection result of the HPD processing from the face detection unit 210 in the normal operation state, the operation control unit 312 transitions the system to the standby state.

[0066] The detection mode control unit 313 controls whether the face detection unit 210 performs HPD processing using the RGB camera 121 or the IR camera 122, based on the detection result of the ambient light sensor 130. Hereinafter, the mode in which the HPD processing is performed using the RGB camera 121 is referred to as the "RGB detection mode," and the mode in which the HPD processing is performed using the IR camera 122 is referred to as the "IR detection mode."

[0067] For example, the detection mode control unit 313 controls to the RGB detection mode in the initial state (default), and controls to the IR detection mode when the surrounding brightness is equal to or lower than a preset "threshold TH1". The threshold TH1 is preset as a brightness threshold at which the presence of a person cannot be detected when an image is captured using the RGB camera 121. The detection mode control unit 313 outputs a mode control signal to the face detection unit 210 to control to either the RGB detection mode or the IR detection mode based on the detection result of the ambient light sensor 130.

[0068] When face detection unit 210 receives a mode control signal from detection mode control unit 313 for control to the RGB detection mode, it performs HPD processing using RGB camera 121. When face detection unit 210 receives a mode control signal from detection mode control unit 313 for control to the IR detection mode, it performs HPD processing using IR camera 122.

[0069] That is, in the initial state (default), the face detection unit 210 detects the presence of a person based on an image captured using the RGB camera 121, and when the surrounding brightness is below the threshold value TH1 (dark surrounding environment), it detects the presence of a person based on an image captured using the IR camera 122.

[0070] [Processing behavior] Next, the operation of the HPD process using the RGB camera 121 or the IR camera 122 depending on the surrounding brightness will be described with reference to FIG. FIG. 7 is a flowchart showing an example of the HPD process according to this embodiment.

[0071] (Step S101) Control unit 310 determines whether the surrounding brightness is higher than threshold TH1 (ambient light>TH1?) based on the detection result of ambient light sensor 130. If control unit 310 determines that the surrounding brightness is higher than threshold TH1 (YES), it outputs a mode control signal to face detection unit 210 for control to RGB detection mode. Then, the process proceeds to step S103. On the other hand, if control unit 310 determines that the surrounding brightness is equal to or lower than threshold TH1 (NO), it outputs a mode control signal to face detection unit 210 for control to IR detection mode. Then, the process proceeds to step S105.

[0072] (Step S103) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, the face detection unit 210 performs HPD processing using the RGB camera 121. Then, the process returns to step S101.

[0073] (Step S105) When face detection unit 210 acquires a mode control signal for control to the IR detection mode, it performs HPD processing using IR camera 122. Then, the process returns to step S101.

[0074] [Summary of the first embodiment] As described above, the information processing device 1 according to this embodiment includes an RGB camera 121 (an example of a first imaging section) that captures images using visible light, an IR camera 122 (an example of a second imaging section) that captures images using infrared light, a memory (e.g., system memory 304) that acquires and temporarily stores captured images (imaging data) captured using the RGB camera 121 or the IR camera 122, and a face detection section 210 (an example of a processor) that detects the presence of a person based on the captured image captured using the RGB camera 121. Then, based on the detection result of the brightness of the surroundings, when the brightness of the surroundings is equal to or lower than a threshold value TH1 (an example of a first threshold value), the face detection section 210 detects the presence of a person based on the captured image captured using the IR camera 122.

[0075] As a result, the information processing device 1 can detect the approach or departure of a person regardless of the brightness of the surroundings by using the RGB camera 121 and the IR camera 122 according to the brightness of the surroundings. Also, since the information processing device 1 uses the IR camera 122 only when the surroundings are dark, it is possible to suppress an increase in power consumption.

[0076] The information processing device 1 includes an ambient light sensor 130, and detects the brightness of the surroundings using the ambient light sensor 130.

[0077] This allows the information processing device 1 to accurately detect the brightness of the surroundings. For example, when detecting the brightness of the surroundings using the imaging unit 120 (for example, the RGB camera 121), it is determined that the surroundings are dark even when the camera shutter is closed, but by using the ambient light sensor 130, it is possible to detect a dark surroundings environment by distinguishing it from a state in which the camera shutter is closed.

[0078] The information processing device 1 may use the RGB camera 121 to detect the surrounding brightness.

[0079] In this case, the information processing device 1 does not need to be provided with a separate ambient light sensor 130, which is advantageous in terms of cost.

[0080] Furthermore, the face detection unit 210 outputs the result of detection of the presence of a person as information for controlling the operating state of the system.

[0081] This allows the information processing device 1 to start up the system in response to the approach of a person, and transition the system to a standby state in response to the departure of a person, regardless of the brightness of the surroundings.

[0082] Furthermore, in accordance with this embodiment, a control method in an information processing device 1 including a first imaging unit that captures images using visible light, a second imaging unit that captures images using infrared light, a memory that acquires and temporarily stores imaging data captured using the first imaging unit or the second imaging unit, and a processor that detects the presence of a person based on the imaging data captured using the first imaging unit includes a step in which a control unit 310 (an example of a processor) detects the brightness of the surroundings, and a step in which a face detection unit 210 (an example of a processor) detects the presence of a person based on the imaging data captured using the second imaging unit when the detected brightness of the surroundings is equal to or lower than a first threshold value.

[0083] As a result, the information processing device 1 can detect the approach or departure of a person regardless of the brightness of the surroundings by using the RGB camera 121 and the IR camera 122 according to the brightness of the surroundings. Also, since the information processing device 1 uses the IR camera 122 only when the surroundings are dark, it is possible to suppress an increase in power consumption.

[0084] <Second embodiment> Next, a second embodiment will be described. In the first embodiment, a configuration has been described in which HPD processing is possible by using the IR camera 122 instead of the RGB camera 121 when the surroundings are in a dark environment, but even in a dark environment, there are cases in which HPD processing is possible with the RGB camera 121 if the face is illuminated by the brightness of the display unit 110. Therefore, in this embodiment, the information processing device 1 performs HPD processing using the RGB camera 121 when the setting value of the screen luminance of the display unit 110 is high even in a dark environment.

[0085] For example, the screen brightness of the display unit 110 is set by the display settings of the OS. The set value of the screen brightness can be arbitrarily changed by the user between 0% and 100%, with the maximum brightness being 100%. Even if the surrounding brightness is equal to or lower than the threshold value TH1, when the set value of the screen brightness is higher than a preset "threshold value TH2", the face detection unit 210 performs HPD processing using the RGB camera 121. The threshold value TH2 is preset based on the amount of light irradiated from the display unit 110 to the face of a user present in front of the information processing device 1, at which the face can be detected from a captured image.

[0086] FIG. 8 is a flowchart showing an example of the HPD process according to this embodiment.

[0087] (Step S201) The control unit 310 determines whether the surrounding brightness is higher than the threshold value TH1 (ambient light>TH1?) based on the detection result of the ambient light sensor 130. If the control unit 310 determines that the surrounding brightness is higher than the threshold value TH1 (YES), it outputs a mode control signal to the face detection unit 210 to control the RGB detection mode. Then, the process proceeds to step S205. On the other hand, if the control unit 310 determines that the surrounding brightness is equal to or lower than the threshold value TH1 (NO), the process proceeds to step S203.

[0088] (Step S203) The control unit 310 judges whether the setting value of the screen luminance is higher than the threshold value TH2 (screen luminance>TH2?). If the control unit 310 judges that the setting value of the screen luminance is higher than the threshold value TH2 (YES), it outputs a mode control signal for controlling the RGB detection mode to the face detection unit 210. Then, the process proceeds to step S205. On the other hand, if the control unit 310 judges that the setting value of the screen luminance is equal to or lower than the threshold value TH2 (NO), the process proceeds to step S207.

[0089] (Step S205) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, it performs HPD processing using the RGB camera 121. Then, the process returns to step S201.

[0090] (Step S207) When face detection unit 210 acquires a mode control signal for control to the IR detection mode, it performs HPD processing using IR camera 122. Then, the process returns to step S201.

[0091] [Summary of the second embodiment] As described above, the display unit 110 displays an image based on the setting value of the screen luminance. Even if the surrounding brightness is equal to or lower than the threshold value TH1 (an example of a first threshold value), when the setting value of the screen luminance is higher than the threshold value TH2 (an example of a second threshold value), the information processing device 1 detects the presence of a person based on a captured image (captured image data) captured using the RGB camera 121 (an example of a first imaging unit). Note that when the setting value of the screen luminance is equal to or lower than the threshold value TH2, the information processing device 1 detects the presence of a person based on a captured image captured using the IR camera 122 (an example of a second imaging unit).

[0092] As a result, the information processing device 1 can detect the approach or departure of a person regardless of the surrounding brightness by using the RGB camera 121 and the IR camera 122 according to the surrounding brightness. Also, the information processing device 1 can suppress an increase in power consumption because it uses the IR camera 122 only when the surroundings are dark and the screen brightness of the display unit 110 is low.

[0093] <Third embodiment> In the second embodiment, a configuration was described in which HPD processing is performed using the RGB camera 121 when the setting value of the screen luminance of the display unit 110 is high even in a dark environment, but when the image displayed on the display unit 110 is a dark image or the user's face is far away, the RGB camera 121 may not be able to perform HPD processing correctly. Therefore, in this embodiment, if the information processing device 1 is in a dark environment and the setting value of the screen luminance is high and the presence of a person is not detected using the RGB camera 121, the information processing device 1 performs HPD processing using the IR camera 122 to confirm whether the detection result is really correct.

[0094] For example, when the surrounding brightness is below a threshold value TH1 and the setting value of the screen brightness of the display unit 110 is higher than a threshold value TH2, if the presence of a person is not detected based on an image captured using the RGB camera 121, the face detection unit 210 detects the presence of a person based on an image captured using the IR camera 122.

[0095] FIG. 9 is a flowchart showing an example of the HPD process according to this embodiment.

[0096] (Step S301) The control unit 310 determines whether the surrounding brightness is higher than the threshold value TH1 (ambient light>TH1?) based on the detection result of the ambient light sensor 130. If the control unit 310 determines that the surrounding brightness is higher than the threshold value TH1 (YES), it outputs a mode control signal to the face detection unit 210 to control the RGB detection mode. Then, the process proceeds to step S303. On the other hand, if the control unit 310 determines that the surrounding brightness is equal to or lower than the threshold value TH1 (NO), the process proceeds to step S305.

[0097] (Step S303) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, the face detection unit 210 performs HPD processing using the RGB camera 121. Then, the process returns to step S301.

[0098] (Step S305) The control unit 310 determines whether the setting value of the screen luminance is higher than the threshold value TH2 (screen luminance>TH2?). If the control unit 310 determines that the setting value of the screen luminance is higher than the threshold value TH2 (YES), it outputs a mode control signal for controlling the RGB detection mode to the face detection unit 210. Then, the process proceeds to step S307. On the other hand, if the control unit 310 determines that the setting value of the screen luminance is equal to or lower than the threshold value TH2 (NO), the process proceeds to step S311.

[0099] (Step S307) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, the face detection unit 210 performs HPD processing using the RGB camera 121. Then, the process proceeds to step S309.

[0100] (Step S309) The face detection unit 210 determines whether or not the detection result of the HPD processing in step S307 is "True" (HPD=True?). If the face detection unit 210 determines that the detection result of the HPD processing is "True" (YES), the process returns to step S301. On the other hand, if the face detection unit 210 determines that the detection result of the HPD processing is "False" (NO), the process proceeds to step S311.

[0101] (Step S311) When face detection unit 210 acquires a mode control signal for control to the IR detection mode, it performs HPD processing using IR camera 122. Then, the process returns to step S301.

[0102] [Summary of the third embodiment] As described above, when the information processing device 1 according to this embodiment does not detect the presence of a person based on an image (imaging data) captured using the RGB camera 121 (an example of a first imaging section) when the ambient brightness is equal to or lower than the threshold value TH1 (an example of a first threshold value) and the setting value of the screen brightness of the display unit 110 is higher than the threshold value TH2 (an example of a second threshold value), the information processing device 1 detects the presence of a person based on an image (imaging data) captured using the IR camera 122 (an example of a second imaging section).

[0103] As a result, the information processing device 1 can detect approaching or leaving a person regardless of the surrounding brightness by using the RGB camera 121 and the IR camera 122 according to the surrounding brightness. Also, the information processing device 1 can suppress an increase in power consumption because it uses the IR camera 122 only when the surroundings are dark and the screen brightness of the display unit 110 is low or when no person is detected even if the screen brightness is bright.

[0104] <Fourth embodiment> In the third embodiment, when the surroundings are in a dark environment, the setting value of the screen luminance of the display unit 110 is checked, and if the presence of a person is not detected using the RGB camera 121 when the setting value of the screen luminance is high, HPD processing is performed using the IR camera 122, but the process of checking the setting value of the screen luminance may be omitted. In this embodiment, even when the surroundings are in a dark environment, the information processing device 1 first performs HPD processing using the RGB camera 121, and performs HPD processing using the IR camera 122 only if the presence of a person is not detected using the RGB camera 121.

[0105] For example, the face detection unit 210 performs HPD processing using the RGB camera 121 even when the surrounding brightness is below the threshold value TH1, and if the presence of a person is not detected based on the image captured using the RGB camera 121, it detects the presence of a person based on the image captured using the IR camera 122.

[0106] FIG. 10 is a flowchart showing an example of the HPD process according to this embodiment.

[0107] (Step S401) Control unit 310 determines whether the surrounding brightness is higher than threshold TH1 (ambient light>TH1?) based on the detection result of ambient light sensor 130. If control unit 310 determines that the surrounding brightness is higher than threshold TH1 (YES), it outputs a mode control signal to control to RGB detection mode to face detection unit 210. Then, it proceeds to processing of step S403. On the other hand, if control unit 310 determines that the surrounding brightness is equal to or lower than threshold TH1 (NO), it also outputs a mode control signal to control to RGB detection mode to face detection unit 210, but proceeds to processing of step S405.

[0108] (Step S403) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, the face detection unit 210 performs HPD processing using the RGB camera 121. Then, the process returns to step S401.

[0109] (Step S405) When the face detection unit 210 acquires a mode control signal for control to the RGB detection mode, the face detection unit 210 performs HPD processing using the RGB camera 121. Then, the process proceeds to step S407.

[0110] (Step S407) The face detection unit 210 determines whether or not the detection result of the HPD processing in step S405 is "True" (HPD=True?). If the face detection unit 210 determines that the detection result of the HPD processing is "True" (YES), the process returns to step S401. On the other hand, if the face detection unit 210 determines that the detection result of the HPD processing is "False" (NO), the process proceeds to step S409.

[0111] (Step S409) If face detection unit 210 acquires a mode control signal for control to the IR detection mode, it performs HPD processing using IR camera 122. Then, the process returns to step S401.

[0112] [Summary of the fourth embodiment] As described above, the information processing device 1 of this embodiment detects the presence of a person based on an image (imaging data) captured using the IR camera 122 (an example of a second imaging section) when the ambient brightness is below the threshold TH1 (an example of a first threshold) and the presence of a person is not detected based on an image (imaging data) captured using the RGB camera 121 (an example of a first imaging section).

[0113] As a result, the information processing device 1 can detect the approach or departure of a person regardless of the brightness of the surroundings by using the RGB camera 121 and the IR camera 122 according to the brightness of the surroundings. Also, the information processing device 1 can suppress an increase in power consumption because it uses the IR camera 122 only when the surroundings are dark and no person is detected.

[0114] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above embodiment, and the present invention also includes designs that do not deviate from the gist of the present invention. For example, the configurations described in the above embodiment can be combined in any combination.

[0115] In the above embodiment, an example has been described in which the RGB camera 121 and the IR camera 122 are switched when performing HPD processing, but the RGB camera 121 may also be used in combination when performing HPD processing using the IR camera 122.

[0116] Also, although a configuration example in which the imaging unit 120 is built into the information processing device 1 has been described, the present invention is not limited to this. For example, the imaging unit 120 does not have to be built into the information processing device 1, and may be configured to be attachable to the information processing device 1 (for example, any one of the side surfaces 10a, 10b, 10c, etc.) as an external accessory of the information processing device 1, and may be communicatively connected to the information processing device 1 wirelessly or via a wire.

[0117] In the above embodiment, an example has been shown in which the face detection unit 210 is provided separately from the CPU 301 and the chipset 303, but a part or all of the face detection unit 210 may be provided in the chipset 303, or may be provided in a processor integrated with the CPU 301 or the chipset 303. For example, the CPU 301, the chipset 303, and the face detection unit 210 may be configured as separate processors, or may be integrated into a single processor. In addition, a part or all of the face detection unit 210 may be provided in the EC 200.

[0118] The above-mentioned standby state may include a hibernation state, a power-off state, etc. The hibernation state corresponds to, for example, the S4 state defined by ACPI. The power-off state corresponds to, for example, the S5 state (shutdown state) defined by ACPI. Among the standby states, the standby state, sleep state, hibernation state, power-off state, etc. are states in which power consumption is lower (states in which power consumption is reduced) than in the normal operating state.

[0119] The information processing device 1 described above has a computer system inside. A program for implementing the functions of each component of the information processing device 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each component of the information processing device 1 described above. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. The "computer system" may also include multiple computer devices connected via a network including a communication line such as the Internet, a WAN, a LAN, or a dedicated line. The "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.

[0120] The recording medium also includes a recording medium installed inside or outside and accessible from a distribution server to distribute the program. The program may be divided into a plurality of parts, downloaded at different times, and then combined by each component of the information processing device 1, or each divided program may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a recording medium that holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network. The program may be a recording medium for implementing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0121] In addition, some or all of the functions of the information processing device 1 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually processed, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.

[0122] Furthermore, the information processing device 1 of the above embodiment is not limited to a notebook PC, but may be, for example, a desktop PC. [Explanation of symbols]

[0123] 1 Information processing device, 10 First housing, 20 Second housing, 15 Hinge mechanism, 110 Display unit, 120 Imaging unit, 121 RGB camera, 122 IR camera, 130 Ambient light sensor, 140 Power button, 150 Input device, 151 Keyboard, 153 Touch pad, 160 Communication unit, 170 Memory unit, 200 EC, 210 Face detection unit, 300 Main processing unit, 301 CPU, 302 GPU, 303 Chipset, 304 System memory, 310 Control unit, 311 HPD information acquisition unit, 312 Operation control unit, 313 Detection mode control unit, 400 Power supply unit

Claims

1. A first imaging unit that captures an image using visible light; A second imaging unit that captures an image using infrared light; a memory that acquires and temporarily stores imaging data captured by the first imaging unit or the second imaging unit; a processor for detecting the presence of a person based on the imaging data captured by the first imaging unit; a display unit that displays an image based on a screen luminance setting that can be changed by a user; Equipped with The processor, based on a result of the detection of the brightness of the surroundings, when the brightness of the surroundings is equal to or lower than a first threshold value and when the set value of the screen brightness is higher than a second threshold value, the presence of a person is detected based on the imaging data captured by the first imaging unit, and when the set value of the screen brightness is equal to or lower than the second threshold value, the presence of a person is detected based on the imaging data captured by the second imaging unit. Information processing device.

2. The processor, when the presence of a person is not detected based on the imaging data captured by the first imaging unit when the ambient brightness is equal to or lower than the first threshold value and the setting value of the screen luminance is higher than a second threshold value, the presence of a person is detected based on the imaging data captured by the second imaging unit. The information processing device according to claim 1 .

3. An ambient light sensor is provided. The processor, Detecting the brightness of the surroundings using the ambient light sensor; 3. The information processing device according to claim 1 or 2.

4. The processor, Detecting the brightness of the surroundings using the first imaging unit; 3. The information processing device according to claim 1 or 2.

5. The processor, outputting the detection result of the presence of a person as information for controlling an operating state of the system; 3. The information processing device according to claim 1 or 2.

6. 1. A control method for an information processing device including a first imaging unit that captures images using visible light, a second imaging unit that captures images using infrared light, a memory that acquires and temporarily stores imaging data captured using the first imaging unit or the second imaging unit, a processor that detects the presence of a person based on the imaging data captured using the first imaging unit, and a display unit that displays an image based on a setting value of a screen luminance that can be changed by a user, The processor, Detecting the brightness of an environment; a step of detecting the presence of a person based on the imaging data captured by the first imaging unit when the detected ambient brightness is equal to or lower than a first threshold value and when the set value of the screen brightness is higher than a second threshold value, and detecting the presence of a person based on the imaging data captured by the second imaging unit when the set value of the screen brightness is equal to or lower than the second threshold value; A control method comprising:

Images