Information processing device and control method

Abstract

The HPD function using the camera should be available even when the privacy shutter is closed, while also taking privacy concerns into consideration. [Solution] The information processing device includes: a first image sensor that captures a color image using visible light incident through a color filter; a second image sensor having a mixture of pixels for capturing an infrared image using infrared light incident through an infrared filter and pixels for capturing a monochrome image using visible light incident through a visible light filter; a shutter unit that switches between a closed state that shields the side from which light is incident on the first image sensor and an open state that opens it; a first processor that performs face recognition processing based on image data output from the first or second image sensor; and a second processor that, when the shutter unit is in the closed state, sends black image data to the first processor in place of the image data output from the second image sensor.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a control method.

Background Art

[0002] Among information processing apparatuses such as PCs (Personal Computers), those equipped with built-in cameras are becoming widespread. Some of these information processing apparatuses equipped with built-in cameras have a privacy shutter (cover) that can physically shield the imaging direction of the built-in camera from the perspective of privacy protection (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the privacy shutter is closed, the imaging direction of the built-in camera is shielded, so the camera function, face authentication at login, and the HPD (Human Presence Detection) function for detecting the presence of a person using the camera become unavailable. On the other hand, even when the privacy shutter is closed, those that perform HPD by mounting a ToF (Time of Flight) sensor or an ultrasonic sensor do not use the camera, so the HPD function can be used without causing concerns about privacy. However, when mounting a ToF or ultrasonic sensor in addition to the camera, there is a problem that the cost or power consumption increases, and it may be difficult to adopt. Therefore, there is a desire to enable the use of the HPD function using the camera while considering privacy concerns without mounting a ToF sensor or an ultrasonic sensor in addition to the camera.

[0005] The present invention has been made in view of the above circumstances, and one of its objectives is to provide an information processing device and control method that enable the use of a camera-based HPD function even when the privacy shutter is closed, while also taking into consideration privacy concerns. [Means for solving the problem]

[0006] The present invention has been made to solve the above problems, and an information processing device according to a first aspect of the present invention comprises: a first image sensor that captures a color image using visible light incident through a color filter that transmits visible light of each color; a second image sensor having a mixture of pixels for capturing an infrared image using infrared light incident through an infrared filter that transmits infrared light, and pixels for capturing a monochrome image using visible light incident through at least a visible light filter that transmits visible light; a shutter unit that switches between a closed state that shields the side from which light is incident on the first image sensor and an open state that opens it; a first processor that performs face recognition processing based on image data output from the first image sensor or image data output from the second image sensor; and a second processor that, when the shutter unit is in the closed state, sends image data determined to be a black image to the first processor in place of the image data output from the second image sensor.

[0007] The above-mentioned information processing device may further include a third processor that performs person detection processing based on imaging data output from the second imaging sensor.

[0008] In the above-described information processing device, the second processor may perform a person detection process based on the imaging data output from the second imaging sensor.

[0009] In the above-described information processing device, the first processor may determine that the shutter unit is in the closed state by acquiring image data that is determined to be a black image, and may not perform the face recognition process, but instead display information corresponding to the closed state on the display unit.

[0010] In the above-described information processing device, the second processor may detect that the shutter unit has entered the closed state based on the imaging data output from the first imaging sensor.

[0011] In the above-described information processing apparatus, the shutter unit further includes a position sensor that moves between an open position and a closed position and detects the position of the shutter unit, and the second processor may use the position sensor to detect that the shutter unit has entered the closed state.

[0012] In the above-described information processing device, the second processor may send the imaging data output from the second imaging sensor to the first processor when the shutter unit is in the open state.

[0013] Furthermore, a control method for an information processing device according to a second aspect of the present invention, comprising: a first image sensor that captures a color image using visible light incident through a color filter that transmits visible light of each color; a second image sensor having a mixture of pixels for capturing an infrared image using infrared light incident through an infrared filter that transmits infrared light and pixels for capturing a monochrome image using visible light incident through at least a visible light filter that transmits visible light; and a shutter unit that switches between a closed state that shields the side of the first image sensor from which light is incident and an open state that opens, includes the steps of: a first processor performing face recognition processing based on image data output from the first image sensor or image data output from the second image sensor; and a second processor, when the shutter unit is in the closed state, sending image data determined to be a black image to the first processor in place of the image data output from the second image sensor. [Effects of the Invention]

[0014] According to the above aspect of the present invention, the HPD function using a camera can be made available even when the privacy shutter is closed, and concerns regarding privacy can also be taken into consideration.

Brief Description of the Drawings

[0015] [Figure 1] A diagram for explaining the outline of the HPD process of the information processing apparatus according to the first embodiment. [Figure 2] A diagram showing an example of the detection range of a person by the information processing apparatus according to the first embodiment. [Figure 3] A perspective view showing a configuration example of the appearance of the information processing apparatus according to the first embodiment. [Figure 4] A schematic diagram showing the difference between the IR-Mono sensor and a general IR sensor according to the first embodiment. [Figure 5] A schematic diagram showing the appearance of the privacy shutter according to the first embodiment. [Figure 6] A schematic diagram showing the open / closed state of the privacy shutter according to the first embodiment. [Figure 7] A diagram showing an example of the hardware configuration of the information processing apparatus according to the first embodiment. [Figure 8] A diagram showing an example of a configuration for performing processing in the shutter closed state according to the first embodiment. [Figure 9] A diagram showing an example of a configuration for performing processing in the shutter open state according to the first embodiment. [Figure 10] A flowchart showing an example of processing associated with the opening and closing of the privacy shutter according to the first embodiment. [Figure 11] A diagram showing an example of a configuration for performing processing in the shutter closed state according to the second embodiment. [Figure 12] A diagram showing an example of a configuration for performing processing in the shutter open state according to the second embodiment.

Modes for Carrying Out the Invention

[0016] Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. <First Embodiment> First, the first embodiment will be described. The information processing apparatus according to this embodiment is, for example, a notebook (clamshell type) PC (Personal Computer). First, an overview of the information processing apparatus according to this embodiment will be described.

[0017] [Overview of Information Processing Apparatus] FIG. 1 is a diagram for explaining the overview of the HPD process of the information processing apparatus 1 according to this embodiment. The information processing apparatus 1 detects a person (i.e., a user) existing in the vicinity of the information processing apparatus 1. This process of detecting the presence of this person is referred to as HPD (Human Presence Detection) processing. The information processing apparatus 1 detects the presence or absence of a person by HPD processing, and controls the operating state of the system of the information processing apparatus 1 based on the detection result.

[0018] The information processing apparatus 1 can transition at least between a normal operating state (power-on state) and a standby state as the operating state of the system. The normal operating state is an operating state in which processing can be executed without particular limitation, and corresponds to, for example, the S0 state defined by ACPI (Advanced Configuration and Power Interface). The standby state is a state in which at least a part of the system processing is restricted. For example, the standby state may be a standby state, a sleep state, etc., and may also be a state corresponding to modern standby (corresponding to the S0ix state) in Windows (registered trademark) or a state corresponding to the S3 state (sleep state) defined by ACPI. The S0ix state corresponds to S0 Low Power Idle, which is an extension of the S0 state defined by ACPI. It has lower power consumption than the S0 state but faster return to the S0 state. For example, the standby state is an operating state with lower power consumption than the normal operating state. Also, the standby state includes at least a state in which the display of the display unit is off (screen off). For example, in the normal operating state, the display of the display unit is on (screen on), and when transitioning to the standby state, the display becomes off (screen off).

[0019] In the following, the transition of the system's operating state from standby to normal operation will sometimes be referred to as "startup." In standby, the system's activity level is generally lower than in normal operation; therefore, starting up the information processing device 1 system activates the system's operation.

[0020] For example, as shown in Figure 1(A), when the information processing device 1 detects a change from a state where no person is present in front of it (Absence) to a state where a person is present (Presence), i.e., when a person approaches the information processing device 1, it determines that a user has approached and automatically starts the system and transitions to the normal operating state. Also, as shown in Figure 1(B), when the information processing device 1 is present in front of it (Presence), it determines that a user is present and continues the normal operating state. Furthermore, as shown in Figure 1(C), when the information processing device 1 detects a change from a state where a person is present in front of it (Presence) to a state where a person is absent (Absence), i.e., when a person leaves the information processing device 1 (Leave), it determines that a user has left and transitions the system to the standby state.

[0021] The information processing device 1 detects the presence of a person within a predetermined range in front of it. Figure 2 shows an example of the person detection range of the information processing device 1 according to this embodiment. In the illustrated example, the detection range FoV (Field of View) in front of the information processing device 1 is the range in which a person can be detected. For example, the information processing device 1 determines whether or not a person (user) is present in front of the information processing device 1 by detecting the area of ​​the face image in which a face is captured (hereinafter referred to as the "face area") from the captured image taken in front of the information processing device 1. The detection range FoV corresponds to the image field of view captured by the information processing device 1. If the information processing device 1 detects a face area from the captured image, it determines that a user is present. On the other hand, if the information processing device 1 does not detect a face area from the captured image, it determines that a user is not present.

[0022] Furthermore, when the system starts up, the information processing device 1 performs a system authentication process to authenticate whether or not the user is a legitimate user through system processing. For example, if the information processing device 1 detects a face region from an captured image while in standby mode and determines that a user exists, it starts the system from standby mode and performs the system authentication process after startup. A legitimate user is a user that has been pre-registered as a user who will use the information processing device 1. If the information processing device 1 determines that the user is a legitimate user, it grants permission to use the system (permits login) and transitions to the normal operating state. On the other hand, if the information processing device 1 determines that the user is not a legitimate user, it does not grant permission to use the system (permits login) and continues in the authentication waiting state. This user authentication by the system authentication process at startup will be referred to as "login authentication" below.

[0023] Login authentication methods include password authentication, where the user enters a password using a keyboard; PIN authentication, where the user enters a PIN (Personal Identification Number); facial authentication, where the user's face is used for authentication; and fingerprint authentication, where the user's fingerprint is used for authentication. When facial authentication is enabled during login authentication, the information processing device 1 performs facial authentication by comparing the feature information based on the facial image of the facial region detected from the captured image with the feature information based on the facial image of a pre-registered legitimate user. As an example of facial authentication during login, Windows® Hello facial authentication can be used as an example.

[0024] Figure 3 is a perspective view showing an example of the external configuration of an information processing device according to this embodiment. The information processing device 1 comprises a first housing 10, a second housing 20, and a hinge mechanism 15. The first housing 10 and the second housing 20 are connected using the hinge mechanism 15. The first housing 10 is rotatable relative to the second housing 20 around the axis of rotation formed by the hinge mechanism 15. The opening angle due to the rotation of the first housing 10 and the second housing 20 is shown as "θ".

[0025] The first enclosure 10 is also called the A cover or display enclosure. The second enclosure 20 is also called the C cover or system enclosure. In the following description, the sides of the first enclosure 10 and the second enclosure 20 on which the hinge mechanism 15 is located are referred to as sides 10c and 20c, respectively. The sides of the first enclosure 10 and the second enclosure 20 opposite to sides 10c and 20c are referred to as sides 10a and 20a, respectively. In the diagram, the direction from side 20a toward side 20c is referred to as "rear," and the direction from side 20c toward side 20a is referred to as "front." The right and left sides relative to the rear are referred to as "right" and "left," respectively. The left sides of the first enclosure 10 and the second enclosure 20 are referred to as sides 10b and 20b, respectively, and the right sides are referred to as sides 10d and 20d, respectively. Furthermore, the state in which the first housing 10 and the second housing 20 overlap and are completely closed (opening angle θ = 0°) is called the "closed state". In the closed state, the faces of the first housing 10 and the second housing 20 facing each other are called the "inner surfaces", and the surface opposite to the inner surface is called the "outer surface". Also, the state in which the first housing 10 and the second housing 20 are open relative to the closed state is called the "open state".

[0026] The external appearance of the information processing device 1 shown in Figure 3 is an example of the open state. In the open state, the side surface 10a of the first housing 10 and the side surface 20a of the second housing 20 are separated. In the open state, the inner surfaces of the first housing 10 and the second housing 20 are exposed. The open state is one of the states in which the user uses the information processing device 1, and is typically used with an opening angle θ of approximately 100 to 130°. The range of the opening angle θ in the open state can be arbitrarily determined according to the range of angles that can be rotated by the hinge mechanism 15, etc.

[0027] A display unit 110 is provided on the inner surface of the first housing 10. The display unit 110 is composed of a liquid crystal display (LCD) or an electroluminescent (EL) display, etc. An imaging unit 120 is provided in the area surrounding the display unit 110 on the inner surface of the first housing 10. For example, the imaging unit 120 is located on the side 20a side of the area surrounding the display unit 110. Note that the position of the imaging unit 120 is just an example, and it may be placed in other positions as long as it is possible to image the direction facing the inner surface of the first housing 10 (forward).

[0028] When in the open state, the imaging unit 120 captures a predetermined imaging range in the direction facing the inner surface of the first housing 10 (forward). The predetermined imaging range is the range of the field of view determined by the image sensor of the imaging unit 120 and the optical lens provided in front of the imaging surface of the image sensor, and corresponds to the person (face) detection range FoV shown in Figure 2. For example, the imaging unit 120 can capture an image that includes a person (e.g., a user) that is in front of (facing) the information processing device 1.

[0029] Furthermore, a power button 140 is provided on the side 20b of the second housing 20. The power button 140 is an operator used by the user to instruct the power on or off, transition from standby state to normal operation state, and transition from normal operation state to standby state. In addition, a keyboard 151 and a touchpad 153 are provided on the inside of the second housing 20 as input devices that accept user input. Note that a touch sensor may be provided instead of, or in addition to, the keyboard 151 and touchpad 153 as input devices, or a mouse or external keyboard may be connected. In the configuration where a touch sensor is provided, the area corresponding to the display surface of the display unit 110 may be configured as a touch panel that accepts operation. Furthermore, the input devices may include a microphone for inputting voice.

[0030] Here, the configuration of the imaging unit 120 will be described in detail. The imaging unit 120 includes an RGB camera 121 and an IR-Mono camera 122. For example, as shown in Figure 3, the RGB camera 121 and the IR-Mono camera 122 are arranged side by side along the periphery of the display unit 110. The RGB camera 121 is an example of a camera that captures a color image using visible light incident on it through color filters that transmit visible light of each color. The RGB camera 121 includes an RGB sensor (an example of a first imaging sensor) in which R pixels equipped with a color filter that transmits the R (red) wavelength band, G pixels equipped with a color filter that transmits the G (green) wavelength band, and B pixels equipped with a color filter that transmits the B (blue) wavelength band are arranged.

[0031] The IR-Mono camera 122 is equipped with an IR-Mono sensor (an example of a second imaging sensor) that has a mixture of IR (InfraRed) pixels and monochrome (hereinafter referred to as "Mono") pixels. Figure 4 is a schematic diagram showing the difference between the IR-Mono sensor according to this embodiment and a general IR sensor.

[0032] Figure 4(A) shows an example of a typical IR sensor. An IR sensor has IR pixels arranged in a matrix to capture an IR image using infrared light incident through an infrared filter that transmits infrared light. On the other hand, Figure 4(B) shows an example of an IR-Mono sensor. An IR-Mono sensor has a mixed arrangement of IR pixels for capturing an IR image using infrared light incident through an infrared filter that transmits infrared light, and pixels for capturing Mono pixels using visible light incident through a visible light filter that transmits at least visible light.

[0033] Furthermore, the pixel arrangement pattern and the mixing ratio of IR pixels and Mono pixels can be determined arbitrarily.

[0034] In this embodiment, the mono pixels of the IR-Mono camera 122 are used during HPD processing. The IR pixels are used for applications such as facial recognition during login authentication, similar to conventional processing using general IR sensors.

[0035] Furthermore, the imaging unit 120 is provided with a privacy shutter capable of physically shielding the side from which light enters (imaging direction). However, the privacy shutter according to this embodiment can physically shield only the RGB camera 121 of the RGB camera 121 and the IR-Mono camera 122.

[0036] Figure 5 is a schematic diagram showing the appearance of the privacy shutter according to this embodiment. The privacy shutter is a mechanism that can physically block the imaging direction of the imaging unit 120 for privacy protection. In this embodiment, the user can manually open and close the privacy shutter. Figure 5(A) shows the state in which the privacy shutter is open (shutter open state). In the shutter open state, the imaging direction of the imaging unit 120 (RGB camera 121 and IR-Mono camera 122) is open and usable. Dark glass is provided on the front side (imaging direction) of the IR-Mono camera 122. The purpose of providing dark glass is mainly for design purposes. The camera lens is not visible with dark glass, creating a sense of unity in the area (bezel) that forms the frame around the periphery of the display unit 110.

[0037] The privacy shutter consists of a sliding part 120SL for manual opening and closing by the user, and a shielding member 120S whose position moves in conjunction with the movement of the sliding part 120SL. The user can close the privacy shutter by sliding the sliding part 120SL in the direction of the arrow, which moves the shielding member 120S. Figure 5(B) shows the privacy shutter in the closed state (shutter closed state).

[0038] Conventionally, in cameras (imaging units) equipped with both an RGB camera and an IR camera, closing the privacy shutter blocks the imaging direction of both the RGB camera and the IR camera. However, in this embodiment, when the shutter is closed, only the front side (imaging direction) of the RGB camera 121 is blocked by the shielding member 120S. Even when the privacy shutter is closed, the front side (imaging direction) of the IR-Mono camera 122 is not blocked by the shielding member 120S, but because it is covered with dark glass, it is not possible to visually detect whether or not it is blocked by the shielding member 120S, thus not causing any discomfort to the user.

[0039] As a result, in this embodiment, the information processing device 1 can perform HPD processing using the IR-Mono camera 122 even when the privacy shutter is closed, and the HPD function can be utilized. The state of the privacy shutter at this time is shown in Figure 6 in comparison with the state of the privacy shutter of a conventional device equipped with an RGB camera and an IR camera.

[0040] Figure 6 is a schematic diagram showing the open and closed states of the privacy shutter according to this embodiment. Conventional privacy shutters blocked both the RGB camera and the IR camera when the shutter was closed. Conventionally, HPD processing was performed using the RGB camera, and HPD processing could not be performed when the shutter was closed, making it impossible to use the HPD function.

[0041] On the other hand, in this embodiment, the privacy shutter shields only the RGB camera 121 when the shutter is closed, and does not shield the IR-Mono camera 122. By performing HPD processing using the mono pixels of the IR-Mono camera 122, HPD processing can be performed not only when the shutter is open but also when the shutter is closed, making it possible to use the HPD function even when the shutter is closed.

[0042] However, since the IR-Mono camera 122 is not obscured when the shutter is closed, it is possible to enable facial recognition during login authentication even when the privacy shutter is closed. If facial recognition during login authentication is enabled, the camera may be perceived as operating despite the privacy shutter being closed, potentially causing privacy concerns for the user. Therefore, when the shutter is closed, the image data captured by the IR-Mono camera 122 is passed to the HPD processing side, but not to the system that performs facial recognition during login authentication. This makes it possible to use the HPD function using the camera even when the privacy shutter is closed, and provides an information processing device 1 that also takes privacy concerns into consideration. The configuration of the information processing device 1 will be described in detail below.

[0043] [Hardware configuration of information processing equipment] Figure 7 is a schematic block diagram showing an example of the hardware configuration of the information processing device 1 according to this embodiment. In Figure 7, the components corresponding to each part in Figure 3 are denoted by the same reference numerals. The information processing device 1 consists of a display unit 110, an imaging unit 120, a power button 140, an input device 150, a communication unit 160, a storage unit 170, an EC (Embedded Controller) 200, a face detection unit 210, a main processing unit 300, and a power supply unit 400.

[0044] The display unit 110 displays display data (images) generated based on system processing performed by the main processing unit 300 and processing by application programs running on the system processing.

[0045] The imaging unit 120 captures an image of an object within a predetermined field of view (for example, the detection range FoV shown in Figure 2) in the direction facing the inner surface of the first housing 10 (forward), and outputs the captured image to the face detection unit 210 and the main processing unit 300. As mentioned above, the imaging unit 120 includes an RGB camera 121 and an IR-Mono camera 122.

[0046] As described above, the RGB camera 121 is a visible light camera that takes images using visible light, and is composed of an RGB sensor in which R pixels, G pixels, and B pixels are arranged. The RGB camera 121 outputs imaging data of an RGB image (visible light image) obtained by photoelectrically converting the visible light incident on the R pixels, G pixels, and B pixels through RGB color filters.

[0047] As mentioned above, the IR-Mono camera 122 is configured to include an IR-Mono sensor in which IR pixels and Mono pixels are mixed. The IR-Mono camera 122 outputs imaging data of an IR image (infrared image) obtained by photoelectric conversion of infrared light incident on the IR pixels, and imaging data of a monochrome image obtained by photoelectric conversion of visible light incident on the Mono pixels.

[0048] During HPD processing, the image data of a monochrome image obtained by photoelectric conversion of visible light incident on the mono pixels of the IR-Mono camera 122 is used. During face recognition processing for login authentication, the image data of an IR image obtained by photoelectric conversion of infrared light incident on the IR pixels is used.

[0049] The power button 140 outputs an operation signal to the EC200 in response to user operation. The input device 150 is an input unit that receives user input and includes, for example, a keyboard 151 and a touchpad 153. The input device 150 outputs an operation signal indicating the content of the operation to the EC200 in response to operation on the keyboard 151 and the touchpad 153.

[0050] The communication unit 160 connects to other devices via a wireless or wired communication network and transmits and receives various types of data. For example, the communication unit 160 is configured to include a wired LAN interface such as Ethernet® and a wireless LAN interface such as Wi-Fi®.

[0051] The storage unit 170 is comprised of storage media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), RAM (Random Access Memory), and ROM (Read Only Memory). The storage unit 170 stores various programs such as the OS, device drivers, and applications, as well as various data acquired through the operation of these programs.

[0052] The power supply unit 400 supplies power to each part of the information processing device 1 according to its operating status. The power supply unit 400 is equipped with a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of the DC power supplied from an AC (Alternate Current) / DC adapter or a battery (battery pack) to the voltage required by each part. The power whose voltage has been converted by the DC / DC converter is supplied to each part via each power supply system. For example, the power supply unit 400 supplies power to each part via each power supply system based on control signals input from the EC200.

[0053] The EC200 is a microcomputer comprising a CPU (Central Processing Unit), RAM, ROM, and I / O (Input / Output) logic circuits. The EC200's CPU reads a control program (firmware) pre-stored in its ROM, executes the read control program, and performs its functions. The EC200 operates independently of the main processing unit 300, controls the operation of the main processing unit 300, and manages its operating state. The EC200 is also connected to the power button 140, input device 150, and power supply unit 400, etc.

[0054] For example, the EC200 communicates with the power supply unit 400 to obtain information about the battery status (such as remaining capacity) from the power supply unit 400, and outputs control signals to the power supply unit 400 to control the power supply according to the operating status of each part of the information processing device 1. The EC200 also obtains operation signals from the power button 140 and the input device 150, and outputs operation signals related to the processing of the main processing unit 300 to the main processing unit 300.

[0055] The face detection unit 210 includes a processor that processes image data of the captured image captured by the imaging unit 120. The face detection unit 210 acquires image data of the captured image captured by the imaging unit 120 and temporarily stores the acquired image data in memory. The memory in which the image data is stored may be the system memory 304 or a memory (not shown) within the face detection unit 210.

[0056] For example, the face detection unit 210 acquires imaging data from the imaging unit 120 and performs face detection processing to detect face regions from the captured image based on the acquired imaging data. The face detection unit 210 also performs HPD processing to detect the presence of a person based on the results of the face detection processing. The face detection unit 210 also transmits the detection results of the face detection processing, the detection results of the HPD processing, etc., to the chipset 303 of the main processing unit 300.

[0057] The face detection unit 210 operates not only in the normal operating state but also in the standby state. In the standby state, the face detection unit 210 performs HPD processing based on the image data of a monochrome image acquired from, for example, the IR-Mono camera 122.

[0058] The main processing unit 300 is comprised of a CPU (Central Processing Unit) 301, a GPU (Graphics Processing Unit) 302, a chipset 303, and system memory 304, and is capable of executing various application programs on the OS (Operating System) through system processing based on the OS.

[0059] The CPU 301 executes processes performed by the BIOS, the OS, and application programs running on the OS. The CPU 301 controls the system's operating state based on instructions from the chipset 303 and other sources. For example, the CPU 301 performs the startup process to wake the system from a standby state. After starting from the standby state, the CPU 301 also performs login authentication to verify whether the user is a legitimate user, and if authentication is successful, transitions to the normal operating state.

[0060] For example, the CPU 301 performs facial recognition authentication during login authentication. At this time, the CPU 301 acquires image data from the imaging unit 120 via the chipset 303 and performs facial recognition authentication based on the acquired image data. The CPU 301 may also perform authentication other than facial recognition (for example, password authentication, PIN authentication, fingerprint authentication, etc.) during login authentication.

[0061] If CPU301 determines that the user is a legitimate user (authentication successful) during login authentication, it grants permission to use (allows login) and transitions to the normal operating state. On the other hand, if CPU301 determines that the user is not a legitimate user (authentication failed) during login authentication, it does not grant permission to use (allows login) and continues in a state waiting for login authentication.

[0062] The GPU 302 is connected to the display unit 110. The GPU 302 performs image processing based on the control of the CPU 301 and generates display data. The GPU 302 outputs the generated display data to the display unit 110.

[0063] The chipset 303 has functions as a memory controller and an I / O controller. For example, the chipset 303 controls the reading and writing of data from the system memory 304 and storage unit 170 by the CPU 301 and GPU 302. The chipset 303 also controls the input and output of data from the communication unit 160, display unit 110, and EC200. Furthermore, the chipset 303 has functions as a sensor hub and image processing (data processing). For example, the chipset 303 acquires imaging data from the imaging unit 120, performs processing as needed, and then passes it to the CPU 301.

[0064] The system memory 304 is used as a reading area for programs executed by the CPU 301 and as a work area for writing processing data. The system memory 304 also temporarily stores image data of images captured by the imaging unit 120.

[0065] The CPU 301, GPU 302, and chipset 303 may be integrated and configured as a single SoC (System on a Chip), or they may be configured as individual processors, either partially or individually. For example, in normal operation, the CPU 301, GPU 302, and chipset 303 are all operational, but in standby mode, at least a portion of the chipset 303 is operational. In standby mode, at least the functions necessary for HPD processing during startup are operational.

[0066] [Configuration for processing associated with opening and closing the privacy shutter] Next, we will describe a configuration that performs processing associated with the opening and closing of the privacy shutter. As mentioned above, the information processing device 1 enables the use of the HPD function by not shielding the IR-Mono camera 122 when the shutter is closed. Therefore, when the shutter is closed, it controls the device so that it does not pass the image data captured by the IR-Mono camera 122 to the system side, thereby preventing facial recognition from functioning during login authentication. In this embodiment, we will describe a configuration in which the chipset 303 executes the processing performed when these privacy shutters are opened and closed.

[0067] The chipset 303 determines whether the shutter is open or closed, and controls whether or not to pass the image data acquired from the IR-Mono camera 122 to the CPU 301 (system side) based on the determination result. First, the processing when the shutter is closed will be explained with reference to Figure 8.

[0068] Figure 8 shows an example of a configuration for processing in the shutter closed state according to this embodiment. The chipset 303 acquires imaging data output from the IR-Mono camera 122, but when the shutter is closed, it passes image data determined to be a black image to the CPU 301 (OS-based face recognition processing side) instead of the imaging data output from the IR-Mono camera 122.

[0069] Image data determined to be a black image may be image data with a value of "0", or image data with a predetermined value close to "0". The predetermined value is a value that can be determined to indicate that the shutter is closed (or that the front of the camera is being obstructed by a hand or object), and is set in advance as a design value. Image data determined to be a black image will be referred to as "black image data" below.

[0070] When the shutter is closed, the CPU 301 receives black image data from the chipset 303 during login authentication, so facial recognition does not function. Also, if the CPU 301 receives black image data from the chipset 303 during login authentication, the OS will display a message on the display unit 110 to notify the user that the area in front of the camera is obstructed (for example, "Please remove anything blocking the camera"). Note that the above message is just an example and can be anything you like.

[0071] Furthermore, the chipset 303 determines whether the shutter is open or closed based on, for example, the image data output from the RGB camera 121. For example, the chipset 303 determines that the shutter is closed if the image data output from the RGB camera 121 is black and its average value is below a predetermined threshold. Image data that is black and has an average value below a predetermined threshold corresponds to image data that can be determined to indicate that the front of the camera is blocked.

[0072] The information processing device 1 may also include a position sensor that detects the position of the shielding member 120S, which moves between a position where the shutter is open and a position where it is closed. In that case, the chipset 303 may use the position sensor to detect that the shutter is in the closed state.

[0073] Figure 9 shows an example of a configuration for processing in the shutter-open state according to this embodiment. When the shutter is open, the chipset 303 passes the image data acquired from the IR-Mono camera 122 to the CPU 301 (OS-based face recognition processing side). When the shutter is open, the CPU 301 acquires image data from the IR-Mono camera 122 (and RGB camera 121) during login authentication. Since the CPU 301 can execute face recognition processing based on the acquired image data, face recognition can be made to work.

[0074] [Processing behavior related to opening and closing the privacy shutter] Next, with reference to Figure 10, the operation of the processes performed by the chipset 303 in conjunction with the opening and closing of the privacy shutter will be explained. Figure 10 is a flowchart showing an example of the processes associated with the opening and closing of the privacy shutter according to this embodiment.

[0075] (Step S101) The chipset 303 acquires the image data output from the RGB camera 121 and proceeds to the process in step S103.

[0076] (Step S103) The chipset 303 determines whether the shutter is closed or closed based on the image data output from the RGB camera 121. If the chipset 303 determines that the shutter is closed (YES), it proceeds to step S105. On the other hand, if the chipset 303 determines that the shutter is not closed (i.e., the shutter is open) (NO), it proceeds to step S107.

[0077] (Step S105) The chipset 303 sends black image data to the CPU 301 (OS-operated face recognition processing side) in place of the image data output from the IR-Mono camera 122. This disables the face recognition function.

[0078] (Step S107) The chipset 303 sends the image data output from the IR-Mono camera 122 to the CPU 301 (the side that processes facial recognition by the OS). This enables the facial recognition function.

[0079] [Summary of the first embodiment] As described above, the information processing device 1 according to this embodiment includes an RGB camera 121, an IR-Mono camera 122, a privacy shutter shielding member 120S (an example of a shutter), a CPU 301 (an example of a first processor), and a chipset 303 (an example of a second processor). The RGB camera 121 includes an RGB sensor (an example of a first imaging sensor) that captures a color image using visible light incident through a color filter that transmits visible light for each color (R, G, B). The IR-Mono camera 122 includes an IR-Mono sensor (an example of a second imaging sensor) which has a mixture of IR pixels for capturing an IR image using infrared light incident through an infrared filter that transmits infrared light, and Mono pixels for capturing a monochrome image using visible light incident through a visible light filter that transmits at least visible light. The shielding member 120S switches between a closed state that shields the side to which light is incident on the RGB sensor and an open state that opens it. The CPU 301 performs face recognition processing based on the image data output from the RGB camera 121 or the image data output from the IR-Mono camera 122. When the shutter is closed by the shielding member 120S, the chipset 303 sends black image data (image data determined to be a black image) to the CPU 301 instead of the image data output from the IR-Mono camera 122.

[0080] As a result, when the privacy shutter is closed, the information processing device 1 keeps only the RGB camera 121 shutter closed while the IR-Mono camera 122 remains usable, thus enabling the use of the HPD function using the camera. Furthermore, when the shutter is closed, the information processing device 1 sends black image data instead of the image data output from the IR-Mono camera 122 to the CPU 301 (face recognition processing side), so face recognition does not function during login authentication, thus preventing users from having privacy concerns. Therefore, the information processing device 1 enables the use of the HPD function using the camera even when the privacy shutter is closed, while also taking privacy concerns into consideration. In addition, since the information processing device 1 does not require a shutter to physically shield the front of the IR-Mono camera 122, it is possible to miniaturize the structure and reduce costs.

[0081] Furthermore, the information processing device 1 includes a face detection unit 210 (an example of a third processor) that performs HPD processing (person detection processing) based on the image data output from the IR-Mono camera 122.

[0082] As a result, even when the privacy shutter is closed, the information processing device 1 can enable the HPD function by performing HPD processing based on the image data output from the IR-Mono camera 122.

[0083] Furthermore, the CPU 301 determines that the shutter is closed by acquiring black image data, and instead of performing face recognition processing, it displays information corresponding to the shutter being closed on the display unit 110.

[0084] As a result, when the privacy shutter is closed, the information processing device 1 receives black image data as output from the IR-Mono camera 122 even if the IR-Mono camera 122 is not physically shielded. Therefore, it recognizes that the privacy shutter is closed and can display a message on the display unit 110 indicating that the privacy shutter is closed without activating face recognition.

[0085] For example, the chipset 303 detects that the shutter is closed based on the image data output from the RGB camera 121.

[0086] This allows the information processing device 1 to appropriately detect that the privacy shutter has been closed.

[0087] The shielding member 120S moves between a position where the shutter is open and a position where the shutter is closed. The information processing device 1 may be equipped with a position sensor to detect the position of the shielding member 120S. In this case, the chipset 303 may use this position sensor to detect that the shutter is in the closed state.

[0088] This allows the information processing device 1 to appropriately detect that the privacy shutter has been closed.

[0089] Furthermore, when the shutter is open, the chipset 303 sends the image data output from the IR-Mono camera 122 to the CPU 301.

[0090] This allows the information processing device 1 to enable the facial recognition function when the privacy shutter is open.

[0091] Furthermore, the control method for the information processing device 1 according to this embodiment, which includes an RGB camera 121 equipped with an RGB sensor (an example of a first imaging sensor), an IR-Mono camera 122 equipped with an IR-Mono sensor (an example of a second imaging sensor), a shielding member 120S (an example of a shutter unit) that switches between a closed state that shields the side into which light enters the RGB sensor and an open state that opens it, a CPU 301 (an example of a first processor), and a chipset 303 (an example of a second processor), includes the steps of: the CPU 301 performing face recognition processing based on imaging data output from the RGB camera 121 or imaging data output from the IR-Mono camera 122; and the chipset 303 sending black image data (image data determined to be a black image) to the CPU 301 in place of imaging data output from the IR-Mono camera 122 when the shutter is closed by the shielding member 120S.

[0092] As a result, the control method in the information processing device 1, when the privacy shutter is closed, only the RGB camera 121 side is kept in the closed state, while the IR-Mono camera 122 side remains usable, thus enabling the use of the HPD function using the camera. Furthermore, when the shutter is closed, the control method in the information processing device 1 sends black image data instead of the image data output from the IR-Mono camera 122 to the CPU 301 (face recognition processing side), so face recognition does not function during login authentication, thus not causing users to have privacy concerns. Therefore, the control method in the information processing device 1 enables the use of the HPD function using the camera even when the privacy shutter is closed, while also taking privacy concerns into consideration. In addition, the control method in the information processing device 1 eliminates the need for a physically shielding shutter in front of the IR-Mono camera 122, enabling miniaturization of the structure and cost reduction.

[0093] <Second Embodiment> In the first embodiment, the chipset 303 performed the processing associated with the opening and closing of the privacy shutter, but in this embodiment, an embodiment in which the face detection unit 210 performs the processing will be described.

[0094] Figure 11 shows an example of a configuration for processing in the shutter closed state according to this embodiment. The face detection unit 210 acquires imaging data output from the IR-Mono camera 122, but when the shutter is closed, it passes black image data in place of the imaging data output from the IR-Mono camera 122 to the CPU 301 (OS-based face recognition processing side) via the chipset 303.

[0095] When the shutter is closed, the CPU 301 receives black image data from the chipset 303 during login authentication, so facial recognition does not function. Also, if the CPU 301 receives black image data from the chipset 303 during login authentication, the OS will display a message on the display unit 110 to notify the user that the area in front of the camera is obstructed (for example, "Please remove anything blocking the camera"). Note that the above message is just an example and can be anything you like.

[0096] Furthermore, the face detection unit 210 determines whether the shutter is open or closed based on, for example, the image data output from the RGB camera 121. For example, the face detection unit 210 determines that the shutter is closed if the value of the image data output from the RGB camera 121 is less than a predetermined value (a value of image data that allows it to be determined that the front of the camera is blocked).

[0097] The information processing device 1 may also include a position sensor that detects the position of the shielding member 120S, which moves between a position where the shutter is open and a position where it is closed. In that case, the face detection unit 210 may use the position sensor to detect that the shutter is in the closed state.

[0098] Figure 12 shows an example of a configuration for processing in the shutter-open state according to this embodiment. In the shutter-open state, the face detection unit 210 passes the image data acquired from the IR-Mono camera 122 to the CPU 301 (OS-based face recognition processing side) via the chipset 303. In the shutter-open state, the CPU 301 acquires image data from the IR-Mono camera 122 (and RGB camera 121) during login authentication. The CPU 301 can perform face recognition processing based on the acquired image data, thus enabling face recognition to function.

[0099] Furthermore, the operation of the processing associated with opening and closing the privacy shutter according to this embodiment differs from the first embodiment in that the face detection unit 210 executes each process in the flowchart shown in Figure 10 instead of the chipset 303, and when sending black image data or imaging data from the face detection unit 210 to the CPU 301 (OS-based face recognition processing side), it is sent via the chipset 303.

[0100] [Summary of the second embodiment] As described above, the information processing device 1 according to this embodiment includes an RGB camera 121, an IR-Mono camera 122, a privacy shutter shielding member 120S (an example of a shutter unit), a CPU 301 (an example of a first processor), and a face detection unit 210 (an example of a second processor). The RGB camera 121 includes an RGB sensor (an example of a first imaging sensor) that captures a color image using visible light incident through a color filter that transmits visible light for each color (R, G, B). The IR-Mono camera 122 includes an IR-Mono sensor (an example of a second imaging sensor) which has a mixture of IR pixels for capturing an IR image using infrared light incident through an infrared filter that transmits infrared light, and Mono pixels for capturing a monochrome image using visible light incident through a visible light filter that transmits at least visible light. The shielding member 120S switches between a closed state that shields the side to which light is incident on the RGB sensor and an open state that opens it. The CPU 301 performs face recognition processing based on the image data output from the RGB camera 121 or the image data output from the IR-Mono camera 122. When the shutter of the face detection unit 210 is closed by the shielding member 120S, it sends black image data (image data determined to be a black image) to the CPU 301 via the chipset 303 instead of the image data output from the IR-Mono camera 122.

[0101] As a result, when the privacy shutter is closed, the information processing device 1 keeps only the RGB camera 121 side in the closed state, while the IR-Mono camera 122 side remains usable, thus enabling the use of the HPD function using the camera. Furthermore, when the shutter is closed, the information processing device 1 sends black image data instead of the image data output from the IR-Mono camera 122 to the CPU 301 (face recognition processing side), so face recognition does not function during login authentication, thus preventing users from having privacy concerns. Therefore, similar to the first embodiment, the information processing device 1 enables the use of the HPD function using the camera even when the privacy shutter is closed, while also taking privacy concerns into consideration. In addition, similar to the first embodiment, the information processing device 1 does not require a shutter to physically shield the front of the IR-Mono camera 122, thus enabling miniaturization of the structure and cost reduction.

[0102] Furthermore, the face detection unit 210 performs HPD processing (human detection processing) based on the image data output from the IR-Mono camera 122.

[0103] As a result, the information processing device 1 can prevent face authentication from functioning during login authentication by sending black image data to the CPU 301 instead of the image data output from the IR-Mono camera 122 when the face detection unit 210 (an example of a second processor) that performs HPD processing is in the shutter closed state.

[0104] Furthermore, the control method for the information processing device 1 according to this embodiment, which includes an RGB camera 121 equipped with an RGB sensor (an example of a first imaging sensor), an IR-Mono camera 122 equipped with an IR-Mono sensor (an example of a second imaging sensor), a shielding member 120S (an example of a shutter unit) that switches between a closed state that shields the side into which light enters the RGB sensor and an open state that opens it, a CPU 301 (an example of a first processor), and a face detection unit 210 (an example of a second processor), includes the steps of: the CPU 301 executing face recognition processing based on imaging data output from the RGB camera 121 or imaging data output from the IR-Mono camera 122; and the face detection unit 210, when the shutter is closed by the shielding member 120S, sending black image data (image data determined to be a black image) to the CPU 301 via the chipset 303 in place of imaging data output from the IR-Mono camera 122.

[0105] As a result, the control method in the information processing device 1, when the privacy shutter is closed, keeps only the RGB camera 121 side in the closed state, while the IR-Mono camera 122 side remains usable, thus enabling the use of the HPD function using the camera. Furthermore, when the shutter is closed, the control method in the information processing device 1 sends black image data to the CPU 301 (face recognition processing side) instead of the image data output from the IR-Mono camera 122, so face recognition does not function during login authentication, thus preventing users from having privacy concerns. Therefore, the control method in the information processing device 1, similar to the first embodiment, enables the use of the HPD function using the camera even when the privacy shutter is closed, while also taking privacy concerns into consideration. In addition, similar to the first embodiment, the control method in the information processing device 1 eliminates the need for a physically shielding shutter in front of the IR-Mono camera 122, enabling miniaturization of the structure and cost reduction.

[0106] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configurations are not limited to the embodiments described above, and include designs and the like that do not depart from the spirit of this invention. For example, the configurations described in the embodiments described above can be combined in any way.

[0107] In the above embodiment, an example of manually opening and closing the privacy shutter was described, but it is not limited to manual operation. The opening and closing of the privacy shutter may also be triggered by operations on specific icons displayed on the display unit 110, operations on specific keys included in the keyboard 151, or specific conditions of processing performed by the CPU 301.

[0108] Furthermore, although the above embodiment describes an example in which the imaging unit 120 is built into the information processing device 1, the invention is not limited to this. For example, the imaging unit 120 does not have to be built into the information processing device 1, but may be configured to be attachable to the information processing device 1 (for example, any of the sides 10a, 10b, 10c, etc.) as an external accessory of the information processing device 1, and may be connected to the information processing device 1 wirelessly or via a wired connection.

[0109] Furthermore, although the above embodiment shows an example in which the face detection unit 210 (an example of a first processor) is provided separately from the chipset 303, part or all of the face detection unit 210 may be provided in the chipset 303, or in a processor integrated with the chipset 303. Also, the face detection unit 210, the chipset 303, and the CPU 301 may be configured as a single integrated processor. In addition, part or all of the face detection unit 210 may be provided in the EC200.

[0110] Furthermore, the standby states described above may include hibernation and power-off states. The hibernation state corresponds to, for example, the S4 state as defined by ACPI. The power-off state corresponds to, for example, the S5 state (shutdown state) as defined by ACPI. Note that among the standby states, the standby state, sleep state, hibernation state, and power-off state are states that consume less power than the normal operating state (states with reduced power consumption).

[0111] The information processing device 1 described above has a computer system inside. The processing in each configuration of the information processing device 1 may be performed by recording a program for realizing the functions of each configuration of the information processing device 1 onto a computer-readable recording medium, loading the program recorded on this recording medium into the computer system, and executing it. Here, "loading the program recorded on the recording medium into the computer system and executing it" includes installing the program into the computer system. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer system" may include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Also, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Thus, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.

[0112] Furthermore, the recording medium also includes internal or external recording media accessible from the distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined in each configuration of the information processing device 1. The distribution servers for each of the divided programs may also be different. Moreover, "computer-readable recording media" includes volatile memory (RAM) within computer systems that act as servers or clients when a program is transmitted over a network, which retains the program for a certain period of time. The program itself may also be intended to implement some of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already recorded in the computer system.

[0113] Furthermore, some or all of the functions of the information processing device 1 in the above-described embodiment may be implemented as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually processorized, or some or all of them may be integrated into a single processor. In addition, the method of implementing the integrated circuit is not limited to LSIs; it may also be implemented using dedicated circuits or general-purpose processors. Furthermore, if an integrated circuit technology that can replace LSIs emerges due to advances in semiconductor technology, an integrated circuit using that technology may be used.

[0114] The information processing device 1 may be any type of information processing device, such as a desktop PC, tablet PC, or smartphone. [Explanation of symbols]

[0115] 1 Information processing unit, 10 First housing, 20 Second housing, 15 Hinge mechanism, 110 Display unit, 120 Imaging unit, 121 RGB camera, 122 IR-Mono camera, 140 Power button, 150 Input device, 151 Keyboard, 153 Touchpad, 160 Communication unit, 170 Memory unit, 200 EC, 210 Face detection unit, 300 Main processing unit, 301 CPU, 302 GPU, 303 Chipset, 304 System memory, 400 Power supply unit

Claims

1. A first imaging sensor captures a color image using visible light incident through a color filter that transmits visible light of each color, A second imaging sensor having a mixture of pixels for capturing infrared images using infrared light incident through an infrared filter that transmits infrared light, and pixels for capturing monochrome images using visible light incident through at least a visible light filter that transmits visible light, A shutter unit that switches between a closed state that shields the side into which light enters the first imaging sensor and an open state that opens it, A first processor that performs face recognition processing based on imaging data output from the first imaging sensor or "imaging data based on the infrared image among imaging data based on the infrared image and the monochrome image output from the second imaging sensor", When the shutter unit is in the closed state, the second processor sends image data determined to be a black image to the first processor in place of the imaging data based on the infrared image and the monochrome image output from the second imaging sensor. An information processing device equipped with the following features.

2. A third processor that performs person detection processing based on imaging data derived from the monochrome image output from the second imaging sensor, The information processing apparatus according to claim 1, comprising:

3. The second processor is, Based on the imaging data derived from the monochrome image output from the second imaging sensor, a person detection process is performed. The information processing apparatus according to claim 1.

4. The first processor is, By acquiring image data that is determined to be a black image, the shutter unit determines that the shutter is in the closed state, and without executing the face recognition process, information corresponding to the closed state is displayed on the display unit. The information processing apparatus according to claim 1.

5. The second processor is, Based on the imaging data output from the first imaging sensor, the shutter unit detects that the shutter has entered the closed state. The information processing apparatus according to claim 1.

6. The shutter unit moves between the position in the open state and the position in the closed state. The system includes a position sensor that detects the position of the shutter section, The second processor is, Using the position sensor, the shutter unit detects that it has entered the closed state. The information processing apparatus according to claim 1.

7. The second processor is, When the shutter unit is in the open state, the imaging data based on the infrared image and the monochrome image output from the second imaging sensor is sent to the first processor. The information processing apparatus according to claim 1.

8. A control method for an information processing device comprising: a first image sensor that captures a color image using visible light incident through a color filter that transmits visible light of each color; a second image sensor having a mixture of pixels for capturing an infrared image using infrared light incident through an infrared filter that transmits infrared light, and pixels for capturing a monochrome image using visible light incident through at least a visible light filter that transmits visible light; and a shutter unit that switches between a closed state that shields the side of the first image sensor from light and an open state that opens it, is The first processor performs face recognition processing based on imaging data output from the first imaging sensor or "imaging data based on the infrared image among imaging data based on the infrared image and the monochrome image output from the second imaging sensor," When the second processor is in the closed state due to the shutter unit, it sends image data determined to be a black image to the first processor in place of the imaging data based on the infrared image and the monochrome image output from the second imaging sensor. A control method including

Images