Information processing device and control method

The device authenticates faces only when they are facing forward after successful login, addressing unauthorized access and optimizing power consumption through strategic camera usage.

JP2024156392AActive Publication Date: 2024-11-06LENOVO IRELAND INTERNATIONAL LTD

Patent Information

Application Number
JP2023070810
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-24
Publication Date
2024-11-06
Estimated Expiration
2043-04-24

AI Technical Summary

Technical Problem

Existing face detection methods for activating information processing devices can mistakenly activate when an unauthorized user approaches, leading to potential unauthorized access due to faces not facing forward during login authentication.

Method used

The information processing device employs a face detection process that registers and authenticates a face image only when it is facing forward within a predetermined time after successful login authentication, using both RGB and IR cameras to enhance accuracy and reduce power consumption.

Benefits of technology

This approach ensures accurate authentication of authorized users by registering face images only when they are facing forward, thereby preventing unauthorized activation and optimizing power usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156392000001_ABST
    Figure 2024156392000001_ABST
Patent Text Reader

Abstract

To properly authenticate the face of a valid user.SOLUTION: An information processing device includes a first processor executing a face detection process for detecting a region of a face image in which a face is captured from a captured image captured by an imaging unit and a face authentication process for performing face authentication triggered by information based on the detected face image and information based on the face image of a valid user, and a second processor for starting the system from a standby state based on the detection of the region of the face image by the face detection process and the success of the face authentication by the face authentication process. After the system is started, the second processor executes a system authentication process for authenticating whether or not the user is a valid user by the process of the system. The first processor registers, as the information based on the face image of the legitimate user, the information based on the face image when the orientation of the face among the face images detected by the face detection process becomes a specific orientation within a predetermined time after the success of the authentication by the system authentication process.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device and a control method. [Background technology]

[0002] There is an information processing device that transitions to a usable state when a person approaches, and transitions to a standby state in which all functions except for some functions are stopped when the person leaves. For example, in Patent Document 1, an infrared sensor is used to detect whether a person is approaching or has moved away.

[0003] In recent years, the accuracy of detecting a face from an image has improved due to the development of computer vision and the like. Therefore, face detection has begun to be used instead of detecting a person using an infrared sensor. Furthermore, when starting up an information processing device, user authentication (e.g., login authentication) is performed to determine whether the user is a legitimate user, and face authentication is becoming more common as this authentication method. For example, when a person approaches, the information processing device starts up by detecting the face of the person, and then performs login authentication such as face authentication to permit login to the system if the user is a legitimate user, transitioning to a usable state. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-148895 A Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned method of starting the device by face detection has a problem that the device may start up even if a person other than the authorized user approaches. To solve this problem, a method may be considered in which a face detected when login authentication is successful is registered as the face of the authorized user, and face authentication is performed in addition to face detection when starting the device. However, since a face is not necessarily facing forward when login authentication is successful, a face that is not facing forward may be registered even if it is the face of an authorized user. In that case, the face of the authorized user may not be properly authenticated, and the device may not start up even if, for example, a legitimate user approaches.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and has as its object to provide an information processing device and a control method capable of appropriately authenticating the face of a legitimate user. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and an information processing device according to a first aspect of the present invention comprises a memory for temporarily storing a system program, a first processor for executing a face detection process for detecting an area of ​​a facial image in which a face is captured from an image captured by an imaging unit, and a facial authentication process for performing facial authentication based on information based on the detected facial image and information based on a facial image of a legitimate user, and a second processor for starting up the system from a standby state by executing the system program based on the area of ​​the facial image being detected by the face detection process and facial authentication by the facial authentication process being successful, wherein after starting up the system, the second processor executes a system authentication process for authenticating whether or not the user is a legitimate user through the system processing, and the first processor registers information based on a facial image detected by the face detection process when the facial orientation is in a specific orientation within a predetermined time after authentication by the system authentication process is successful, as information based on the facial image of the legitimate user.

[0008] In the information processing device, the specific orientation may be an orientation based on a direction of the imaging unit.

[0009] In the above information processing device, the first processor may detect a face angle of a face image detected from the captured image within the specified time, and determine that the face orientation has become the specific orientation when the detected face angle is within a specified angle range based on the direction of the imaging unit.

[0010] In the above information processing device, the first processor may detect a face image area from within the captured image within the specified time period more frequently than the frequency of detecting a face image area from within the captured image in the standby state.

[0011] In the above information processing device, when registering information based on the facial image of the legitimate user, the first processor may register new information based on the facial image of the legitimate user if the information based on the facial image of the legitimate user is not registered, and may update the registered information if the information based on the facial image of the legitimate user is already registered.

[0012] In the above information processing device, if information based on a facial image of the legitimate user is not registered, the second processor may start the system from the standby state based on the detection of a facial image area in the captured image captured by the imaging unit in the standby state.

[0013] In the above information processing device, the second processor may transition to the standby state when, after startup of the system, a face image area is no longer detected in the captured image captured by the imaging unit.

[0014] In the above information processing device, the second processor may transition to the standby state if, after startup of the system, face authentication by face authentication processing fails even if a face image area is detected from the captured image captured by the imaging unit.

[0015] In addition, according to a second aspect of the present invention, a control method in an information processing device having a memory for temporarily storing a system program, a first processor, and a second processor includes the steps of: the first processor performing a face detection process to detect an area of ​​a facial image in which a face is captured from an image captured by an imaging unit; the first processor performing a face authentication process to perform face authentication based on information based on the detected facial image and information based on a facial image of a legitimate user; the second processor starting up the system from a standby state by executing a program of the system based on the detection of the area of ​​the facial image by the face detection process and success of face authentication by the face authentication process; the second processor executing a system authentication process to authenticate whether or not the user is a legitimate user by the system processing after starting up the system; and the first processor registering information based on a facial image detected by the face detection process when the facial orientation is in a specific orientation within a predetermined time after authentication by the system authentication process is successful, as information based on the facial image of the legitimate user. Effect of the Invention

[0016] According to the above-described aspects of the present invention, the face of a legitimate user can be properly authenticated. [Brief description of the drawings]

[0017] [Figure 1] 4 is a view for explaining an overview of HPD processing of the information processing device according to the embodiment. [Diagram 2] 5 is a diagram showing an example of a person detection range of the information processing device according to the embodiment. [Diagram 3] 4 is a diagram showing an overview of HPD processing using an HPD user ID of the information processing device according to the embodiment. [Figure 4] FIG. 1 is a perspective view showing an example of the external configuration of an information processing apparatus according to an embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 6]FIG. 2 is a block diagram showing an example of a functional configuration of an information processing apparatus according to an embodiment. [Figure 7] 11 is a flowchart showing an example of an HPD process in a standby state according to the embodiment. [Figure 8] 10 is a flowchart showing an example of login authentication processing by face authentication according to the embodiment. [Figure 9] 11 is a flowchart showing an example of an HPD user ID registration / update process according to the embodiment. [Figure 10] 11 is a flowchart showing an example of an HPD user ID registration process according to the embodiment. [Figure 11] 11 is a flowchart showing an example of an HPD user ID update process according to the embodiment. [Figure 12] 11 is a flowchart showing an example of an HPD process in a normal operation state according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, an overview of the information processing device according to the present embodiment will be described. The information processing device 1 according to the present embodiment is, for example, a notebook type (clamshell type) PC (Personal Computer). The information processing device 1 may be any type of information processing device, such as a desktop type PC, a tablet type PC, or a smartphone.

[0019] The information processing device 1 can transition between at least a normal operation state (power-on state) and a standby state as the system operation state. The normal operation state is an operation state in which processing can be executed without particular restrictions, and corresponds to, for example, the S0 state defined by ACPI (Advanced Configuration and Power Interface). The standby state is a state in which at least a part of the system processing is restricted. For example, the standby state may be a standby state, a sleep state, or the like, and may be a state equivalent to modern standby in Windows (registered trademark) or the S3 state (sleep state) defined by ACPI. For example, the standby state is an operation state in which power consumption is lower than that of the normal operation state.

[0020] In the following, the transition of the system's operating state from the standby state to the normal operating state may be referred to as "activation." In the standby state, the activity of the operation is generally lower than in the normal operating state, so activating the system of the information processing device 1 activates the operation of the system in the information processing device 1.

[0021] FIG. 1 is a diagram for explaining an outline of the HPD process of the information processing device 1 according to the present embodiment. The information processing device 1 detects a person (i.e., a user) existing in the vicinity of the information processing device 1. This process of detecting the presence of a person is called HPD (Human Presence Detection) process. The information processing device 1 detects the presence or absence of a person by the HPD process, and controls the operation state of the system of the information processing device 1 based on the detection result. For example, as shown in FIG. 1(A), when the information processing device 1 detects a change from a state in which no person is present in front of the information processing device 1 (Absence) to a state in which a person is present (Presence), that is, when it detects that a person has approached the information processing device 1 (Approach), it determines that a user has approached, automatically starts the system, and transitions to a normal operation state. Also, as shown in FIG. 1(B), when a person is present in front of the information processing device 1 (Presence), it determines that a user is present, and continues the normal operation state. Then, when the information processing device 1 detects a change from a state in which a person is present (Presence) in front of the information processing device 1 (front) to a state in which a person is not present (Absence), that is, when the information processing device 1 detects that the person has left the information processing device 1 (Leave), as shown in FIG. 1(C), it determines that the user has left and transitions the system to a standby state.

[0022] The information processing device 1 detects the presence of a person within a predetermined range ahead (on the front side). FIG. 2 is a diagram showing an example of a detection range of a person of the information processing device 1 according to the present embodiment. In the illustrated example, a detection range FoV (Field of View: detection viewing angle) in front (front side) of the information processing device 1 is a range in which a person can be detected. For example, the information processing device 1 detects a face image area (hereinafter referred to as a "face area") in which a face is captured from a captured image capturing the front (front side) to determine whether or not a person (user) is present in front (front) of the information processing device 1. The detection range FoV corresponds to an imaging angle of view captured by the information processing device 1. When a face area is detected from the captured image, the information processing device 1 determines that a user is present. On the other hand, when a face area is not detected from the captured image, the information processing device 1 determines that a user is not present.

[0023] For example, when the information processing device 1 detects a face area from a captured image in a standby state and determines that a user is present, the information processing device 1 starts the system from the standby state. After the system is started, the information processing device 1 executes a system authentication process for authenticating whether or not the user is a legitimate user through system processing. A legitimate user is a user who is registered in advance as a user who uses the information processing device 1. When the information processing device 1 determines that the user is a legitimate user, it permits use (permits login) and transitions to a normal operation state. On the other hand, when the information processing device 1 determines that the user is not a legitimate user, it does not permit use (permits login) and continues the state waiting for authentication. This user authentication by the system authentication process at the time of startup is hereinafter referred to as "login authentication".

[0024] Login authentication methods include password authentication in which a user inputs a password from a keyboard, PIN authentication in which a user inputs a PIN (Personal Identification Number), face authentication in which a user is authenticated by his / her face, fingerprint authentication in which a user is authenticated by his / her fingerprint, etc. When login authentication using face authentication is enabled, the information processing device 1 performs face authentication by comparing feature information based on a face image of a face area detected from a captured image with feature information based on a face image of a preregistered authorized user.

[0025] Here, in the case where the information processing device 1 starts up the system from a standby state only by face detection that detects a face area from a captured image in the HPD processing, the information processing device 1 also starts up when a person other than a legitimate user approaches. Therefore, the information processing device 1 registers a user ID (hereinafter referred to as "HPD user ID") for authenticating whether or not the user is a legitimate user in the HPD processing based on the authentication result at the time of login authentication.

[0026] For example, when login authentication after system startup is successful, it is assumed that a legitimate user is present in front of the information processing device 1. Therefore, the information processing device 1 detects a face area from a captured image captured when login authentication is successful, regards the detected face image as a face image of the legitimate user, and registers feature information based on the face image as an HPD user ID. When the information processing device 1 detects a face area from a captured image in HPD processing, if the face area is registered as an HPD user ID, the information processing device 1 performs control such as starting up the system, or putting the system into a standby state or putting the system into a standby state if the face area is not registered as an HPD user ID.

[0027] However, when face authentication is performed during login authentication, high power consumption is permitted to enable highly accurate face authentication, but when HPD processing is performed in standby mode, it is desirable to keep power consumption as low as possible. For example, face authentication during login authentication uses an image captured by an IR (Infrared Rays) camera, but face authentication during HPD processing in standby mode uses an image captured by an RGB camera. Also, the frame rate at which images are captured for use in face authentication during HPD processing in standby mode is set lower than the frame rate () at which images are captured for use in face authentication during login authentication.

[0028] Due to the difference between the face authentication during login authentication and the face authentication in HPD processing, for example, face authentication during login authentication is possible even if the face is tilted about ±20° from the front, but face authentication in HPD processing can only recognize the same face when the face is tilted about ±15° from the front. Therefore, it is desirable to register feature information based on a face image of the front face as much as possible as an HPD user ID. Note that the face facing forward means that the face faces the direction of the imaging unit such as the IR camera or RGB camera, and the captured image shows the front face.

[0029] For example, the information processing device 1 registers, as an HPD user ID, feature information based on a face image taken with the face facing forward within a predetermined time (e.g., within 5 seconds) after login authentication is successful. The reason for limiting the time to within the predetermined time (e.g., within 5 seconds) is to reduce the possibility of a user other than the authorized user being replaced.

[0030] Fig. 3 is a diagram showing an overview of the HPD process using the HPD user ID according to this embodiment. With reference to Fig. 3, an overview of the HPD process according to this embodiment will be described in the order of (1) to (8). Hereinafter, a captured image captured by the RGB camera of the front (front side) when the information processing device 1 is in a standby state will be referred to as a "first captured image". Meanwhile, a captured image captured by the information processing device 1 of the front (front side) with the IR camera during login authentication after system startup will be referred to as a "second captured image". (1) The information processing device 1 starts up the system by detecting a face area from a first captured image captured by an RGB camera in a standby state. At this point, the HPD user ID is not yet registered.

[0031] (2) After booting, the information processing device 1 performs login authentication to allow only authorized users to use the device. For example, the information processing device 1 detects a face area from a second captured image captured by the IR camera during a login authentication event after booting, and performs face authentication processing based on the face image of the detected face area. Note that the authentication method for login authentication may be an authentication method other than face authentication.

[0032] (3) The information processing device 1 detects a face area from a first captured image captured by the RGB camera within a predetermined time (for example, within 5 seconds) after the login authentication is successful.

[0033] (4) The information processing device 1 registers, within a predetermined time (e.g., within 5 seconds), feature information based on a facial image of the face facing forward based on a facial area detected from a first captured image captured by an RGB camera as an HPD user ID.

[0034] (5) Next, when the information processing device 1 detects a face region in the first captured image captured by the RGB camera in the standby state, the information processing device 1 performs face authentication to determine whether or not the user is a legitimate user using the HPD user ID registered in (4) above. For example, when the information processing device 1 detects a face region in the first captured image in the standby state, the information processing device 1 performs face authentication by comparing feature information based on the face image of the detected face region with the HPD user ID.

[0035] (6) If the face authentication in (5) fails, the information processing device 1 determines that the user is not a legitimate user, and continues the standby state without starting the system.

[0036] (7) On the other hand, if the face authentication in (5) is successful, the information processing device 1 determines that the user is an authorized user and starts up the system.

[0037] (8) After startup, the information processing device 1 performs the login authentication process in the same manner as in (2) above.

[0038] (9) (10) Then, in the same manner as in (3) and (4) above, within a predetermined time (e.g., within 5 seconds) after the login authentication is successful, the information processing device 1 registers (updates) as an HPD user ID feature information based on a facial image of the face facing forward based on a facial area detected from a first captured image captured by the RGB camera.

[0039] Thereafter, steps (5) to (10) are repeated. In this manner, the information processing device 1 registers, as an HPD user ID, feature information based on a facial image of the user facing forward within a predetermined time (e.g., within 5 seconds) after login authentication is successful, and performs facial authentication using the HPD user ID in HPD processing, so that the face of a legitimate user can be appropriately authenticated.

[0040] Next, the configuration of the information processing device 1 according to this embodiment will be described in detail. [External configuration of information processing device] FIG. 4 is a perspective view showing an example of the external configuration of the information processing device 1 according to the present embodiment. The information processing device 1 includes a first housing 10, a second housing 20, and a hinge mechanism 15. The first housing 10 and the second housing 20 are connected using the hinge mechanism 15. The first housing 10 is rotatable relative to the second housing 20 around a rotation axis defined by the hinge mechanism 15. The opening angle resulting from the rotation of the first housing 10 and the second housing 20 is illustrated as "θ".

[0041] The first housing 10 is also called the A cover or the display housing. The second housing 20 is also called the C cover or the system housing. In the following description, the sides of the first housing 10 and the second housing 20 on which the hinge mechanism 15 is provided are called the sides 10c and 20c, respectively. The sides of the first housing 10 and the second housing 20 opposite the sides 10c and 20c are called the sides 10a and 20a, respectively. In the drawings, the direction from the side 20a to the side 20c is called the "rear", and the direction from the side 20c to the side 20a is called the "front". The right and left sides of the rear are called the "right" and "left", respectively. The left sides of the first housing 10 and the second housing 20 are called the sides 10b and 20b, respectively, and the right sides are called the sides 10d and 20d, respectively. Moreover, a state in which the first housing 10 and the second housing 20 overlap and are completely closed (a state in which the opening angle θ=0°) is referred to as a "closed state." In the closed state, the surfaces of the first housing 10 and the second housing 20 that face each other are referred to as the "inner surfaces," and the surfaces opposite the inner surfaces are referred to as the "outer surfaces." Moreover, a state in which the first housing 10 and the second housing 20 are open relative to the closed state is referred to as an "open state."

[0042] The external appearance of the information processing device 1 shown in FIG. 4 shows an example of an open state. In the open state, the side surface 10a of the first housing 10 and the side surface 20a of the second housing 20 are separated. In the open state, the inner surfaces of the first housing 10 and the second housing 20 are exposed. The open state is one of the states when the user uses the information processing device 1, and is typically used in a state where the opening angle θ is about 100 to 130°. The range of the opening angle θ in the open state can be arbitrarily determined depending on the range of angles that can be rotated by the hinge mechanism 15, etc.

[0043] A display unit 110 is provided on the inner surface of the first housing 10. The display unit 110 is configured to include a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display. An imaging unit 120 is provided in a peripheral region of the display unit 110 on the inner surface of the first housing 10. For example, the imaging unit 120 is disposed on the side surface 20a side of the peripheral region of the display unit 110. Note that the position where the imaging unit 120 is disposed is only an example, and the imaging unit 120 may be disposed in another position as long as it is possible to capture an image in the direction facing the inner surface of the first housing 10 (forward).

[0044] In the open state, the imaging unit 120 captures an image of a predetermined imaging range in a direction (forward) facing the inner surface of the first housing 10. The predetermined imaging range is a range of an angle of view determined by an imaging element of the imaging unit 120 and an optical lens provided in front of the imaging surface of the imaging element, and corresponds to the person (face) detection range FoV shown in Fig. 2. For example, the imaging unit 120 can capture an image including a person (e.g., a user) present in front of (on the front side of) the information processing device 1.

[0045] A power button 140 is provided on the side surface 20b of the second housing 20. The power button 140 is an operator for the user to instruct turning the power on or off, transitioning from a standby state to a normal operation state, transitioning from a normal operation state to a standby state, and the like. A keyboard 151 and a touch pad 153 are provided on the inner surface of the second housing 20 as input devices for accepting user operation inputs. Note that, as the input devices, instead of or in addition to the keyboard 151 and the touch pad 153, a touch sensor may be provided, or a mouse or an external keyboard may be connected. In the case of a configuration in which a touch sensor is provided, an area corresponding to the display surface of the display unit 110 may be configured as a touch panel for accepting operations. The input device may also include a microphone into which voice is input.

[0046] [Hardware configuration of information processing device] 5 is a schematic block diagram showing an example of a hardware configuration of the information processing device 1 according to this embodiment. The information processing device 1 includes a display unit 110, an imaging unit 120, a power button 140, an input device 150, a communication unit 160, a storage unit 170, an EC (Embedded Controller) 200, a face detection unit 210, a main processing unit 300, and a power supply unit 400. The display unit 110 displays display data (images) generated based on system processing executed by the main processing unit 300 and processing of application programs running on the system processing.

[0047] The imaging unit 120 captures an image of an object within a predetermined angle of view (for example, the detection range FoV shown in FIG. 2) in the direction facing the inner surface of the first housing 10 (forward), and outputs the captured image to the face detection unit 210 and the main processing unit 300. For example, the imaging unit 120 includes an RGB camera and an IR (Infrared Rays) camera. The RGB camera is a normal camera that captures images based on visible light. The IR camera is a camera that captures images based on infrared light emitted from an object.

[0048] An IR camera, for example, irradiates infrared rays when capturing an image, and therefore consumes more power when capturing an image than an RGB camera. Therefore, when detecting a face area from the first captured image in a standby state, an RGB camera, for example, is used in order to minimize standby power consumption. On the other hand, when detecting a face area from the second captured image for face authentication in login authentication using face authentication, an IR camera, for example, is used in order to improve authentication accuracy. Note that both an IR camera and an RGB camera may be used in login authentication using face authentication.

[0049] The power button 140 outputs an operation signal to the EC 200 in response to a user operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to accepting an operation on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC 200.

[0050] The communication unit 160 is communicably connected to other devices via a wireless or wired communication network, and transmits and receives various data. For example, the communication unit 160 includes a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark).

[0051] The storage unit 170 includes storage media such as a hard disk drive (HDD), a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), etc. The storage unit 170 stores various programs such as an OS, device drivers, and applications, as well as various data acquired by the operation of the programs.

[0052] The power supply unit 400 supplies power to each unit according to the operating state of each unit of the information processing device 1. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of direct current power supplied from an AC (Alternate Current) / DC adapter or a battery (battery pack) into a voltage required by each unit. The power whose voltage has been converted by the DC / DC converter is supplied to each unit via each power supply system. For example, the power supply unit 400 supplies power to each unit via each power supply system based on a control signal input from the EC 200.

[0053] The EC200 is a microcomputer including a CPU (Central Processing Unit), RAM, ROM, and an I / O (Input / Output) logic circuit. The CPU of the EC200 reads a control program (firmware) prestored in its own ROM, and executes the read control program to fulfill its functions. The EC200 operates independently of the main processing unit 300, controls the operation of the main processing unit 300, and manages its operating state. The EC200 is also connected to a power button 140, an input device 150, a power supply unit 400, and the like.

[0054] For example, the EC 200 communicates with the power supply unit 400 to obtain information on the battery state (such as remaining capacity) from the power supply unit 400 and outputs control signals and the like for controlling the supply of power according to the operating state of each unit of the information processing device 1 to the power supply unit 400. The EC 200 also obtains operation signals from the power button 140 and the input device 150, and outputs to the main processing unit 300 operation signals related to the processing of the main processing unit 300 among the obtained operation signals.

[0055] The face detection unit 210 includes a processor that processes image data of a captured image captured by the imaging unit 120. The face detection unit 210 acquires image data of a captured image captured by the imaging unit 120, and temporarily stores the acquired image data in a memory. The memory that stores the image data may be the system memory 304, or may be a memory (not shown) in the face detection unit 210.

[0056] For example, the face detection unit 210 performs face detection processing to detect a face area from the captured image and face authentication processing to authenticate a face image in the detected face area by processing image data of the captured image acquired from the imaging unit 120. The face detection unit 210 transmits the detection result of the face detection processing, the authentication result of the face authentication processing, and the like to the chipset 303 of the main processing unit 300.

[0057] The face detection unit 210 operates not only in a normal operating state but also in a standby state. In the standby state, the face detection unit 210 obtains image data of a first captured image captured by, for example, the RGB camera of the imaging unit 120, and detects a face area. By capturing an image using the RGB camera without using an IR camera, it is possible to reduce power consumption in the standby state (standby power). On the other hand, in the login authentication process after startup, the face detection unit 210 obtains image data of a second captured image captured by the IR camera (or both the RGB camera and the IR camera) of the imaging unit 120, and detects a face area and performs face authentication, thereby improving the accuracy of face authentication.

[0058] For example, the face detection unit 210 performs face authentication using the HPD user ID registered when authentication by the login authentication process is successful in the HPD process described with reference to Fig. 3. The functional configuration of this HPD process will be described in detail later.

[0059] The main processing unit 300 is composed of a CPU (Central Processing Unit) 301, a GPU (Graphic Processing Unit) 302, a chipset 303, and a system memory 304, and is capable of executing processing of various application programs on an OS (Operating System) through system processing based on the OS.

[0060] The CPU 301 executes processes by the BIOS, processes by the OS, processes by application programs running on the OS, etc. The CPU 301 controls the operating state of the system based on instructions from the chipset 303, etc. For example, the CPU 301 executes a startup process to start the system from a standby state. After starting from the standby state, the CPU 301 also executes a login authentication process to authenticate whether or not the user is a legitimate user, and transitions to a normal operating state if the authentication is successful.

[0061] For example, the CPU 301 executes authentication processing by face authentication in login authentication. Note that the CPU 301 may execute authentication processing other than face authentication (for example, password authentication, PIN authentication, fingerprint authentication, etc.) in login authentication.

[0062] When CPU 301 determines that the user is a legitimate user (authentication successful) in the login authentication, it permits use (permits login) and transitions to a normal operation state. On the other hand, when CPU 301 determines that the user is not a legitimate user (authentication failed) in the login authentication, it does not permit use (permits login) and continues the state waiting for login authentication.

[0063] The GPU 302 is connected to the display unit 110. The GPU 302 executes image processing and generates display data under the control of the CPU 301. The GPU 302 outputs the generated display data to the display unit 110.

[0064] The chipset 303 has a function as a memory controller and a function as an I / O controller. For example, the chipset 303 controls the reading and writing of data from the system memory 304, the storage unit 170, etc. by the CPU 301 and the GPU 302. The chipset 303 also controls the input and output of data from the communication unit 160, the display unit 110, and the EC 200. The chipset 303 also has a function as a sensor hub. For example, in the HPD process, the chipset 303 obtains a face detection result from the face detection unit 210, detects the presence of a person (user) based on the detection result, and controls the operating state of the system.

[0065] The system memory 304 is used as a reading area for the programs executed by the CPU 301, a working area for writing processing data, etc. The system memory 304 also temporarily stores image data of images captured by the imaging unit 120.

[0066] The CPU 301, GPU 302, and chipset 303 may be configured as a single integrated processor, or some or each may be configured as individual processors. For example, in a normal operation state, the CPU 301, GPU 302, and chipset 303 are all in operation, but in a standby state, at least a part of the chipset 303 is in operation. In the standby state, at least the functions necessary for the HPD processing at the time of startup are in operation.

[0067] [Functional configuration of information processing device] Next, a functional configuration in which the information processing device 1 controls the operating state of the system through HPD processing will be described.

[0068] FIG. 6 is a block diagram showing an example of a functional configuration of the information processing device 1 according to this embodiment. The information processing device 1 includes a face detection unit 210, a system processing unit 310, and an HPD control processing unit 330. The face detection unit 210 corresponds to the face detection unit 210 shown in FIG. 5, and is a functional configuration realized by executing a program in the face detection unit 210. The system processing unit 310 and the HPD control processing unit 330 are functional configurations realized by the main processing unit 300 shown in FIG. 5 executing an OS and a program running on the OS. For example, the system processing unit 310 is a functional configuration realized by the CPU 301 executing a program of the OS. Also, the HPD control processing unit 330 is a functional configuration realized by the chipset 303 executing a program running on the OS.

[0069] The face detection unit 210 includes a face detection processing unit 211, a face authentication processing unit 212, an HPD user ID registration unit 213, and an HPD processing unit 214. The system processing unit 310 includes an operation control unit 311 and an authentication processing unit 312. The HPD control processing unit 330 includes an operation instruction unit 331 and an authentication result notification unit 332.

[0070] The face detection processing unit 211 reads out image data of captured images captured by the imaging unit 120 at a predetermined frame rate (predetermined frequency) from the system memory 304, and detects a face area from each captured image. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects a face based on facial feature information, or face detection using learning data (trained model) machine-learned based on facial feature information or a face detection library. In addition, the frame rate at which the imaging unit 120 captures an image is controlled to, for example, 4 FPS in a standby state, 15 FPS during login authentication, 15 FPS within a predetermined time (for example, within 5 seconds) after login authentication is successful, and 1 FPS in a normal operating state.

[0071] For example, in a standby state, the face detection processing unit 211 detects a face area from a first captured image (RGB image) captured at 4 FPS using the RGB camera of the imaging unit 120, and outputs coordinate information of the face area, etc., as a detection result. Also, for example, during login authentication, the face detection processing unit 211 detects a face area from a second captured image (IR image) captured at 15 FPS using the IR camera of the imaging unit 120, and outputs coordinate information of the face area, etc., as a detection result. Note that, during login authentication, the face detection processing unit 211 may detect a face area from a second captured image (IR image and RGB image) captured using the IR camera and RGB camera of the imaging unit 120.

[0072] Furthermore, for example, within a predetermined time (for example, within 5 seconds) after login authentication is successful, the face detection processing unit 211 detects a face area from a first captured image (RGB image) captured at 15 FPS using the RGB camera of the imaging unit 120, and outputs the coordinate information of the face area and information on the face direction as the detection result. The information on the face direction is the face angle of the face image in the detected face area. For example, the face angle is an angle corresponding to the face direction with the direction of the imaging unit 120 (the direction in which the face faces forward) as the reference angle (for example, 0°).

[0073] Also, for example, in a normal operating state, the face detection processing unit 211 detects a face area from a first captured image (RGB image) captured at 1 FPS using the RGB camera of the imaging unit 120, and outputs coordinate information of the face area as the detection result.

[0074] The face authentication processing unit 212 executes face authentication processing to determine whether or not a face image in the detected face area is a face image of a legitimate user. For example, during login authentication by system processing, the face authentication processing unit 212 performs face authentication by comparing a face image detected from a second captured image (IR image) captured at 15 FPS using the IR camera of the imaging unit 120 with a face image of a legitimate user registered in advance as user information (account information) for logging in to the system. Specifically, for example, during login authentication, the face authentication processing unit 212 performs face authentication by comparing feature information based on the face image detected from the second captured image (IR image) with feature information based on the face image of a legitimate user registered in advance.

[0075] The facial image of a legitimate user registered as user information (account information) is, for example, a facial image detected from a second captured image (IR image) captured at 15 FPS using the IR camera of the imaging unit 120 in the facial recognition image registration menu of the system settings.

[0076] Furthermore, the face authentication processing unit 212 performs face authentication based on feature information based on a face image detected from a first captured image (RGB image) captured at 15 FPS using the RGB camera of the imaging unit 120 in the HPD processing and feature information based on a face image of a legitimate user registered as an HPD user ID. For example, the face authentication processing unit 212 performs face authentication by comparing feature information based on a face image detected from a first captured image (RGB image) captured at 15 FPS using the RGB camera of the imaging unit 120 in the HPD processing with feature information based on a face image of a legitimate user registered as an HPD user ID.

[0077] The HPD user ID registration unit 213 registers, within a predetermined time (for example, within 5 seconds) after authentication by the login authentication process is successful, feature information based on a face image when the face is facing forward among face images detected by the face detection processing unit 211 as an HPD user ID (feature information based on a face image of a legitimate user). For example, the HPD user ID registration unit 213 stores and registers the HPD user ID in the system memory 304 or a memory (not shown) in the face detection unit 210.

[0078] For example, the HPD user ID registration unit 213 determines that the face is facing forward when the face angle of the face image detected by the face detection processing unit 211 is within a predetermined angle range (e.g., ±15°) based on the direction of the imaging unit 120 (the direction in which the face faces forward) within a predetermined time (e.g., within 5 seconds) after authentication by the login authentication process is successful. When registering an HPD user ID, the HPD user ID registration unit 213 registers a new HPD user ID if it is not registered, and updates the registered HPD user ID if it is already registered.

[0079] The HPD processing unit 214 outputs information indicating the detection result of the HPD processing based on the detection result of the face detection processing by the face detection processing unit 211 and the authentication result of the face authentication processing by the face authentication processing unit 212. For example, if an HPD user ID has been registered, and if a face area is detected from the first captured image (RGB image) by the face detection processing unit 211 and face authentication by the face authentication processing unit 212 is successful, the HPD processing unit 214 outputs presence information indicating that a legitimate user is present in front of the information processing device 1.

[0080] When the HPD user ID is not registered, the HPD processing unit 214 outputs presence information based on the detection of a face area in the first captured image (RGB image) by the face detection processing unit 211 in the standby state, assuming that a user is present in front of the information processing device 1, although it cannot determine whether the user is a legitimate user.

[0081] On the other hand, when the face detection processing unit 211 no longer detects a face area in the first captured image (RGB image) after the system is started, the HPD processing unit 214 outputs absence information indicating that a user is not present in front of the information processing device 1. Furthermore, when the face detection processing unit 211 detects a face area in the first captured image (RGB image) after the system is started, but face authentication by the face authentication processing unit 212 fails, the HPD processing unit 214 outputs absence information because the user present in front of the information processing device 1 is not a legitimate user.

[0082] In the standby state, when operation instruction unit 331 acquires presence information output from HPD processing unit 214, it issues an instruction to start the system from the standby state. For example, operation instruction unit 331 outputs start-up instruction information indicating an instruction to start the system from the standby state to operation control unit 311. Furthermore, in the standby state, while operation instruction unit 331 acquires absence information from HPD processing unit 214, it does not issue an instruction to start the system.

[0083] In addition, in the normal operation state, the operation instruction unit 331 issues an instruction to transition the system to a standby state when it acquires absence information from the HPD processing unit 214. For example, the operation instruction unit 331 outputs, to the operation control unit 311, standby instruction information indicating an instruction to transition the system to the standby state.

[0084] When the operation control unit 311 acquires the start-up instruction information output from the operation instruction unit 331, it executes a system program and starts the system from a standby state. After starting the system from the standby state, the operation control unit 311 generates a login authentication event. Then, the operation control unit 311 waits without permitting login until authentication is successful. If authentication is successful, the operation control unit 311 permits login and transitions to a normal operation state.

[0085] In addition, in the normal operation state, when the operation control section 311 receives standby instruction information output from the operation instruction section 331, it transitions the system from the normal operation state to the standby state.

[0086] When a login authentication event occurs, the authentication processing unit 312 executes login authentication processing to authenticate whether or not the user is a legitimate user. For example, the authentication processing unit 312 executes login authentication processing by any one of face authentication, password authentication, PIN authentication, and fingerprint authentication. For example, when executing authentication processing by face authentication, the authentication processing unit 312 acquires the authentication result of face authentication from the face detection unit 210, and executes login authentication processing based on the acquired authentication result. If authentication is successful, the authentication processing unit 312 determines that the user is a legitimate user, and if authentication is unsuccessful, determines that the user is not a legitimate user.

[0087] In the case of password authentication or PIN authentication, the authentication processing unit 312 acquires an operation signal based on a user's operation on the keyboard 151 via the EC 200. Then, based on the acquired operation signal, the authentication processing unit 312 executes the authentication process by comparing the password or PIN entered by the user's operation with a password or PIN registered in advance. In the case of fingerprint authentication, the authentication processing unit 312 executes the authentication process by comparing a fingerprint acquired using a fingerprint sensor (not shown) with a fingerprint registered in advance.

[0088] The authentication result notification unit 332 notifies the face detection unit 210 of the authentication result of the login authentication by the authentication processing unit 312. The HPD user ID registration unit 213 of the face detection unit 210 is triggered by the notification of the authentication result of the login authentication to register, as an HPD user ID, feature information based on a face image when facing forward within a predetermined time (e.g., within 5 seconds) after authentication by the login authentication process is successful.

[0089] [HPD processing operation] Next, the operation of the process of authenticating a legitimate user in the HPD process and controlling the operating state of the system will be described. First, the operation of the HPD process in the standby state will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the HPD process in the standby state according to this embodiment.

[0090] (Step S101) The face detection unit 210 acquires a first captured image (RGB image) captured by the imaging unit 120 in a standby state. In the standby state, the first captured image (RGB image) is captured by the imaging unit 120 at a frame rate of, for example, 4 FPS. Then, the process proceeds to step S103.

[0091] (Step S103) The face detection unit 210 detects a face area (area of ​​a face image) from the first captured image (RGB image) acquired in step S101. If the face detection unit 210 does not detect a face area (NO), the process returns to step S101. On the other hand, if the face detection unit 210 detects a face area (YES), the process proceeds to step S105.

[0092] (Step S105) The face detection unit 210 determines whether or not the HPD user ID has been registered. If the face detection unit 210 determines that the HPD user ID has not been registered (NO), the process proceeds to step S111. On the other hand, if the face detection unit 210 determines that the HPD user ID has been registered (YES), the process proceeds to step S107.

[0093] (Step S107) The face detection unit 210 performs face authentication based on feature information based on the face image detected in step S103 and feature information based on the face image of the authorized user registered as the HPD user ID. Then, the process proceeds to step S109.

[0094] (Step S109) The face detection unit 210 determines whether or not the face authentication was successful in step S107. If the face detection unit 210 determines that the face authentication was successful (YES), the process proceeds to step S111. On the other hand, if the face detection unit 210 determines that the face authentication was unsuccessful (NO), the process returns to step S101.

[0095] (Step S111) Face detection unit 210 outputs presence information. On acquiring the presence information output from face detection unit 210, HPD control processing unit 330 outputs start-up instruction information indicating an instruction to start the system from a standby state to system processing unit 310 as a start-up instruction by HPD processing. On acquiring the start-up instruction information output from HPD control processing unit 330, system processing unit 310 executes a system program to start the system from the standby state. Note that even if there is a start-up instruction other than the start-up instruction by HPD processing (such as an operation on power button 140), system processing unit 310 executes a system program to start the system from the standby state. Then, the process proceeds to step S113.

[0096] (Step S113) After starting up from a standby state, the system processing unit 310 generates a login authentication event and executes authentication processing to authenticate whether or not the user is a valid user. For example, the system processing unit 310 executes authentication processing by any one of face authentication, password authentication, PIN authentication, and fingerprint authentication.

[0097] Here, as an example of the login authentication process in step S113, the operation of the login authentication process using face authentication will be described with reference to FIG. FIG. 8 is a flowchart showing an example of login authentication processing by face authentication according to this embodiment.

[0098] (Step S151) When the system processing unit 310 generates a login authentication event, the face detection unit 210 executes face detection processing and face authentication processing in the login authentication. First, the face detection unit 210 acquires a second captured image (IR image, or IR image and RGB image) captured by the imaging unit 120. In the login authentication processing, the second captured image (IR image, or IR image and RGB image) is captured by the imaging unit 120 at a frame rate of, for example, 15 FPS. Then, the process proceeds to step S153.

[0099] (Step S153) The face detection unit 210 detects a face area from the second captured image (IR image, or IR image and RGB image) acquired in step S151. If the face detection unit 210 does not detect a face area (NO), the process returns to step S151. On the other hand, if the face detection unit 210 detects a face area (YES), the process proceeds to step S155.

[0100] (Step S155) Face detection unit 210 performs face recognition processing based on the face image of the face region detected in step S153. For example, face detection unit 210 performs face recognition processing based on feature information of the face image of the detected face region and feature information of the face image of a regular user registered in advance as user information (account information) for logging in to the system, and outputs the recognition result to system processing unit 310.

[0101] (Step S157) The system processing unit 310 judges whether or not the login authentication is successful based on the authentication result of the face authentication processing output from the face detection unit 210. For example, if the authentication result of the face authentication processing is failure (face authentication is unsuccessful), the system processing unit 310 judges that the login authentication is unsuccessful (NO), does not permit the login, and returns to step S151. On the other hand, if the authentication result of the face authentication processing is success (face authentication is successful), the system processing unit 310 judges that the login authentication is successful (YES), permits the login, and ends the login authentication processing. Also, if the login authentication by the system processing unit 310 is successful, the HPD control processing unit 330 notifies the face detection unit 210 of the authentication result of the login authentication. Then, the process proceeds to step S115 in FIG. 7.

[0102] Although an example of login authentication processing using face authentication has been described here, in the case of login authentication processing using password authentication, PIN authentication, fingerprint authentication, or the like, if login authentication is successful, login is permitted, the login authentication processing is terminated, and processing proceeds to steps S115A and S115B in Fig. 7. There is no particular order in which the processing in steps S115A and S115B is performed, and they are performed in parallel.

[0103] (Step S115A) When face detection section 210 receives a notification of the authentication result of login authentication in response to a success of login authentication by system processing section 310, face detection section 210 executes HPD user ID registration / update processing.

[0104] (Step S115B) The system processing unit 310 executes login processing with the account of the authorized user whose login authentication has been successful, and transitions to a normal operation state. In the normal operation state, the face detection unit 210 detects a face area from the first captured image (RGB image) captured by the imaging unit 120 at a frame rate of, for example, 1 FPS, and executes HPD processing.

[0105] Here, the specific operation of the HPD user ID registration / update process in step S115 will be described with reference to FIG. 9 to FIG. FIG. 9 is a flowchart showing an example of the HPD user ID registration / update process according to this embodiment.

[0106] The face detection unit 210 determines whether the HPD user ID has been registered (step S201). If the face detection unit 210 determines that the HPD user ID has not been registered (NO), the process proceeds to step S203, where the HPD user ID registration process is executed. On the other hand, if the face detection unit 210 determines that the HPD user ID has been registered (YES), the process proceeds to step S205, where the HPD user ID update process is executed.

[0107] The HPD user ID registration process in step S203 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the HPD user ID registration process according to this embodiment. In a normal operation state, the face detection unit 210 detects a face area from a first captured image (RGB image) captured by the imaging unit 120 at a frame rate of, for example, 1 FPS and executes the HPD process, but when executing the HPD user ID registration process and the HPD user ID update process, the frame rate is increased.

[0108] (Step S211) The face detection unit 210 increases the frame rate to, for example, 15 FPS, and proceeds to the process of step S213.

[0109] (Step S213) The face detection unit 210 acquires a first captured image (RGB image) captured by the imaging unit 120 at a frame rate of, for example, 15 FPS, and proceeds to the process of step S103.

[0110] (Step S215) The face detection unit 210 detects a face area (area of ​​a face image) from the first captured image (RGB image) acquired in step S213. If the face detection unit 210 does not detect a face area (NO), the process proceeds to step S223. On the other hand, if the face detection unit 210 detects a face area (YES), the process proceeds to step S217.

[0111] (Step S217) The face detection unit 210 detects the face direction (face angle) of the face image in the face area detected in step S215, and then proceeds to the process of step S219.

[0112] (Step S219) Face detection unit 210 determines whether the face detected in step S217 faces forward. For example, if the face angle detected in step S217 is within a predetermined angle range (for example, within ±5°) with respect to the direction of imaging unit 120 (forward, face angle=0°), face detection unit 210 determines that the face faces forward. If face detection unit 210 determines that the face does not face forward (NO), it proceeds to processing of step S223. On the other hand, if face detection unit 210 determines that the face faces forward (YES), it proceeds to processing of step S221A.

[0113] (Step S221A) The face detection unit 210 registers the feature information based on the face image in which the face direction is determined to be the front as an HPD user ID (feature information based on the face image of a legitimate user). Then, the process proceeds to step S225.

[0114] (Step S223) The face detection unit 210 determines whether or not a predetermined time (e.g., 5 seconds) has elapsed since the login authentication was successful. If the face detection unit 210 determines that the predetermined time (e.g., 5 seconds) has not elapsed (NO), the process returns to step S213. On the other hand, if the face detection unit 210 determines that the predetermined time (e.g., 5 seconds) has elapsed (YES), the process proceeds to step S225.

[0115] (Step S225) The face detection unit 210 lowers the frame rate to, for example, 1 FPS, and ends the HPD user ID registration process.

[0116] Next, the HPD user ID update process in step S205 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the HPD user ID update process according to this embodiment. The HPD user ID update process shown in this figure differs from the HPD user ID registration process shown in Fig. 10 only in the process of step S221B. In step S221B, the face detection unit 210 registers feature information based on a face image in which the face is determined to be facing forward as the HPD user ID instead of the registered HPD user ID, and updates it. Note that the processes other than step S221B of the HPD user ID update process are similar to the HPD user ID registration process shown in Fig. 10, and therefore description thereof will be omitted.

[0117] Next, the operation of the HPD process in the normal operation state will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the HPD process in the normal operation state according to this embodiment.

[0118] (Step S301) The face detection unit 210 acquires a first captured image (RGB image) captured by the imaging unit 120 in a normal operating state. In the normal operating state, the first captured image (RGB image) is captured by the imaging unit 120 at a frame rate of, for example, 1 FPS. Then, the process proceeds to step S303.

[0119] (Step S303) The face detection unit 210 detects a face area (area of ​​a face image) from the first captured image (RGB image) acquired in step S301. If the face detection unit 210 does not detect a face area (NO), the process proceeds to step S311 since no user is present. On the other hand, if the face detection unit 210 detects a face area (YES), the process proceeds to step S305.

[0120] (Step S305) The face detection unit 210 determines whether or not the HPD user ID has been registered. If the face detection unit 210 determines that the HPD user ID has not been registered (NO), the process returns to step S301. On the other hand, if the face detection unit 210 determines that the HPD user ID has been registered (YES), the process proceeds to step S307.

[0121] (Step S307) The face detection unit 210 performs face authentication based on feature information based on the face image detected in step S303 and feature information based on the face image of the authorized user registered as the HPD user ID. Then, the process proceeds to step S309.

[0122] (Step S309) The face detection unit 210 determines whether or not the face authentication was successful in step S307. If the face detection unit 210 determines that the face authentication was successful (YES), the process returns to step S301 since a legitimate user exists. On the other hand, if the face detection unit 210 determines that the face authentication was unsuccessful (NO), the process proceeds to step S311 since the user is not a legitimate user.

[0123] (Step S311) The face detection unit 210 outputs the absence information. When the HPD control processing unit 330 acquires the absence information output from the face detection unit 210, it outputs to the system processing unit 310, as an instruction to the standby state by the HPD processing, standby instruction information indicating a standby instruction to transition the system to the standby state. For example, the HPD control processing unit 330 outputs standby instruction information to the system processing unit 310 after a certain time has elapsed since the state in which the absence information output from the face detection unit 210 has been acquired. When the system processing unit 310 acquires the standby instruction information output from the HPD control processing unit 330, it transitions the system from the normal operation state to the standby state. Note that the system processing unit 310 also transitions the system from the normal operation state to the standby state when there is a standby instruction other than the standby instruction by the HPD processing. The standby instruction other than the HPD is, for example, a state of no operation elapses for a certain time, a user performs an operation to transition to a sleep state, and the like. In addition, in the standby state, the face detection unit 210 acquires the first captured image (RGB image) captured by the imaging unit 120 in the same manner as in the normal operation state, but switches the frame rate to, for example, 4 FPS. That is, in the standby state, the face detection unit 210 detects a face area from the first captured image (RGB image) captured by the imaging unit 120 at a frame rate of, for example, 4 FPS, and executes the HPD process.

[0124] [Summary of the embodiment] As described above, the information processing device 1 according to the present embodiment includes a system memory 304 (an example of a memory) that temporarily stores a system program, a face detection unit 210 (an example of a first processor), and a system processing unit 310 (for example, a CPU 301, an example of a second processor). The face detection unit 210 executes a face detection process for detecting a region (face region) of a face image in which a face is captured from an image captured by the imaging unit 120, and a face authentication process for performing face authentication based on feature information (an example of information) based on the detected face image and feature information (an example of information) based on a face image of a legitimate user. The system processing unit 310 starts the system from a standby state by executing a system program based on the detection of a face region by the face detection process and the success of face authentication by the face authentication process. In addition, the system processing unit 310 executes a login authentication process (an example of a system authentication process) for authenticating whether or not the user is a legitimate user by system processing after the system is started. Furthermore, the face detection unit 210 registers feature information based on a face image detected by the face detection process when the face is facing forward (an example of a specific orientation) within a predetermined time (e.g., within 5 seconds) after authentication by the login authentication process is successful, as feature information based on the face image of the authorized user (e.g., an HPD user ID).

[0125] As a result, when the information processing device 1 uses the face image of a user detected when login authentication is successful as the face image of a legitimate user for face authentication, the face image takes into consideration the direction of the face for face authentication, so that the face of the legitimate user can be properly authenticated.

[0126] The face facing forward (an example of a specific direction) is a direction based on the direction of the imaging unit 120.

[0127] As a result, the information processing device 1 uses a face image of a front face facing the direction of the imaging unit 120 for face authentication, and therefore the accuracy of face authentication can be improved.

[0128] For example, the face detection unit 210 detects the face angle of a face image detected from a captured image within a specified time (e.g., within 5 seconds) after successful login authentication, and determines that the face is facing forward (an example of a specific orientation) if the detected face angle is within a specified angle range (e.g., within ±5°) based on the direction of the imaging unit 120.

[0129] As a result, the information processing device 1 determines the direction of the face by detecting the face angle of the detected face image and uses the front face image for face authentication, thereby improving the accuracy of face authentication.

[0130] In addition, the face detection unit 210 sets a frame rate (an example of a frequency) for detecting a face area from a captured image within a predetermined time (e.g., within 5 seconds) after successful login authentication to be higher than the frame rate for detecting a face area from a captured image in a standby state.

[0131] This allows the information processing device 1 to accurately detect the face angle of a face image detected from a captured image within a predetermined time (for example, within 5 seconds) after successful login authentication.

[0132] In addition, when registering an HPD user ID (feature information based on a facial image of a legitimate user), the face detection unit 210 registers a new HPD user ID if the HPD user ID has not been registered, and updates the registered information if the HPD user ID has already been registered.

[0133] This allows the information processing device 1 to update the HPD user ID to the latest information, and therefore allows the face of a legitimate user to be properly authenticated.

[0134] Furthermore, if the HPD user ID is not registered, the system processing unit 310 starts up the system from the standby state based on the fact that a face area is detected from within an image captured by the imaging unit 120 in the standby state.

[0135] As a result, when the HPD user ID is not registered, the information processing device 1 cannot determine whether the user is a legitimate user or not, so it starts up both when a legitimate user approaches and when a user other than a legitimate user approaches, and then, if the user is a legitimate user, the login authentication is successful and the HPD user ID can be registered.

[0136] Furthermore, after the system is started up, if a face area is no longer detected in an image captured by the imaging section 120, the system processing section 310 transitions to a standby state.

[0137] This allows the information processing device 1 to save power when not in use and improve security.

[0138] In addition, after the system is started, if face authentication by the face authentication process fails even if a face area is detected in the captured image captured by the imaging unit 120, the system processing unit 310 transitions to a standby state.

[0139] This allows the information processing device 1 to save power when not being used by an authorized user, and also improves security.

[0140] Further, the control method in the information processing device 1 according to the present embodiment includes a step in which the face detection unit 210 (an example of a first processor) performs face detection processing to detect a region (face region) of a face image in which a face is captured from a captured image captured by the imaging unit 120, a step in which the face detection unit 210 performs face authentication processing based on information based on the detected face image (e.g., feature information) and information based on a face image of a legitimate user (e.g., feature information), and a step in which the system processing unit 310 (e.g., the CPU 301, an example of a second processor) performs system authentication based on the fact that the face region has been detected by the face detection processing and that face authentication by the face authentication processing has been successful. the step of starting the system from a standby state by executing the program, the step of the system processing unit 310 executing a login authentication process (one example of a system authentication process) for authenticating whether or not the user is a legitimate user through system processing after the system is started, and the step of the face detection unit 210 registering, within a predetermined time (e.g., within 5 seconds) after authentication by the login authentication process is successful, information (e.g., feature information) based on a face image when the face is facing forward (one example of a specific direction) among the face images detected by the face detection process as information (e.g., feature information) based on the face image of the legitimate user.

[0141] As a result, when the control method in the information processing device 1 uses the facial image of a user detected when login authentication is successful as the facial image of a legitimate user for facial authentication, the facial image takes into account the facial orientation, thereby enabling the face of the legitimate user to be properly authenticated.

[0142] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above embodiment, and the present invention also includes designs that do not deviate from the gist of the present invention. For example, the configurations described in the above embodiment can be combined in any combination.

[0143] The imaging unit 120 may include either an IR camera or an RGB camera. For example, an IR camera may be used in both the HPD process and the login authentication process, or an RGB camera may be used in both the HPD process and the login authentication process.

[0144] The frame rate of the imaging unit 120 in each operation state and process is merely an example and is not limited to the above-mentioned frame rate example. For example, it is preferable to set the frame rate in the HPD user registration / update process higher than the frame rate in the standby state and normal operation state, but the frame rate may be set to a different frame rate from the above-mentioned frame rate example.

[0145] In the above embodiment, the configuration example in which the imaging unit 120 is built into the information processing device 1 has been described, but the present invention is not limited to this. For example, the imaging unit 120 does not have to be built into the information processing device 1, and may be configured to be attachable to the information processing device 1 (for example, any one of the side surfaces 10a, 10b, 10c, etc.) as an external accessory of the information processing device 1, and may be communicatively connected to the information processing device 1 wirelessly or via a wire.

[0146] Furthermore, the CPU 301 (an example of a third processor) and the chipset 303 (an example of a second processor) may be configured as separate processors, or may be integrated into a single processor.

[0147] In the above embodiment, an example has been shown in which the face detection unit 210 (an example of a first processor) is provided separately from the chipset 303, but a part or all of the face detection unit 210 may be provided in the chipset 303, or in a processor integrated with the chipset 303. The face detection unit 210, the chipset 303, and the CPU 301 may be integrated into one processor. The face detection unit 210 may be provided in part or all of the EC 200.

[0148] The above-mentioned standby state may include a hibernation state, a power-off state, etc. The hibernation state corresponds to, for example, the S4 state defined by ACPI. The power-off state corresponds to, for example, the S5 state (shutdown state) defined by ACPI. Among the standby states, the standby state, sleep state, hibernation state, power-off state, etc. are states in which power consumption is lower (states in which power consumption is reduced) than in the normal operating state.

[0149] The information processing device 1 described above has a computer system inside. A program for implementing the functions of each component of the information processing device 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each component of the information processing device 1 described above. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. The "computer system" may also include multiple computer devices connected via a network including a communication line such as the Internet, a WAN, a LAN, or a dedicated line. The "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.

[0150] The recording medium also includes a recording medium installed inside or outside and accessible from a distribution server to distribute the program. The program may be divided into a plurality of parts, downloaded at different times, and then combined by each component of the information processing device 1, or each divided program may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a recording medium that holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network. The program may be a recording medium for implementing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0151] In addition, some or all of the functions of the information processing device 1 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually processed, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]

[0152] 1 Information processing device, 10 First housing, 20 Second housing, 15 Hinge mechanism, 110 Display unit, 120 Imaging unit, 140 Power button, 150 Input device, 151 Keyboard, 153 Touch pad, 160 Communication unit, 170 Memory unit, 200 EC, 210 Face detection unit, 211 Face detection processing unit, 212 Face authentication processing unit, 213 HPD user ID registration unit, 214 HPD processing unit, 300 Main processing unit, 301 CPU, 302 GPU, 303 Chipset, 304 System memory, 310 System processing unit, 311 Operation control unit, 312 Authentication processing unit, 330 HPD control processing unit, 331 Operation instruction unit, 332 Authentication result notification unit, 400 Power supply unit

Claims

1. A memory for temporarily storing system programs; a first processor that executes a face detection process for detecting a region of a face image in which a face is captured from an image captured by an imaging unit, and a face authentication process for performing face authentication based on information based on the detected face image and information based on a face image of a legitimate user; a second processor that starts up the system from a standby state by executing a program of the system based on the fact that the area of ​​the face image is detected by the face detection processing and the face authentication by the face authentication processing is successful; Equipped with The second processor After the system is started, a system authentication process is executed to authenticate whether the user is a regular user or not by the processing of the system; The first processor, registering, within a predetermined time period after the authentication by the system authentication process is successful, information based on a face image detected by the face detection process when the face is oriented in a specific direction as information based on the face image of the authorized user; Information processing device.

2. The specific orientation is an orientation based on the direction of the imaging unit. The information processing device according to claim 1 .

3. The first processor, detecting a face angle of a face image detected from the captured image within the predetermined time period, and determining that the face orientation has become the specific orientation when the detected face angle is within a predetermined angle range based on a direction of the imaging unit; The information processing device according to claim 2 .

4. The first processor, a frequency at which a face image area is detected from the captured image within the predetermined time period is set to be higher than a frequency at which a face image area is detected from the captured image in the standby state; The information processing device according to claim 3 .

5. The first processor, When registering information based on the face image of the authorized user, if the information based on the face image of the authorized user has not been registered, the information is newly registered, and if the information based on the face image of the authorized user has already been registered, the registered information is updated. The information processing device according to claim 1 .

6. If the information based on the face image of the authorized user is not registered, The second processor activating the system from the standby state based on the detection of a face image area from the captured image captured by the imaging unit in the standby state; The information processing device according to claim 1 .

7. The second processor when a face image area is no longer detected in the captured image captured by the imaging unit after the system is started up, transitioning to the standby state. The information processing device according to claim 1 .

8. The second processor After the system is started, if face authentication by face authentication processing fails even if a face image area is detected from the captured image captured by the imaging unit, the system transitions to the standby state. The information processing device according to claim 1 .

9. A control method for an information processing device including a memory for temporarily storing a system program, a first processor, and a second processor, comprising: a step of performing face detection processing by the first processor to detect a face image area in which a face is captured from a captured image captured by an imaging unit; A step in which the first processor performs a face recognition process for performing face recognition based on information based on the detected face image and information based on a face image of a legitimate user; a step in which the second processor starts up the system from a standby state by executing a program of the system based on the fact that the area of ​​the face image has been detected by the face detection processing and the face authentication by the face authentication processing has been successful; a step of executing a system authentication process by the second processor after starting up the system, for authenticating whether or not the user is a legitimate user through processing of the system; a step of registering, by the first processor, information based on a face image detected by the face detection process within a predetermined time period after authentication by the system authentication process has been successful, the face image being in a specific direction, as information based on the face image of the authorized user; A control method comprising:

Citation Information

Patent Citations

  • Implementation of biometric authentication

    CN110020521A

  • Information processing system, information processing device, and information processing method

    JP2016225824A

  • Electronic apparatus and control method

    JP2021179831A

  • Authentication device and method for controlling the same, and authentication system

    JP2023032239A

  • Instrument with face authentication function

    JP2016148895A

Cited By

  • Information processing device and control method

    JP7869377B1