Information processing device and control method
The information processing device uses face detection and authentication to balance power saving and security by adjusting screen brightness and locking the system based on user presence, ensuring quick recovery and secure operation.
Patent Information
- Application Number
- JP2024070601
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-04-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-04-24
AI Technical Summary
Existing systems face challenges in balancing power saving and security when a user is briefly absent, as reducing screen brightness for power saving can lead to security risks if the system remains unlocked for an extended period.
An information processing device that uses face detection and authentication to determine if a user is present and paying attention, transitioning to low power mode if the user is absent briefly, and locking the system only if the user is not a registered user after a short absence, ensuring security without prolonged unlocking.
The device achieves both power saving and security by quickly adjusting screen brightness and locking the system based on user presence and authentication, preventing unauthorized access while minimizing user inconvenience.
Smart Images

Figure 0007590613000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device and a control method. [Background technology]
[0002] For example, Patent Document 1 discloses a PC (personal computer) that uses an infrared sensor to detect when a person approaches or leaves, starts up the system when it detects that a person has approached, and turns off the display and transitions to a standby state when it detects that the person has left. In the standby state, in addition to reducing power consumption by turning off the display, security is also ensured by locking the system.
[0003] Furthermore, in recent years, the accuracy of face detection from an image has improved due to developments in computer vision and the like. Therefore, instead of detecting people using infrared sensors, people are also detected by detecting faces from captured images. Person detection by face detection can not only detect people but also detect the direction of the face, making it possible to perform control according to the direction of the face. For example, when the face is not facing forward (toward the screen), control is performed to reduce the screen brightness of the display unit to save power. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2020-102151 A Summary of the Invention [Problem to be solved by the invention]
[0005] For example, if the screen brightness is lowered to save power when the user's face is turned to the side, the screen brightness can be restored immediately when the user turns to face forward again, because this is the control that only adjusts the brightness. On the other hand, if the system is locked when the person leaves and is absent, user authentication is required to unlock the system the next time the system is used, which is troublesome. Therefore, although it is acceptable to perform power saving when the user turns to the side based on a short-term decision since the system can be restored immediately, there is a demand to not lock the system if the user is absent for only a short time. However, if the time between detecting the absence of a person and locking the system is lengthened, there is a concern about security during that time.
[0006] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objectives is to provide an information processing device and a control method that, when locking the system in response to the absence of a person, controls the system so that it is not locked if the person is absent for a short period of time while ensuring security. [Means for solving the problem]
[0007] The present invention has been made to solve the above-mentioned problems, and an information processing device according to a first aspect of the present invention includes a memory that temporarily stores a system program, a first processor that controls an operation of the system by executing the system program, and a second processor that performs a detection process of detecting a person present in a direction facing a display unit and detecting whether or not the person is looking at the display unit based on an image captured by an imaging unit, and a face authentication process of comparing a face image of the person present in the direction facing the display unit with a face image of a pre-registered user, wherein the first processor performs a power reduction process of transitioning to a low power consumption mode when the presence of a person looking at the display unit is no longer detected from a state in which the presence of the person is detected by the detection process, a determination process of determining whether or not the person is a pre-registered user based on an authentication result of the face authentication process, if the presence of a person looking at the display unit is detected by the detection process after transitioning to the low power consumption mode, and a power recovery process of returning from the low power consumption mode when the presence of a person looking at the display unit is detected by the detection process and it is determined by the determination process that the person is a pre-registered user. After transitioning to the low power consumption mode, if the detection process detects the presence of a person looking at the display unit and the judgment process determines that the person is not a pre-registered user, a lock process is performed to lock the system.
[0008] In the above information processing device, the first processor performs an unlocking process of unlocking the system on condition that authentication is successful in a system authentication process for authenticating whether or not the user is a regular user pre-registered in the system after locking the system; may be carried out.
[0009] In the above-mentioned information processing device, the first processor may, in the locking process, lock the system when the presence of a person looking at the display unit is no longer detected in the direction facing the display unit after the detection process has detected the presence of a person, and when a first time has elapsed without detecting the presence of a person, and when, after transitioning to the low power consumption mode, the detection process detects the presence of a person looking at the display unit and the determination process determines that the person is not a pre-registered user, lock the system in response to the detection and determination without waiting for the first time.
[0010] In the above-mentioned information processing device, the first processor may, in the power reduction process, transition to the low power consumption mode when the presence of a person looking at the display unit is no longer detected by the detection process from a state in which the presence of a person looking at the display unit was detected, or when a second time shorter than the first time has elapsed while the presence of a person looking at the display unit is not detected.
[0011] In the above-mentioned information processing device, the first processor may, in the locking process, lock the system when a third time period has elapsed without detecting any user input regardless of the transition to the low power consumption mode, and, after the transition to the low power consumption mode, when the detection process detects the presence of a person looking at the display and the determination process determines that the person is not a pre-registered user, lock the system in response to the detection and determination without waiting for the third time period.
[0012] In the above-mentioned information processing device, the first processor may, in the power reduction process, transition to the low power consumption mode when the presence of a person looking at the display unit is no longer detected by the detection process from a state in which the presence of a person looking at the display unit was detected, or when a second time shorter than the third time has elapsed while the presence of a person looking at the display unit is not detected.
[0013] In the information processing device, the low power consumption mode may be an operation mode in which a screen brightness of the display unit is reduced.
[0014] Also, an information processing device according to a second aspect of the present invention includes a memory that temporarily stores a system program, a first processor that controls an operation of the system by executing the system program, and a second processor that performs a detection process to detect a person present in a direction facing a display unit based on an image captured by an imaging unit, the second processor performs a first face authentication process to compare a first face image of the person present in the direction facing the display unit with a first face image for authentication of a pre-registered user, and the first processor performs a lock process to lock the system when the presence of a person is detected in the direction facing the display unit by the detection process and the first face authentication process determines that the person is not a pre-registered user, and a second face authentication process to compare a second face image of the person present in the direction facing the display unit, which is obtained based on an image captured by an imaging unit under conditions different from the first face image, with a second face image for authentication of a pre-registered user. When the second face authentication process determines that the person is a pre-registered user, an unlock process is performed to unlock the system.
[0015] According to a third aspect of the present invention, there is provided a control method for an information processing device including a memory for temporarily storing a system program, a first processor for controlling an operation of the system by executing the system program, and a second processor for performing a detection process for detecting a person present in a direction facing a display unit and detecting whether or not the person is looking at the display unit based on an image captured by an imaging unit, and a face authentication process for comparing a face image of the person present in the direction facing the display unit with a face image of a user registered in advance, the control method comprising the steps of: when the first processor detects the presence of a person looking at the display unit by the detection process but no longer detects the presence of the person, the control method comprising the steps of: the step of switching the system to the low power consumption mode; if, after the transition to the low power consumption mode, the detection process detects the presence of a person looking at the display unit, determining whether or not the person is a pre-registered user based on an authentication result by the face recognition process; if, after the transition to the low power consumption mode, the detection process detects the presence of a person looking at the display unit and it is determined that the person is a pre-registered user, returning from the low power consumption mode; and if, after the transition to the low power consumption mode, the detection process detects the presence of a person looking at the display unit and it is determined that the person is not a pre-registered user, locking the system.
[0016] In addition, according to a fourth aspect of the present invention, a control method in an information processing device including a memory for temporarily storing a system program, a first processor for controlling operation of the system by executing the system program, and a second processor for performing a detection process for detecting a person present in a direction facing a display unit based on an image captured by an imaging unit includes a first face authentication step in which the second processor compares a first face image of the person present in the direction facing the display unit with a first authentication face image of a pre-registered user, a step in which the first processor locks the system when the presence of a person is detected in the direction facing the display unit by the detection process and the first face authentication step determines that the person is not a pre-registered user, a second face authentication step in which the first processor compares a second face image of the person present in the direction facing the display unit, obtained based on an image captured by an imaging unit under conditions different from the first face image, with a second authentication face image of the pre-registered user, and a step of unlocking the system when the second face authentication step determines that the person is a pre-registered user. Effect of the Invention
[0017] According to the above-described aspects of the present invention, when the system is locked in response to the absence of a person, it is possible to ensure security while controlling so that the system is not locked if the person is absent for a short period of time. [Brief description of the drawings]
[0018] [Figure 1] FIG. 1 is a perspective view showing an example of the external configuration of an information processing apparatus according to an embodiment. [Diagram 2] 5 is a diagram showing an example of a person detection range of the information processing device according to the embodiment. [Diagram 3] 4 is a view for explaining an overview of HPD processing of the information processing device according to the embodiment. [Figure 4] 5A and 5B are schematic diagrams illustrating control of screen luminance according to the face direction according to the embodiment. [Diagram 5]6A and 6B are diagrams illustrating a comparison between a screen brightness reduction process and a system lock process according to the embodiment. [Figure 6] 6 is a diagram showing an overview of a process for returning the screen luminance from low luminance according to the embodiment; [Figure 7] FIG. 1 is a schematic block diagram showing an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 8] FIG. 1 is a schematic block diagram showing an example of a functional configuration of an information processing apparatus according to an embodiment. [Figure 9] 6 is a flowchart showing an example of a screen brightness reduction process and a system lock process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a perspective view showing an example of the external configuration of an information processing device 1 according to the present embodiment.
[0020] The information processing device 1 is, for example, a notebook (clamshell) type PC (Personal Computer). The information processing device 1 includes a first housing 10, a second housing 20, and a hinge mechanism 15. The first housing 10 and the second housing 20 are connected using the hinge mechanism 15. The first housing 10 is rotatable relative to the second housing 20 around a rotation axis formed by the hinge mechanism 15. The opening angle resulting from the rotation of the first housing 10 and the second housing 20 is illustrated as "θ".
[0021] The first housing 10 is also called the A cover or the display housing. The second housing 20 is also called the C cover or the system housing. In the following description, the sides of the first housing 10 and the second housing 20 on which the hinge mechanism 15 is provided are called the sides 10c and 20c, respectively. The sides of the first housing 10 and the second housing 20 opposite the sides 10c and 20c are called the sides 10a and 20a, respectively. In the drawings, the direction from the side 20a to the side 20c is called the "rear", and the direction from the side 20c to the side 20a is called the "front". The right and left sides of the rear are called the "right" and "left", respectively. The left sides of the first housing 10 and the second housing 20 are called the sides 10b and 20b, respectively, and the right sides are called the sides 10d and 20d, respectively. Moreover, a state in which the first housing 10 and the second housing 20 overlap and are completely closed (a state in which the opening angle θ=0°) is referred to as a "closed state." In the closed state, the surfaces of the first housing 10 and the second housing 20 that face each other are referred to as the "inner surfaces," and the surfaces opposite the inner surfaces are referred to as the "outer surfaces." Moreover, a state in which the first housing 10 and the second housing 20 are open relative to the closed state is referred to as an "open state."
[0022] The external appearance of the information processing device 1 shown in FIG. 1 shows an example of an open state. In the open state, the side surface 10a of the first housing 10 and the side surface 20a of the second housing 20 are separated. In the open state, the inner surfaces of the first housing 10 and the second housing 20 are exposed. The open state is one of the states when the user uses the information processing device 1, and is typically used in a state where the opening angle θ is about 100 to 130°. The range of the opening angle θ in the open state can be arbitrarily determined depending on the range of angles that can be rotated by the hinge mechanism 15, etc.
[0023] A display unit 110 is provided on the inner surface of the first housing 10. The display unit 110 is configured to include a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display. An imaging unit 120 is provided in a peripheral region of the display unit 110 on the inner surface of the first housing 10. For example, the imaging unit 120 is disposed on the side surface 20a side of the peripheral region of the display unit 110. Note that the position where the imaging unit 120 is disposed is merely an example, and the imaging unit 120 may be disposed in another location as long as it can face in a direction facing the display screen of the display unit 110.
[0024] In the open state, the imaging unit 120 captures an image of a predetermined imaging range in a direction facing the display screen of the display unit 110 (i.e., in front of the information processing device 1). The predetermined imaging range is a range of an angle of view determined by an imaging element of the imaging unit 120 and an optical lens provided in front of the imaging surface of the imaging element. For example, the imaging unit 120 can capture an image including a person (user) present in front of the information processing device 1 (on the front side).
[0025] A power button 140 is provided on the side surface 20b of the second housing 20. The power button 140 is an operator for the user to instruct power on or off, transition from a standby state to a normal operation state, transition from the normal operation state to a standby state, etc. The normal operation state is an operation state of the system in which processing can be executed without particular restrictions, and corresponds to, for example, the S0 state defined by ACPI (Advanced Configuration and Power Interface).
[0026] The standby state is a state in which at least a part of the system processing is restricted, and in which power consumption is lower than in the normal operation state. For example, the standby state is a state in which the display of the display unit 110 is turned off (screen off) and the system is locked, and may be a standby state, a sleep state, or the like, or may be a state equivalent to modern standby in Windows (registered trademark) or the S3 state (sleep state) defined by ACPI. The locked state of the system means a state in which the system cannot be used unless it is unlocked, and in order to unlock it, user authentication of the system is required.
[0027] Further, on the inner surface of the second housing 20, a keyboard 151 and a touch pad 153 are provided as input devices that accept operation inputs from the user. Note that, instead of or in addition to the keyboard 151 and the touch pad 153, a touch sensor may be provided as the input device, or a mouse or an external keyboard may be connected. In the case where a touch sensor is provided, an area corresponding to the display screen of the display unit 110 may be configured as a touch panel that accepts operations. Furthermore, the input device may include a microphone into which voice is input.
[0028] In addition, when the first housing 10 and the second housing 20 are in a closed state, the display unit 110 and the imaging unit 120 provided on the inner surface of the first housing 10, and the keyboard 151 and the touchpad 153 provided on the inner surface of the second housing 20 are covered by the surface of the other housing, and are unable to function.
[0029] The information processing device 1 executes a human presence detection (HPD) process to detect a person present in front of the information processing device 1, based on a captured image captured by the imaging unit 120.
[0030] 2 is a diagram showing an example of a person detection range of the information processing device 1 according to this embodiment. In the example shown in the figure, a detection range FoV (Field of View: detection viewing angle) in front of the information processing device 1 is a range in which a person can be detected.
[0031] For example, the information processing device 1 detects a face area in which a face is captured from a captured image captured by the imaging unit 120, thereby determining whether or not a person (user) is present in front of the information processing device 1. The detection range FoV corresponds to, for example, the imaging angle of view captured by the imaging unit 120. Note that the detection range FoV may be a range narrower than the imaging angle of view captured by the imaging unit 120. When a face area is detected from the captured image, the information processing device 1 determines that a person is present. On the other hand, when a face area is not detected from the captured image, the information processing device 1 determines that a person is not present.
[0032] The information processing device 1 controls the operation state of the system of the information processing device 1 according to the presence or absence of a person through HPD processing. For example, when a person is present in front of the information processing device 1, the information processing device 1 controls the system to a normal operation state, and when no person is present in front of the information processing device 1, the information processing device 1 transitions to a standby state and locks the system.
[0033] FIG. 3 is a diagram for explaining an outline of the HPD process of the information processing device 1 according to the present embodiment. The information processing device 1 detects a person present in front of the information processing device 1 by the HPD process, and controls the operation state of the system of the information processing device 1 based on the presence or absence of the person. For example, as shown in FIG. 3A, when the information processing device 1 detects a change from a state in which no person is present in front of the information processing device 1 (No presence) to a state in which a person is present (Presence), that is, when the information processing device 1 detects an approach of a person to the information processing device 1 (Approach), in the standby state, the system is automatically started and transitioned to a normal operation state. Also, as shown in FIG. 3B, in the normal operation state, when the information processing device 1 detects a state in which a person is present in front of the information processing device 1 (Presence), the normal operation state is continued. Also, as shown in FIG. 3C, when the information processing device 1 detects a change from a state in which a person is present in front of the information processing device 1 (Presence) to a state in which a person is not present (No presence), that is, when the information processing device 1 detects a departure of the person from the information processing device 1 (Leave), the system transitions to a standby state and locks the system.
[0034] Furthermore, in the HPD process, the information processing device 1 detects the face direction of a person present in front of the information processing device 1. The face direction here refers to, for example, a direction corresponding to a rotation angle in the left-right direction and a direction corresponding to a rotation angle in the up-down direction of the face. In the following, a state in which the face is facing in the direction of the information processing device 1 (the direction of the display unit 110 and the imaging unit 120) is defined as a state in which the face is facing forward.
[0035] FIG. 4 is a schematic diagram for explaining the control of screen luminance according to the face direction according to the present embodiment. As shown in this figure, the information processing device 1 performs a process (screen luminance reduction process) of reducing the screen luminance of the display unit 110 according to the face direction of a person (user). Specifically, the information processing device 1 reduces the screen luminance of the display unit 110 when the face direction is not directed to the front as shown in FIG. 4(B) (for example, when the face is turned to the side) as compared with when the face direction is directed to the front as shown in FIG. 4(A). Since the purpose of reducing the screen luminance is to save power, it can also be said to be a transition to a low power consumption mode. In addition, when the face direction is directed to the front again, the information processing device 1 returns the screen luminance to the original level before reduction.
[0036] Here, the original screen luminance before reduction is, for example, the screen luminance when the system is in a normal operating state, and is based on a luminance setting value initially set in the system or a luminance setting value changed by the user from the luminance setting value. Hereinafter, the screen luminance based on the initially set luminance setting value or a luminance setting value changed by the user from the luminance setting value is referred to as "standard luminance". In contrast, the screen luminance reduced from the standard luminance is referred to as "low luminance". Low luminance is at least a luminance lower than the standard luminance, but the lower the luminance, the greater the effect of power saving. The luminance setting value of low luminance is preset, and is set to, for example, a value of about 0 to 10% of the standard luminance value. A luminance of 0% is visually equivalent to a screen-off state. Note that when the face is not facing forward, this is not limited to when the face is facing sideways, but also includes when the face is facing up, down, or backwards.
[0037] Also, the state in which the face is facing forward is referred to as "Attention" since the face is focused on the display screen of the display unit 110 of the information processing device 1. On the other hand, the state in which the face is not facing forward is referred to as "No attention" since the face is not focused on the display screen of the display unit 110 of the information processing device 1.
[0038] For example, when the state changes from "Attention" to "No attention", the information processing device 1 changes the screen brightness of the display unit 110 from standard brightness to low brightness (transition to low power consumption mode). In addition, when the state changes to "No presence" (i.e., when "Leave" is detected), the information processing device 1 also changes to the "No attention" state, so first, the screen brightness of the display unit 110 is changed from standard brightness to low brightness (transition to low power consumption mode), and then, after a predetermined time has elapsed, the information processing device 1 transitions to a standby state, turns off the screen, and locks the system. In this way, when the user is not paying attention to the display screen of the display unit 110 or the user is absent, the information processing device 1 saves power by lowering the screen brightness of the display unit 110 to low brightness, and further, when a predetermined time has elapsed while the user is absent, locks the system to ensure security.
[0039] Here, referring to Fig. 5, the screen brightness reduction process performed by the information processing device 1 depending on the direction of a person's face and absence will be compared with the system lock process performed due to continued absence of a person. Fig. 5 is a diagram showing the comparison between the screen brightness reduction process and the system lock process. As described above, the screen brightness reduction process is a process aimed at power saving, and performs an operation of reducing the screen brightness. The screen brightness reduction process only adjusts the brightness, so that the screen brightness quickly returns after being reduced. Therefore, the time (timer) from when "No attention" (or "No presence") is detected until the screen brightness is reduced is set to a relatively short time (for example, 10 seconds).
[0040] On the other hand, as described above, the system lock process is a process for security purposes, and performs the operation of turning off the screen and locking the system. The system lock process is time-consuming because it requires unlocking when returning to the system. If the system is locked simply because of a short absence, it will be time-consuming to unlock it each time. Therefore, the time (timer) from when "no presence" is detected until the system is locked is often set relatively long (for example, about 30 seconds to 5 minutes) so that the system will not be locked if the absence is too short.
[0041] However, although the brightness of the screen is reduced to save power when the user is away, there are concerns about security if the system is left unlocked for a long period of time. For example, if the system is left unlocked for a long period of time while the user is away, there is a growing concern that the system may be accessed by people other than authorized users.
[0042] Therefore, in the information processing device 1 of this embodiment, when "Attention" is detected during a period in which the screen brightness is reduced due to the detection of "No attention" (or "No presence"), the information processing device 1 restores the screen brightness to standard brightness only if the person is a registered user (i.e., a legitimate user), and if the person is an unregistered user, immediately locks the system and makes it unusable until the lock is released by user authentication or the like.
[0043] FIG. 6 is a diagram showing an overview of the process of returning the screen brightness from low brightness according to this embodiment. (1) When the information processing device 1 detects "No attention" from a state where "Attention" is detected, (2) it changes the screen brightness of the display unit 110 from standard brightness to low brightness. (3) When the information processing device 1 detects "Attention" in a state where the screen brightness is low (a state where "No attention" is detected), (4) it determines whether or not the user is a registered user. (5) When the information processing device 1 determines that the user is a registered user (YES), it returns the screen brightness to standard brightness. (6) On the other hand, when the information processing device 1 determines that the user is an unregistered user (NO), it turns off the screen and locks the system.
[0044] In this way, the information processing device 1 reduces the screen brightness when the user is not paying attention to the display screen or when the user is absent, and then immediately locks the system if an unregistered user is detected before the system is locked. Thus, the information processing device 1 can achieve both power saving and security when the user is absent for a short period of time.
[0045] This ensures security until the system is locked even if the time (timer) from when the absence of a person is detected until when the system is locked is set long so that the system is not locked if the person is absent (absent) for a short time (so that the user does not have to go through the trouble of unlocking the system when the system is used). Therefore, the information processing device 1 can control the system so that the system is not locked if the user is absent for a short time while ensuring security.
[0046] The configuration of the information processing device 1 according to this embodiment will be described in detail below. [Hardware configuration of information processing device] Fig. 7 is a schematic block diagram showing an example of a hardware configuration of an information processing device 1 according to this embodiment. In Fig. 7, components corresponding to those in Fig. 1 are given the same reference numerals. The information processing device 1 includes a display unit 110, an imaging unit 120, a power button 140, an input device 150, a communication unit 160, a storage unit 170, an EC (Embedded Controller) 200, a face detection unit 210, a main processing unit 300, and a power supply unit 400.
[0047] The display unit 110 displays display data (images) generated based on system processing executed by the main processing unit 300 and processing of application programs running on the system processing.
[0048] The imaging unit 120 captures an image of an object within a predetermined imaging range (angle of view) in a direction (forward) facing the inner surface of the first housing 10, and outputs the captured image to the main processing unit 300 and the face detection unit 210. For example, the imaging unit 120 includes a visible light camera (RGB camera) that captures images using visible light, and an infrared camera (IR camera) that captures images using infrared light.
[0049] The imaging section 120 may be configured to include either a visible light camera or an infrared camera, or may be configured to include both.
[0050] The power button 140 outputs an operation signal to the EC 200 in response to a user operation. The input device 150 is an input unit that accepts user input, and is configured to include, for example, a keyboard 151 and a touchpad 153. In response to accepting an operation on the keyboard 151 and the touchpad 153, the input device 150 outputs an operation signal indicating the operation content to the EC 200.
[0051] The communication unit 160 is communicably connected to other devices via a wireless or wired communication network, and transmits and receives various data. For example, the communication unit 160 includes a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark).
[0052] The storage unit 170 includes storage media such as a hard disk drive (HDD), a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), a flash ROM, etc. The storage unit 170 stores various programs such as an OS, device drivers, and applications, as well as various data acquired by the operation of the programs.
[0053] The power supply unit 400 supplies power to each unit according to the operating state of each unit of the information processing device 1. The power supply unit 400 includes a DC (Direct Current) / DC converter. The DC / DC converter converts the voltage of direct current power supplied from an AC (Alternate Current) / DC adapter or a battery (battery pack) into a voltage required by each unit. The power whose voltage has been converted by the DC / DC converter is supplied to each unit via each power supply system. For example, the power supply unit 400 supplies power to each unit via each power supply system based on a control signal input from the EC 200.
[0054] The EC200 is a microcomputer including a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and an input / output (I / O) logic circuit. The CPU of the EC200 reads a control program (firmware) prestored in its own ROM, and executes the read control program to fulfill its functions. The EC200 operates independently of the main processing unit 300, controls the operation of the main processing unit 300, and manages its operating state. The EC200 is also connected to a power button 140, an input device 150, a power supply unit 400, and the like.
[0055] For example, the EC 200 communicates with the power supply unit 400 to obtain information on the battery state (such as remaining capacity) from the power supply unit 400 and outputs control signals and the like for controlling the supply of power according to the operating state of each unit of the information processing device 1 to the power supply unit 400. The EC 200 also obtains operation signals from the power button 140 and the input device 150, and outputs to the main processing unit 300 operation signals related to the processing of the main processing unit 300 among the obtained operation signals.
[0056] The face detection unit 210 is configured to include a processor that executes HPD processing by face detection based on image data of a captured image captured by the imaging unit 120. The face detection unit 210 acquires image data of a captured image captured by the imaging unit 120, and temporarily stores the acquired image data in a memory. The memory that stores the image data may be the system memory 304, or may be a memory (not shown) in the face detection unit 210.
[0057] For example, the face detection unit 210 performs face detection processing to detect a face area from the captured image and detect the face orientation of a face image included in the detected face area by processing image data of the captured image acquired from the imaging unit 120. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects a face based on facial feature information, or face detection using learning data (trained model) or a face detection library that is machine-learned based on facial feature information.
[0058] The main processing unit 300 is composed of a CPU (Central Processing Unit) 301, a GPU (Graphic Processing Unit) 302, a chipset 303, and a system memory 304, and is capable of executing processing of various application programs on an OS (Operating System) through system processing based on the OS.
[0059] The CPU 301 is a processor that executes processes based on BIOS programs, processes based on OS programs, processes based on application programs that run on the OS, etc. For example, the CPU 301 executes a startup process that starts the system from a standby state and transitions it to a normal operating state, a sleep process that transitions it from the normal operating state to a standby state, etc.
[0060] The GPU 302 is connected to the display unit 110. The GPU 302 executes image processing and generates display data under the control of the CPU 301. The GPU 302 outputs the generated display data to the display unit 110.
[0061] The chipset 303 has a function as a memory controller and a function as an I / O controller. For example, the chipset 303 controls the reading and writing of data from the system memory 304, the storage unit 170, etc. by the CPU 301 and the GPU 302. The chipset 303 also controls the input and output of data from the communication unit 160, the display unit 110, and the EC 200. The chipset 303 also has a function as a sensor hub. For example, the chipset 303 acquires detection results by face detection processing acquired from the face detection unit 210, and outputs the results to the CPU 301.
[0062] The system memory 304 is used as a reading area for the programs executed by the CPU 301, a working area for writing processing data, etc. The system memory 304 also temporarily stores image data of images captured by the imaging unit 120.
[0063] The CPU 301, the GPU 302, and the chipset 303 may be configured as a single integrated processor, or some or each of them may be configured as individual processors. For example, in a normal operation state, the CPU 301, the GPU 302, and the chipset 303 are all in operation, but in a standby state, only at least a part of the chipset 303 is in operation.
[0064] [Functional configuration of information processing device] Next, a detailed description will be given of the functional configuration of the information processing device 1 that controls the screen luminance of the display unit 110 by the HPD process.
[0065] FIG. 8 is a schematic block diagram showing an example of a functional configuration of the information processing device 1 according to the present embodiment. The information processing device 1 includes a face detection unit 210 and a main processing unit 300. The face detection unit 210 corresponds to the face detection unit 210 shown in FIG. 7, and is a functional configuration realized by executing a program in the face detection unit 210. The main processing unit 300 corresponds to the main processing unit 300 shown in FIG. 7, and includes a system processing unit 310 and an HPD control processing unit 330. The system processing unit 310 and the HPD control processing unit 330 are functional configurations realized by the main processing unit 300 executing an OS and a program running on the OS. For example, the system processing unit 310 is a functional configuration realized by the CPU 301 executing a program of the OS.
[0066] The face detection unit 210 includes a detection processing unit 211 and a face authentication processing unit 212. The detection processing unit 211 detects a face area in which a face is captured from the captured image by processing image data of the captured image acquired from the imaging unit 120. For example, the detection processing unit 211 reads image data of the captured image captured by the imaging unit 120 at a predetermined frame rate (predetermined frequency) from the system memory 304, and detects a face area from each captured image. As a face detection method, any detection method can be applied, such as a face detection algorithm that detects a face based on facial feature information, or face detection using learning data (trained model) or a face detection library that is machine-learned based on facial feature information.
[0067] For example, when performing HPD processing, the detection processing unit 211 detects a face area from a captured image (RGB image) captured using the RGB camera of the imaging unit 120, and outputs coordinate information of the face area as the detection result.
[0068] Furthermore, when the detection processing unit 211 detects a face region from the captured image in the HPD processing, it outputs "Presence" information indicating that a person is present in front of the information processing device 1. On the other hand, when the detection processing unit 211 does not detect a face region from the captured image, it outputs "No presence" information indicating that no person is present in front of the information processing device 1.
[0069] Furthermore, when the detection processing unit 211 detects a face region from the captured image, it detects the face orientation of the face image included in the detected face region. For example, the detection processing unit 211 detects an orientation corresponding to a rotation angle in the left-right direction and an orientation corresponding to a rotation angle in the up-down direction of the face, and determines whether the face is facing forward or not. When the detection processing unit 211 determines that the face is facing forward, it outputs "Attention" information indicating that the user is paying attention to the information processing device 1. On the other hand, when the detection processing unit 211 determines that the face is not facing forward, it outputs "No attention" information indicating that the user is not paying attention to the information processing device 1.
[0070] The face authentication processing unit 212 executes face authentication processing to compare a face image in the detected face area with a face image of a legitimate user. For example, the face authentication processing unit 212 performs face authentication processing (first face authentication processing) by comparing a face image (first face image) detected from a captured image (RGB image) captured using the RGB camera of the imaging unit 120 in the HPD processing with a face image (first face image for authentication) of a legitimate user registered as a user ID in the HPD processing.
[0071] This first face authentication process is used in the process of determining whether or not the user is a registered user when "Attention" is detected in a state where the screen brightness is low by the screen brightness reduction process (a state where "No attention" is detected), for example (see (5) in FIG. 6). The face authentication processing unit 212 outputs the authentication result at the time of user authentication in the HPD process (the authentication result of the first face authentication process) to the HPD processing unit 330.
[0072] Note that the face image of the authorized user registered as user information (account information) in the HPD process is different from the face image of the authorized user registered as user information (account information) of the system. For example, it is the face image of the authorized user detected from the captured image (RGB image) captured by the RGB camera of the imaging unit 120 in the registration menu of the face authentication image in the HPD process.
[0073] The HPD processing unit 330 has a screen brightness change unit 331, a Dim timer 332, a user determination unit 333, a system operation instruction unit 334, a Leave timer 335, and an HID input determination unit 336, as functional configurations realized when the CPU 301 or chipset 303 (e.g., a sensor hub) executes an HPD processing program.
[0074] The screen luminance change unit 331 changes the screen luminance of the display unit 110 by the HPD process. For example, when the screen luminance change unit 331 changes the screen luminance, it changes the value of the luminance setting value stored in the storage unit 170. For example, when the screen luminance of the display unit 110 is controlled to the standard luminance and the screen luminance change unit 331 acquires "No attention" information or "No presence" information from the face detection unit 210, the screen luminance change unit 331 starts measuring the time TD until the screen luminance is reduced using the Dim timer 332. Then, after the time TD has elapsed, the screen luminance change unit 331 changes the luminance setting value to a low luminance value and executes a screen luminance reduction process for changing the screen luminance from the standard luminance to the low luminance. That is, when the face detection unit 210 detects the presence of a person who is watching the display unit 110 but no longer detects it, the screen luminance change unit 331 transitions to a low power consumption mode, which is an operation mode in which the screen luminance of the display unit 110 is reduced.
[0075] The Dim timer 332 is a timer that measures the time TD (e.g., 10 seconds) from when the "No attention" information or "No presence" information is acquired from the face detection unit 210 until the screen luminance change unit 331 reduces (dimming) the screen luminance of the display unit 110. For example, the Dim timer 332 starts measuring the time TD when it acquires the "No attention" information or "No presence" information from the face detection unit 210. Furthermore, when the measurement of the time TD by the Dim timer 332 is completed, the screen luminance change unit 331 changes the screen luminance from standard luminance to low luminance and transitions to a low power consumption mode.
[0076] When the user determination unit 333 acquires "Attention" information from the face detection unit 210 after the screen brightness change unit 331 changes the screen brightness of the display unit 110 from standard brightness to low brightness, the user determination unit 333 acquires an authentication result by the first face authentication process (an authentication result at the time of user authentication in the HPD process) from the face detection unit 210, and performs a determination process to determine whether or not the person is a pre-registered authorized user based on the acquired authentication result. That is, after transition to a low power consumption mode in which the screen brightness of the display unit 110 is reduced, when the face detection unit 210 detects the presence of a person looking at the display unit 110, the user determination unit 333 determines whether or not the person is a pre-registered authorized user based on the authentication result by the first face authentication process.
[0077] When the user determination unit 333 determines that the person looking at the display unit 110 is a pre-registered legitimate user, the screen brightness change unit 331 returns the screen brightness of the display unit 110 from low brightness to standard brightness and returns the display unit 110 from the low power consumption mode.
[0078] On the other hand, if the user determination unit 333 determines that the person looking at the display unit 110 is not a pre-registered legitimate user, the system operation instruction unit 334 outputs an instruction to the system processing unit 310 to transition to a standby state (standby state transition instruction) in order to lock the system.
[0079] Furthermore, when the system operation instruction unit 334 acquires "No presence" information from the face detection unit 210, the system operation instruction unit 334 starts measuring the time TL until the system is locked using the leave timer 335. Then, when the time TL has elapsed without acquiring "Attention" information (or "Presence" information), the system operation instruction unit 334 determines that the person has left (Leave) and outputs a standby state transition instruction to the system processing unit 310 to lock the system. That is, when the face detection unit 210 detects the presence of a person who is looking at the display unit 110, but the presence of the person is no longer detected, the system operation instruction unit 334 also outputs a standby state transition instruction to the system processing unit 310 to lock the system when the time TL has elapsed without detecting the presence of the person. The time TL until the system is locked at the time of this leave is set longer than the time TD until the screen luminance is changed to low luminance in the screen luminance reduction process.
[0080] The Leave timer 335 is a timer that measures the time TL (e.g., 30 seconds to 5 minutes) from when the "No presence" information is acquired from the face detection unit 210 until the system operation instruction unit 334 outputs a standby state transition instruction to the system processing unit 310 to lock the system. For example, the Leave timer 335 starts measuring the time TL when it acquires the "No presence" information from the face detection unit 210. Furthermore, when the measurement of the time TL by the Leave timer 335 is completed, the system operation instruction unit 334 outputs a standby state transition instruction to the system processing unit 310.
[0081] The HID input determination unit 336 determines whether or not there is an operation input to an HID (Human Interface Device) such as the input device 150 (e.g., the keyboard 151, the touchpad 153, etc.) (hereinafter referred to as "HID input"). For example, when there is no HID input, the HPD processing unit 330 enables and executes the above-mentioned screen brightness reduction processing and system lock processing, and when there is an HID input, if the Dim timer 332 or the Leave timer 335 is timing, it stops timing and resets it, and if the screen brightness has been changed to low brightness, it returns it to standard brightness.
[0082] The system processing unit 310 is a functional configuration realized by the CPU 11 executing the BIOS and OS programs or the programs executed on the OS. For example, the system processing unit 310 includes an operation control unit 311, an OS timer (sleep timer) 312, a display control unit 313, and a system authentication unit 314 as functional configurations realized by executing the OS program.
[0083] The operation control unit 311 controls the operation state of the system. For example, when the power button 140 is operated in the standby state, the operation control unit 311 starts the system from the standby state based on an operation signal acquired from the power button 140 via the EC 200. In addition, the operation control unit 311 performs processes such as shutting down the system, transitioning to a standby state, and restarting the system based on an operation on a power menu (shutdown, sleep, restart, etc.) of the OS displayed on the display unit 110 in the normal operation state.
[0084] Furthermore, the operation control unit 311 controls the operation state of the system according to the HPD processing executed by the HPD processing unit 330 based on the detection result by the face detection unit 210. For example, when the operation control unit 311 acquires a standby state transition instruction from the HPD processing unit 320 in the normal operation state, it turns off the screen by issuing a screen-off instruction to the display control unit 313, and then locks the system to transition to the standby state.
[0085] Furthermore, when no HID input is detected, the operation control unit 311 uses the OS timer 312 to measure the time TS until the system transitions to a standby state. Then, when the time TS has elapsed without detecting an HID input, the operation control unit 311 turns off the screen by issuing a screen-off instruction to the display control unit 313, and then locks the system and transitions it to the standby state.
[0086] If an HID input is detected before the time TS has elapsed, the operation control unit 311 stops and resets the OS timer 312, and continues the normal operation state without transitioning to the standby state.
[0087] The OS timer 312 is a timer that measures the time TS (e.g., 30 seconds to 5 minutes) until transition to a standby state occurs when a state in which no HID input is detected continues. For example, the OS timer 312 is reset every time an HID input is detected, and starts measuring the time TS (from the time of the last HID input) when no HID input is detected. When the OS timer 312 has finished measuring the time TS, the operation control unit 311 issues a screen-off instruction to the display control unit 313, locks the system, and transitions to a standby state.
[0088] The display control unit 313 controls the display on and off of the display unit 110. For example, when the display control unit 313 receives a screen-off instruction from the operation control unit 311, it controls the display unit 110 to turn off the display, thereby turning off the screen.
[0089] Furthermore, the display control unit 313 controls the screen luminance of the display unit 110 in the display-on state. Specifically, the display control unit 313 controls the screen luminance of the display unit 110 based on a luminance setting value stored in the storage unit 170. This luminance setting value is initially set in the system, and can be changed from the initial setting by the user according to the usage environment or preference. Furthermore, this luminance setting value is changed by the screen luminance change unit 331 of the HPD processing unit 320.
[0090] When a user authentication event occurs, such as when logging in to the system or unlocking the system, the system authentication unit 314 executes an authentication process to authenticate whether or not the user is a legitimate user. For example, the system authentication unit 314 executes an authentication process using any one of face authentication, password authentication, PIN authentication, and fingerprint authentication.
[0091] For example, when performing authentication processing by face authentication during user authentication of the system, the system authentication unit 314 performs face authentication processing (second face authentication processing) by comparing a face image (second face image) detected from an image (IR image + RGB image) captured using the IR camera and RGB camera of the imaging unit 120 with a face image (second authentication face image) of a legitimate user registered in advance as user information (account information) of the system. If the authentication result of the second face authentication processing is successful, the system authentication unit 314 determines that the user is a legitimate user, and if the authentication is unsuccessful, determines that the user is not a legitimate user. Then, the system authentication unit 314 outputs the authentication result during user authentication of the system (authentication result of the second face authentication processing) to the system processing unit 310.
[0092] The facial image of a legitimate user (second authentication facial image) registered as user information (account information) of the system is, for example, a facial image of a legitimate user detected from an image (IR image + RGB image) captured using the IR camera and RGB camera of the imaging unit 120 in the facial authentication image registration menu of the system settings.
[0093] In this way, the second face image used during user authentication of the system is a face image captured under different conditions from the first face image used during user authentication in the HPD process. For example, as described above, the first face image and the second face image are captured under different conditions, the first face image is a face image detected from an RGB image, and the second face image is a face image detected from an IR image and an RGB image.
[0094] In addition, during system user authentication (e.g., login authentication), the system authentication unit 314 may perform authentication processing using a face image detected from an image (IR image or RGB image) captured using either the IR camera or the RGB camera of the imaging unit 120 (e.g., only the IR camera).
[0095] Furthermore, in the case of password authentication or PIN authentication, the system authentication unit 314 acquires an operation signal based on a user's operation on the keyboard 151 via the EC 200. Then, based on the acquired operation signal, the system authentication unit 314 executes authentication processing by comparing the password or PIN entered by the user's operation with a password or PIN registered in advance. In the case of fingerprint authentication, the system authentication unit 314 executes authentication processing by comparing a fingerprint acquired using a fingerprint sensor (not shown) with a fingerprint registered in advance.
[0096] It should be noted that the above-mentioned Dim timer 332, Leave timer 335, and OS timer 312 operate separately.
[0097] [Screen brightness reduction processing operation] Next, the operation of the above-mentioned screen brightness reduction process and system lock process will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the screen brightness reduction process and system lock process according to the present embodiment. Here, it is assumed that the information processing device 1 is in a normal operating state, in the "Presence" and "Attention" states, and the screen brightness of the display unit 110 is controlled to standard brightness.
[0098] (Step S101) The HPD processing unit 330 determines whether or not there is an HID input. If the HPD processing unit 330 determines that there is an HID input (YES), it performs the process of step S101 again. On the other hand, if the HPD processing unit 330 determines that there is no HID input (NO), it proceeds to the process of step S103.
[0099] (Step S103) If it is determined in step S101 that there is no HID input, the system processing unit 310 starts counting the time of the OS timer 312. Then, the process proceeds to step S105.
[0100] (Step S105) The HPD processing unit 330 determines whether or not "Prsence" information has been acquired from the face detection unit 210. If the HPD processing unit 330 determines that "Prsence" information has not been acquired (NO), the process proceeds to step S107. On the other hand, if the HPD processing unit 330 determines that "Prsence" information has been acquired (YES), the process proceeds to step S109.
[0101] (Step S107) The HPD processing unit 330 starts counting the time of the Dim timer 332 and the Leave timer 335. Then, the process proceeds to step S113.
[0102] (Step S109) The HPD processing unit 330 determines whether or not "Attention" information has been acquired from the face detection unit 210. If the HPD processing unit 330 determines that "Attention" information has been acquired (YES), the process returns to step S105. On the other hand, if the HPD processing unit 330 determines that "Attention" information has not been acquired (NO), the process proceeds to step S111.
[0103] (Step S111) The HPD processing unit 330 starts counting the time of the Dim timer 332. Then, the process proceeds to step S113.
[0104] (Step S113) The HPD processing unit 330 determines whether or not "Attention" information has been acquired from the face detection unit 210. If the HPD processing unit 330 determines that "Attention" information has been acquired (YES), the process proceeds to step S115. On the other hand, if the HPD processing unit 330 determines that "Attention" information has not been acquired (NO), the process proceeds to step S117.
[0105] (Step S115) The HPD processing unit 330 stops and resets the Dim timer 332 and the Leave timer 335, and returns to the process of step S105.
[0106] (Step S117) When the HPD processing unit 330 has completed timing of the Dim timer 332, the process proceeds to step S119.
[0107] (Step S119) The HPD processing unit 330 changes the screen luminance of the display unit 110 from standard luminance to low luminance. Then, the process proceeds to step S121.
[0108] (Step S121) The HPD processing unit 330 determines whether or not "Attention" information has been acquired from the face detection unit 210. If the HPD processing unit 330 determines that "Attention" information has been acquired (YES), that is, if it determines that there is a person looking at the display unit 110, the process proceeds to step S123. On the other hand, if the HPD processing unit 330 determines that "Attention" information has not been acquired (NO), the process proceeds to step S127.
[0109] (Step S123) The HPD processing unit 330 judges whether or not the person looking at the display unit 110 is a registered user (i.e., a legitimate user) based on the authentication result by the face detection unit 210. If the HPD processing unit 330 judges that the person is not a registered user (NO), the process proceeds to step S129. On the other hand, if the HPD processing unit 330 judges that the person is a registered user (YES), the process proceeds to step S125.
[0110] (Step S125) The HPD processing unit 330 returns the screen luminance of the display unit 110 from low luminance to standard luminance, and then returns to the process of step S105.
[0111] (Step S127) The HPD processing unit 330 determines whether or not the timing of the OS timer 312 or the Leave timer 335 has completed. If the HPD processing unit 330 determines that neither the OS timer 312 nor the Leave timer 335 has completed timing (NO), the HPD processing unit 330 returns to the process of step S121. On the other hand, if the HPD processing unit 330 determines that the timing of the OS timer 312 or the Leave timer 335 has completed timing (YES), the HPD processing unit 330 proceeds to the process of step S129.
[0112] (Step S129) The HPD processing unit 330 outputs a standby state transition instruction to the system processing unit 310 to lock the system. When the system processing unit 310 acquires the standby state transition instruction from the HPD processing unit 330, it controls the display unit 110 to turn off the display and turn off the screen. Then, the process proceeds to step S129.
[0113] (Step S131) The system processing unit 310 locks the system and transitions it to a standby state. After that, when using the information processing device 1, it is necessary to unlock it by user authentication.
[0114] [Summary of the embodiment] As described above, the information processing device 1 according to the present embodiment includes a memory (e.g., system memory 304) that temporarily stores a program of the system (e.g., OS), a first processor (e.g., CPU 301, chipset 303, etc.) that controls the operation of the system by executing the program of the system, and a second processor (e.g., face detection unit 210) that performs a detection process of detecting a person present in the direction facing the display unit 110 and detecting whether the person is looking at the display unit 110 based on an image captured by the imaging unit 120, and a face authentication process (e.g., first face authentication process) of matching a face image of the person present in the direction facing the display unit 110 with a face image (e.g., first face image) of a user registered in advance. When the presence of a person looking at the display unit 110 is no longer detected from a state in which the presence of the person is detected by the detection process, the information processing device 1 performs a power reduction process of transitioning to a low power consumption mode. When the information processing device 1 detects the presence of a person looking at the display unit 110 after transitioning to the low power consumption mode, the information processing device 1 performs a determination process to determine whether or not the person is a pre-registered user based on an authentication result by face authentication processing (for example, a first face authentication processing). When the information processing device 1 detects the presence of a person looking at the display unit 110 after transitioning to the low power consumption mode and determines that the person is a pre-registered user, the information processing device 1 performs a power recovery process to return from the low power consumption mode. When the information processing device 1 detects the presence of a person looking at the display unit 110 after transitioning to the low power consumption mode and determines that the person is not a pre-registered user, the information processing device 1 performs a lock process to lock the system.
[0115] This allows the information processing device 1 to achieve both power saving and security during a short absence (absence), so even if the time (timer) from when a person's absence is detected until the system is locked is set long, security is ensured until the system is locked. Therefore, the information processing device 1 can control so as not to lock the system when a person (for example, a regular user) is absent for a short time while ensuring security.
[0116] The low power consumption mode is, for example, an operation mode in which the screen luminance of the display unit 110 is reduced. For example, when the information processing device 1 detects no longer the presence of a person looking at the display unit 110, the information processing device 1 performs a screen luminance reduction process for reducing the screen luminance of the display unit 110 (changing from standard luminance to low luminance) as the power reduction process.
[0117] This allows the information processing device 1 to reduce the screen brightness when the user is away (absent) for a short period of time, thereby saving power.
[0118] In addition, after locking the system, the information processing device 1 performs an unlocking process to unlock the system, on the condition that authentication is successful in a system authentication process (e.g., a user authentication process) that authenticates whether the user is a legitimate user pre-registered in the system.
[0119] As a result, after the information processing device 1 has locked the system, user authentication is required the next time the system is used, which, although it takes time, ensures security.
[0120] Furthermore, in a lock process for locking the system, the information processing device 1 locks the system when the presence of a person looking at the display unit 110 is no longer detected in the direction facing the display unit 110 after the presence of the person has been detected, and when a time TL (an example of a first time) has elapsed without the presence of a person being detected. Furthermore, after transitioning to the low power consumption mode, when the information processing device 1 detects the presence of a person looking at the display unit 110 and determines that the person is not a pre-registered user, the information processing device 1 locks the system in response to the detection and determination without waiting for the time TL.
[0121] As a result, when the information processing device 1 is in power saving mode due to a short absence (absence), even if the system is not locked, if an unregistered user is detected, the system will be locked immediately, so security can be ensured until the system is locked even if the time (timer) from when a person's absence is detected to when the system is locked is set long.
[0122] Furthermore, in a power reduction process (e.g., a screen brightness reduction process), the information processing device 1 transitions to a low power consumption mode (e.g., reduces the screen brightness) when the presence of a person looking at the display unit 110 is no longer detected after being detected, or when a time TD (an example of a second time) shorter than the time TL has elapsed while the presence of a person looking at the display unit 110 is not detected.
[0123] This allows the information processing device 1 to save power (e.g., reduce screen brightness) relatively quickly when there is no person looking at the display unit 110, thereby improving the effect of power saving. For example, even if the information processing device 1 reduces the screen brightness, it can immediately return to normal without causing the user any trouble by simply changing the screen brightness when returning to normal, so that the time TD from when it is detected that there is no person looking at the display unit 110 to when power saving (e.g., reduce screen brightness) can be set to be short.
[0124] Furthermore, in a lock process for locking the system, the information processing device 1 locks the system when a time TS (an example of a third time) has elapsed without detecting an HID input (an input by a user) regardless of a transition to the low power consumption mode. Furthermore, after transitioning to the low power consumption mode, if the information processing device 1 detects the presence of a person looking at the display unit 110 and determines that the person is not a pre-registered user, the information processing device 1 locks the system in response to the detection and determination without waiting for the time TS.
[0125] As a result, when the information processing device 1 detects an unregistered user even if the system is not locked while in power saving mode due to a short absence (absence), the system is immediately locked, so security can be ensured until the system is locked even if the time (timer) until the OS locks the system after determining that a person is absent due to continued no operations is set long.
[0126] Furthermore, in a power reduction process (e.g., a screen brightness reduction process), the information processing device 1 transitions to a low power consumption mode (e.g., reduces the screen brightness) when the presence of a person looking at the display unit 110 is no longer detected after being detected, or when a time TD (an example of a second time) shorter than the time TS has elapsed while the presence of a person looking at the display unit 110 is not detected.
[0127] This allows the information processing device 1 to save power (e.g., reduce screen brightness) relatively quickly when there is no person looking at the display unit 110, thereby improving the effect of power saving. For example, even if the information processing device 1 reduces the screen brightness, it can immediately return to normal without causing the user any trouble by simply changing the screen brightness when returning to normal, so that the time TD from when it is detected that there is no person looking at the display unit 110 to when power saving (e.g., reduce screen brightness) can be set to be short.
[0128] Further, according to the present embodiment, a control method for an information processing device 1 including a memory (e.g., system memory 304) for temporarily storing a program of a system (e.g., an OS), a first processor (e.g., CPU 301, chipset 303, etc.) for controlling the operation of the system by executing the program of the system, and a second processor (e.g., face detection unit 210) for performing a detection process for detecting a person present in a direction facing the display unit 110 and detecting whether or not the person is paying attention to the display unit 110 based on an image captured by the imaging unit 120, and a face authentication process for matching a face image of the person present in the direction facing the display unit 110 with a face image of a user registered in advance, comprises the steps of: The method includes a step of transitioning to a low power consumption mode when the presence of a person looking at display unit 110 is no longer detected from a state in which the presence was detected; a step of determining whether or not a person looking at display unit 110 is a pre-registered user based on an authentication result by face authentication processing if the presence of a person looking at display unit 110 is detected after the transition to the low power consumption mode, and it is determined that the person is a pre-registered user; a step of returning from the low power consumption mode if the presence of a person looking at display unit 110 is detected after the transition to the low power consumption mode and it is determined that the person is a pre-registered user; and a step of locking the system if the presence of a person looking at display unit 110 is detected after the transition to the low power consumption mode and it is determined that the person is not a pre-registered user.
[0129] As a result, the control method in the information processing device 1 can achieve both power saving and security during a short absence (absence), so even if the time (timer) from when a person's absence is detected until the system is locked is set long, security is ensured until the system is locked. Therefore, the control method in the information processing device 1 can control so as not to lock the system when a person (for example, a regular user) is absent for a short time while ensuring security.
[0130] Moreover, the information processing device 1 according to the present embodiment includes a memory (e.g., system memory 304) that temporarily stores a program of the system (e.g., OS), a first processor (e.g., CPU 301, chipset 303, etc.) that controls the operation of the system by executing the program of the system, and a second processor (e.g., face detection unit 210) that performs a detection process to detect a person present in the direction facing the display unit 110 based on an image captured by the imaging unit 120. The information processing device 1 performs a first face authentication process to compare a first face image of a person present in the direction facing the display unit 110 with a first authentication face image of a user registered in advance. Moreover, when the presence of a person is detected in the direction facing the display unit 110 by the detection process and the person is determined not to be a user registered in advance by the first face authentication process, the information processing device 1 performs a lock process to lock the system. Furthermore, the information processing device 1 performs a second face authentication process to compare a second face image of a person present in a direction facing the display unit 110, which is acquired based on an image captured by the imaging unit 120 under conditions different from those of the first face image, with a second authentication face image of a pre-registered user. Then, when the information processing device 1 determines by the second face authentication process that the person is a pre-registered user, it performs an unlock process to unlock the system.
[0131] That is, the information processing device 1 has two types of face authentication processes for matching two types of face images (for example, a first face image and a second face image of the same user captured under different imaging conditions), and locks the system when it is determined using the first face authentication process that a person present in the direction facing the display unit 110 is not a pre-registered user, and unlocks the system when it is determined using the second face authentication process that a person present in the direction facing the display unit 110 is a pre-registered user. This allows the information processing device 1 to appropriately control locking and unlocking of the system depending on the situation, such as the operating state of the system or the presence of a person, and therefore allows the system to be locked or not locked depending on the situation while ensuring security.
[0132] In addition, according to the present embodiment, a control method in an information processing device 1 including a memory (e.g., system memory 304) that temporarily stores a program of a system (e.g., an OS), a first processor (e.g., CPU 301, chipset 303, etc.) that controls the operation of the system by executing the program of the system, and a second processor (e.g., face detection unit 210) that performs a detection process to detect a person present in a direction facing the display unit 110 based on an image captured by the imaging unit 120, includes the steps of: the first processor detects the presence of a person in the direction facing the display unit 110 by the detection process and determines by the first facial authentication process that the person is not a pre-registered user; a second facial authentication step of comparing a second facial image of the person present in the direction facing the display unit 110, which is acquired based on an image captured by the imaging unit 120 under conditions different from the first facial image, with a second authentication facial image of a pre-registered user; and a step of unlocking the system if the second facial authentication step determines that the person is a pre-registered user.
[0133] That is, the control method in the information processing device 1 includes two types of face authentication processes for matching two types of face images (for example, a first face image and a second face image of the same user captured under different imaging conditions), and locks the system when it is determined using the first face authentication process that a person present in the direction facing the display unit 110 is not a pre-registered user, and unlocks the system when it is determined using the second face authentication process that a person present in the direction facing the display unit 110 is a pre-registered user. As a result, the control method in the information processing device 1 can appropriately control locking and unlocking of the system depending on the situation, such as the operating state of the system or the presence of a person, and can therefore lock or not lock the system depending on the situation while ensuring security.
[0134] Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configurations are not limited to the above-mentioned embodiments, and the present invention also includes designs that do not deviate from the gist of the present invention. For example, the configurations described in the above-mentioned embodiments can be combined in any combination.
[0135] In the above embodiment, an example of saving power by reducing the screen brightness of the display unit 110 has been described as the low power consumption mode, but the power saving method is not limited to this, and any power saving method can be applied. For example, a configuration may be adopted in which power is saved by lowering the refresh rate of the display of the display unit 110. Also, in the case where the display unit 110 is provided with a touch panel that accepts touch operations on the display screen, a configuration may be adopted in which power is saved by turning off the touch panel.
[0136] Also, although a configuration example in which the imaging unit 120 is built into the information processing device 1 has been described, the present invention is not limited to this. For example, the imaging unit 120 does not have to be built into the information processing device 1, and may be configured to be attachable to the information processing device 1 (for example, any one of the side surfaces 10a, 10b, 10c, etc.) as an external accessory of the information processing device 1, and may be communicatively connected to the information processing device 1 wirelessly or via a wire.
[0137] In the above embodiment, an example has been shown in which the face detection unit 210 is provided separately from the CPU 301 and the chipset 303, but a part or all of the face detection unit 210 may be provided in the chipset 303, or may be provided in a processor integrated with the CPU 301 or the chipset 303. For example, the CPU 301, the chipset 303, and the face detection unit 210 may be configured as separate processors, or may be integrated into a single processor. In addition, a part or all of the face detection unit 210 may be provided in the EC 200.
[0138] The above-mentioned standby state may include a hibernation state, a power-off state, etc. The hibernation state corresponds to, for example, the S4 state defined by ACPI. The power-off state corresponds to, for example, the S5 state (shutdown state) defined by ACPI. Among the standby states, the standby state, sleep state, hibernation state, power-off state, etc. are states in which power consumption is lower (states in which power consumption is reduced) than in the normal operating state.
[0139] The information processing device 1 described above has a computer system inside. A program for implementing the functions of each component of the information processing device 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each component of the information processing device 1 described above. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. The "computer system" may also include multiple computer devices connected via a network including a communication line such as the Internet, a WAN, a LAN, or a dedicated line. The "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM.
[0140] The recording medium also includes a recording medium installed inside or outside and accessible from a distribution server to distribute the program. The program may be divided into a plurality of parts, downloaded at different times, and then combined by each component of the information processing device 1, or each divided program may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a recording medium that holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network. The program may be a recording medium for implementing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.
[0141] In addition, some or all of the functions of the information processing device 1 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually processed, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.
[0142] Furthermore, the information processing device 1 of the above embodiment is not limited to a notebook PC, but may be, for example, a desktop PC. [Explanation of symbols]
[0143] 1 Information processing device, 10 First housing, 20 Second housing, 15 Hinge mechanism, 110 Display unit, 120 Imaging unit, 140 Power button, 150 Input device, 151 Keyboard, 153 Touch pad, 160 Communication unit, 170 Memory unit, 200 EC, 210 Face detection unit, 211 Detection processing unit, 212 Face authentication processing unit, 300 Main processing unit, 301 CPU, 302 GPU, 303 Chip set, 304 System memory, 310 System processing unit, 311 Operation control unit, 312 OS timer, 313 Display control unit, 314 System authentication unit, 330 HPD processing unit, 331 Screen brightness change unit, 332 Dim timer, 333 User determination unit, 334 System operation instruction unit, 335 Leave timer, 336 HID input determination unit, 400 Power supply unit
Claims
1. A memory for temporarily storing system programs; a first processor that controls the operation of the system by executing a program of the system; a second processor that performs a detection process of detecting a person present in a direction facing the display unit and detecting whether or not the person is paying attention to the display unit based on an image captured by the imaging unit, and a face authentication process of matching a face image of the person present in a direction facing the display unit with a face image of a user registered in advance; Equipped with The first processor, a power reduction process of transitioning to a low power consumption mode when the presence of a person watching the display unit is no longer detected by the detection process; a determination process for determining, when the presence of a person looking at the display unit is detected by the detection process after the transition to the low power consumption mode, whether or not the person is a pre-registered user based on an authentication result by the face authentication process; a power recovery process for recovering from the low power consumption mode when, after the transition to the low power consumption mode, the presence of a person looking at the display unit is detected by the detection process and the determination process determines that the person is a pre-registered user; and a locking process for locking the system when the detection process detects the presence of a person looking at the display unit after the transition to the low power consumption mode and the determination process determines that the person is not a pre-registered user; and An information processing device that performs the above.
2. The first processor, an unlocking process for unlocking the system on the condition that authentication is successful in a system authentication process for authenticating whether or not the user is a regular user preregistered in the system after the system has been locked; The information processing apparatus according to claim 1 .
3. The first processor, in the locking process, when the presence of a person in a direction facing the display unit is no longer detected from a state in which the presence of a person looking at the display unit has been detected by the detection process, and when a first time period has elapsed without the presence of a person being detected, the system is locked; after transition to the low power consumption mode, when the detection process detects the presence of a person looking at the display unit and the determination process determines that the person is not a pre-registered user, locking the system in response to the detection and determination without waiting for the first time period. The information processing device according to claim 1 .
4. The first processor, in the power reduction process, when the presence of a person looking at the display unit is no longer detected in the detection process, and when a second time shorter than the first time has elapsed in a state in which the presence of a person looking at the display unit is not detected in the detection process, transitioning to the low power consumption mode. The information processing device according to claim 3 .
5. The first processor, locking the system when a third time period has elapsed without detecting a user input, regardless of the transition to the low power consumption mode; after transition to the low power consumption mode, when the detection process detects the presence of a person looking at the display unit and the determination process determines that the person is not a pre-registered user, locking the system in response to the detection and determination without waiting for the third time period. The information processing device according to claim 1 .
6. The first processor, in the power reduction process, when the presence of a person looking at the display unit is no longer detected in the detection process, and when a second time shorter than the third time has elapsed while the presence of a person looking at the display unit is not detected in the detection process, transitioning to the low power consumption mode. The information processing device according to claim 5 .
7. The low power consumption mode is an operation mode in which the screen brightness of the display unit is reduced. The information processing device according to claim 1 .
8. A memory for temporarily storing system programs; a first processor that controls the operation of the system by executing a program of the system; a second processor that performs a detection process to detect a person present in a direction facing the display unit based on an image captured by the imaging unit; Equipped with The second processor performing a first face authentication process for matching a first face image of a person present in a direction facing the display unit with a first authentication face image of a user registered in advance; The first processor, a lock process for locking the system when the presence of a person is detected in a direction facing the display unit by the detection process and the person is determined to be not a pre-registered user by the first face authentication process; a second face authentication process for comparing a second face image of a person present in a direction facing the display unit, which is acquired based on an image captured by an imaging unit under conditions different from those of the first face image, with a second authentication face image of a user that is registered in advance; an unlocking process for unlocking the system when the person is determined to be a pre-registered user by the second face authentication process; An information processing device that performs the above.
9. A control method for an information processing device including a memory for temporarily storing a system program, a first processor for controlling an operation of the system by executing the system program, and a second processor for performing a detection process for detecting a person present in a direction facing a display unit and detecting whether or not the person is paying attention to the display unit based on an image captured by an imaging unit, and a face authentication process for matching a face image of the person present in the direction facing the display unit with a face image of a user registered in advance, The first processor, transitioning to a low power consumption mode when the presence of a person watching the display unit is no longer detected by the detection process; a step of determining, when the presence of a person looking at the display unit is detected by the detection process after the transition to the low power consumption mode, whether or not the person is a pre-registered user based on an authentication result by the face authentication process; a step of returning from the low power consumption mode when, after transition to the low power consumption mode, the presence of a person looking at the display unit is detected by the detection process and the person is determined to be a pre-registered user; a step of locking the system when, after the transition to the low power consumption mode, the presence of a person looking at the display unit is detected by the detection process and the person is determined to be not a pre-registered user; A control method comprising:
10. A control method for an information processing device including a memory that temporarily stores a system program, a first processor that controls an operation of the system by executing the system program, and a second processor that performs a detection process to detect a person present in a direction facing a display unit based on an image captured by an imaging unit, the method comprising: a first face authentication step in which the second processor compares a first face image of a person present in a direction facing the display unit with a first authentication face image of a user registered in advance; The first processor, a step of locking the system when the presence of a person is detected in a direction facing the display unit by the detection process and the first face authentication step determines that the person is not a pre-registered user; a second face authentication step of comparing a second face image of a person present in a direction facing the display unit, which is acquired based on an image captured by an imaging unit under conditions different from those of the first face image, with a second authentication face image of a user registered in advance; and unlocking the system if the second face authentication step determines that the person is a pre-registered user.
Citation Information
Patent Citations
Multi-granularity authentication technique
JP2018512083A
Image forming apparatus, control program, and control method
JP2019103075A
Electronic device, control method, and program
JP2020102151A
JPP7454635B
Electronic device and method for controlling electronic device
US20210247812A1
Cited By
Information processing device and control method
JP2026123547A
Information processing device and control method
JP7805497B1
