Computer-implemented method and computer program product for access control of a terminal device
Patent Information
- Application Number
- DE502018015300
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2017-06-16
- Filing Date
- 2018-06-15
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2038-06-15
AI Technical Summary
Existing access control methods for terminal devices do not differentiate between restricted and unrestricted access, allowing unauthorized users to access all data when the authorized user is authenticated, and do not provide a visual indication of access status to unauthorized users.
A computer-implemented method that captures an image signal with a camera, performs face detection, and continuously monitors the number of faces, restricting access if more than one face is detected, while displaying a visual indication of unrestricted access to authorized users and hiding private data during restricted access.
Ensures that private data is kept secure by restricting access when multiple users are present and providing a visual cue only to authorized users, maintaining the privacy of sensitive information.
Description
[0001] The invention relates to a computer-implemented method for access control of a terminal device, wherein private data and other data are stored on the terminal device and access to the private data can be restricted or unrestricted, comprising the following steps in the case of unrestricted access: capturing an image signal with a camera of the terminal device; carrying out face detection with the captured image signal; continuously monitoring the number of faces in the captured image signal determined during face detection, wherein access is restricted if the number of faces is greater than one, wherein the terminal device has a screen for displaying private and other data and the private data is completely hidden in the case of restricted access and only the other data is displayed.
[0002] The end device can be a computer, terminal, in particular a mobile device, e.g., a smartphone or a tablet computer. Private data in this context is data marked as private, stored on the end device, or received and temporarily stored by the end device. For example, a contact from a smartphone's address book can be marked as private (a private contact), and as a result, all data associated with the contact marked as private (e.g., contact information, chat displays, call signaling, location, etc.) are automatically marked as private as well. Such private data is typically stored locally on the end device and fully encrypted. This can make circumvention of access control more difficult or even impossible.
[0003] In this context, face detection is understood to be a well-known image analysis method. Such methods are often used, for example, in digital cameras, to locate a face in an image (face localization). In this context, the position of the faces is only of secondary importance. Essentially, face detection refers to the automatic recognition (in the sense of detection, not identification) of one or more faces in an image. This is usually achieved, for example, based on movement, typical shapes (oval) and colors (skin pigments) and / or the recognition and evaluation of the arrangement and distance between the eyes. In contrast, face identification concerns the assignment of a face to a specific person. Face identification is a biometric method and can be used, for example, as a biometric factor for the authentication of persons.For example, biometric features of a face are compared with the stored features of a known face, and if there is a match, the person's identity is confirmed. The term facial recognition can refer to both facial detection and facial identification and will therefore be largely avoided in the following.
[0004] Access control essentially controls which data a user has access to and, in particular, which data is visible to the user at all. A typical access control distinguishes between blocked access (access lock), which is characterized by the fact that the blocked data is not visible, and unrestricted access. The access lock can, for example, be a screen lock, which can only be lifted by user authentication. With access lock, access to all data (i.e., private and other data) on the end device is blocked. Accordingly, when access lock is active, generally none of the data stored on the end device is visible. At most, access is possible to public status information (e.g., time, reception signal, etc. are displayed and thus visible despite the active access lock).
[0005] Such access control is known, for example, from US 8,370,639 B2. This describes a method for authenticating and continuously tracking a user (e.g., a mobile phone user) using facial recognition. Facial recognition can be initiated by entering other authentication features, e.g., a password. However, this access control does not distinguish between blocked and restricted access or between restricted and unrestricted access: as long as a user can be authenticated, unrestricted access remains. A second user who is in the vicinity of the terminal device thus essentially receives the same unrestricted access as the authorized user. In particular, all displayed data can be read.
[0006] A similar method is also shown in US Pat. No. 8,660,322 B. In this method, access is blocked when the user leaves the field of view of a camera; as soon as they are recognized again, unrestricted access is granted. Here, too, no restricted access is provided.
[0007] US 2017 / 046507 A1 describes a generic method in which a camera, e.g., a smartphone, continuously captures an image and applies facial recognition to it. The smartphone can store both public and private data. Depending on the result of the facial recognition, either only the public data (restricted access) or all data (unrestricted access) is displayed. Specifically, facial recognition distinguishes between registered and unregistered faces and activates restricted access if at least one unregistered face is detected.
[0008] US 2010 / 124363 A1 concerns a method for controlling a privacy filter. This is a hardware-based measure that limits the viewing angle at which the screen content of the device to be protected can be read. This means that all data is always visible, but with the privacy filter active, only from a narrow angle. In this context, US 2010 / 124363 A1 describes that images are repeatedly taken with a camera and fed into a facial recognition system, which determines the number of detected faces. If more than one face is detected, the privacy filter is activated.
[0009] In a method of the type mentioned above, the number of faces in the captured image signal determined during face detection is continuously monitored, with access being restricted if the number of faces is greater than one. This ensures that unrestricted access is only granted as long as only one person is detected in the camera's field of view (i.e. the area captured by the camera). As soon as the face of a second person enters the camera's field of view, access is restricted, i.e. the terminal device is switched to restricted access. The users of the terminal device then continue to have access to the other data stored on the terminal device that is not marked as private. This makes it more difficult for a second person to spy on private data.For example, private contacts and all references to communication with them can be kept secret from anyone other than the authorized user of the device. Marking data as private is done by the user and stored on the device. For example, the user can mark an existing contact as "private" or create a new contact with the "private" tag. The device then treats all data associated with this contact as private data, e.g., previous calls, text messages, emails, chat messages, the contact information itself, or images or videos associated with the contact.
[0010] The device has a screen for displaying private and other data. When access is restricted, the private data is completely hidden, and only the other data is displayed. This means that access to the private data is not simply restricted by obscuring it, but rather in such a way that, when access is restricted, no indication of the existence of this data is visible. In particular, unauthorized users should not be able to detect that they have only restricted access.
[0011] Sophie Weiß's B.Sc. thesis, "Confidential Desktop," proposes a Windows application that can implement various configurable privacy protection measures by changing the screen content. Among other things, it is possible to close certain applications, minimize a browser window, or close an email client depending on various triggers. One trigger mentioned is facial recognition using a webcam of two or more faces.
[0012] In this context, it is an object of the invention that authorized users can recognize whether access is currently restricted, but not unauthorized users.
[0013] The invention solves this problem with the features according to claim 1.
[0014] When access is unrestricted, a visual indicator indicating unrestricted access is displayed. This allows a user to see that both other and private data are now being displayed, giving them complete, unfettered access to all data. The visual indicator could be, for example, a green border on the screen. Restricted access is thus recognizable to authorized users by the absence of the visual indicator; however, the restricted access is not recognizable to unauthorized users because they are not familiar with or expecting the visual indicator.
[0015] Preferably, events related to private data can be displayed on the screen in a restricted (e.g., censored) format when access is restricted. This function should enable the user to be notified of events related to private data (e.g., a message from a private contact) even when access is restricted (e.g., when the device is not being used or not being used alone). For example, a call from a private contact can be displayed as an "Unknown Number" if, for example, a smartphone is lying openly on a table.
[0016] With regard to the start of continuous verification, it is advantageous to perform user authentication to obtain unrestricted access. Before granting unrestricted access, an image signal is captured with the device's camera, face detection is performed with the captured image signal, and the number of faces in the captured image signal is determined. This is done only if the number of faces is exactly one. Even with successful authentication, unrestricted access is only granted if no observers or potential eavesdroppers are detected.
[0017] In this context, it is particularly advantageous if authentication includes biometric authentication, in particular facial and / or iris identification. In this case, authentication and the determination of the number of faces can be performed based on the same image data. Alternatively or additionally, other factors can of course also be used for authentication, e.g., a password or fingerprint.
[0018] In a preferred variant of the present invention, facial identification can be repeated regularly or randomly in parallel with the verification of the number of faces. This ensures that an unauthorized user who appears in the camera's field of view at the same time as the authorized user leaves the camera's field of view is granted unrestricted access.
[0019] The invention will be further explained below using particularly preferred embodiments, to which it is not intended to be limited, and with reference to the drawings. In detail: Fig. 1 schematically shows the sequence of a simple variant of the method according to the invention; Fig. 2 schematically shows the sequence of an extended variant of the method according to the invention; Fig. 3a schematic of a mobile device with blocked access; Fig. 3b schematic of a mobile device with unrestricted access; Fig. 3c schematic of a mobile device with restricted access; Fig. 4a schematically shows a use case for unrestricted access on a mobile device; and Fig. 4b schematic of a use case for restricted access on a mobile device.
[0020] Fig. 1 shows the sequence of a computer-implemented method for access control of a terminal device (cf. Fig. 3a-c Private and other data are stored on the device. Access to the private data may be restricted or unrestricted.
[0021] Based on unrestricted access 1 (cf. Fig. 3b ) includes the Fig. 1 The method shown follows the following steps, which are represented as a functional block. First, an image signal is captured with a camera of the terminal device 2 (see Fig. 4a-b ). Subsequently, face detection is carried out using the captured image signal 3. After face detection is completed, the number of faces determined during face detection is checked in the captured image signal 4. If the number of faces is greater than one (cf. Fig. 4b ), access is restricted 5 (cf. Fig. 3c Otherwise, 6 the next image signal is captured, allowing continuous monitoring 7 of the number of faces in the camera's field of view during unrestricted access. Once access is restricted, monitoring can be aborted.
[0022] If the device has a screen for displaying private and other data, the private data will be completely hidden in restricted access, and only the other data will be displayed. In the case of unrestricted access, a visual indication of unrestricted access will be displayed on the screen in the form of a colored (e.g., green) frame. If an event involving private data occurs in restricted access, the event will be displayed on the screen in a restricted format.
[0023] Fig. 2 shows an extended variant of the method according to the invention. During the monitoring 7, a facial identification 8 is additionally performed. This is followed by a check 9 of the user's authorization. If the user is not authorized, access is restricted 5. The representation in Fig. 2 is simplified; facial identification 8 can, for example, only be carried out randomly or, for example, every tenth run of the verification 7 in order to conserve the resources of the terminal device (CPU time, battery).
[0024] According to a further variant of the method according to the invention, a user can be biometrically authenticated using facial identification and, optionally, iris identification to obtain unrestricted access. Before granting unrestricted access, an image signal is captured with the device's camera, face detection is performed using the captured image signal, and the number of faces in the captured image signal is determined. Subsequently, access is only granted if the number of faces is exactly one.
[0025] In Fig. 3a-c The various states of a terminal device 10 are shown, which can be assumed within the scope of the application of the method according to the invention. The terminal device 10 in this example is a smartphone with a screen 11. Fig. 3a The terminal 10 is shown with access blocked. A general symbol is displayed on the screen 11, which symbolizes and identifies the access block. In this state, access to the data stored on the terminal 10 is generally not granted; in particular, no personal data of a user of the terminal is displayed. At most, technical data (such as time, signal, network information, etc.) may be displayed. If an authorized user authenticates himself, e.g., by proving his identity, and is successfully authenticated, the terminal usually changes directly to a state with unrestricted access to all data stored on the terminal. This state is in Fig. 3b In this state, messages from four contacts are displayed on the screen 11 of the terminal 10: a first message 12 from contact "Anthony," a second message 13 from contact "Beatrice," a third message 14 from contact "Chris," and a fourth message 15 from contact "David." The contact "Chris" is marked as a private contact. The third message 14 is therefore displayed with the visual indicator 16 "PRIVATE"; the indicator 16 allows the user of the terminal 10 to recognize that unrestricted access is granted.
[0026] As soon as more than one face is detected in the image signal of a camera within the scope of the method according to the invention carried out on the terminal device 10, the terminal device 10 changes to a state with restricted access. This state is Fig. 3c In this state, the first message 12, the second message 13, and the fourth message 15 are displayed. The third message 14 is from the contact "Chris" marked as private and is therefore also marked as private and belongs to the private data. Accordingly, in the Fig. 3c With the restricted access shown, no indication of the content or existence of the private third message 14 is displayed. An observer who observes the screen 11 together with the user therefore does not learn of the existence of the third message 14. Since the visual indication 16 is also hidden, the observer cannot recognize that access is currently restricted.
[0027] Fig. 4a und 4b show the conditions according to Fig. 3b und 3c corresponding use cases. In Fig. 4a A first application case is shown in which an authorized user 17 uses the terminal device 10. The terminal device 10 has already been unlocked and is in a state with unrestricted access to private data 18 and other data 19. Using a front camera 20 of the terminal device 10, which is arranged on the same side as the screen 11, an image is repeatedly captured and the number of faces in the captured image is detected (cf. Fig. 1 ). In the first use case, only the authorized user 17 is in the field of view of the front camera 20, so that only one face is recognized during each check, thus maintaining unrestricted access.
[0028] In Fig. 4b A second application case is shown. An observer 21 steps to the side of the authorized user 17 and into the field of view of the front camera 20. The check immediately detects that the number of detected faces is greater than one and puts the terminal device 10 into the restricted access state (see Fig. 3c ). Only the other data 19 is displayed on the screen. All private data 18 has been hidden from the display, so that their existence and content are concealed from the observer 21.
Claims
1. Computer-implemented method for the access control of a terminal, private data and other data being stored on the terminal and access to the private data being able to be limited or unlimited, comprising the following steps in the case of unlimited access (1): - capturing (2) an image signal by means of a camera of the terminal; - performing (3) a face detection process with the captured image signal; and - continuous monitoring (4) of the number of faces detected in the detected image signal during the face detection process, wherein access is limited (5) if the number of faces is greater than one, wherein the terminal comprises a screen for displaying private data and other data and the private data is completely hidden when access is limited (5) and only the other data is displayed, characterised in that a visual indication of the unlimited access is displayed when access is unlimited (1).
2. Method according to Claim 1, characterized in that, in the case of limited access (5), events pertaining to private data are displayed on the screen in a limited form.
3. Method according to one of the Claims 1 or 2, characterized in that an authentication of a user is performed in order to obtain unlimited access (1), wherein, before the unlimited access (1) is granted, an image signal is captured using the camera of the terminal, a face detection process is carried out using the captured image signal and a determination of the number of faces in the captured image signal is carried out, wherein unlimited access is only granted if the number of faces is exactly one.
4. Method according to Claim 3, characterized in that the authentication comprises biometric authentication, in particular, face identification and / or iris identification.
5. Computer program product that can be loaded directly into the internal memory of a digital computer and comprises software code sections, by means of which the steps are carried out according to one of Claims 1 to 4 when the computer program is running on a computer.
