Authentication enhancement and status transmission system using combined facial and gesture recognition

Abstract

To provide technical support for generally using a space such as a hotel or an office in a convenient and safe manner.SOLUTION: The present invention relates to an authentication enhancement and status transmission technology that combines facial and gesture recognition to attain functions for authenticating an entry / exit person by using a facial recognition device of a security door in order to authenticate a normal entry / exit person who performs a specific gesture, or for transmitting a current status (normal, abnormal, danger, etc.) to the outside. By combining the facial and gesture recognition, an entry / exit authentication can be enhanced. By registering the gesture, a user can confidentially communicate to a security control system that the user is currently in danger.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention generally relates to a technology for providing technical support for convenient and safe use of spaces such as hotels and offices. In particular, the present invention relates to a technology for strengthening authentication and transmitting status using the composite recognition of face and gestures, which combines face recognition and gesture recognition to provide a function for composite authentication of a person requesting entry or exit as a normal person by making a specific gesture when authenticating the person using a face recognition device at a security door, or for transmitting the current status (normal, abnormal, threat, etc.) to the outside. [Background technology]

[0002] Previously, efforts have been made to safely use spaces such as hotels and offices. The general approach is to assign a manager to manage entrance and exit, but this is undesirable from the standpoints of user convenience, management costs, and virus infection control. Therefore, a method of managing entrance and exit using electronic devices has been introduced.

[0003] First, a method was used in which access cards were issued to authorized users or QR codes (registered trademark) were issued and the access cards or QR codes were tagged to the smart gate to open the doors.

[0004] Also used is a system in which fingerprint and facial information of authorized users is registered in advance, and a fingerprint or facial recognition device is used to confirm the fingerprint or face of the person requesting entry, and the door is opened only if the user is a registered user.

[0005] Thus, various techniques have been proposed for safely utilizing spaces, which generally aim to verify whether or not a person requesting entry is a legitimate person. However, in reality, various threats exist. In particular, a person who is authorized to enter may be requested to enter under undesirable circumstances, for example, due to a threat from someone. Conventional technical means have the drawback of being unable to distinguish between these situations or to respond appropriately.

[0006] Biometric-based access control methods also have their drawbacks. Fingerprint recognition has the drawback that people feel uncomfortable touching their fingers and that it is difficult to recognize fingers when they are wet or dry. Facial recognition has the drawback of being unable to distinguish between people with very similar faces, such as relatives or twins, and may allow the wrong person to enter.

[0007] There is a need for technological development to resolve these issues. Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention generally aims to provide a technology that provides technical support for enabling spaces such as hotels and offices to be used conveniently and safely. In particular, the present invention aims to provide a technology for strengthening authentication and transmitting status using the composite recognition of face and gestures, which combines face recognition and gesture recognition to provide a function for composite authentication of a person requesting entry or exit as a normal entry or exit, or for transmitting the current status (normal, abnormal, threat, etc.) to the outside, by having the person make a specific gesture when authenticating the person requesting entry or exit using a face recognition device at a security door. [Means for solving the problem]

[0009] In order to achieve the above object, the authentication reinforcement and status transmission system using composite face and gesture recognition according to the present invention comprises a security door 100, which is installed to separate the inside and outside of a specific space, and has a face recognition device 200 fixedly installed facing outdoors at a height corresponding to the face of a person in a standing state, and opens and closes the door according to the face recognition result and gesture verification result linked to the face recognition device 200; a face analysis server 310 which receives a face photo from a normal access person B via an access management computer 400, stores the face photo in a face database 310A, analyzes the face image of an access requester A acquired by the face recognition device 200 of the security door 100 to extract face attributes, and determines whether or not the access requester A is a normal access person permitted to enter the space according to the result of querying the face attributes in the face database 310A, and provides the result to the security door 100; and and a gesture analysis server 320 which provides a specific gesture action and the corresponding matching relationship of the verification of the normal entry / exit or the status information from a normal entry / exit person B via the gesture analysis server 320 and stores the gesture action in the gesture database 320A; identifies the gesture of the entry / exit person A (hereinafter referred to as the "user gesture") by analyzing the photographed image of the entry / exit person A acquired by the face recognition device 200 of the security door 100; and verifies whether the entry / exit person A is a normal entry / exit person permitted to enter the space in accordance with the matching relationship information pre-stored in the gesture database 320A based on the result of querying the user gesture to the gesture database 320A, and provides the result to the security door 100 or transmits the status information matched with the user gesture to the administrator terminal 500.

[0010] At this time, the gesture analysis server 320 may be configured to provide a successful verification result to the security door 100 if the verification of a normal entry / exit person based on the user gesture is successful, and to provide a failed verification result to the security door 100 if the verification of a normal entry / exit person based on the user gesture is unsuccessful, and to transmit status information that has been matched with the user gesture to the administrator terminal 500. Effect of the Invention

[0011] According to the present invention, there is an advantage that the entrance / exit authentication can be strengthened by combining face recognition and gesture recognition. Furthermore, according to the present invention, there is an advantage that, by registering a gesture, the user can secretly inform the security management system that he or she is currently in a dangerous situation. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a general access control system based on face recognition. [Diagram 2] 1 is an operational flowchart of a general access control system based on face recognition. [Diagram 3] 1 is an overall configuration diagram of a system for strengthening authentication and transmitting status using combined face and gesture recognition according to the present invention; [Figure 4] FIG. 2 is a conceptual diagram of authentication of valid visitors based on face recognition in the present invention. [Diagram 5] FIG. 2 is a conceptual diagram of authentication reinforcement and status transfer based on gesture recognition in the present invention. [Figure 6] 1 is an exemplary diagram of a gesture-based status transfer in the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present invention will now be described in detail with reference to the drawings. In the description of the present invention, detailed description of parts overlapping with the prior art will be omitted. FIG. 1 is a diagram showing the overall configuration of a general access control system based on face recognition, and FIG. 2 is a flowchart showing the operation of the general access control system based on face recognition.

[0014] 1, a typical entrance / exit control system is composed of a security door 10 equipped with a face recognition device 20, a face analysis server 30, and an entrance / exit control computer 40. The entrance / exit control computer 40 is a device for registering or deleting the faces of people permitted to enter the space, and the security door 10 controls opening and closing depending on the result of the face recognition device 20 confirming whether or not the person A requesting entrance is registered.

[0015] The face analysis server 30 extracts face attributes of normal visitors in advance and stores them in the face database 30A. When the face image (video) of the visitor A is sent from the face recognition device 20, the face analysis server 30 analyzes the face image (video) and consults the face database 30A to determine whether the visitor A is a normal visitor.

[0016] When the security door 10 receives a determination result from the face analysis server 30 that the person is a normal visitor, it opens the door. Facial recognition is generally carried out through the following steps. First, the facial recognition device 20 acquires an image of the exterior front of the security door 10, and if a face is present in the image, the facial portion (facial image) is detected. Then, facial attributes are extracted from the facial image. Typically, facial attributes are acquired after detecting landmarks (points at specific positions). Next, the facial database 30A is consulted, and a determination is made by facial matching as to whether or not the person requesting entry / exit A is a person permitted to enter the space (a normal entrant, a registered user).

[0017] It is also possible to implement a method in which the face analysis server 30 and the face database 30A are integrated into the face recognition device 20. In that case, even if the network is disconnected, the security door 10 can be opened and closed by face recognition. The face analysis server 30 and the face database 30A can also be implemented on a cloud basis.

[0018] 2, the face recognition device 20 of the security door 10 detects the face of the person A requesting entry (S100), and the face analysis server 30 extracts the face attributes of the person A requesting entry (S110), and performs face matching to compare with the face database 30A (S120). If the face matching result (S130) indicates that the person A requesting entry is a normal person (registered user), an authentication log is stored, and the face matching result (face recognition result) is sent to the security door 10 (S140, S150). Depending on the face recognition result, the security door 10 is opened (S160). Conversely, if the face matching result (S130) indicates that the person A requesting entry is not a normal person, the security door 10 does not open.

[0019] FIG. 3 is a diagram showing the overall configuration of a system for strengthening authentication and transmitting status using combined face and gesture recognition according to the present invention. In the description of the present invention, detailed description of parts overlapping with the prior art will be omitted.

[0020] The authentication reinforcement and status transmission system according to the present invention includes a security door 100 equipped with a face recognition device 200, a face analysis server 310 that performs face recognition on a person A requesting entry and exit, and a gesture analysis server 320 that performs gesture analysis on the person A requesting entry and exit. The face analysis server 310 operates a face database 310A, and the gesture analysis server 320 operates a gesture database 320A.

[0021] On the other hand, the entrance / exit management computer 400 is a device for registering or deleting the faces of people who are permitted to enter and exit the space, and the administrator terminal 500 is a device provided to an administrator of the space (e.g., a security guard) or a police station.

[0022] With this configuration, according to the present invention, by combining face recognition and gesture recognition, entrance / exit authentication can be strengthened and the user can secretly inform the security management system that he or she is currently in a dangerous situation.

[0023] First, the security door 100 is a door structure installed to separate the inside and outside of a specific space. A face recognition device 200 is fixedly installed on the security door 100 facing the outside at a height corresponding to the face of a person in a standing state, and the security door 100 opens and closes the door according to the face recognition result and gesture verification result of a face analysis server 310 and a gesture analysis server 320 linked to the face recognition device 200.

[0024] The face analysis server 310 is a component that determines whether or not to permit entry to the access requester A based on face recognition, and provides the security door 100 with the result of the determination. FIG. 4 is a conceptual diagram of authentication of valid visitors based on face recognition performed by the face analysis server 310 in the present invention.

[0025] First, the entrance / exit management computer 400 receives a facial photograph of a person B who normally enters the space where the security door 100 is installed. For example, a selfie taken with a smartphone may be received. The facial photograph is sent to the face analysis server 310, and the face analysis server 310 stores the facial photograph of the person B in the face database 310A.

[0026] Thereafter, when the person A arrives at the security door 100, the security door 100 must determine whether the door can be opened or closed. At that time, the face recognition device 200 of the security door 100 captures the face of the person A and obtains trial data for face recognition therefrom. In one embodiment, the trial data is a face image of the person A.

[0027] The face analysis server 310 analyzes the face image (trial data) of the person requesting entry A, extracts face attributes, and queries the face attributes in the face database 310A. If the query shows that the face attributes are registered in the face database 310A, the face analysis server 310 determines that the person requesting entry A is a normal person B who is permitted to enter the space. On the other hand, if the person is not registered in the face database 310A, the face analysis server 310 determines that the person requesting entry A is not a normal person B.

[0028] The gesture analysis server 320 is a component that strengthens entrance / exit authentication for the entrance requester A based on gesture recognition or transmits a status. FIG. 5 is a conceptual diagram of enhanced authentication and status transfer based on gesture recognition in the present invention.

[0029] First, the entrance / exit control computer 400 preferably receives a facial photograph from a normal entrance / exit person B, and at the same time receives a specific gesture action. Then, for example, by a menu operation, the entrance / exit control computer 400 receives a user operation for setting the gesture action to perform additional verification as to whether or not the person is a normal entrance / exit person, or for setting the gesture action to correspond to a specific situation (e.g., normal, abnormal, threat, etc.).

[0030] Such information is sent to the gesture analysis server 320, which stores a matching relationship between the gesture action and the corresponding valid visitor verification or status information in the gesture database 320A.

[0031] At that time, if a gesture action to be used for verifying a valid person is set, the security door 100 is opened only when both the authentication of the valid person based on face recognition by the face analysis server 310 and the verification of the valid person based on gesture recognition by the gesture analysis server 320 are successful. Since the gesture used for verifying a valid person is known only to the person in question, if another person tries to enter or exit using a fake face (e.g., a photograph, a wax mask, etc.), the security door 100 will not be opened.

[0032] In addition, if a gesture action for conveying a situation is set, when the entrance requester A makes that gesture, the matching-set situation information is transmitted to the administrator terminal 500.

[0033] As described above, when the person A who requests entry arrives at the security door 100 and requests the door to be unlocked, the face recognition device 200 of the security door 100 captures an image of the face of the person A who requests entry, and authenticates the person as a normal entry person based on the face recognition by the face analysis server 310. Preferably, the face recognition device 200 simultaneously captures an image of the person A who requests entry, and transmits the captured image to the gesture analysis server 320.

[0034] The gesture analysis server 320 analyzes the captured image of the person A requesting entry, identifies the gesture of the person A requesting entry (hereinafter, referred to as a "user gesture"), and queries the gesture database 320A for the user gesture.

[0035] According to the matching relationship information stored in advance in the gesture database 320A, the gesture analysis server 320 can use the user gesture to verify whether the person A is a valid person who is permitted to enter the space. In this case, the result of the verification as to whether the person A is a valid person is provided to the security door 100.

[0036] In addition, in response to the matching relationship information pre-stored in the gesture database 320A, the gesture analysis server 320 can also obtain situation information that is matched with the user gesture from the gesture database 320A and transmit it to the administrator terminal 500.

[0037] Fig. 6 is an example diagram of the situation transmission based on gestures in the present invention. In Fig. 6, (f1) and (f2) shown in blue are areas used by the face analysis server 310 for face recognition, and (g1) and (g2) shown in red are areas used by the gesture analysis server 320 for the situation transmission based on gestures.

[0038] As an example, matching settings can be made such that when a V-shaped gesture as shown in (a) of Fig. 6 is made, it indicates that the access requester A is currently in a normal state, and when a gesture of spreading and waving the whole palm as shown in (b) of Fig. 6 is made, it indicates that the access requester A is currently in a threatening state. Such matching settings can be provided by a normal access person B via the access control computer 400 as shown in Fig. 5. In some embodiments, a face recognition device 200 may be used instead of the access control computer 400.

[0039] In FIG. 5, when person A making an entry / exit request makes the gesture of FIG. 6(b) in front of the security door 100, the gesture analysis server 320 obtains information that person A is currently in a threatening state from the result of querying the gesture database 320A for that gesture, and transmits the status information to the administrator terminal 500 so that appropriate measures can be taken by the security management system.

[0040] In the above, the configuration for verifying whether a person is a valid entrant based on a gesture and the configuration for transmitting status information have been described as being optional. However, the present invention may be configured to combine the verification of whether a person is a valid entrant and the transmission of status information.

[0041] As an example, assume that a normal entrance / exit person B in FIG. 5 has registered the normal notification gesture in FIG. 6(a) and the danger notification gesture in FIG. 6(b). When the access requester A performs a normal access gesture as shown in Fig. 6(a), the gesture analysis server 320 determines that the authentication has been successful and provides the authentication success result to the security door 100. In other words, the normal access gesture is used as a gesture for authenticating the normal access.

[0042] Conversely, if the person requesting entry / exit A fails to be verified as a normal entry / exit user because he / she does not perform any gesture other than the normal notification gesture, the result of the failed verification may be provided to the security door 100, and status information that is set to match the user gesture performed by the person requesting entry / exit A may be obtained from the gesture database 320A and transmitted to the administrator terminal 500.

[0043] The face analysis server 310, face database 310A, gesture analysis server 320, and gesture database 320A may be implemented integrally in the face recognition device 200 or security door 100, or may be implemented on the cloud. In this specification, as shown in FIG. 3, these configurations 310, 310A, 320, and 320A are described based on a form in which they are connected via a network. This is for convenience, and does not exclude other methods. In addition, the entrance and exit management computer 400 and the administrator terminal 500 may be implemented integrally or separately.

Claims

1. A security door (100) that is installed to separate the inside and outside of a specific space, has a face recognition device (200) fixedly installed facing the outside at a height corresponding to the face of a person in a standing state, and opens and closes the door according to a face recognition result and a gesture verification result linked to the face recognition device (200); a face analysis server (310) which receives a face photograph from a normal entering / exiting person (B) via an entrance / exit management computer (400), stores the face photograph in a face database (310A), analyzes the face image of the person requesting entrance / exit (A) acquired by the face recognition device (200) of the security door (100) to extract face attributes, and judges whether the person requesting entrance / exit (A) is a normal entering / exiting person permitted to enter the corresponding space according to the result of querying the face attributes in the face database (310A), and provides the result to the security door (100); a gesture analysis server (320) for receiving a specific gesture action and a corresponding matching relationship of a normal entrance / exit person verification or status information from a normal entrance / exit person (B) through an entrance / exit management computer (400), storing the gesture action in a gesture database (320A), identifying a user gesture, which is a gesture of the entrance / exit person (A), by analyzing a photographed image of the entrance / exit person (A) acquired by the face recognition device (200) of the security door (100), and verifying whether the entrance / exit person (A) is a normal entrance / exit person permitted to enter the corresponding space in accordance with the matching relationship information previously stored in the gesture database (320A) based on the result of querying the user gesture in the gesture database (320A), and providing the result to the security door (100) or transmitting the status information set to be matched with the user gesture to an administrator terminal (500); A system for enhancing authentication and transmitting status using combined face and gesture recognition, comprising:

2. The system for enhancing authentication and transmitting status using composite recognition of face and gesture as described in claim 1, characterized in that the gesture analysis server (320) is configured to provide a successful verification result to the security door (100) when the user gesture is successful in verifying the validity of the user, and to provide a failed verification result to the security door (100) when the user gesture is unsuccessful in verifying the validity of the user, and to transmit status information that is matched with the user gesture to an administrator terminal (500).

Images