Contactless palm vein gesture recognition method and access control system

Abstract

The present application provides a contactless palm vein gesture recognition method and access control system, the contactless palm vein gesture recognition method comprising the following steps: obtaining a user's palm vein image and performing palm vein recognition; after palm vein recognition is passed, randomly generating a gesture action instruction; obtaining a user's gesture action image, inputting the gesture action image into a gesture action multi-classification neural network for category prediction, and outputting the category probability corresponding to the gesture action instruction and the category probability of the alarm gesture action; if the category probability corresponding to the gesture action instruction is greater than a first threshold, the user's identity is verified and an unlock command is sent; if the category probability corresponding to the gesture action instruction is less than the first threshold, and there is an alarm gesture action with a category probability greater than a second threshold, an alarm message is sent. The contactless palm vein gesture recognition method of the present application improves the anti-attack capability of identity recognition and authentication, and ensures the personal safety of users.

Description

Technical Field

[0001] The present application belongs to the field of biometric recognition technology, and more specifically, relates to a contactless palm vein gesture recognition method and an access control system. Background Art

[0002] Biometric recognition technology, unlike traditional identity authentication technologies based on passwords, is gaining increasing attention. It has become widely adopted in our daily lives, with applications such as facial recognition, fingerprint login, and vein recognition impacting every aspect of our daily security. Vein recognition is a contactless authentication method that uses a vein reader to obtain a map of an individual's veins under near-infrared light. The method then uses a specialized matching algorithm to extract characteristic values ​​from the map for matching and authentication. Currently, the mainstream vein recognition method primarily involves palm vein recognition, specifically identifying veins in the palm area.

[0003] In some high-security scenarios with existing technologies, such as banks, customs, border inspections, prisons, etc., although vein recognition can overcome most prosthetic attacks, there are still potential security risks. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a contactless palm vein gesture recognition method and an access control system to solve the technical problem of insufficient security in the palm vein application process in the prior art.

[0005] To achieve the above objectives, the technical solution adopted in this application is to provide a non-contact palm vein gesture recognition method, including:

[0006] Obtain the user's palm vein image and perform palm vein recognition;

[0007] After palm vein recognition is passed, gesture action commands are randomly generated;

[0008] Obtaining a user's gesture image, inputting the gesture image into a gesture multi-classification neural network for category prediction, and outputting the category probability corresponding to the gesture instruction and the category probability of the alarm gesture;

[0009] If the category probability corresponding to the gesture action instruction is greater than the first threshold, the user identity is verified and the unlock command is sent; if the category probability corresponding to the gesture action instruction is less than the first threshold, and the category probability of the alarm gesture action is greater than the second threshold, an alarm message is sent.

[0010] Preferably, if the category probability corresponding to the gesture action instruction is less than a first threshold, and the category probability of the alarm gesture action is less than a second threshold, a gesture action instruction is randomly regenerated.

[0011] Preferably, the method for obtaining a user's palm vein image and performing palm vein recognition comprises the steps of:

[0012] Acquire the user's palm vein image under infrared light;

[0013] encoding the acquired palm vein map of the user to generate a first feature vector;

[0014] Matching the first feature vector with the user permission record database to generate a first matching score;

[0015] It is determined whether the first matching score is greater than a third threshold.

[0016] Preferably, the user authority record library includes a second feature vector, wherein the second feature vector is generated by encoding a pre-collected palm vein record image of an authorized user.

[0017] Preferably, the user authority registration library includes a third feature vector, wherein the third feature vector is extracted and segmented from the palm vein registration image of the authorized user to obtain a palm vein target segmentation image, the palm vein target segmentation image is expanded, and the expanded registration palm vein image is encoded and generated.

[0018] Preferably, if the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, the value of the first threshold is increased.

[0019] Preferably, if the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, the value of the second threshold is reduced.

[0020] Preferably, the alarm gesture action includes a non-unlock alarm gesture action and an unlock alarm gesture action. After sending the alarm information, if the alarm gesture action is recognized as a non-unlock alarm gesture action, a gesture action instruction is randomly regenerated. If the alarm gesture action is recognized as an unlock alarm gesture action, an unlock command is also sent.

[0021] Preferably, after sending the alarm information, the method further comprises the steps of:

[0022] Get the surveillance video of the user's area.

[0023] The present application also provides an access control system, including an access control device, a palm vein recognition terminal and a background monitoring device. The palm vein recognition terminal can send an unlock command to the access control device or send an alarm message to the background monitoring device according to the recognition result based on the non-contact palm vein gesture recognition method as described above.

[0024] Compared with the existing technology, the contactless palm vein gesture recognition method provided in this application improves the anti-attack capability of identity recognition and authentication by combining palm vein recognition with gesture recognition. It uses a gesture multi-classification neural network for category prediction and outputs the category probability corresponding to the gesture instruction and the category probability of the alarm gesture. On the one hand, it improves the smoothness of gesture recognition, and on the other hand, it can quickly issue alarm information through hidden gestures to ensure the personal safety of users.

[0025] Compared with the existing technology, the access control system provided by the present application, when the access control device is equipped with the palm vein recognition terminal, can simultaneously monitor multiple access control devices and palm vein recognition terminals through the background monitoring device, and the staff can remotely monitor through the background monitoring device, thereby improving the security of the access control device. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 A flowchart of a contactless palm vein gesture recognition method according to an embodiment of the present application;

[0028] Figure 2 A flowchart of another form of the contactless palm vein gesture recognition method provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of a multi-classification neural network for gesture actions provided in an embodiment of the present application;

[0030] Figure 4 A schematic block diagram of matching the first feature vector provided in an embodiment of the present application with a user rights record library;

[0031] Figure 5 A palm vein target segmentation image corresponding to the second eigenvector provided in an embodiment of the present application;

[0032] Figure 6 A palm vein target segmentation image corresponding to the third eigenvector provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of the access control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0035] It should be noted that when an element is referred to as being “fixed on” or “disposed on” another element, it may be directly on the other element or indirectly on the other element. When an element is referred to as being “connected to” another element, it may be directly connected to the other element or indirectly connected to the other element.

[0036] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0037] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0038] Please also refer to Figures 1 to 3 The contactless palm vein gesture recognition method provided by the embodiment of the present application is now described. The contactless palm vein gesture recognition method includes:

[0039] Step S1: Acquire a user's palm vein image and perform palm vein recognition.

[0040] It is understandable that palm vein images can be collected using an infrared camera. For example, a laser sensor and an infrared fill light are set next to the infrared camera. When the user's palm is placed on the infrared camera, the laser sensor is triggered and the infrared fill light is used to fill light the palm. The infrared camera can collect a clear palm vein image to identify and authenticate whether the user has permission.

[0041] Step S2: After the palm vein recognition is passed, a gesture action instruction is randomly generated.

[0042] It is understood that after palm vein recognition is successful, gesture instructions can be randomly generated through the display, text, or voice to prompt the user to imitate the gesture. This can prevent attacks that exploit the user's unconsciousness and improve the security of palm vein recognition.

[0043] Step S3: obtaining a user's gesture image, inputting the gesture image into a gesture multi-classification neural network for category prediction, and outputting the category probability corresponding to the gesture instruction and the category probability of the alarm gesture.

[0044] Step S4: If the category probability corresponding to the gesture action instruction is greater than the first threshold, the user identity is verified and the unlock command is sent; if the category probability corresponding to the gesture action instruction is less than the first threshold, and the category probability of the alarm gesture action is greater than the second threshold, an alarm message is sent.

[0045] It is understood that after the user performs a corresponding gesture according to the gesture instruction, the gesture image can be captured through the same infrared camera or a separate visible light camera. The backbone network of the gesture multi-classification neural network is preferably EfficientNet-b3, which includes categories corresponding to all gesture instructions and alarm gesture categories. That is, each randomly generated gesture instruction has a corresponding category.

[0046] If the user is in a normal state and makes a corresponding gesture according to the gesture instruction, and the category probability corresponding to the output gesture instruction is greater than the first threshold, the user identity is verified and the unlock command is sent.

[0047] If a user is threatened, they can perform an alarm gesture. This will cause the probability of the category corresponding to the gesture instruction to be less than the first threshold, and the probability of the category of the alarm gesture to be greater than the second threshold, and then send an alarm message. This allows for quick and easy alarm messages through concealed gestures, ensuring user safety.

[0048] Of course, the alarm information includes user information and location information. The user information can be obtained through the palm vein recognition result in step S1. Therefore, the backstage personnel who receive the alarm information can take better security measures.

[0049] Unlike conventional multi-classification neural networks, the gesture multi-classification neural network does not directly output a prediction result. Instead, it outputs the category probability corresponding to the gesture instruction and the category probability of the alarm gesture. This ensures that the output of the gesture multi-classification neural network is not affected by all categories and does not need to be compared with the probabilities of all categories. Instead, it only needs to compare the category probability corresponding to the gesture instruction with a first threshold. Even if the category probability corresponding to the gesture instruction is not the highest, as long as the category probability corresponding to the gesture instruction is greater than the first threshold, the user's identity can be verified and the unlock command can be sent. Similarly, as long as the category probability corresponding to the gesture instruction is less than the first threshold and the category probability of the alarm gesture is greater than the second threshold, an alarm message can be sent. This is because the purpose of gesture instructions is primarily to prevent attacks that exploit the user's unconsciousness. Simply performing an action similar to the gesture instruction can eliminate this attack method. This can solve the problem of frequent gesture recognition failures caused by differences in different users' hands and limbs, improving recognition efficiency and user experience.

[0050] In addition, after palm vein recognition is passed, the user's hand target can be tracked until the user's gesture action image is obtained to prevent the hand target from being changed in the middle.

[0051] Compared with the existing technology, the contactless palm vein gesture recognition method provided in this application improves the anti-attack capability of identity recognition and authentication by combining palm vein recognition with gesture recognition. It uses a gesture multi-classification neural network for category prediction and outputs the category probability corresponding to the gesture instruction and the category probability of the alarm gesture. On the one hand, it improves the smoothness of gesture recognition, and on the other hand, it can quickly issue alarm information through hidden gestures to ensure the personal safety of users.

[0052] In another embodiment of the present application, if the category probability corresponding to the gesture action instruction is less than a first threshold, and the category probability of the alarm gesture action is less than a second threshold, a gesture action instruction is randomly regenerated.

[0053] It can be understood that if the category probability corresponding to the gesture action instruction is less than the first threshold, and the category probability of the alarm gesture action is less than the second threshold, then it is judged that the gesture action is neither similar to the gesture action instruction nor an alarm gesture action, and the gesture action instruction is randomly regenerated to prompt the user to make the gesture action again.

[0054] In another embodiment of this application, please refer to Figure 4 The method for obtaining a user's palm vein image and performing palm vein recognition comprises the steps of:

[0055] Acquire the user's palm vein image under infrared light;

[0056] encoding the acquired palm vein map of the user to generate a first feature vector;

[0057] Matching the first feature vector with the user permission record database to generate a first matching score;

[0058] It is determined whether the first matching score is greater than a third threshold.

[0059] It is understood that, before encoding the acquired user's palm vein map, preprocessing steps may be included, such as image enhancement, cropping, brightness adjustment, and binarization of the palm vein map. This will make the generated first feature vector more representative. The user authority record library is a collection of palm vein features of all authorized users. If the current user is an authorized user, the first matching score will be greater than the third threshold, and palm vein recognition will pass; otherwise, it will fail.

[0060] Further, please also refer to Figure 4 The user authority record library includes a second feature vector, wherein the second feature vector is generated by encoding a pre-collected palm vein record image of an authorized user.

[0061] It is understood that by pre-collecting a palm vein image of an authorized user, encoding the palm vein image of the authorized user to generate a second feature vector, the second feature vector may include the user's ID, palm vein features, face, identity information, etc., and is stored in the user rights record library. During operation, the first feature vector is matched with the second feature vector in the user rights record library to determine whether the current user has permission.

[0062] Further, please also refer to Figure 5 and Figure 6 The user authority registration library includes a third feature vector, wherein the third feature vector is extracted and segmented by the palm vein registration image of the authorized user to obtain a palm vein target segmentation image, the palm vein target segmentation image is expanded, and the expanded registration palm vein image is encoded and generated.

[0063] It is understandable that when the human body is in a state of emotional excitement or after strenuous exercise, the heartbeat will be rapidly accelerated, the heart's contraction force will be strengthened, blood circulation will be accelerated, and the amount of blood returning to the heart through the veins will increase accordingly, resulting in venous dilation, which is what we often call "blue veins". This phenomenon will be more obvious in the hands and feet of the human body.

[0064] Palm vein recognition is based on identifying the vein features of the human palm. However, since venous dilation is generally not captured, the present application first extracts and segments the palm vein record image of the authorized user using the third eigenvector to obtain a palm vein target segmentation image with interference removed. The palm vein target segmentation image is then dilated to thicken the subtle vein features, generating a palm vein record image under venous dilation. The dilated record palm vein image is then encoded to generate the third eigenvector. Therefore, the palm vein features of the third eigenvector can include more subtle vein features.

[0065] In this way, during the application process, the first feature vector can be matched with the second feature vector and the third feature vector in the user authority registration library at the same time. If the first feature vector is obtained in a state of venous dilation, it can also pass the palm vein recognition by matching the third feature vector, thereby improving the success rate of palm vein recognition.

[0066] Furthermore, if the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, the value of the first threshold is increased.

[0067] It is understandable that when a person is in a dangerous situation, such as duress, their heart rate will accelerate rapidly, causing the physiological phenomenon of venous dilation. Therefore, when the palm vein recognition algorithm finds that the user's first match score is obtained by matching the first eigenvector with the third eigenvector in the user permission record library, the algorithm can understand that the probability of the person being in a dangerous situation, such as duress, has increased. Because the first match score is the highest eigenvector match score, the value of the first threshold is increased, increasing the probability of the category corresponding to the gesture action instruction required to send the unlock command.

[0068] Furthermore, if the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, the value of the second threshold is reduced.

[0069] It can be understood that, similarly, when the front palm vein recognition finds that the user's first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, it can be understood at the algorithm level that the probability of the human body being in a dangerous situation such as duress has increased, then the value of the second threshold is reduced, and the category probability of the alarm gesture action required to send the alarm information is reduced.

[0070] In another embodiment of this application, please refer to Figure 3The alarm gesture action includes a non-unlock alarm gesture action and an unlock alarm gesture action. After sending the alarm information, if the alarm gesture action is recognized as a non-unlock alarm gesture action, a gesture action instruction is randomly regenerated. If the alarm gesture action is recognized as an unlock alarm gesture action, an unlock command is also sent.

[0071] It is understandable that when the user is in a dangerous situation such as duress, the user can make an unlocking alarm gesture or an unlocking alarm gesture according to the actual situation. That is, when unlocking may cause immeasurable harm, the user can choose not to make an unlocking alarm gesture, only send an alarm message, delay time and wait for rescue. If the personal safety may be in danger if the unlocking is not carried out, the user can choose to make an unlocking alarm gesture, send the alarm message, and also send an unlocking command to ensure personal safety.

[0072] In another embodiment of the present application, after sending the alarm information, the step of obtaining surveillance video of the area where the user is located is also included.

[0073] It is understandable that after receiving the alarm information, the back-end staff can also analyze the on-site situation through the surveillance video of the user's area and make corresponding countermeasures based on the actual situation.

[0074] Please also refer to Figure 7 The present application also provides an access control system, including an access control device, a palm vein recognition terminal and a background monitoring device. The palm vein recognition terminal can send an unlock command to the access control device or send an alarm message to the background monitoring device according to the recognition result based on the non-contact palm vein gesture recognition method as described above.

[0075] Compared with the existing technology, the access control system provided by the present application, when the access control device is equipped with the palm vein recognition terminal, can simultaneously monitor multiple access control devices and palm vein recognition terminals through the background monitoring device, and the staff can remotely monitor through the background monitoring device, thereby improving the security of the access control device.

[0076] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A non-contact palm vein gesture recognition method, characterized in that: include: Obtain the user's palm vein image and perform palm vein recognition; The method for obtaining a user's palm vein image and performing palm vein recognition comprises the steps of: Acquire the user's palm vein image under infrared light; encoding the acquired palm vein map of the user to generate a first feature vector; Matching the first feature vector with the user permission record database to generate a first matching score; Determining whether the first matching score is greater than a third threshold; The user authority record library includes a second feature vector, wherein the second feature vector is generated by encoding a pre-collected palm vein record image of an authorized user; The user authority record library includes a third feature vector, wherein the third feature vector is extracted and segmented by the palm vein record image of the authorized user to obtain a palm vein target segmentation image, the palm vein target segmentation image is expanded, and the expanded record palm vein image is encoded and generated; After palm vein recognition is passed, gesture action commands are randomly generated; Obtaining a user's gesture image, inputting the gesture image into a gesture multi-classification neural network for category prediction, and outputting the category probability corresponding to the gesture instruction and the category probability of the alarm gesture; If the category probability corresponding to the gesture action instruction is greater than the first threshold, the user identity is verified and the unlock command is sent; if the category probability corresponding to the gesture action instruction is less than the first threshold, and the category probability of the alarm gesture action is greater than the second threshold, an alarm message is sent.

2. The contactless palm vein gesture recognition method according to claim 1, wherein: If the category probability corresponding to the gesture action instruction is less than the first threshold, and the category probability of the alarm gesture action is less than the second threshold, the gesture action instruction is randomly regenerated.

3. The contactless palm vein gesture recognition method according to claim 1, wherein: If the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority registration library, the value of the first threshold is increased.

4. The contactless palm vein gesture recognition method according to claim 1, wherein: If the first matching score is obtained by matching the first feature vector with the third feature vector of the user authority record library, the value of the second threshold is reduced.

5. The contactless palm vein gesture recognition method according to any one of claims 1 to 4, characterized in that: The alarm gesture action includes a non-unlock alarm gesture action and an unlock alarm gesture action. After sending the alarm information, if the alarm gesture action is recognized as a non-unlock alarm gesture action, a gesture action instruction is randomly regenerated. If the alarm gesture action is recognized as an unlock alarm gesture action, an unlock command is also sent.

6. The contactless palm vein gesture recognition method according to any one of claims 1 to 4, characterized in that: After sending the alarm information, the following steps are also included: Get the surveillance video of the user's area.

7. An access control system, characterized in that: It includes an access control device, a palm vein recognition terminal and a background monitoring device. The palm vein recognition terminal can send an unlock command to the access control device or send an alarm message to the background monitoring device according to the recognition result based on the contactless palm vein gesture recognition method as claimed in claim 1.

Images