Identification methods, identification devices, electronic devices and storage media

Abstract

This application provides an identity recognition method, identity recognition device, electronic device, and storage medium, applied to augmented reality devices, belonging to the field of image processing technology. It acquires a first hand image, performs hand keypoint detection on the first hand image to obtain first keypoint position features, extracts a first region of interest from the first hand image based on the first keypoint position features, extracts palmprint features from the first region of interest to obtain first palmprint features, extracts palmprint features from a preset template to obtain second palmprint features, calculates similarity data between the first and second palmprint features, and performs identity recognition based on the similarity data and a preset similarity threshold to log in to the augmented reality device. This reduces the cost of identity recognition while achieving secure and fast identity recognition, thereby improving login efficiency and security.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an identity recognition method, identity recognition device, electronic device and storage medium. Background Technology

[0002] Head-mounted augmented reality (AR) devices require identity verification upon login to enhance security. Currently, the primary login method for AR devices is password-based. Some AR devices are equipped with iris detection elements, enabling login based on iris features. Password login verifies the user's identity using a standard password or a Personal Identification Number (PIN). However, password verification requires users to set custom passwords, and most platforms and devices require passwords for login. Among the many passwords a user may have, the AR device login password may be forgotten over time, posing a risk of forgetting. When a user forgets their password, quick login is not possible. Passwords are multi-character strings, making them vulnerable to leakage and decryption, thus introducing security risks into password verification. Furthermore, each password entry takes time, negatively impacting user experience and overall interactivity. Iris recognition login requires capturing the user's iris information, which necessitates incorporating a series of closely cooperating hardware components into the AR device, such as a dedicated infrared camera and an iris recognition chip, resulting in high implementation costs. Summary of the Invention

[0003] The main objective of this application is to propose an identity recognition method, identity recognition device, electronic device, and storage medium, which aims to enable convenient and quick login of head-mounted augmented reality devices upon power-on, thereby reducing costs while improving login efficiency and security.

[0004] To achieve the above objectives, a first aspect of this application proposes an identity recognition method applied to an augmented reality device, the method comprising:

[0005] Obtain the first hand image;

[0006] Hand key point detection is performed on the first hand image to obtain the position features of the first key point;

[0007] Based on the location features of the first key points, a first region of interest is extracted from the first hand image;

[0008] Palmprint features are extracted from the first region of interest to obtain the first palmprint features, and palmprint features are extracted from the preset template to obtain the second palmprint features.

[0009] Calculate the similarity data between the first palmprint feature and the second palmprint feature;

[0010] Identity verification is performed based on the similarity data and a preset similarity threshold to log in to the augmented reality device.

[0011] In some embodiments, the first keypoint location features include first keypoint coordinates, second keypoint coordinates, and third keypoint coordinates. Extracting a first region of interest from the first hand image based on the first keypoint location features includes:

[0012] Use the coordinates of the first key point as the coordinates of the first vertex, and use the coordinates of the second key point as the coordinates of the second vertex;

[0013] The third vertex coordinates are obtained by predicting the vertex position based on the first key point coordinates, the second key point coordinates, and the third key point coordinates; the fourth vertex coordinates are obtained by predicting the vertex position based on the first key point coordinates, the second key point coordinates, and the third key point coordinates.

[0014] The first region of interest is obtained by connecting the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex of the first hand image.

[0015] In some embodiments, the step of predicting the vertex position to obtain the third vertex coordinates based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates, and predicting the vertex position to obtain the fourth vertex coordinates based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates, includes:

[0016] Distance data is obtained by evaluating the distance based on the coordinates of the first key point, the second key point, and the third key point.

[0017] The third vertex coordinates are obtained by predicting the vertex position based on the first keypoint coordinates, the second keypoint coordinates, and the distance data. The fourth vertex coordinates are obtained by predicting the vertex position based on the first keypoint coordinates, the second keypoint coordinates, and the distance data.

[0018] In some embodiments, the step of extracting palmprint features from the first region of interest to obtain the first palmprint features includes:

[0019] Multi-directional palmprint feature extraction is performed on the first region of interest to obtain multiple palmprint feature maps;

[0020] Compare the pixel values ​​of pixels at each corresponding position in multiple palm print feature maps;

[0021] The pixel with the largest pixel value at each corresponding position is taken as the first palmprint feature.

[0022] In some embodiments, calculating the similarity data between the first palmprint feature and the second palmprint feature includes:

[0023] The first palmprint feature is encoded to obtain a first code, and the second palmprint feature is encoded to obtain a second code;

[0024] The similarity data is obtained by calculating the angular distance based on the first code and the second code.

[0025] In some embodiments, identity recognition is performed based on the similarity data and a preset similarity threshold, including:

[0026] If the similarity data is less than or equal to the preset similarity threshold, then identity recognition fails;

[0027] The number of times identity verification failed;

[0028] If the number of times is less than a preset threshold, then a second hand image is obtained;

[0029] Identification is performed based on the second hand image and the preset template.

[0030] In some embodiments, the preset template is obtained according to the following steps:

[0031] Obtain the third hand image;

[0032] Hand key point detection is performed on the third hand image to obtain the second key point position features;

[0033] Extract a second region of interest from the third hand image based on the location features of the second key point;

[0034] The second region of interest is used as the preset template.

[0035] To achieve the above objectives, a second aspect of this application provides an identity recognition device applied to an augmented reality device, the device comprising:

[0036] Image acquisition module, used to acquire the first hand image;

[0037] The key point detection module is used to detect key points of the hand in the first hand image and obtain the position features of the first key points;

[0038] The region of interest extraction module is used to extract a first region of interest from the first hand image based on the positional features of the first key point;

[0039] The palmprint feature extraction module is used to extract palmprint features from the first region of interest to obtain the first palmprint feature, and to extract palmprint features from a preset template to obtain the second palmprint feature.

[0040] The calculation module is used to calculate the similarity data between the first palmprint feature and the second palmprint feature;

[0041] An identity recognition module is used to perform identity recognition based on the similarity data and a preset similarity threshold in order to log in to the augmented reality device.

[0042] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the identity recognition method described in the first aspect.

[0043] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the identity recognition method described in the first aspect.

[0044] The identity recognition method, device, electronic device, and storage medium proposed in this application acquire a first hand image through the camera built into the head-mounted augmented reality device, eliminating the need for a series of specialized hardware components within the augmented reality device and reducing costs. Key hand points are detected in the first hand image to obtain the first key point location features. A first region of interest is extracted from the first hand image based on the first key point location features. Palmprint features are extracted from the first region of interest to obtain the first palmprint features. Palmprint features are then extracted from a preset template to obtain the second palmprint features. Similarity data between the first and second palmprint features is calculated. Identity recognition is performed based on the similarity data and a preset similarity threshold. This method uses the similarity between palmprint features to perform login operations on the augmented reality device. Compared to identity recognition methods using passwords or iris recognition, it avoids the drawbacks of forgotten passwords, password leaks, and high costs. It achieves rapid login while reducing costs, improving login efficiency and security. Attached Figure Description

[0045] Figure 1 This is a flowchart of the identity recognition method provided in the embodiments of this application;

[0046] Figure 2 This is a rendering of the key hand points provided in the embodiments of this application;

[0047] Figure 3yes Figure 1 The flowchart of step S130 in the process;

[0048] Figure 4 It is an image of the region of interest provided in the embodiments of this application;

[0049] Figure 5 yes Figure 3 The flowchart of step S320 in the text;

[0050] Figure 6 yes Figure 1 The flowchart of step S140 in the middle;

[0051] Figure 7 This is a schematic diagram of palmprint feature extraction provided in an embodiment of this application;

[0052] Figure 8 yes Figure 1 The flowchart of the process of obtaining the preset template in step S140;

[0053] Figure 9 yes Figure 1 The flowchart of step S150 in the middle;

[0054] Figure 10 yes Figure 1 The flowchart of step S160 in the process;

[0055] Figure 11 This is a schematic diagram of the structure of the identity recognition device provided in the embodiments of this application;

[0056] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0060] First, let's analyze some of the terms used in this application:

[0061] Application Programming Interface (API): These are predefined functions that enable communication between computer software.

[0062] In related technologies, identity verification is performed using ordinary passwords, PIN pads, and iris recognition to log in to augmented reality devices. However, password-based identity verification requires users to set custom passwords, which are prone to being forgotten or leaked. When users forget their passwords, quick login is not possible, and if the password is leaked, the security of device data cannot be guaranteed. Iris recognition requires a series of specialized hardware components on the augmented reality device to collect iris information, which incurs extremely high costs.

[0063] Based on this, embodiments of this application provide an identity recognition method, an identity recognition device, an electronic device, and a computer-readable storage medium, aiming to achieve fast login while reducing costs and improving login efficiency and security.

[0064] The identity recognition method, identity recognition device, electronic device, and computer-readable storage medium provided in the embodiments of this application are specifically described through the following embodiments. First, the identity recognition method in the embodiments of this application is described.

[0065] The identity recognition method provided in this application relates to the field of image processing technology. The identity recognition method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the identity recognition method, but is not limited to the above forms.

[0066] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0067] Figure 1 This is an optional flowchart of the identity recognition method provided in the embodiments of this application. Figure 1 The method described herein, when applied to an augmented reality device, may include, but is not limited to, steps S110 to S160.

[0068] Step S110: Obtain the first hand image;

[0069] Step S120: Perform hand key point detection on the first hand image to obtain the position features of the first key point;

[0070] Step S130: Extract the first region of interest from the first hand image based on the location features of the first key point;

[0071] Step S140: Extract palm print features from the first region of interest to obtain the first palm print features; extract palm print features from the preset template to obtain the second palm print features.

[0072] Step S150: Calculate the similarity data between the first palm print feature and the second palm print feature;

[0073] Step S160: Perform identity verification based on similarity data and a preset similarity threshold to log in to the augmented reality device.

[0074] Steps S110 to S160, as illustrated in this embodiment, extract palmprint features from a hand image and utilize the similarity between palmprint features for identity verification to perform a login operation. Compared to password-based identity verification, palmprint-based identity verification avoids the drawbacks of passwords being difficult to remember or easily leaked, enabling fast login and improving login efficiency and security. Furthermore, the acquisition of hand images relies entirely on the augmented reality device's built-in camera and does not require additional hardware components, thus reducing the cost of augmented reality device login compared to iris-based identity verification.

[0075] In step S110 of some embodiments, after the user opens the terminal device and enters the login interface, the terminal device starts the hand detection program, acquires the hand position information and the position information of the camera mounted on the terminal device in real time, calculates the distance between the hand and the camera based on the hand position information and the camera position information, and determines whether the object's hand appears in the area in front of the camera based on the distance. If the distance is less than a preset distance threshold, it means that the hand appears in the area, and the camera is called to capture the external environment scene to obtain a first hand image, which includes the palm print area of ​​the hand. If the distance is greater than or equal to the preset distance threshold, it means that the hand does not appear in the area, and hand detection continues. The terminal device can be an augmented reality device. The hand detection program is a hand detection module API. This module uses the `TryGetJointPose` function, which is encapsulated in the `MixedReality.Toolkit.Utilities` package for augmented reality devices in Unity. It can obtain real-time hand position information and 3D position points in the virtual scene. The function's inputs are `TrackedHandJoint.Palm`, `Handedness.Left`, `out`, `MixedRealityPose`, and `palmPose`. `palmPose` is a set of triples storing the 3D coordinates of the hand's key points in the world coordinate system. A preset distance threshold can be set according to actual conditions, for example, 0.2m. In essence, if x represents the horizontal coordinate, y represents the vertical coordinate, and z represents the vertical coordinate, the 3D coordinates of the hand's key points in the world coordinate system can be represented as [x1, y1, z1], and the camera's 3D coordinates in the world coordinate system can be represented as [x2, y2, z2]. Therefore, the distance between the hand and the camera can be represented as:

[0076] In step S120 of some embodiments, for the first hand image captured by the camera, the HandDetector provided in cvzone is called to detect hand key points and obtain the first key point position features. The first key point position features are the position information of 21 hand key points, such as... Figure 2 As shown, the hand detector is a neural network model trained on a large dataset that can analyze input hand images to obtain the location information of key hand points.

[0077] Please see Figure 3 In some embodiments, the location features of the first key point include the coordinates of the first key point, the coordinates of the second key point, and the coordinates of the third key point. Step S130 may include, but is not limited to, steps S310 to S330:

[0078] Step S310: Use the coordinates of the first key point as the coordinates of the first vertex, and use the coordinates of the second key point as the coordinates of the second vertex;

[0079] Step S320: Based on the coordinates of the first key point, the second key point, and the third key point, the vertex position is predicted to obtain the coordinates of the third vertex; based on the coordinates of the first key point, the second key point, and the third key point, the vertex position is predicted to obtain the coordinates of the fourth vertex.

[0080] Step S330: Connect the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex of the first hand image to obtain the first region of interest.

[0081] In step S310 of some embodiments, such as Figure 2As shown, from the 21 detected hand keypoints, keypoint 1 below the little finger, keypoint 2 below the index finger, and keypoint 3 at the bottom of the palm are selected. The first region of interest (ROI) is calculated using these three keypoints. The location information of the keypoint below the little finger is the first keypoint coordinate, the location information of the keypoint below the index finger is the second keypoint coordinate, and the location information of the keypoint at the bottom of the palm is the third keypoint coordinate. All three keypoint coordinates are two-dimensional coordinates in the image coordinate system. The first ROI is a rectangular region, and the first and second vertex coordinates are used to represent the position information of the two vertices of the rectangular region. If the first keypoint coordinates are represented as (x1, y1), the second keypoint coordinates as (x2, y2), the first vertex coordinates as (n1, m1), and the second vertex coordinates as (n2, m2), the x-coordinate of the first keypoint (x1) is used as the x-coordinate of the first vertex (n1), and the y-coordinate of the first keypoint (y1) is used as the y-coordinate of the first vertex (m1). The x-coordinate x2 of the second key point is taken as the x-coordinate n2 of the second vertex, and the y-coordinate y2 of the second key point is taken as the y-coordinate m2 of the second vertex. The calculation method of the first vertex coordinate is shown in formula (1), and the calculation method of the second vertex coordinate is shown in formula (2).

[0082]

[0083]

[0084] In step S320 of some embodiments, the coordinates of the first key point and the second key point are connected, and the growth is parallel to the direction of the third key point coordinate to predict the position of the other two vertices of the rectangular area, thereby obtaining the coordinates of the third vertex and the fourth vertex.

[0085] In step S330 of some embodiments, a rectangular frame is formed by connecting the first vertex coordinates, second vertex coordinates, third vertex coordinates, and fourth vertex coordinates of the first hand image. The image region of the first hand image within the rectangular frame is then subjected to grayscale processing to obtain a first region of interest (ROI), which is then stored in a database. The first ROI is as follows: Figure 4 As shown.

[0086] Steps S310 to S330 described above can locate the region of interest (ROI) from the first hand image. ROI extraction eliminates interference from irrelevant data in identity recognition, improving its accuracy and reliability. Furthermore, ROI extraction eliminates the need for subsequent processing of irrelevant data, reducing computational load and increasing the speed of identity recognition.

[0087] Please see Figure 5 In some embodiments, step S320 may include, but is not limited to, steps S510 to S520:

[0088] Step S510: Perform distance assessment based on the coordinates of the first key point, the second key point, and the third key point to obtain distance data;

[0089] Step S520: Based on the coordinates of the first key point, the coordinates of the second key point, and the distance data, predict the vertex position to obtain the coordinates of the third vertex. Based on the coordinates of the first key point, the coordinates of the second key point, and the distance data, predict the vertex position to obtain the coordinates of the fourth vertex.

[0090] In step S510 of some embodiments, if the coordinates of the third key point are represented as (x3, y3), the distance evaluation calculation method based on the coordinates of the first key point (x1, y1), the coordinates of the second key point (x2, y2), and the coordinates of the third key point (x3, y3) is as shown in formula (3).

[0091]

[0092] Where *dis* represents distance data, indicating the inclination of the line connecting the coordinates of the third and fourth vertices relative to the line connecting the coordinates of the first and second vertices; *d* is the distance between the keypoints below the little finger and the keypoints below the index finger, expressed as...

[0093] In step S520 of some embodiments, vertex position prediction is performed based on the coordinates of the first key point (x1, y1), the coordinates of the second key point (x2, y2), and the distance data dis, to obtain the coordinates of the third vertex (n3, m3) and the coordinates of the fourth vertex (n4, m4). The calculation method of the third vertex coordinate is shown in formula (4), and the calculation method of the fourth vertex coordinate is shown in formula (5).

[0094]

[0095]

[0096] λ is a weighting factor. In formulas (4) and (5), through experimental registration analysis, the value of λ is predefined as 0.7 in the embodiments of this application.

[0097] As shown in formulas (4) and (5), the process of predicting the position of the third and fourth vertices includes: calculating the first distance y1-y2 between the ordinate y1 of the first keypoint and the ordinate y2 of the second keypoint. The distance d from the keypoint below the little finger to the keypoint below the index finger is taken as the second distance. The ratio of the first distance to the second distance is calculated to obtain the first ratio. The weight factor 0.7, the distance data dis, and the first ratio are multiplied to obtain the first offset. The first offset is used to represent the deviation between the abscissa n3 of the third vertices and the abscissa x1 of the first keypoint, and the deviation between the abscissa n4 of the fourth vertices and the abscissa x2 of the second keypoint. The abscissa x1 of the first keypoint is added to the first offset to obtain the abscissa n3 of the third vertices. The abscissa x2 of the second keypoint is added to the first offset to obtain the abscissa n4 of the fourth vertices. The third distance x2-x1 between the abscissa x2 of the second keypoint and the abscissa x1 of the first keypoint is calculated. Calculate the ratio of the third distance to the second distance to obtain the second ratio. Multiply the weight factor 0.7, the distance data dis, and the second ratio to obtain the second offset. The second offset represents the deviation between the ordinate m3 of the third vertex and the ordinate y1 of the first keypoint, and the deviation between the ordinate m4 of the fourth vertex and the ordinate y2 of the second keypoint. Add the second offset to the ordinate y1 of the first keypoint to obtain the ordinate m3 of the third vertex. Add the second offset to the ordinate y2 of the second keypoint to obtain the ordinate m4 of the fourth vertex.

[0098] Steps S510 to S520 above determine the region of interest using coordinates, accurately locating areas in the hand image that contain palm print information. Furthermore, using two-dimensional coordinates instead of texture or color features to locate the region of interest reduces computational and data storage requirements, thus improving the efficiency of region of interest extraction.

[0099] Please see Figure 6 In some embodiments, step S140 may include, but is not limited to, steps S610 to S630:

[0100] Step S610: Extract palmprint features from the first region of interest in multiple directions to obtain multiple palmprint feature maps;

[0101] Step S620: Compare the pixel values ​​of the pixels at each corresponding position of the multiple palm print feature maps.

[0102] Step S630: The pixel with the largest pixel value at each corresponding position is taken as the first palm print feature.

[0103] In step S610 of some embodiments, in image digitization processing, palmprint lines can be represented as a series of digital values, some of which are positive and some are negative. Positive values indicate that the pixel values of the palmprint lines in a region are higher than those of the pixel points in the surrounding region, and negative values indicate that the pixel values of the palmprint lines in a region are lower than those of the pixel points in the surrounding region. This way of representing palmprint lines with positive and negative values can better distinguish palmprint lines from the surrounding region, thereby more accurately extracting and identifying palmprint features. Palmprint lines include geometric shapes such as curves and arcs formed between the palm center and fingers. Since the negative part of the palmprint lines can better express palmprint features and can suppress noise interference, the negative part of the palmprint lines is used for palmprint feature extraction. Palmprint features can be texture features, morphological features, etc. of the palmprint.

[0104] Competitive coding has higher palmprint verification accuracy. In the embodiments of this application, palmprint recognition is performed by means of competitive coding. Specifically, a two-dimensional Gabor filter is used to filter the region of interest to obtain a palmprint feature map in multiple directions. Since the palmprint lines are negative, the negative real part of the two-dimensional Gabor function is used as the two-dimensional Gabor filter. The two-dimensional Gabor filter is shown in formula (6).

[0105]

[0106] Where, (x, y) is the coordinate of a certain pixel point in the region of interest; (x0, y0) is the center of the filter; ω is the radial frequency per unit length of the filter; θ is the direction of the filter; x′ = (x − x0)cosθ+(y − y0)sinθ; y′ = −(x − x0)sinθ+(y − y0)cosθ; δ is a fixed value, which is the half-amplitude bandwidth of the frequency response.

[0107] It should be noted that the center of the filter is half of the filter step size. For example, for a 6×6 filter, both x0 and y0 are 3.

[0108] Six two-dimensional Gabor filters in different directions are used to process the region of interest, and these six directions are represented as [[ID=?]] Each two-dimensional Gabor filter in each direction is respectively convolved with the region of interest to obtain multiple feature responses, that is, palmprint feature maps in six directions. If the two-dimensional Gabor filters in different directions are represented as ψ(x, y, θ , , ,

[0107] , , i , Figure 7 , i ,

[0109] ,

[0106] , ,

[0108] , , ) and the region of interest is represented as I(x, y), then the palmprint feature map can be represented as W(x, y) = I(x, y) * ψ(x, y, θ i )

[0109] In step S620 of some embodiments, such as Figure 7As shown, during the competitive encoding process, each feature response is compared with other feature responses, and only the strongest feature response is retained. Specifically, the pixel values ​​of the pixels at each corresponding position in multiple palmprint feature maps are compared. For example, if six palmprint feature maps each have two pixels with coordinates (x1, y1) and (x2, y2), the pixel values ​​of the six pixels at position (x1, y1) and the six pixels at position (x2, y2) in these six palmprint feature maps are compared.

[0110] In step S630 of some embodiments, the pixel with the largest pixel value at each corresponding position is selected as the first palmprint feature. For example, the pixel with the largest pixel value is selected from the six pixels at position (x1, y1) as the pixel at position (x1, y1), and the pixel with the largest pixel value is selected from the six pixels at position (x2, y2) as the pixel at position (x2, y2), until the comparison of the last pixel at the last position of the palmprint feature map is completed, and the first palmprint feature is obtained. It can be understood that since palmprints are negative values, the pixel with the largest pixel value is the pixel with the largest absolute value in the palmprint feature map. The representation of the first palmprint feature is shown in formula (7).

[0111]

[0112] The extraction method for the second palm print feature is the same as that for the first palm print feature, and can be referred to steps S610 to S630, which will not be repeated here.

[0113] Through the above steps S610 to S630, palm print features can be extracted from the region of interest for palm print feature matching to complete the login operation of the terminal device.

[0114] Please see Figure 8 In some embodiments, the process of obtaining the preset template in step S140 may include, but is not limited to, steps S810 to S840:

[0115] Step S810: Obtain the third hand image;

[0116] Step S820: Perform hand key point detection on the third hand image to obtain the second key point position features;

[0117] Step S830: Extract the second region of interest from the third hand image based on the location features of the second key point;

[0118] Step S840: Use the second region of interest as a preset template.

[0119] In step S810 of some embodiments, the registration interface of the augmented reality device consists of two parts: a hand model and prompt text. The hand model defines the approximate position of the user's palm to prevent the user's hand from leaving the camera's range, thus ensuring hand image capture. The prompt text prompts the user to open their palm and face the device. Once the user is ready, a registration command is sent to the augmented reality device to enter the registration process. The registration command can be made via voice, keystrokes, etc., and the voice command may include "register" or "sign up". In response to the registration command, the hand model and prompt text are displayed. The user places their palm in the area of ​​the hand model, and the hand detection API is called to determine whether the user's palm is their left or right hand. The hand detection API uses the functions `TryGetJointPose(TrackedHandJoint.Palm, Handedness.Left, out MixedRealityPose palmPose)` and `TryGetJointPose(TrackedHandJoint.Palm, Handedness.Right, out MixedRealityPose palmPoseRight)` from the `MixedReality.Toolkit.Utilities` package. If `TryGetJointPose(TrackedHandJoint.Palm, Handedness.Left, out MixedRealityPose palmPose)` returns false, the user's hand is determined to be left-handed. If `TryGetJointPose(TrackedHandJoint.Palm, Handedness.Right, out MixedRealityPose palmPoseRight)` returns false, the user's hand is determined to be right-handed. The augmented reality device adjusts the hand model in real time to adapt to changes in the user's hand position. If the user's hand is left-handed, the left-hand model is used; if the user's hand is right-handed, the left-hand model is rotated 180° around the central axis to obtain the right-hand model. The augmented reality device uses a camera to capture an image of the current scene, resulting in a third hand image. This third hand image can be either a vertical or horizontal palm print image.

[0120] In some embodiments, a vertical palmprint image is first acquired. If the vertical palmprint image contains a complete vertical palmprint, the vertical palmprint acquisition is successful, and a horizontal palmprint image can be acquired. If the horizontal palmprint image contains a complete horizontal palmprint, the horizontal palmprint acquisition is successful, and palmprint registration is complete. If the vertical palmprint image does not contain a complete vertical palmprint, the vertical palmprint acquisition effect is poor, and it is necessary to revert to acquiring a vertical palmprint image until the vertical palmprint is successfully acquired before proceeding with the horizontal palmprint image acquisition. If the horizontal palmprint image does not contain a complete horizontal palmprint, the horizontal palmprint acquisition effect is poor, and it is necessary to revert to acquiring a horizontal palmprint image until the horizontal palmprint is successfully acquired.

[0121] In step S820 of some embodiments, the cvzone hand detection function is called to detect key points of the hand, and 21 key points are obtained. These 21 key points are used as the second key point position features.

[0122] In step S830 of some embodiments, the positional information of three key points—the key point below the index finger, the key point below the little finger, and the wrist key point at the bottom of the palm—are selected from the second key point positional features. A second region of interest is then extracted from the third hand image based on the positional information of these three key points. The method for extracting the region of interest is the same as in steps S310 to S330, and will not be repeated here.

[0123] In step S840 of some embodiments, the second region of interest is saved to the backend database as a preset template.

[0124] Through the above steps S810 to S840, a preset template can be obtained, and identity recognition can be performed based on template matching.

[0125] Please see Figure 9 In some embodiments, step S150 may include, but is not limited to, steps S910 to S920:

[0126] Step S910: Encode the first palm print feature to obtain the first code, and encode the second palm print feature to obtain the second code;

[0127] Step S920: Angular distance is calculated based on the first and second codes to obtain similarity data.

[0128] In step S910 of some embodiments, feature matching is performed on the first palmprint feature and the second palmprint feature. The distance between the two palmprints is determined by calculating the angular distance based on a Boolean operator, where the distance between the two palmprints is defined as the sum of the angular distances of all sampling points, i.e., the dissimilarity. To speed up feature matching, the filter direction θ of each pixel in the first palmprint feature is adjusted. iThe index value i is encoded as 3 bits, with direction index values ​​of 0, 1, 2, 3, 4, and 5. The 3-bit index value and the 1-bit first mask are combined to form a competition code, resulting in the first code. The first mask is used to indicate non-palmprint pixels in the first palmprint feature. Each bit of the first code is a binary value. The second code, like the first code, is a competition code, also composed of a 3-bit index value and a 1-bit second mask, and uses the same encoding method as the first code, which will not be elaborated further here.

[0129] In step S920 of some embodiments, the angular distance between the two competing codes, i.e., the dissimilarity, is calculated, and the similarity data is obtained by subtracting the dissimilarity from 1. The method for calculating the similarity data is shown in formula (8).

[0130]

[0131] Where P is the first code; Q is the second code; P i (x, y) is the i-th bit of the competition code of the pixel at position (x, y) in the first encoding; Indicates bitwise XOR; M is the length of the region of interest; N is the width of the region of interest; Q i (x, y) is the i-th bit of the competition code of the pixel at position (x, y) in the second encoding.

[0132] Through the above steps S910 to S920, the degree of matching between the two palm print features can be obtained to identify the identity of the user to be logged in.

[0133] Please see Figure 10 In some embodiments, step S160 may include, but is not limited to, steps S1010 to S1040:

[0134] Step S1010: If the similarity data is less than or equal to the preset similarity threshold, then the identity recognition fails.

[0135] Step S1020: Obtain the number of times identity verification failed;

[0136] Step S1030: If the number of attempts is less than a preset threshold, then acquire the second hand image;

[0137] Step S1040: Perform identity recognition based on the second hand image and the preset template.

[0138] In step S1010 of some embodiments, if the similarity data is greater than a preset similarity threshold, the identity recognition is successful and the login operation is performed. If the similarity data is less than or equal to the preset similarity threshold, the identity recognition fails.

[0139] In step S1020 of some embodiments, when identity recognition fails, the number of times identity recognition fails is obtained in order to determine whether palmprint recognition needs to be performed again.

[0140] In step S1030 of some embodiments, if the number of attempts is less than a preset threshold, the hand image is re-acquired to obtain a second hand image. If the number of attempts is greater than or equal to the preset threshold, it indicates that the maximum tolerance number of attempts has been exceeded, and the palmprint login process is exited, preventing the user from logging into the terminal device using palmprint login and forcing them to use other login methods. It is understood that the preset threshold can be modified to control the maximum number of matches.

[0141] In step S1040 of some embodiments, hand keypoint detection, region of interest extraction, and palmprint feature extraction are performed on the second hand image to obtain first palmprint features. Palmprint features are extracted from a preset template to obtain second palmprint features. Similarity data between the first and second palmprint features is calculated, and identity recognition is performed based on the similarity data and a preset similarity threshold. If the similarity is greater than the preset similarity threshold, it means that the palmprint of the login object is similar to the preset template, and identity recognition is successful. Then, a login operation is performed to log in to the augmented reality device for virtual interaction. If the similarity is less than or equal to the preset similarity threshold, it means that the palmprint of the login object differs significantly from the preset template, and the login object may not be the owner of the augmented reality device. The identity recognition of the login object fails, and successful login cannot be achieved.

[0142] Through steps S1010 to S1040, when identity recognition fails, it can be re-performed without exceeding the maximum number of errors, thereby improving the accuracy of identity recognition.

[0143] As AR devices become increasingly popular, users are placing higher demands on their security. Preventing unauthorized use of their devices has become a real need for users. Furthermore, the convenience and speed of the identity verification process is also a crucial factor affecting user experience. Therefore, developing a convenient, fast, and robust login verification technology has significant application value.

[0144] In related technologies, AR devices primarily rely on password verification for identity recognition. Some AR devices are equipped with iris detection elements, enabling identity recognition based on iris features. Password verification requires users to set passwords, and most platforms and devices require password logins. Users may forget their AR device login passwords over time. Passwords are multi-character strings, making them vulnerable to leakage and decryption, thus posing a security risk. Furthermore, each password entry takes time, negatively impacting user experience and overall interactivity. Iris recognition login, on the other hand, requires capturing the user's iris information. This necessitates incorporating a series of tightly integrated hardware components within the AR device, such as a dedicated infrared camera and an iris recognition chip, which incurs significant implementation costs.

[0145] This application proposes an identity recognition method to enable convenient and quick login for head-mounted augmented reality devices upon startup, while reducing costs and ensuring login security.

[0146] The identity recognition method in this application acquires hand images using a regular camera mounted on a head-mounted augmented reality device, without adding any additional equipment, thus reducing design and deployment costs. The cvzone hand detector is used to detect key points in the hand image, obtaining the location information of 21 hand key points. From these 21 key points, three key points are selected: the key point below the index finger, the key point below the little finger, and the wrist key point at the bottom of the palm. A rectangular region of interest is calculated based on the location information of these three key points. A two-dimensional Gabor filter with six directions is used to extract palmprint features from the rectangular region of interest in multiple directions, resulting in palmprint feature maps in six directions: 0°, 30°, 60°, 90°, 120°, and 150°. The pixel values ​​of the pixels at each corresponding position in the multiple palmprint feature maps are compared, and the filter direction index value of the pixel with the largest pixel value at each corresponding position is taken as the first palmprint feature. Palmprint features are extracted from a preset template used in the palmprint registration phase, resulting in palmprint feature maps in six directions: 0°, 30°, 60°, 90°, 120°, and 150°. The filter direction index value of the pixel with the highest pixel value at each corresponding position is used as the second palmprint feature. The similarity between the first and second palmprint features is calculated. If the similarity is greater than a preset similarity threshold, a login operation is performed. Competitive coding for palmprint feature matching improves the speed of identity recognition and enhances the user experience.

[0147] Please see Figure 11 This application also provides an identity recognition device for use in augmented reality devices, which can implement the above-described identity recognition method. The device includes:

[0148] Image acquisition module 1110 is used to acquire a first hand image;

[0149] The key point detection module 1120 is used to detect key points of the hand in the first hand image and obtain the position features of the first key points;

[0150] The region of interest extraction module 1130 is used to extract a first region of interest from the first hand image based on the positional features of the first key point;

[0151] The palmprint feature extraction module 1140 is used to extract palmprint features from a first region of interest to obtain a first palmprint feature, and to extract palmprint features from a preset template to obtain a second palmprint feature.

[0152] Calculation module 1150 is used to calculate the similarity data between the first palm print feature and the second palm print feature;

[0153] The identity recognition module 1160 is used to perform identity recognition based on similarity data and a preset similarity threshold in order to log in to the augmented reality device.

[0154] The specific implementation of this identity recognition device is basically the same as the specific implementation of the above-described identity recognition method, and will not be repeated here.

[0155] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned identity recognition method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0156] Please see Figure 12 , Figure 12 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0157] The processor 1210 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0158] The memory 1220 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1220 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1220 and is called and executed by the processor 1210 to execute the identity recognition method of the embodiments of this application.

[0159] The input / output interface 1230 is used to implement information input and output.

[0160] The communication interface 1240 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0161] Bus 1250 transmits information between various components of the device (e.g., processor 1210, memory 1220, input / output interface 1230, and communication interface 1240);

[0162] The processor 1210, memory 1220, input / output interface 1230 and communication interface 1240 are connected to each other within the device via bus 1250.

[0163] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described identity recognition method.

[0164] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0165] The identity recognition method, identity recognition device, electronic device, and computer-readable storage medium provided in this application extract palm print features from hand images and use the similarity between palm print features to perform identity recognition and login operations. Compared with identity recognition through passwords, it does not have the disadvantages of forgetting passwords or password leakage, and can achieve fast login, improving login efficiency and security.

[0166] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0167] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0168] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0170] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0171] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0173] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0175] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An identity recognition method, characterized in that, Applied to augmented reality devices, the method includes: Obtain the first hand image; Hand key point detection is performed on the first hand image to obtain the position features of the first key point; The first region of interest is extracted from the first hand image based on the location features of the first key point; Palmprint features are extracted from the first region of interest to obtain the first palmprint features, and palmprint features are extracted from the preset template to obtain the second palmprint features. Calculate the similarity data between the first palmprint feature and the second palmprint feature; Identity verification is performed based on the similarity data and a preset similarity threshold to log in to the augmented reality device; The first key point location features include first key point coordinates, second key point coordinates, and third key point coordinates. Extracting the first region of interest from the first hand image based on the first key point location features includes: The first keypoint coordinates are used as the first vertex coordinates, and the second keypoint coordinates are used as the second vertex coordinates; the vertex position is predicted based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates to obtain the third vertex coordinates; the vertex position is predicted based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates to obtain the fourth vertex coordinates; the first vertex coordinates, the second vertex coordinates, the third vertex coordinates, and the fourth vertex coordinates of the first hand image are connected to obtain the first region of interest; The preset template is obtained according to the following steps: A third hand image is acquired, wherein the third hand image is a hand image registered on the registration interface of the augmented reality device; key point detection is performed on the third hand image to obtain second key point position features; a second region of interest is extracted from the third hand image based on the second key point position features; and the second region of interest is used as the preset template.

2. The identity recognition method according to claim 1, characterized in that, The step of predicting the vertex position to obtain the third vertex coordinates based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates, and predicting the vertex position to obtain the fourth vertex coordinates based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates, includes: Distance data is obtained by evaluating the distance based on the coordinates of the first key point, the second key point, and the third key point. The third vertex coordinates are obtained by predicting the vertex position based on the first keypoint coordinates, the second keypoint coordinates, and the distance data. The fourth vertex coordinates are obtained by predicting the vertex position based on the first keypoint coordinates, the second keypoint coordinates, and the distance data.

3. The identity recognition method according to claim 1, characterized in that, The step of extracting palmprint features from the first region of interest to obtain the first palmprint features includes: Multi-directional palmprint feature extraction is performed on the first region of interest to obtain multiple palmprint feature maps; Compare the pixel values ​​of pixels at each corresponding position in multiple palm print feature maps; The pixel with the largest pixel value at each corresponding position is taken as the first palmprint feature.

4. The identity recognition method according to claim 1, characterized in that, The calculation of similarity data between the first palmprint feature and the second palmprint feature includes: The first palmprint feature is encoded to obtain a first code, and the second palmprint feature is encoded to obtain a second code; The similarity data is obtained by calculating the angular distance based on the first code and the second code.

5. The identity recognition method according to any one of claims 1 to 4, characterized in that, Identity recognition is performed based on the similarity data and a preset similarity threshold, including: If the similarity data is less than or equal to the preset similarity threshold, then identity recognition fails; The number of times identity verification failed; If the number of times is less than a preset threshold, then a second hand image is obtained; Identification is performed based on the second hand image and the preset template.

6. An identity recognition device, characterized in that, Applied to augmented reality devices, the device includes: Image acquisition module, used to acquire the first hand image; The key point detection module is used to detect key points of the hand in the first hand image and obtain the position features of the first key points; The region of interest extraction module is used to extract a first region of interest from the first hand image based on the positional features of the first key point; The palmprint feature extraction module is used to extract palmprint features from the first region of interest to obtain the first palmprint feature, and to extract palmprint features from a preset template to obtain the second palmprint feature. The calculation module is used to calculate the similarity data between the first palmprint feature and the second palmprint feature; An identity recognition module is used to perform identity recognition based on the similarity data and a preset similarity threshold in order to log in to the augmented reality device; The first key point location features include first key point coordinates, second key point coordinates, and third key point coordinates. Extracting the first region of interest from the first hand image based on the first key point location features includes: The first keypoint coordinates are used as the first vertex coordinates, and the second keypoint coordinates are used as the second vertex coordinates; the vertex position is predicted based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates to obtain the third vertex coordinates; the vertex position is predicted based on the first keypoint coordinates, the second keypoint coordinates, and the third keypoint coordinates to obtain the fourth vertex coordinates; the first vertex coordinates, the second vertex coordinates, the third vertex coordinates, and the fourth vertex coordinates of the first hand image are connected to obtain the first region of interest; The preset template is obtained according to the following steps: A third hand image is acquired, wherein the third hand image is a hand image registered on the registration interface of the augmented reality device; key point detection is performed on the third hand image to obtain second key point position features; a second region of interest is extracted from the third hand image based on the second key point position features; and the second region of interest is used as the preset template.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the identity recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the identity recognition method according to any one of claims 1 to 5.

Images