Face recognition-based identity verification method and device

Abstract

The present application can be used in the technical field of artificial intelligence technology applied in finance, and provides a face recognition-based identity verification method and device.The face recognition-based identity verification method comprises: extracting feature values of a pre-acquired user face image; the face image comprises a two-dimensional image and a three-dimensional model; determining information gain of the feature values according to information entropy of the feature values; determining similarity between the user face image and a real user face image according to the feature values and the information gain; and verifying the identity of the user according to the similarity.The present application uses holographic projection to collect human body appearance data information as an auxiliary standard for identity authentication.The present application uses the feature of holographic projection that full-body data can be acquired to expand the data collection range of identity verification from a human body part to the whole body, further perfects the comprehensiveness of data, and finally increases the authenticity and reliability of identity verification technology by using other supplementary authentication methods.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an identity verification method and device based on facial recognition. Background Technology

[0002] With the rapid development of internet technology, especially 5G communication and multimedia technologies, mobile terminals are becoming increasingly feature-rich, possessing ever-more powerful data processing and transmission capabilities. Consequently, network-based financial services are becoming increasingly prevalent due to their convenience. In this context, user authentication has become a crucial element in ensuring the security of financial transactions.

[0003] While facial recognition technology has matured, it suffers from low accuracy when multiple users share high facial similarity. Currently, some mobile devices possess 3D communication capabilities, making it possible to improve accuracy using digital holography. Therefore, how to comprehensively utilize both 2D and 3D data to enhance user identification accuracy is a pressing issue that needs to be addressed. Summary of the Invention

[0004] This invention can be used in the technical field of applying artificial intelligence technology in finance, and can also be used in any field other than finance. This invention comprehensively utilizes the two-dimensional image and three-dimensional video data of users collected during financial transactions, and uses artificial intelligence algorithms for dual verification to improve the accuracy of identity verification.

[0005] To address the technical problems in the background section of this application, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides an identity verification method based on facial recognition, comprising:

[0007] Extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0008] The information gain of the feature value is determined based on the information entropy of the feature value;

[0009] The similarity between the user's face image and the user's real face image is determined based on the feature value and the information gain.

[0010] The user's identity is verified based on the similarity score.

[0011] In one embodiment, the step of obtaining the three-dimensional model includes:

[0012] The user's laser pattern is captured using an infrared camera;

[0013] The user's depth pixel value is determined based on the emission and reception times of light to the user and the laser pattern.

[0014] Merge multiple depth pixel values ​​to determine the user's depth image;

[0015] The 3D model is obtained from the depth image.

[0016] In one embodiment, obtaining the 3D model based on the depth image includes:

[0017] The depth image is converted into 3D point cloud data according to the intrinsic parameter formula of the infrared camera;

[0018] Remove outliers from the 3D point cloud data to generate downsampled 3D point cloud data;

[0019] The 3D point cloud data is used to perform surface reconstruction to generate the three-dimensional model.

[0020] In one embodiment, extracting feature values ​​from the user's face image includes:

[0021] Using the convolutional layers of a convolutional neural network, feature maps of different regions of the user's face image are generated based on the length, width, and color of the user's face image.

[0022] The feature map is reduced in dimensionality using the pooling layer of the convolutional neural network;

[0023] The feature values ​​are generated using the fully connected layers of the convolutional neural network and the reduced-dimensional feature map.

[0024] In one embodiment, determining the information gain of the feature value based on its information entropy includes:

[0025] Determine the total information entropy corresponding to the original data set generated by all feature values;

[0026] Determine the sub-information entropy of the data set corresponding to the feature value;

[0027] The information gain is determined based on the total information entropy and the sub-information entropy.

[0028] In one embodiment, the face recognition-based identity verification method further includes: generating the information entropy of the feature values, including:

[0029] Determine the number of times the feature value appears;

[0030] Calculate the probability mass function of the feature value based on the number of times;

[0031] The information entropy of the feature value is generated based on the probability mass function.

[0032] In one embodiment, determining the similarity between the user's face image and the user's real face image based on the feature value and the information gain includes:

[0033] A comparison vector is generated based on the eigenvalues ​​and their corresponding information gains;

[0034] Generate a standard vector based on the real face image;

[0035] The Euclidean distance between the comparison vector and the standard vector is calculated to determine the similarity.

[0036] Secondly, the present invention provides an identity verification device based on facial recognition, the device comprising:

[0037] The feature extraction module is used to extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0038] An information gain determination module is used to determine the information gain of the feature value based on the information entropy of the feature value.

[0039] A similarity determination module is used to determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain;

[0040] An identity verification module is used to verify the user's identity based on the similarity.

[0041] In one embodiment, the face recognition-based identity verification device further includes: a 3D model acquisition module, used to acquire the 3D model, the 3D model acquisition module comprising:

[0042] A laser pattern acquisition unit is used to acquire the user's laser pattern via an infrared camera;

[0043] A depth pixel value determination unit is used to determine the user's depth pixel value based on the user's emission time, reception time, and the laser pattern.

[0044] A depth image determination unit is used to merge multiple depth pixel values ​​to determine the user's depth image;

[0045] A 3D model acquisition unit is used to acquire the 3D model based on the depth image.

[0046] In one embodiment, the three-dimensional model acquisition unit includes:

[0047] A cloud data generation unit is used to convert the depth image into 3D point cloud data according to the intrinsic parameter formula of the infrared camera.

[0048] An outlier removal unit is used to remove outliers from the 3D point cloud data to generate downsampled 3D point cloud data.

[0049] A surface reconstruction unit is used to perform surface reconstruction on the 3D point cloud data to generate the three-dimensional model.

[0050] In one embodiment, the feature value extraction module includes:

[0051] The feature map generation unit is used to generate feature maps of different regions of the user's face image based on the length, width, and color of the user's face image using the convolutional layer of the convolutional neural network.

[0052] The feature map dimensionality reduction unit is used to reduce the dimensionality of the feature map using the pooling layer of the convolutional neural network;

[0053] The feature value generation unit is used to generate the feature values ​​using the fully connected layers of the convolutional neural network and the reduced-dimensional feature map.

[0054] In one embodiment, the information gain determination module includes:

[0055] The total information entropy generation unit is used to determine the total information entropy corresponding to the original data set generated by all feature values;

[0056] A sub-information entropy determination unit is used to determine the sub-information entropy of the data set corresponding to the feature value;

[0057] An information gain determination unit is used to determine the information gain based on the total information entropy and the sub-information entropy.

[0058] In one embodiment, the face recognition-based identity verification device further includes: an information entropy generation module, used to generate the information entropy of the feature value, the information entropy generation module comprising:

[0059] A frequency determination unit is used to determine the frequency of occurrence of the feature value;

[0060] A function calculation unit is used to calculate the probability mass function of the feature value based on the number of times;

[0061] An information entropy generation unit is used to generate the information entropy of the feature value based on the probability mass function.

[0062] In one embodiment, the similarity determination module includes:

[0063] A contrast vector generation unit is used to generate a contrast vector based on the feature value and its corresponding information gain.

[0064] A standard vector generation unit is used to generate standard vectors based on the real face image;

[0065] A similarity determination unit is used to calculate the Euclidean distance between the comparison vector and the standard vector to determine the similarity.

[0066] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a face recognition-based authentication method.

[0067] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a face recognition-based authentication method.

[0068] As described above, embodiments of the present invention provide an identity verification method and apparatus based on face recognition, comprising: firstly, extracting feature values ​​from a pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model; nextly, determining the information gain of the feature values ​​based on their information entropy; determining the similarity between the user face image and the user's real face image based on the feature values ​​and the information gain; and finally verifying the user's identity based on the similarity. This invention utilizes the ability of holographic projection to collect human physical appearance data as an auxiliary standard for identity verification. By leveraging the ability of holographic projection to acquire full-body data, this invention expands the data collection scope for identity verification from a localized part of the human body to the entire body, further improving the comprehensiveness of the data. Combined with other supplementary authentication methods, this ultimately increases the authenticity and reliability of the identity verification technology. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating the identity verification method based on face recognition in an embodiment of the present invention. Figure 1 ;

[0071] Figure 2 This is a flowchart illustrating the identity verification method based on face recognition in an embodiment of the present invention. Figure 2 ;

[0072] Figure 3 This is a flowchart illustrating step 500 of the identity verification method based on face recognition in an embodiment of the present invention;

[0073] Figure 4This is a flowchart illustrating step 504 of the identity verification method based on face recognition in an embodiment of the present invention;

[0074] Figure 5 This is a flowchart illustrating step 100 of the identity verification method based on face recognition in an embodiment of the present invention;

[0075] Figure 6 This is a flowchart illustrating step 200 of the identity verification method based on face recognition in an embodiment of the present invention;

[0076] Figure 7 This is a flowchart illustrating the identity verification method based on face recognition in an embodiment of the present invention. Figure 3 ;

[0077] Figure 8 This is a flowchart illustrating step 600 of the identity verification method based on face recognition in an embodiment of the present invention;

[0078] Figure 9 This is a flowchart illustrating the identity verification method based on face recognition in a specific embodiment of the present invention.

[0079] Figure 10 The square is an example of a face recognition-based identity verification device in an embodiment of the present invention. Figure 1 ;

[0080] Figure 11 The square is an example of a face recognition-based identity verification device in an embodiment of the present invention. Figure 2 ;

[0081] Figure 12 This is a block diagram of the three-dimensional model acquisition module 50 in an embodiment of the present invention;

[0082] Figure 13 This is a block diagram of the three-dimensional model acquisition unit 504 in an embodiment of the present invention;

[0083] Figure 14 This is a block diagram of the feature value extraction module 10 in an embodiment of the present invention;

[0084] Figure 15 This is a block diagram of the information gain determination module 20 in an embodiment of the present invention;

[0085] Figure 16 The square is an example of a face recognition-based identity verification device in an embodiment of the present invention. Figure 3 ;

[0086] Figure 17 This is a block diagram of the similarity determination module 30 in an embodiment of the present invention;

[0087] Figure 18This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] It should be noted that the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0091] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0092] Based on the aforementioned technical limitations of the prior art, embodiments of the present invention provide a specific implementation of a face recognition-based identity verification method, see below. Figure 1 The method specifically includes the following:

[0093] Step 100: Extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0094] Specifically, the process begins by using a holographic acquisition device to collect holographic data from the customer's location. This holographic acquisition device refers to an electronic terminal device that includes an infrared camera, a visible light camera, or a depth-sensing camera. It can be a mobile phone, tablet, laptop, smart wearable device such as a smart bracelet or smartwatch, or recording equipment used in business outlets. The visible light camera is used to acquire a two-dimensional image of the target space, while the infrared camera captures the laser pattern modulated by the target object.

[0095] Preferably, the feature values ​​in step 100 include age (feature values: born in the 1980s, 1990s, 1970s), height (feature values: tall, short), gender (feature values: male, female), eyes (feature values: large eyes, small eyes, single eyelids, double eyelids), nose bridge (feature values: high nose bridge, low nose bridge), hair (feature values: long hair, medium-long hair, medium hair, medium-short hair, short hair), facial proportions (feature values: 1:1, 1:1.54), etc. Preferably, each feature value is identified using a feature classification model. For example, for the age classification model: the training samples are images of people of different ages. A convolutional neural network is trained using the training sample images to obtain the age feature classification model. Different feature models are trained sequentially to obtain the final feature values ​​for each feature.

[0096] Step 200: Determine the information gain of the feature value based on its information entropy;

[0097] It is understandable that different feature values ​​have different weights in the process of calculating the similarity between a user image and its real user image. Therefore, it is necessary to determine the weight of each feature value, i.e., information gain.

[0098] Step 300: Determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain.

[0099] Understandably, a user's real facial image can be a photo of the user's face on their ID card, or a photo of the user's face taken on-site with the user's prior consent.

[0100] Step 400: Verify the user's identity based on the similarity score.

[0101] Based on expert experience, a preset threshold is set. Then, the similarity determined in step 300 is compared with the threshold, and the user's identity is determined to be real based on the relationship between the two.

[0102] As can be seen from the above description, the identity verification method based on face recognition provided in this embodiment of the invention comprehensively utilizes two-dimensional image and three-dimensional video data collected during financial transactions, uses convolutional neural network algorithm to extract features, uses decision tree algorithm to calculate feature weights, uses cosine similarity, Euclidean distance and other algorithms to calculate similarity, and finally uses logistic regression algorithm to calculate total similarity.

[0103] In one embodiment, see Figure 2 Face recognition-based identity verification methods also include:

[0104] Step 500: Obtain the three-dimensional model.

[0105] First, an infrared camera is used to capture the laser pattern. Then, an image matching algorithm is used to calculate the depth image of the laser pattern.

[0106] See Figure 3 Step 500 further includes:

[0107] Step 501: Acquire the user's laser pattern using an infrared camera;

[0108] Step 502: Determine the user's depth pixel value based on the emission and reception times of the light for the user and the laser pattern;

[0109] Step 503: Merge multiple depth pixel values ​​to determine the user's depth image;

[0110] In steps 501 to 503, specifically, the laser projector projects uniform light into the target space, the infrared camera receives the reflected light and records the time of light emission and the time of light reception, calculates the depth pixel value corresponding to the object in the target space based on the difference between the two times and the speed of light, and merges multiple depth pixel values ​​to obtain a depth image.

[0111] Step 504: Obtain the three-dimensional model based on the depth image.

[0112] In one embodiment, see Figure 4 Step 504 includes:

[0113] Step 5041: Convert the depth image into 3D point cloud data according to the intrinsic parameter formula of the infrared camera;

[0114] Step 5042: Remove outliers from the 3D point cloud data to generate downsampled 3D point cloud data;

[0115] Step 5043: Perform surface reconstruction on the 3D point cloud data to generate the three-dimensional model.

[0116] Specifically, in steps 5041 to 5043, the depth image is first converted into a 3D point cloud according to the intrinsic parameter formula of the infrared camera. Then, the 3D point cloud is downsampled to remove outliers and the surface is reconstructed to build a three-dimensional model of the customer. Preferably, the three-dimensional model can be divided into a face model, a half-body model and a full-body model according to the different photos taken by the user.

[0117] In one embodiment, see Figure 5 Step 100 specifically includes:

[0118] Step 101: Using the convolutional layers of a convolutional neural network, generate feature maps of different regions of the user's face image based on the length, width, and color of the user's face image;

[0119] Step 100 essentially involves using a Convolutional Neural Network (CNN) to extract features from the user's face image. A CNN includes convolutional layers, pooling layers, and fully connected layers. Each two-dimensional image is composed of pixels, for example, 100×100 pixels, and pixels are composed of different colors. Therefore, pixels can be used as the length and width of the image, and colors as different channels, becoming the input to the CNN. Each image uses filters (convolutional kernels) to filter different small regions of the image, and features are extracted from each region of the image through a sliding motion, resulting in feature maps of different parts.

[0120] Step 102: Reduce the dimensionality of the feature map using the pooling layer of the convolutional neural network;

[0121] Step 103: Generate the feature values ​​using the fully connected layers of the convolutional neural network and the reduced-dimensional feature map.

[0122] In steps 102 and 103, the feature maps output by the convolutional layers are used as input to the pooling layers to reduce the dimensionality of the feature maps and retain more important features. Finally, the features sampled after convolution are integrated through fully connected layers, normalized to obtain one-dimensional features, and a probability is obtained for each category. The probability can be used as a feature value to classify different features.

[0123] In one embodiment, see Figure 6 Step 200 specifically includes:

[0124] Step 201: Determine the total information entropy corresponding to the original data set generated by all feature values;

[0125] Step 202: Determine the sub-information entropy of the data set corresponding to the feature value;

[0126] In steps 201 and 202, information entropy refers to uncertainty. The greater the information entropy, the greater the uncertainty of the corresponding eigenvalue. If the eigenvalue has n possible values: U1…U2…U3…U4…U5…U6…U7…U8…U9 ...i …U n The corresponding probabilities are: P1…P i …P n Furthermore, the occurrence of each eigenvalue is independent of the others. In this case, the average uncertainty of the eigenvalues ​​should be the uncertainty of a single sign - logP. i The statistical average (E) is called the information entropy, i.e.:

[0127]

[0128] In the above formula, the logarithm is generally taken to be base 2, and the unit is bits.

[0129] Step 203: Determine the information gain based on the total information entropy and the sub-information entropy.

[0130] Specifically, subtracting the sub-information entropy from the total information entropy yields the information gain of the corresponding feature value.

[0131] In one embodiment, see Figure 7 Face recognition-based identity verification methods also include:

[0132] Step 600: Generate the information entropy of the feature values. Further, see... Figure 8 Step 600 includes:

[0133] Step 601: Based on the number of times the feature value appears;

[0134] Step 602: Calculate the probability mass function of the feature value based on the number of times;

[0135] The probability of an eigenvalue is calculated based on the frequency of its occurrence, i.e., the probability mass function probability P(x). Specifically, it can be determined using the following formula, assuming X is a discrete random variable defined on a countable sample space S. Then its probability mass function fX(x) is:

[0136]

[0137] Step 603: Generate the information entropy of the feature value based on the probability mass function.

[0138] It is understood that the feature values ​​involved in this application are discrete random variables, so a probability mass function is used to generate information entropy.

[0139] In one embodiment, step 300 specifically includes:

[0140] Step 301: Generate a contrast vector based on the feature values ​​and their corresponding information gains;

[0141] Step 302: Generate a standard vector based on the real face image;

[0142] Step 303: Calculate the Euclidean distance between the comparison vector and the standard vector to determine the similarity.

[0143] In steps 301 to 303, for two-dimensional images, similarity calculation can employ algorithms such as cosine similarity and Euclidean distance. Specifically, firstly, the word2vec algorithm is used to convert the extracted two-dimensional features and their corresponding feature weights into vectors, and then the similarity between these vectors is calculated using the specified similarity algorithm.

[0144] Since 3D features contain spatial features, similarity can be calculated using Euclidean distance. First, the word2vec algorithm is used to convert the extracted 3D features and their corresponding weights into vectors. Then, the distance between these vectors is calculated using Euclidean distance; the smaller the distance, the more similar the two vectors are.

[0145] See Figure 9 The present invention also provides a specific implementation of an identity verification method based on facial recognition, which specifically includes the following:

[0146] S1: Data Acquisition.

[0147] Obtain financial product order requests from client terminals. These requests may include, but are not limited to, customer information such as customer ID, product code, purchase amount, and transaction account. Based on the customer ID in the order request, obtain basic customer information, including but not limited to: Basic information: name, gender, ID type, ID number, age, industry, occupation, contact information, marital status, asset and liability information, held products, risk tolerance, height, weight, health status, clothing style, appearance, investment ratio, investment experience, investment mentality, family income, social security status, and preferences (travel and entertainment, education and development, dining information, transportation and communication, medical aesthetics, interpersonal relationships, and other information). Relationship information: related persons, specific relationships, and key information about the related persons (including: name, gender, age, ID type, ID number, industry, occupation, and contact information). Holographic image: Obtain two-dimensional and depth images of the customer's historical transactions stored in the database. ID photo: Use the obtained ID number to apply to the public security system to obtain the customer's ID photo.

[0148] S2: Feature extraction.

[0149] For 2D images, a Convolutional Neural Network (CNN) is used to extract features from the 2D images acquired by the data acquisition module. The CNN consists of convolutional layers, pooling layers, and fully connected layers. Pixels are used as the length and width of the image, and color is used as different channels, becoming the input to the CNN. Each image uses a filter (convolutional kernel) to filter different small regions of the image, and features are extracted from each region in a sliding manner, resulting in feature maps of different parts. The feature maps output from the convolutional layers are used as input to the pooling layers to reduce the dimensionality of the feature maps and retain more important features. Finally, the fully connected layers integrate the features sampled after convolution, normalize them to obtain one-dimensional features, and obtain a probability for each category. This probability can be used as a feature value to classify different features.

[0150] For 3D models, an irregular 3D point cloud is transformed into a regularly distributed rasterized representation using a voxel-based method, and then a 3D convolutional neural network (3D-CNN) is used for feature extraction. 3D-CNN can better capture temporal and spatial features. Compared to 2D-CNN, each input channel of 3D-CNN undergoes convolution with a 3D convolution kernel, and dimensionality reduction is performed using 3D pooling layers. The convolution output is not compressed into a 2D image, preserving the spatial features of the image. Feature extraction using 3D-CNN yields more precise features, including: eyes (interpupillary distance, expression), nose (nose shape, columella, bridge, nostrils), ears (contour, earlobes), shoulders (shoulder width), and hands (fingers, palm width). The convolutional neural network is trained using 3D images to obtain feature values ​​for different body parts.

[0151] S3: Feature Weight Calculation

[0152] For 2D images, the ID3 decision tree algorithm is used. The ID3 algorithm selects the splitting attribute by calculating the information gain of each feature; that is, the attribute with the larger gain value is the optimal splitting attribute. Since the similarity of each feature in an image affects the final image similarity classification (similar or dissimilar), and the degree of influence varies for each feature, the ID3 algorithm can be used to calculate the information gain of different features based on this principle, which serves as the feature weight. The specific steps are as follows:

[0153] Calculate the information entropy of feature values: Each feature has a corresponding feature value, and the sample set consisting of the same feature is denoted as D. For example, the feature values ​​of the age feature include: those born in the 1990s, 1980s, and 1970s, which can be expressed by the formula a = {a1, a2, a3}, and this set is denoted as d. Calculate the information entropy of different feature values: Ent(d1), Ent(d2), Ent(d3). Calculate the feature information gain: Calculate the information gain of the feature based on the information entropy of the feature values ​​above. Information gain is the weight value of each feature.

[0154] Correspondingly, for the 3D model, the weight calculation uses the C4.5 machine learning decision tree algorithm. Because the number of feature categories varies significantly for each part of the extracted 3D features, the ID3 algorithm cannot handle continuous features. Therefore, the C4.5 algorithm is used to calculate the weights of the 3D features based on the ID3 algorithm. The specific steps are as follows: Referring to the ID3 algorithm processing procedure, the information gain Gain(G,a) of each feature is obtained. The ratio of each feature category to all features is then calculated. Based on the information gain and feature ratio, calculate the information gain ratio for each feature of the information: These are the weight values ​​of each feature.

[0155] S4: Similarity calculation.

[0156] For two-dimensional images, similarity calculation can employ algorithms such as cosine similarity and Euclidean distance. First, the word2vec algorithm is used to convert the extracted two-dimensional features and their corresponding weights into vectors. Then, the similarity between these vectors is calculated using a specified similarity algorithm. For example, the cosine similarity algorithm calculates cosine values ​​within the range [0,1]. The closer the cosine value is to 1, the closer the angle is to 0, indicating greater similarity between the two vectors. The calculated cosine value represents the similarity between the features.

[0157] Since 3D features contain spatial features, similarity can be calculated using Euclidean distance. First, the word2vec algorithm is used to convert the extracted 3D features and their corresponding weights into vectors. Then, the distance between these vectors is calculated using Euclidean distance; the smaller the distance, the more similar the two vectors are. The Euclidean distance calculation result needs to be normalized using a normalization function. The feature similarity is obtained by mapping the values ​​to a corresponding percentage range (0% to 100%). The calculation method is as follows: Calculate the maximum Euclidean distance and set a corresponding percentage (e.g., p1 = 95%), and set a corresponding percentage for the minimum Euclidean distance (e.g., p2 = 5%). Substitute the set variable values ​​X1 and X2 into the normalization formula to obtain the corresponding variable values ​​α and β.

[0158] S5: Identity verification.

[0159] Specifically, the system retrieves financial product order requests from client terminals and calculates the similarity (x1) between the client's 2D facial image and their ID photo. Next, it calculates the similarity (x2) between the user's 2D facial image and their historical facial data, and then calculates the similarity (x3) between the user's 3D model and their actual headshot data.

[0160] Calculate the total similarity. If the customer is an existing customer, use the 2D and depth images of the customer or those from past successful transactions as the training set to build a logistic regression model. The model training module calculates three similarities in this identification, namely x1, x2, and x3. Whether the customer is actually the same person is represented by Y, with values ​​of 0 and 1. The weights of the three similarities are trained. The total similarity Y for this transaction is calculated by combining the similarity obtained in this transaction with the customer's weight coefficient. The total similarity is compared with a pre-set threshold. If it is higher than or equal to the threshold, the identity verification passes; if it is lower, the verification fails.

[0161] Based on the same inventive concept, this application also provides a face recognition-based authentication device, which can be used to implement the method described in the above embodiments, as shown in the following embodiments. Since the principle of the face recognition-based authentication device in solving the problem is similar to that of the face recognition-based authentication method, the implementation of the face recognition-based authentication device can refer to the implementation of the face recognition-based authentication method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0162] This invention provides a specific implementation of a face recognition-based identity verification device capable of implementing a face recognition-based identity verification method. See [link to relevant documentation]. Figure 10 The facial recognition-based identity verification device specifically includes the following:

[0163] The feature extraction module 10 is used to extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0164] Information gain determination module 20 is used to determine the information gain of the feature value based on the information entropy of the feature value;

[0165] The similarity determination module 30 is used to determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain;

[0166] The identity verification module 40 is used to verify the user's identity based on the similarity.

[0167] In one embodiment, see Figure 11 The face recognition-based identity verification device also includes: a 3D model acquisition module 50, used to acquire the 3D model, see [link to documentation]. Figure 12 The 3D model acquisition module 50 includes:

[0168] The laser pattern acquisition unit 501 is used to acquire the user's laser pattern through an infrared camera;

[0169] The depth pixel value determination unit 502 is used to determine the user's depth pixel value based on the user's emission time, reception time and the laser pattern.

[0170] The depth image determination unit 503 is used to merge multiple depth pixel values ​​to determine the user's depth image;

[0171] The 3D model acquisition unit 504 is used to acquire the 3D model based on the depth image.

[0172] In one embodiment, see Figure 13 The three-dimensional model acquisition unit 504 includes:

[0173] The cloud data generation unit 5041 is used to convert the depth image into 3D point cloud data according to the intrinsic parameter formula of the infrared camera.

[0174] Outlier removal unit 5042 is used to remove outliers from the 3D point cloud data to generate downsampled 3D point cloud data;

[0175] The surface reconstruction unit 5043 is used to perform surface reconstruction on the 3D point cloud data to generate the three-dimensional model.

[0176] In one embodiment, see Figure 14 The feature value extraction module 10 includes:

[0177] The feature map generation unit 101 is used to generate feature maps of different regions of the user's face image based on the length, width, and color of the user's face image using the convolutional layer of the convolutional neural network.

[0178] Feature map dimensionality reduction unit 102 is used to reduce the dimensionality of the feature map using the pooling layer of the convolutional neural network;

[0179] The feature value generation unit 103 is used to generate the feature value using the fully connected layer of the convolutional neural network and the reduced feature map.

[0180] In one embodiment, see Figure 15 The information gain determination module 20 includes:

[0181] The total information entropy generation unit 201 is used to determine the total information entropy corresponding to the original data set generated by all feature values;

[0182] Sub-information entropy determination unit 202 is used to determine the sub-information entropy of the data set corresponding to the feature value;

[0183] The information gain determination unit 203 is used to determine the information gain based on the total information entropy and the sub-information entropy.

[0184] In one embodiment, see Figure 16 The face recognition-based identity verification device further includes: an information entropy generation module 60, used to generate the information entropy of the feature value, wherein the information entropy generation module 60 includes:

[0185] The frequency determination unit 601 is used to determine the frequency of occurrence of the feature value;

[0186] The function calculation unit 602 is used to calculate the probability mass function of the feature value based on the number of times.

[0187] The information entropy generation unit 603 is used to generate the information entropy of the feature value according to the probability mass function.

[0188] In one embodiment, see Figure 17 The similarity determination module 30 includes:

[0189] The comparison vector generation unit 301 is used to generate a comparison vector based on the feature value and its corresponding information gain.

[0190] The standard vector generation unit 302 is used to generate a standard vector based on the real face image;

[0191] The similarity determination unit 303 is used to calculate the Euclidean distance between the comparison vector and the standard vector to determine the similarity.

[0192] As described above, embodiments of the present invention provide an identity verification device based on facial recognition, comprising: firstly, extracting feature values ​​from a pre-acquired user facial image; wherein the facial image includes a two-dimensional image and a three-dimensional model; then, determining the information gain of the feature values ​​based on their information entropy; determining the similarity between the user's facial image and the user's real facial image based on the feature values ​​and the information gain; and finally verifying the user's identity based on the similarity. This invention utilizes the ability of holographic projection to collect human physical appearance data as an auxiliary standard for identity verification. By leveraging the ability of holographic projection to acquire full-body data, this invention expands the data collection scope for identity verification from a localized part of the human body to the entire body, further improving the comprehensiveness of the data. Combined with other supplementary authentication methods, this ultimately increases the authenticity and reliability of the identity verification technology.

[0193] The embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps of the face recognition-based identity verification method in the above embodiments, see [link to implementation details]. Figure 18 The electronic devices specifically include the following:

[0194] Processor 1201, memory 1202, communications interface 1203, and bus 1204;

[0195] The processor 1201, memory 1202, and communication interface 1203 communicate with each other via bus 1204; the communication interface 1203 is used to realize information transmission between server-side devices and client-side devices and other related devices.

[0196] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, it implements all the steps in the face recognition-based authentication method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0197] Step 100: Extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0198] Step 200: Determine the information gain of the feature value based on its information entropy;

[0199] Step 300: Determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain;

[0200] Step 400: Verify the user's identity based on the similarity score.

[0201] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the face recognition-based authentication method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the face recognition-based authentication method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0202] Step 100: Extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model;

[0203] Step 200: Determine the information gain of the feature value based on its information entropy;

[0204] Step 300: Determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain;

[0205] Step 400: Verify the user's identity based on the similarity score.

[0206] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0207] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0208] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed sequentially as shown in the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0209] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0210] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0211] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0212] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0213] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0214] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0215] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. A face recognition-based identity verification method, characterized in that, include: Extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model; The information gain of the feature value is determined based on the information entropy of the feature value; The similarity between the user's face image and the user's real face image is determined based on the feature value and the information gain. Verify the user's identity based on the similarity score; The authentication method further includes: generating the information entropy of the feature value; The information entropy used to generate the feature values ​​includes: Determine the number of times the feature value appears; Calculate the probability mass function of the feature value based on the number of occurrences; The information entropy of the feature value is generated based on the probability mass function.

2. The identity verification method based on face recognition as described in claim 1, characterized in that, The steps for obtaining the 3D model include: The user's laser pattern is captured using an infrared camera; The user's depth pixel value is determined based on the emission and reception times of light to the user and the laser pattern. Merge multiple depth pixel values ​​to determine the user's depth image; The 3D model is obtained from the depth image.

3. The identity verification method based on face recognition as described in claim 2, characterized in that, The step of obtaining the 3D model based on the depth image includes: The depth image is converted into 3D point cloud data according to the intrinsic parameter formula of the infrared camera; Remove outliers from the 3D point cloud data to generate downsampled 3D point cloud data; The 3D point cloud data is used to perform surface reconstruction to generate the three-dimensional model.

4. The identity verification method based on face recognition as described in claim 1, characterized in that, The extraction of feature values ​​from the user's facial image includes: Using the convolutional layers of a convolutional neural network, feature maps of different regions of the user's face image are generated based on the length, width, and color of the user's face image. The feature map is reduced in dimensionality using the pooling layer of the convolutional neural network; The feature values ​​are generated using the fully connected layers of the convolutional neural network and the reduced-dimensional feature map.

5. The identity verification method based on face recognition as described in claim 1, characterized in that, The step of determining the information gain of the feature value based on the information entropy of the feature value includes: Determine the total information entropy corresponding to the original data set generated by all feature values; Determine the sub-information entropy of the data set corresponding to the feature value; The information gain is determined based on the total information entropy and the sub-information entropy.

6. The identity verification method based on face recognition as described in claim 1, characterized in that, Determining the similarity between the user's face image and the user's real face image based on the feature value and the information gain includes: A comparison vector is generated based on the eigenvalues ​​and their corresponding information gains; Generate a standard vector based on the real face image; The Euclidean distance between the comparison vector and the standard vector is calculated to determine the similarity.

7. An identity verification device based on facial recognition, characterized in that, include: The feature extraction module is used to extract feature values ​​from the pre-acquired user face image; wherein the face image includes a two-dimensional image and a three-dimensional model; An information gain determination module is used to determine the information gain of the feature value based on the information entropy of the feature value. A similarity determination module is used to determine the similarity between the user's face image and the user's real face image based on the feature value and the information gain; An identity verification module is used to verify the user's identity based on the similarity. The authentication device further includes: an information entropy generation module, used to generate the information entropy of the feature value; The information entropy generation module includes: A frequency determination unit is used to determine the frequency of occurrence of the feature value; A function calculation unit is used to calculate the probability mass function of the feature value based on the number of times; An information entropy generation unit is used to generate the information entropy of the feature value based on the probability mass function.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the face recognition-based identity verification method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the face recognition-based authentication method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the face recognition-based authentication method according to any one of claims 1 to 6.

Images