Identity authentication method and device, model training method and device, equipment and storage medium

Abstract

The application provides an identity authentication method and device, a model training method and device, equipment and a storage medium, and relates to the field of artificial intelligence. The method comprises: receiving an identity authentication request comprising a face image to be identified; obtaining a plurality of fingerprint images for identity reference; extracting a face feature vector of the face image through a first image encoder; extracting a plurality of fingerprint feature vectors of the plurality of fingerprint images through a second image encoder; screening a target fingerprint image from the plurality of fingerprint images according to the feature similarity of the face feature vector and the plurality of fingerprint feature vectors; and determining the identity information corresponding to the face image as the identity information corresponding to the target fingerprint image. The training image group for training the first image encoder and the second image encoder comprises a fingerprint training image, a face positive sample image and a face negative sample image, and the face negative sample image comprises a similar image of the face positive sample image. Thus, the accuracy of identity authentication is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to an identity authentication method, a model training method, a device, an equipment, and a storage medium. Background Technology

[0002] Biometric identification technology has been widely used in fields such as identity verification, access control, and payment security.

[0003] Biometric identification technology is relatively mature. In the process of identity verification, training data can be used to train deep learning algorithms, and based on the trained deep learning algorithms, biometric features can be extracted and identities can be identified.

[0004] However, with technological advancements, biometric features may be replaced by synthetic ones. For example, real facial images may be replaced with synthetic ones, making it difficult to distinguish between real and fake biometric features and resulting in low accuracy in identity verification. Summary of the Invention

[0005] This application provides an identity authentication method, model training method, apparatus, device, and storage medium to solve the problem of low accuracy in identity authentication caused by the difficulty in distinguishing between real and fake facial images.

[0006] Firstly, this application provides a method for identity authentication, including:

[0007] Receive an identity verification request, wherein the identity verification request includes a facial image to be verified;

[0008] Obtain multiple fingerprint images for identity reference from the identity authentication reference library;

[0009] The facial image is processed by a first image encoder to extract features, resulting in a facial feature vector.

[0010] The second image encoder extracts features from the multiple fingerprint images to obtain multiple fingerprint feature vectors.

[0011] Determine the feature similarity between the facial feature vector and the plurality of fingerprint feature vectors;

[0012] Based on the feature similarity, a target fingerprint image is obtained by filtering from the plurality of fingerprint images;

[0013] The identity information corresponding to the facial image is determined to be the identity information corresponding to the target fingerprint image;

[0014] The first image encoder and the second image encoder are trained together based on multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face.

[0015] Secondly, this application provides a model training method, including:

[0016] Acquire training data, which includes multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face.

[0017] The parameters of the first image encoder and the second image encoder are adjusted using training data and a contrastive loss function to obtain the trained first image encoder and the trained second image encoder.

[0018] The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrast loss function is used to maximize the feature similarity between fingerprint images and facial images of the same person and minimize the feature similarity between fingerprint images and facial images of different people.

[0019] Thirdly, this application provides an identity authentication device, comprising:

[0020] A request receiving module is used to receive an identity authentication request, wherein the identity authentication request includes a facial image to be authenticated;

[0021] The fingerprint acquisition module is used to acquire multiple fingerprint images for identity reference from the identity authentication reference library;

[0022] A facial feature extraction module is used to extract features from the facial image using a first image encoder to obtain a facial feature vector;

[0023] The fingerprint feature extraction module is used to extract features from the multiple fingerprint images respectively through the second image encoder to obtain multiple fingerprint feature vectors;

[0024] A similarity determination module is used to determine the feature similarity between the facial feature vector and the plurality of fingerprint feature vectors;

[0025] A fingerprint image filtering module is used to filter a target fingerprint image from a plurality of fingerprint images based on the feature similarity.

[0026] An identity determination module is used to determine that the identity information corresponding to the facial image is the same as the identity information corresponding to the target fingerprint image;

[0027] The first image encoder and the second image encoder are trained together based on multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face.

[0028] Fourthly, this application provides a model training apparatus, comprising:

[0029] The training data acquisition module is used to acquire training data, which includes multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face.

[0030] The parameter adjustment module is used to adjust the parameters of the first image encoder and the second image encoder using training data and a contrastive loss function, so as to obtain the trained first image encoder and the trained second image encoder.

[0031] The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrast loss function is used to maximize the feature similarity between fingerprint images and facial images of the same person and minimize the feature similarity between fingerprint images and facial images of different people.

[0032] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the identity authentication method as described in the first aspect and / or the model training method as described in the second aspect.

[0033] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the identity authentication method described in the first aspect and / or the model training method described in the second aspect.

[0034] In a seventh aspect, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the identity authentication method provided in the first aspect and / or the model training method described in the second aspect.

[0035] The identity authentication method, model training method, apparatus, device, and storage medium provided in this application train a first image encoder and a second image encoder using multiple training image sets. The training image sets include fingerprint training images, positive facial sample images belonging to the same person as the fingerprint training images, and negative facial sample images belonging to different persons. This ensures that facial features and fingerprint features extracted by the first and second image encoders for the same person are similar, while facial features and fingerprint features extracted for different persons are dissimilar. The negative facial sample images include similar images to the positive facial sample images, enabling the first image encoder to distinguish similar facial images for different tasks. Thus, identity authentication can be performed using the first and second image encoders. Based on the feature similarity between the facial feature vector extracted by the first image encoding model and the fingerprint feature vector extracted by the second image encoding model, a target fingerprint image is selected from multiple fingerprint images, and the identity information corresponding to the facial image is determined as the identity information corresponding to the target fingerprint image. Therefore, by using fingerprints as a biometric feature to assist in determining the identity information corresponding to facial images, the accuracy of identity authentication is effectively improved. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0037] Figure 1 This diagram illustrates an application scenario to which the embodiments of this application are applicable.

[0038] Figure 2 A flowchart illustrating the identity authentication method provided in this application embodiment;

[0039] Figure 3 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0040] Figure 4 Example diagrams illustrating the model training and application process provided in the embodiments of this application;

[0041] Figure 5 This is a schematic diagram of the structure of the identity authentication device provided in the embodiments of this application;

[0042] Figure 6 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application;

[0043] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0044] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0046] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0047] It should be noted that the identity authentication method, model training method, device, equipment and storage medium provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application fields of the identity authentication method, model training method, device, equipment and storage medium provided in this application are not limited.

[0048] Using deep learning for person identification has the following drawbacks:

[0049] 1. High data collection cost: Identity authentication usually requires the collection of a large amount of labeled data to train deep learning machines. Professional personnel are needed to label the data one by one, which is costly.

[0050] 2. Poor robustness: With the development of video face-swapping technology, identity authentication technology has difficulty distinguishing whether a face image is a real face image or a synthesized face image, resulting in low accuracy of identity recognition.

[0051] To address the aforementioned problems, this application provides an identity authentication method, a model training method, an apparatus, a device, and a storage medium. In this application, considering the advantages of fingerprint features such as uniqueness, difficulty in forgery, and ease of storage, image information corresponding to fingerprint images is used to assist in determining the identity information corresponding to facial images. A feature vector of the fingerprint image is extracted using an image encoder for fingerprint feature extraction, and a feature vector of the facial image is extracted using an image encoder for facial feature extraction. Based on the feature similarity between the feature vectors of the fingerprint image and the facial image, fingerprint images belonging to the same person as the facial image are identified. Therefore, the identity information corresponding to the facial image can be determined based on the identity information corresponding to the fingerprint image. The image encoder can be trained using fingerprint training images, positive sample facial images belonging to the same person as the fingerprint training images, and negative sample facial images belonging to different persons. The negative sample facial images include similar images to the positive sample facial images, enabling the image encoder to distinguish similar facial images, improving the generalization ability of the image encoder, and enhancing the robustness and accuracy of identity authentication. The fingerprint training images, positive sample facial images, and negative sample facial images do not require annotation, reducing data acquisition costs.

[0052] For ease of distinction, the image encoder used to extract facial features will be referred to as the first image encoder, and the image encoder used to extract fingerprint features will be referred to as the second image encoder.

[0053] The specific application scenario for this application is identity recognition. Figure 1 These are example diagrams illustrating application scenarios applicable to the embodiments of this application. For example... Figure 1 As shown, the application scenario may involve an identity authentication device 101, an identity authentication reference library 102, and a model training device 103. The identity authentication device 101 can be a server or a terminal. Figure 1 Taking a server as an example, the model training device 103 can be a server. The first image encoder and the second image encoder can be trained on the model training device 103, and the trained first image encoder and the trained second image encoder can be deployed on the identity authentication device 101.

[0054] The identity authentication device 101 can obtain facial feature vectors by using a first image encoder to extract features from a face image from the identity authentication reference library 102, and fingerprint feature vectors by using a second image encoder to extract features from a fingerprint image. Based on the feature similarity between the face feature vector and the fingerprint feature vector and the identity information corresponding to the fingerprint image, the identity information corresponding to the face image is determined.

[0055] Optionally, the application scenario may also include an interactive device 104. The interactive device 104 and the identity authentication device 101 can communicate with each other. The interactive device 104 can be a terminal. Users can input facial images on the interactive device 104. The identity authentication device 101 can identify whether the facial images in the video belong to the same person and send the identification result to the interactive device 104. The identity authentication device 101 can also obtain facial images from a database.

[0056] The terminal can be a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities (such as a smartphone or tablet), a computing device (such as a personal computer (PC)), a wearable device (such as a smartwatch or smart bracelet), or a smart home device (such as a smart speaker or smart display device). The server can be a standalone server or a server cluster, and can be a local server or a cloud server.

[0057] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0058] Figure 2 This is a flowchart illustrating the identity authentication method provided in an embodiment of this application. Figure 2 As shown, the identity verification method includes the following steps:

[0059] S201, Receive an identity authentication request, which includes a facial image to be authenticated.

[0060] The identity verification request is used to request identity verification of a facial image. The identity verification request may include one or more facial images, which may belong to the same person or different people.

[0061] Optionally, the identity verification request may include video data, which may include a facial image of the person to be identified. This allows for the identification of individuals within the video data.

[0062] In this embodiment, an identity authentication request can be received from user input, or from other devices. Alternatively, an identity authentication request can be actively triggered upon receiving a facial image or video data containing a facial image, or an identity authentication task can be configured to actively trigger the identity authentication request when the identity authentication task begins execution.

[0063] S202, Obtain multiple fingerprint images for identity reference from the identity authentication reference library.

[0064] The identity authentication reference library may include multiple fingerprint images and the corresponding identity information for each fingerprint image. Different fingerprint images can correspond to different identity information.

[0065] In this embodiment, the identity information corresponding to the fingerprint image in the identity authentication reference library is the authenticated identity information. Multiple fingerprint images for identity reference are obtained from the identity authentication reference library. The identity information corresponding to these multiple fingerprint images can be used as reference identities for facial image identity recognition, thereby improving the reliability and accuracy of identity recognition.

[0066] S203, the first image encoder extracts features from the face image to obtain a face feature vector, and the second image encoder extracts features from multiple fingerprint images to obtain multiple fingerprint feature vectors.

[0067] The first and second image encoders are trained together based on multiple training image sets. Each training image set includes a fingerprint training image, positive facial sample images belonging to the same person as the fingerprint training image, and negative facial sample images belonging to different people. The negative facial sample images include similar images to the positive facial sample images. During training, the first image encoder extracts features from the positive and negative facial sample images separately, while the second image encoder extracts features from the fingerprint training image. The encoder compares and learns the facial feature vectors extracted from the positive and negative facial sample images by the first image encoder, and the fingerprint feature vectors extracted from the fingerprint images by the second image encoder. This ensures that the facial feature vectors and fingerprint feature vectors of the same person are close (i.e., similar), while the facial feature vectors and fingerprint feature vectors of different people are far apart (i.e., dissimilar). Since negative face images include similar images to positive face images, the first image encoder can distinguish similar face images during training and generate different face feature vectors for similar images. This enables it to distinguish similar face images during identity recognition, thereby improving the generalization ability and robustness of identity recognition.

[0068] In this embodiment, a facial image can be input into a first image encoder, or the facial image can be preprocessed and then input into the first image encoder, where facial features are extracted to obtain a facial feature vector. Similarly, a fingerprint image can be input into a second image encoder, or the fingerprint image can be preprocessed and then input into the second image encoder, where fingerprint features are extracted to obtain a fingerprint feature vector.

[0069] S204, determine the feature similarity between the facial feature vector and multiple fingerprint feature vectors.

[0070] In this embodiment, the facial feature vector is compared with multiple fingerprint feature vectors to obtain the feature similarity between the facial feature vector and the multiple fingerprint feature vectors. The higher the feature similarity between the facial feature vector and the fingerprint feature vector, the higher the probability that the facial image corresponding to the facial feature vector and the fingerprint image corresponding to the fingerprint feature vector belong to the same person; conversely, the lower the similarity, the lower the probability that the facial image corresponding to the facial feature vector and the fingerprint image corresponding to the fingerprint feature vector belong to the same person.

[0071] S205. Based on feature similarity, the target fingerprint image is obtained by filtering from multiple fingerprint images.

[0072] In this embodiment, fingerprint images that meet the filtering criteria in terms of feature similarity to facial feature vectors can be selected from multiple fingerprint images. For ease of distinction, the fingerprint feature vector is referred to as the target fingerprint image.

[0073] In one possible implementation, the fingerprint image with the highest feature similarity between its fingerprint feature vector and facial feature vector can be selected from multiple fingerprint images. The target fingerprint image is then determined as the one with the highest feature similarity between its fingerprint and facial feature vectors. Therefore, by selecting the fingerprint image with the highest similarity, the probability that the fingerprint image and the facial image belong to the same person is maximized, improving the accuracy of identity recognition.

[0074] In another possible implementation, a fingerprint image with the highest similarity to a facial feature vector can be selected from multiple fingerprint images. If the similarity between the fingerprint feature vector of this fingerprint image and the facial feature vector of the face image is greater than a similarity threshold, then the target fingerprint image is determined to be this fingerprint image. Thus, by combining the selection of the fingerprint image with a similarity threshold, the accuracy of identity recognition is improved.

[0075] S206, determine that the identity information corresponding to the face image is the identity information corresponding to the target fingerprint image.

[0076] In this embodiment, after selecting the target fingerprint image from multiple fingerprint images, it can be determined that the target fingerprint image and the facial image belong to the same person. The identity information corresponding to the target fingerprint image can then be obtained from the identity authentication reference library, and the identity information corresponding to the facial image can be identified as the identity information corresponding to the target fingerprint image. Thus, by using fingerprint images, identity recognition of facial images can be achieved, improving the accuracy of identity recognition.

[0077] In this embodiment, a first image encoder and a second image encoder are trained through comparative learning and multiple training image sets. Each training image set includes a fingerprint training image, a positive facial sample image belonging to the same person as the fingerprint training image, and a negative facial sample image belonging to a different person. This ensures that after feature extraction by the first and second image encoders, the fingerprint feature vectors and facial feature vectors belonging to the same person are closer in the feature space, while the fingerprint feature vectors and facial feature vectors belonging to different people are further apart in the feature space. This also improves the ability of the first image encoder to distinguish facial features of similar people. During the identity recognition process, the first image encoder extracts the facial feature vector corresponding to the facial image, and the second image encoder extracts the fingerprint feature vectors corresponding to multiple fingerprint images. Based on the feature similarity between the facial feature vector and the multiple fingerprint feature vectors, a target fingerprint image belonging to the same person as the facial image is selected from the multiple fingerprint images, and the identity information corresponding to the facial image is determined as the identity information corresponding to the target fingerprint image. Thus, on the one hand, by improving the ability of the first image encoder to distinguish facial features of similar people, the generalization ability and robustness of identity recognition are improved; on the other hand, by using fingerprint images that have been authenticated, the similarity or difference between different biometric features is compared through contrastive learning, thereby improving the reliability and accuracy of identity recognition.

[0078] In some embodiments, an authenticated facial image can also be used to identify the identity information corresponding to a fingerprint image. Specifically, an identity authentication request is received, which includes a fingerprint image to be identified; multiple facial images for identity reference are obtained from an identity authentication reference library; features are extracted from the multiple facial images using a first image encoder to obtain multiple facial feature vectors; features are extracted from the fingerprint image using a second image encoder to obtain a fingerprint feature vector; the feature similarity between the fingerprint feature vector and the multiple facial feature vectors is determined; based on the feature similarity, a target facial image is selected from the multiple facial images; and the identity information corresponding to the fingerprint image is determined to be the identity information corresponding to the target facial image. The specific process can be referred to the foregoing embodiments and will not be repeated here.

[0079] In some embodiments, the first image encoder is a convolutional neural network employing an attention mechanism. The first image encoder includes convolutional layers and pooling layers. Feature extraction from a facial image using the first image encoder to obtain facial features may include: resizing the facial image according to the input image size of the first image encoder; inputting the resized facial image into the first image encoder, where an attention mechanism is used in the convolutional layers of the first image encoder to extract features from the facial image, obtaining a first output feature; and inputting the first output feature into the pooling layers of the first image encoder, where pooling is performed on the first output feature to obtain a facial feature vector. Thus, by utilizing the attention mechanism and the convolutional neural network, the accuracy of facial feature extraction is improved, thereby enhancing the accuracy of identity recognition.

[0080] In this embodiment, the image size of the face image is converted to the input image size of the first image encoder. For example, if the input image size of the first image encoder is 224x224 pixels, the image size of the face image is converted to 224x224 pixels. The first image encoder may include multiple convolutional layers. In these convolutional layers, an attention mechanism is used to extract features from the face image to obtain a first output feature. To facilitate matching between the face feature vector and the fingerprint feature vector, the vector length of the face feature vector can be kept consistent with the vector length of the fingerprint feature vector. Therefore, the first output feature can be input into a pooling layer, where pooling processing is performed to obtain a face feature vector of fixed length. Thus, the normalization processing of the face feature vector is achieved through the pooling layer.

[0081] Optionally, the pooling layer is an average pooling layer. In the average pooling layer, the first output feature is average pooled to obtain a facial feature vector of fixed length, thereby improving the normalization effect of facial features through average pooling.

[0082] Optionally, the convolutional neural network employing an attention mechanism can be a vision transformer (ViT) encoder. The ViT encoder uses a self-attention mechanism to extract and encode features from the input image. Compared to traditional convolutional neural networks, the ViT encoder can better capture global information and contextual relationships within the image. Therefore, using a ViT encoder can improve the accuracy of facial feature extraction from facial images, extracting richer facial features, and thus contributing to improved accuracy in identity recognition.

[0083] In this optional approach, the step of using the ViT encoder to extract features from the face image may include: converting the image size of the face image to a size suitable for the ViT encoder, such as converting it to 224x224 pixels; and using the ViT encoder to extract features from the face image, converting the face image into a fixed-length feature vector, i.e., the face feature vector. Specifically, in the ViT encoder, the face image is passed forward to obtain the output features of the last encoding layer, and then average pooled to obtain a fixed-length feature vector, i.e., the face feature vector.

[0084] In some embodiments, the second image encoder is a deep residual network including multiple residual structures and pooling layers. The second image encoder extracts features from multiple fingerprint images to obtain multiple fingerprint feature vectors. This includes: resizing the multiple fingerprint images according to the input image size of the second image encoder; inputting the resized fingerprint images into the second image encoder, where multiple residual structures are used to extract features from the fingerprint images to obtain second output features; and inputting the resized second output features into the pooling layers of the second image encoder, where pooling is performed to obtain the corresponding fingerprint feature vectors. Thus, the residual network improves the accuracy of fingerprint feature extraction, thereby improving the accuracy of identity recognition.

[0085] In this embodiment, for each fingerprint image: the image size of the fingerprint image is converted to the input image size of the second image encoder. For example, if the input image size of the second image encoder is 224x224 pixels, the image size of the fingerprint image is converted to 224x224 pixels. In the second image encoder, features are extracted from the fingerprint image through multiple residual structures to obtain the second output features. To facilitate matching between the facial feature vector and the fingerprint feature vector, the vector length of the facial feature vector can be kept consistent with the vector length of the fingerprint feature vector. Therefore, the second output features can be input into a pooling layer, where pooling processing is performed to obtain a fingerprint feature vector of fixed length. Thus, the normalization processing of the fingerprint feature vector is achieved through the pooling layer.

[0086] Optionally, the pooling layer is an average pooling layer. In the average pooling layer, the second output feature is average pooled to obtain a fingerprint feature vector of fixed length, thereby improving the normalization effect of fingerprint features through average pooling.

[0087] Optionally, the deep residual network can be the RestNet50 network, which includes multiple stacked residual structures (also known as residual blocks) and pooling layers. Specifically, the network includes multiple convolutional layers and pooling layers, and uses residual connections to enhance the depth and performance of the network. In residual connections, the input features and the output features of the convolutional layers are summed to build a deeper network structure and improve the fingerprint feature extraction effect.

[0088] In this optional approach, feature extraction from the fingerprint image using the RestNet50 network may include the following steps: converting the fingerprint image to a size suitable for input to the RestNet50 network, for example, 224x224 pixels; inputting the converted fingerprint image into the RestNet50 network; and using the RestNet50 network to extract features from the fingerprint image, converting the fingerprint image into a fixed-length feature vector. Within the RestNet50 network, the fingerprint image can be forward-passed to obtain the output features of the last convolutional layer, which are then average-pooled to obtain a fixed-length feature vector.

[0089] In some embodiments, determining the feature similarity between a facial feature vector and multiple fingerprint feature vectors may include: determining the cosine distance between the facial feature vector and the multiple fingerprint feature vectors, wherein the cosine distance reflects the feature similarity, with a smaller cosine distance indicating higher feature similarity and a larger cosine distance indicating lower feature similarity. Therefore, using cosine distance to represent feature similarity improves the accuracy of feature similarity determination.

[0090] It should be noted that, in addition to cosine distance, other methods can be used to determine feature similarity, such as one-to-one matching of feature values ​​in feature vectors.

[0091] The training examples for the first and second image encoders are given below. It should be noted that the training process and the aforementioned identity authentication process can be executed on the same device or on different devices.

[0092] Figure 3 This is a schematic flowchart illustrating the model training method provided in an embodiment of this application. Figure 3 As shown, the model training method includes the following steps:

[0093] S301, acquire training data. The training data includes multiple training image groups. Each training image group includes a fingerprint training image, a positive face sample image belonging to the same person as the fingerprint training image, and a negative face sample image belonging to a different person than the fingerprint training image. The negative face sample images include similar images to the positive face sample images.

[0094] Each training image group may include a fingerprint training image, one or more positive face sample images, and one or more negative face sample images. In particular, it may include a face training image, a positive face sample image, and multiple negative face sample images, with the multiple negative face sample images including multiple similar images of the positive face sample image. Thus, the ability of the first image encoder to distinguish similar face images is improved by using multiple similar images.

[0095] In this embodiment, training data can be obtained from a database. The training data can be pre-collected and processed. During the processing, similar images to positive facial sample images can be identified among various images to select images that are more difficult to classify (with close similarity) into the negative sample queue, thereby improving the ability of the first image encoder to distinguish similar facial images.

[0096] S302, by using training data and a contrastive loss function, the parameters of the first image encoder and the second image encoder are adjusted to obtain the trained first image encoder and the trained second image encoder.

[0097] The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrastive loss function is used to maximize the feature similarity between the fingerprint image and the facial image of the same person and to minimize the feature similarity between the fingerprint image and the facial image of different people. The contrastive loss function can be the info noise contrastive estimation loss (InfoNCE loss).

[0098] Contrastive learning is an unsupervised learning method that learns how to compare the similarity between two input samples, thereby grouping feature vectors of samples of the same category together and separating feature vectors of samples of different categories. Contrastive learning eliminates the need for labeling training data, reducing the cost of collecting training data.

[0099] In this embodiment, a training image group used in the current training iteration can be selected from the training data. One or more training image groups can be used in a single training iteration. In each training iteration, the fingerprint training image, positive face sample image, and negative face sample image in the training image group used in the current training iteration can be extracted using the first image encoder and the second image encoder. Based on the extracted fingerprint feature vector and face feature vector, the loss value of the contrastive loss function is calculated. The first image encoder and the second image encoder are adjusted according to the loss value to achieve one training iteration of the first image encoder and the second image encoder. In this way, the first image encoder and the second image encoder can be trained multiple times until the loss value is less than or equal to the loss threshold or the number of training iterations exceeds the number of iterations threshold.

[0100] In this embodiment, the first and second image encoders are trained multiple times using a contrastive learning approach and multiple training image sets. This ensures that the facial and fingerprint feature vectors extracted by the first and second image encoders satisfy the following: the fingerprint and facial images of the same person have greater feature similarity, while the fingerprint and facial images of different people have less feature similarity. By adding similar positive images to the negative samples in the training image sets, the ability of the first image encoder to distinguish similar faces is improved. Thus, the accuracy and robustness of identity recognition using the first and second image encoders are enhanced.

[0101] In some embodiments, a training process for the first image encoder and the second image encoder includes: extracting features from positive face sample images using the first image encoder to obtain positive sample feature vectors; extracting features from negative face sample images using the first image encoder to obtain negative sample feature vectors; extracting features from fingerprint training images using the second image encoder to obtain fingerprint training feature vectors; determining a loss value based on a contrastive loss function, the number of negative face sample images, positive sample feature vectors, and negative sample feature vectors; and adjusting the parameters of the first image encoder and the second image encoder based on the loss value. Thus, by performing feature extraction using the first image encoder and the second image encoder respectively, and based on the contrastive loss function, the number of negative face sample images, positive sample feature vectors, and negative sample feature vectors, the accuracy of the loss value is improved, thereby improving the accuracy of adjusting the first image encoder and the second image encoder.

[0102] In this embodiment, for each training image group: positive face sample images from the training image group can be input into the first image encoder, where features are extracted from the positive face sample images to obtain positive sample feature vectors; negative face sample images from the training image group can be input into the first image encoder, where features are extracted from the negative face sample images to obtain negative sample feature vectors; fingerprint training images can be input into the second image encoder, where features are extracted from the fingerprint training images to obtain fingerprint training feature vectors; the fingerprint training feature vectors, positive sample feature vectors, negative sample feature vectors, and the number of negative sample images are input into the contrast loss function to calculate the loss value; and the parameters of the first and second image encoders are adjusted based on the loss value.

[0103] Optionally, the loss function can be expressed as:

[0104]

[0105] Where q represents the fingerprint training feature vector, and k + K represents the feature vector of positive samples. i Let represent the feature vector of the i-th negative sample, k represent the negative sample image, and τ be a hyperparameter.

[0106] In one possible implementation, there are multiple negative face images and multiple negative sample feature vectors. During the process of determining the loss value based on the contrastive loss function, the number of negative face images, and the positive and negative sample feature vectors, the feature similarity between each positive sample feature vector and each negative sample feature vector can be determined. The feature similarity between each positive sample feature vector and each negative sample feature vector, along with the number of negative face images (i.e., the number of negative sample feature vectors), are input into the contrastive loss function to calculate the loss value. Thus, incorporating feature similarity into the contrastive loss function improves the accuracy of the loss value calculation.

[0107] Optionally, the feature similarity between the positive sample feature vector and each negative sample feature vector may include the cosine distance between the positive sample feature vector and each negative sample feature vector, so as to represent the feature similarity by the cosine distance and improve the accuracy of feature similarity.

[0108] Optional, according to the cosine formula:

[0109]

[0110] q·k + Can be converted to:

[0111] q·k + =cos(a,b)·(|a|||b|)

[0112] The loss function can be expressed as:

[0113]

[0114] As can be seen, for each training image group, the first image encoder and the second image encoder can be used to extract the feature vectors of the fingerprint training image, the positive face sample image, and the negative face sample image, respectively. The cosine distance is used to calculate the feature similarity between the fingerprint training image and the positive face sample image, and the feature similarity between the fingerprint training image and the negative face sample image. By minimizing the loss value during the training process, the similarity between the facial features and fingerprint features of different people can be maximized, while the similarity between the facial features and fingerprint features of the same person can be minimized.

[0115] In some embodiments, images similar to positive face sample images in negative face sample images are obtained as follows: Image pairs and a set of face images are acquired. The image pairs include fingerprint training images and positive face sample images, and the set of face image images includes multiple negative sample candidate images. Features are extracted from the positive face sample images using a first image encoder to obtain positive sample feature vectors. Features are extracted from the negative sample candidate images using the first image encoder to obtain candidate feature vectors. The similarity between the positive sample feature vectors and the candidate feature vectors is determined. Based on the similarity between the positive sample feature vectors and the candidate feature vectors, negative face sample images are selected from the negative sample candidate images. Thus, by using the similarity between feature vectors to select similar images to positive face sample images, the accuracy of similar image selection is improved.

[0116] The similarity between positive sample feature vectors and candidate feature vectors can be determined by referring to the method for determining the similarity between fingerprint feature vectors and facial feature vectors in the aforementioned embodiments, and will not be repeated here.

[0117] The first image encoder and the second image encoder can be pre-trained. For example, the first image encoder can be pre-trained through other tasks (such as face recognition tasks or face detection tasks), and the second image encoder can be pre-trained through other tasks (such as fingerprint recognition tasks). The first image encoder and the second image encoder can also be models that have undergone one or more training processes as described above.

[0118] Optionally, after adjusting the model parameters of the first image encoder during training, the face negative sample images are updated using the adjusted first image encoder. Thus, with the adjustment of the first image encoder, more accurate face negative sample images can be obtained, improving the accuracy of face negative sample image selection, increasing the quality of negative samples, and also improving the training effect of both the first and second image encoders.

[0119] In this optional method, during the process of updating the negative face image through the adjusted first image encoder, features can be extracted from the positive face image through the adjusted first image encoder to obtain the positive sample feature vector; features can be extracted from the negative sample candidate image through the adjusted first image encoder to obtain the candidate feature vector; the similarity between the positive sample feature vector and the candidate feature vector is determined; and new negative face images are selected from the negative sample candidate images based on the similarity between the positive sample feature vector and the candidate feature vector.

[0120] like Figure 4 As shown, during training, a face image encoder (i.e., the first image encoder in the aforementioned embodiment) extracts features from the face training images (including positive and negative face sample images) to obtain face feature vectors. For example, n feature vectors of length 512 are obtained, including the feature vector corresponding to one positive face sample image and the feature vectors corresponding to n-1 negative face sample images. Similarly, a fingerprint encoder (i.e., the second image encoder in the aforementioned embodiment) extracts features from the fingerprint training images to obtain fingerprint feature vectors, for example, a single feature vector of length 512. The cosine distances between the feature vectors corresponding to the face training images and the feature vectors corresponding to the fingerprint training images are calculated. Loss values ​​are then calculated based on these cosine distances, and the parameters of the image encoder and fingerprint encoder are adjusted based on the loss values ​​to achieve model training.

[0121] like Figure 4 As shown, in the application process, a facial image to be identified is acquired, and features are extracted from the facial image using a facial image encoder to obtain the corresponding feature vector. A fingerprint image is then obtained from an identity authentication reference library, and features are extracted from the fingerprint image using a fingerprint encoder to obtain the corresponding feature vector. The feature vectors corresponding to the fingerprint image and the facial image are compared to obtain the cosine distance between them. Based on the cosine distance, fingerprint images belonging to the same person as the facial image are identified, and thus the identity of the person corresponding to the facial image can be determined based on the identity of the person corresponding to the fingerprint image.

[0122] Figure 5 This is a schematic diagram of the structure of the identity authentication device provided in an embodiment of this application. Figure 5As shown, the identity authentication device 500 provided in this embodiment includes: a request receiving module 501, used to receive an identity authentication request, the identity authentication request including a facial image to be authenticated; a fingerprint acquisition module 502, used to acquire multiple fingerprint images for identity reference from an identity authentication reference library; a facial feature extraction module 503, used to extract features from the facial image using a first image encoder to obtain a facial feature vector; a fingerprint feature extraction module 504, used to extract features from the multiple fingerprint images using a second image encoder to obtain multiple fingerprint feature vectors; and a similarity determination module 505, used to determine the facial feature vector. The fingerprint image filtering module 506 is used to filter the target fingerprint image from multiple fingerprint images based on the feature similarity; the identity determination module 507 is used to determine the identity information corresponding to the face image as the identity information corresponding to the target fingerprint image; wherein, the first image encoder and the second image encoder are trained together based on multiple training image groups, and each training image group includes a fingerprint training image, a positive face sample image belonging to the same person as the fingerprint training image, and a negative face sample image belonging to a different person than the fingerprint training image, and the negative face sample image includes similar images to the positive face sample image.

[0123] In some embodiments, the first image encoder is a convolutional neural network employing an attention mechanism. The first image encoder includes convolutional layers and pooling layers. The face feature extraction module 503 is specifically used to: adjust the size of the face image according to the input image size of the first image encoder; input the adjusted face image into the first image encoder, and use the attention mechanism to extract features from the face image in the convolutional layer of the first image encoder to obtain a first output feature; input the first output feature into the pooling layer of the first image encoder, and perform pooling processing on the first output feature in the pooling layer to obtain a face feature vector.

[0124] In some embodiments, the second image encoder is a deep residual network including multiple residual structures and pooling layers. The fingerprint feature extraction module 504 is specifically used to: adjust the size of multiple fingerprint images according to the input image size of the second image encoder; input the adjusted fingerprint images into the second image encoder, and extract features from the fingerprint images using multiple residual structures in the second image encoder to obtain second output features; input the adjusted fingerprint images into the pooling layer of the second image encoder, and pool the second output features in the pooling layer to obtain the corresponding fingerprint feature vector.

[0125] In some embodiments, images similar to positive face sample images in negative face sample images are obtained as follows: image pairs and face image sets are acquired, the image pairs including fingerprint training images and positive face sample images, and the face image sets including multiple negative sample candidate images; features are extracted from the positive face sample images using a first image encoder to obtain positive sample feature vectors; features are extracted from the negative sample candidate images using the first image encoder to obtain candidate feature vectors; the similarity between the positive sample feature vectors and the candidate feature vectors is determined; and negative face sample images are selected from the negative sample candidate images based on the similarity between the positive sample feature vectors and the candidate feature vectors.

[0126] In some embodiments, after the model parameters of the first image encoder are adjusted during the training process of the first image encoder, the negative sample image of the face is updated by the adjusted first image encoder.

[0127] Figure 6 This is a schematic diagram of the structure of the model training device provided in an embodiment of this application. Figure 6 As shown, the model training device 600 provided in this embodiment includes:

[0128] The training data acquisition module 601 is used to acquire training data, which includes multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face.

[0129] The parameter adjustment module 602 is used to adjust the parameters of the first image encoder and the second image encoder using training data and a contrastive loss function, so as to obtain the trained first image encoder and the trained second image encoder.

[0130] The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrastive loss function is used to maximize the feature similarity between fingerprint images and facial images of the same person and to minimize the feature similarity between fingerprint images and facial images of different people.

[0131] In some embodiments, during a training process of the first image encoder and the second image encoder, the parameter adjustment module 602 is specifically used to: extract features from positive face sample images using the first image encoder to obtain positive sample feature vectors; extract features from negative face sample images using the first image encoder to obtain negative sample feature vectors; extract features from fingerprint training images using the second image encoder to obtain fingerprint training feature vectors; determine a loss value based on the contrast loss function, the number of negative face sample images, the positive sample feature vectors, and the negative sample feature vectors; and adjust the parameters of the first image encoder and the second image encoder based on the loss value.

[0132] In some embodiments, images similar to positive face sample images in negative face sample images are obtained as follows: image pairs and face image sets are acquired, the image pairs including fingerprint training images and positive face sample images, and the face image sets including multiple negative sample candidate images; features are extracted from the positive face sample images using a first image encoder to obtain positive sample feature vectors; features are extracted from the negative sample candidate images using the first image encoder to obtain candidate feature vectors; the similarity between the positive sample feature vectors and the candidate feature vectors is determined; and negative face sample images are selected from the negative sample candidate images based on the similarity between the positive sample feature vectors and the candidate feature vectors.

[0133] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 may include at least one processor 701 and a memory 702. Figure 7 The example shown is an electronic device using a processor.

[0134] The memory 702 is used to store the program of the processor 701. Specifically, the program may include program code, which includes computer operation instructions.

[0135] The memory 702 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0136] The processor 701 is configured to execute computer programs stored in the memory 702 to implement the steps of the identity authentication methods in the above method embodiments.

[0137] The processor 701 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0138] Optionally, the memory 702 can be either independent or integrated with the processor 701. When the memory 702 is a device independent of the processor 701, the electronic device 700 may further include a bus 703 for connecting the processor 701 and the memory 702. The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. Buses can be classified as address buses, data buses, control buses, etc., but this does not mean that there is only one bus or one type of bus.

[0139] Optionally, in a specific implementation, if the memory 702 and the processor 701 are integrated on a single chip, then the memory 702 and the processor 701 can communicate through an internal interface.

[0140] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores computer-executable instructions. When at least one processor of the electronic device executes the computer-executable instructions, the electronic device executes the various steps of the identity authentication methods provided in the above-described embodiments.

[0141] This application also provides a computer program product comprising a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the processor executes the computer program to cause the electronic device to perform the various steps of the authentication methods provided in the above-described embodiments.

[0142] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0143] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for identity verification, characterized in that, include: Receive an identity verification request, wherein the identity verification request includes a facial image to be verified; Obtain multiple fingerprint images for identity reference from the identity authentication reference library; The facial image is processed by a first image encoder to extract features, resulting in a facial feature vector. The second image encoder extracts features from the multiple fingerprint images to obtain multiple fingerprint feature vectors; the first image encoder is a convolutional neural network with an attention mechanism, and the second image encoder is a deep residual network including multiple residual structures and pooling layers. Determine the feature similarity between the facial feature vector and the plurality of fingerprint feature vectors; Based on the feature similarity, a target fingerprint image is obtained by filtering from the plurality of fingerprint images; The identity information corresponding to the facial image is determined to be the identity information corresponding to the target fingerprint image; The first image encoder and the second image encoder are trained together based on multiple training image groups and a contrastive loss function. The contrastive loss function is used to maximize the feature similarity between fingerprint images and face images of the same person and minimize the feature similarity between fingerprint images and face images of different people. Each training image group includes a fingerprint training image, a positive face sample image of the same person as the fingerprint training image, and a negative face sample image of a different person as the fingerprint training image. The negative face sample image includes similar images to the positive face sample image. During the training process of the first image encoder, the model parameters of the first image encoder are adjusted, and the face negative sample image is updated by the adjusted first image encoder. The images similar to the positive face sample images in the negative face sample images are obtained in the following way: Obtain image pairs and a set of face images, wherein the image pairs include the fingerprint training images and the positive face sample images, and the set of face images includes multiple negative sample candidate images; The first image encoder extracts features from the positive sample image of the face to obtain a positive sample feature vector; The negative sample candidate image is subjected to feature extraction by the first image encoder to obtain a candidate feature vector; Determine the similarity between the positive sample feature vector and the candidate feature vector; The face negative sample image is obtained by filtering from the negative sample candidate images based on the similarity between the positive sample feature vector and the candidate feature vector.

2. The identity authentication method according to claim 1, characterized in that, The first image encoder includes convolutional layers and pooling layers. The step of extracting features from the face image using the first image encoder to obtain a face feature vector includes: The face image is resized according to the input image size of the first image encoder; The adjusted facial image is input into the first image encoder, and the facial image is used to extract features in the convolutional layer of the first image encoder using an attention mechanism to obtain the first output feature. The first output feature is input into the pooling layer of the first image encoder, and the first output feature is pooled in the pooling layer to obtain the facial feature vector.

3. The identity authentication method according to claim 1, characterized in that, The step involves extracting features from the multiple fingerprint images using a second image encoder to obtain multiple fingerprint feature vectors, including: The sizes of the plurality of fingerprint images are adjusted according to the input image size of the second image encoder; For the adjusted fingerprint images, the fingerprint images are input into the second image encoder. The fingerprint images are then used to extract features from the fingerprint images using the multiple residual structures in the second image encoder to obtain the second output features. For the adjusted fingerprint images, the second output feature is input into the pooling layer of the second image encoder, and the second output feature is pooled in the pooling layer to obtain the corresponding fingerprint feature vector.

4. A model training method, characterized in that, include: Acquire training data, which includes multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face. The parameters of the first image encoder and the second image encoder are adjusted using training data and a contrastive loss function to obtain the trained first image encoder and the trained second image encoder. The first image encoder is a convolutional neural network with an attention mechanism, and the second image encoder is a deep residual network including multiple residual structures and pooling layers. The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrast loss function is used to maximize the feature similarity between fingerprint images and facial images of the same person and minimize the feature similarity between fingerprint images and facial images of different people. During the training process of the first image encoder, the model parameters of the first image encoder are adjusted, and the face negative sample image is updated by the adjusted first image encoder. The images similar to the positive face sample images in the negative face sample images are obtained in the following way: Obtain image pairs and a set of face images, wherein the image pairs include the fingerprint training images and the positive face sample images, and the set of face images includes multiple negative sample candidate images; The first image encoder extracts features from the positive sample image of the face to obtain a positive sample feature vector; The negative sample candidate image is subjected to feature extraction by the first image encoder to obtain a candidate feature vector; Determine the similarity between the positive sample feature vector and the candidate feature vector; The face negative sample image is obtained by filtering from the negative sample candidate images based on the similarity between the positive sample feature vector and the candidate feature vector.

5. The model training method according to claim 4, characterized in that, One training process for the first image encoder and the second image encoder includes: The first image encoder extracts features from the positive sample image of the face to obtain a positive sample feature vector; The negative sample image of the face is processed by the first image encoder to extract features and obtain a negative sample feature vector. The fingerprint training image is used to extract features by the second image encoder to obtain the fingerprint training feature vector; The loss value is determined based on the contrast loss function, the number of negative face images, the positive sample feature vector, and the negative sample feature vector; The parameters of the first image encoder and the second image encoder are adjusted based on the loss value.

6. An identity verification device, characterized in that, include: A request receiving module is used to receive an identity authentication request, wherein the identity authentication request includes a facial image to be authenticated; The fingerprint acquisition module is used to acquire multiple fingerprint images for identity reference from the identity authentication reference library; A facial feature extraction module is used to extract features from the facial image using a first image encoder to obtain a facial feature vector; The fingerprint feature extraction module is used to extract features from the multiple fingerprint images respectively through the second image encoder to obtain multiple fingerprint feature vectors; the first image encoder is a convolutional neural network with an attention mechanism, and the second image encoder is a deep residual network including multiple residual structures and pooling layers; A similarity determination module is used to determine the feature similarity between the facial feature vector and the plurality of fingerprint feature vectors; A fingerprint image filtering module is used to filter a target fingerprint image from a plurality of fingerprint images based on the feature similarity. An identity determination module is used to determine that the identity information corresponding to the facial image is the same as the identity information corresponding to the target fingerprint image; The first image encoder and the second image encoder are trained together based on multiple training image groups and a contrastive loss function. The contrastive loss function is used to maximize the feature similarity between fingerprint images and face images of the same person and minimize the feature similarity between fingerprint images and face images of different people. Each training image group includes a fingerprint training image, a positive face sample image of the same person as the fingerprint training image, and a negative face sample image of a different person as the fingerprint training image. The negative face sample image includes similar images to the positive face sample image. During the training process of the first image encoder, the model parameters of the first image encoder are adjusted, and the face negative sample image is updated by the adjusted first image encoder. The images similar to the positive face sample images in the negative face sample images are obtained in the following way: Obtain image pairs and a set of face images, wherein the image pairs include the fingerprint training images and the positive face sample images, and the set of face images includes multiple negative sample candidate images; The first image encoder extracts features from the positive sample image of the face to obtain a positive sample feature vector; The negative sample candidate image is subjected to feature extraction by the first image encoder to obtain a candidate feature vector; Determine the similarity between the positive sample feature vector and the candidate feature vector; The face negative sample image is obtained by filtering from the negative sample candidate images based on the similarity between the positive sample feature vector and the candidate feature vector.

7. A model training device, characterized in that, include: The training data acquisition module is used to acquire training data, which includes multiple training image groups. Each training image group includes a fingerprint training image, a positive sample image of a face belonging to the same person as the fingerprint training image, and a negative sample image of a face belonging to a different person than the fingerprint training image. The negative sample image of the face includes similar images to the positive sample image of the face. The parameter adjustment module is used to adjust the parameters of the first image encoder and the second image encoder using training data and a contrastive loss function to obtain the trained first image encoder and the trained second image encoder; the first image encoder is a convolutional neural network using an attention mechanism, and the second image encoder is a deep residual network including multiple residual structures and pooling layers. The first image encoder is used to extract facial features, the second image encoder is used to extract fingerprint features, and the contrast loss function is used to maximize the feature similarity between fingerprint images and facial images of the same person and minimize the feature similarity between fingerprint images and facial images of different people. During the training process of the first image encoder, the model parameters of the first image encoder are adjusted, and the face negative sample image is updated by the adjusted first image encoder. The images similar to the positive face sample images in the negative face sample images are obtained in the following way: Obtain image pairs and a set of face images, wherein the image pairs include the fingerprint training images and the positive face sample images, and the set of face images includes multiple negative sample candidate images; The first image encoder extracts features from the positive sample image of the face to obtain a positive sample feature vector; The negative sample candidate image is subjected to feature extraction by the first image encoder to obtain a candidate feature vector; Determine the similarity between the positive sample feature vector and the candidate feature vector; The face negative sample image is obtained by filtering from the negative sample candidate images based on the similarity between the positive sample feature vector and the candidate feature vector.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the identity authentication method as described in any one of claims 1 to 3 or the model training method as described in any one of claims 4 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the identity authentication method as described in any one of claims 1 to 3 or the model training method as described in any one of claims 4 to 5.

Images