Method and system for identifying digital identity of accessor based on multiple models

Through multi-model recognition technology based on artificial intelligence, fine-grained deconstruction of fingerprint and face features and bidirectional global attention joint perception, combined with multiple identity recognition models, the accuracy of traditional single biometric recognition methods in complex environments is solved. The problem of insufficient has improved the intelligence and reliability of digital identity verification.

CN119068536AActive Publication Date: 2024-12-03NAT CERTIFICATION TECH (HANGZHOU) CO LTD

Patent Information

Application Number
CN202411559168.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-12-03
Estimated Expiration
2044-11-04

AI Technical Summary

Technical Problem

The traditional single biometric recognition method has the problem of insufficient identification accuracy in complex environments and differences between different individuals, especially when facing potential fraudulent behaviors, factors such as light, angle, and occlusion have a great impact, resulting in the security of identity verification. Sex and accuracy are difficult to guarantee.

Method used

Using image processing and analysis algorithms based on artificial intelligence and machine learning, fingerprint images and face images are analyzed, and through fine-grained deconstructed two-way whole-domain attention joint perception, combined with different identity recognizer models, fingerprints, faces and people are analyzed. The face-fingerprint joint feature is processed to generate the corresponding identity recognition probability distribution, and the legality of the accessor is determined through the fusion mechanism.

Benefits of technology

It improves the intelligence level and reliability of accessor digital identity verification. Through multi-model and feature interaction perception technology, the accuracy and security of identity recognition are enhanced and adapted to diverse application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068536A_ABST
    Figure CN119068536A_ABST
Patent Text Reader

Abstract

The invention discloses a method and system for identifying digital identity of an access person based on multiple models, and the method comprises the steps: carrying out the analysis of a fingerprint image and a face image through employing an image processing and analysis algorithm based on artificial intelligence and machine learning; fine-grained deconstruction-based bidirectional universe attention joint perception is carried out on the fingerprint features and the face features, and furthermore, different identity recognizer models are used for respectively processing the different features. The method comprises the following steps: acquiring a fingerprint, a face and a face-fingerprint combined fine-grained perceptual semantics to generate identity recognition class probability distributions corresponding to the fingerprint, the face and the face-fingerprint combined fine-grained perceptual semantics, and fusing the identity recognition class probability distributions by adopting a fusion mechanism to obtain a recognition result of the digital identity of an access person so as to determine whether the access person is a legal user or not. In this way, the digital identity of an access person can be recognized and verified by using various identity recognition models and a feature interaction perception technology based on artificial intelligence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent identification, and more specifically, to a method and system for identifying a digital identity of an access user based on multiple models. Background Art

[0002] With the rapid development of information technology and the advent of a digital society, identity authentication has become increasingly important in various fields. Traditional identity authentication methods, such as passwords and ID cards, face problems such as insufficient security, easy forgery, and poor user experience. Therefore, biometric recognition technology has gradually become an effective alternative, especially in the fields of finance, security, social media, etc. Biometric recognition uses an individual's unique physiological or behavioral characteristics for identity authentication. Common features include fingerprints, faces, irises, etc. These features are highly unique and stable, and can effectively improve the security and accuracy of identity authentication.

[0003] Although biometric recognition technology provides significant advantages in many aspects, in actual applications, a single biometric recognition method still has certain limitations when facing complex environments, individual differences, and potential fraud. For example, fingerprint recognition may be affected by factors such as the moisture, cleanliness, or presence of wounds on the finger. Face recognition may be affected by light, angle, obstructions (such as glasses, masks), makeup, or age changes. In addition, the biometrics of different individuals may have different manifestations. For example, the fingerprints of some people may not be clear enough, or the iris features of some people's eyes may not be obvious enough. These factors will affect the accuracy of the final digital identity recognition of the access user.

[0004] Therefore, a solution for identifying the digital identity of an access user based on multiple models is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for identifying the digital identity of the accessor based on multiple models, which analyzes the fingerprint image and the face image by using an image processing and analysis algorithm based on artificial intelligence and machine learning, and performs two-way global attention joint perception based on fine-grained deconstruction on these fingerprint features and face features. Furthermore, different identity identifier models are used to process these different features separately to generate identity recognition class probability distributions corresponding to fingerprints, faces, and face-fingerprint joint fine-grained perception semantics, and then a fusion mechanism is used to fuse these identity recognition class probability distributions to obtain the recognition result of the accessor's digital identity to determine whether the accessor is a legitimate user. In this way, multiple identity recognition models and feature interaction perception technology based on artificial intelligence can be used to identify and verify the digital identity of the accessor, thereby improving the intelligence level and reliability of the accessor's digital identity authentication.

[0006] According to one aspect of the present application, a method for identifying an accessor's digital identity based on multiple models is provided, which includes: obtaining a fingerprint image of the accessor collected by a fingerprint module; obtaining a facial image of the accessor collected by a camera; performing feature analysis on the fingerprint image of the accessor and the facial image of the accessor to obtain accessor fingerprint features and accessor facial features; obtaining accessor face-fingerprint joint fine-grained interactive perception features based on fine-grained global interaction analysis between the accessor fingerprint features and the accessor facial features; performing identity recognition based on the accessor fingerprint features to determine a first identity recognition class probability distribution; performing identity recognition based on the accessor facial features to determine a second identity recognition class probability distribution; performing identity recognition based on the accessor face-fingerprint joint fine-grained interactive perception features to determine a third identity recognition class probability distribution; and determining whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution, and the third identity recognition class probability distribution.

[0007] According to another aspect of the present application, a system for identifying a digital identity of an accessor based on multiple models is provided, comprising: a fingerprint image acquisition module, used to acquire a fingerprint image of the accessor acquired by a fingerprint module; a face image acquisition module, used to acquire a face image of the accessor acquired by a camera; an image feature analysis module, used to perform feature analysis on the fingerprint image of the accessor and the face image of the accessor to obtain a fingerprint feature of the accessor and a face feature of the accessor; a fine-grained global interaction analysis module, used to obtain a joint fine-grained interaction perception feature of the accessor's face-fingerprint based on a fine-grained global interaction analysis between the fingerprint feature of the accessor and the face feature of the accessor; an identity recognition module, used to perform identity recognition based on the fingerprint feature of the accessor to determine a first identity recognition class probability distribution; perform identity recognition based on the face feature of the accessor to determine a second identity recognition class probability distribution; perform identity recognition based on the joint fine-grained interaction perception feature of the face-fingerprint of the accessor to determine a third identity recognition class probability distribution; and an identity authentication module, used to determine whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution.

[0008] Compared with the prior art, the present application provides a method and system for identifying the digital identity of the access user based on multiple models. The method and system analyze the fingerprint image and the face image by using an image processing and analysis algorithm based on artificial intelligence and machine learning, and perform two-way global attention joint perception based on fine-grained deconstruction on these fingerprint features and face features. Furthermore, different identity identifier models are used to process these different features separately to generate identity recognition class probability distributions corresponding to fingerprints, faces, and face-fingerprint joint fine-grained perception semantics, and then a fusion mechanism is used to fuse these identity recognition class probability distributions to obtain the recognition result of the access user's digital identity to determine whether the access user is a legitimate user. In this way, multiple identity recognition models and feature interaction perception technology based on artificial intelligence can be used to identify and verify the digital identity of the access user, thereby improving the intelligence level and reliability of the access user's digital identity authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 The present invention is a flowchart of a method for identifying a digital identity of an access user based on multiple models according to an embodiment of the present application.

[0011] Figure 2 A data flow diagram of a method for identifying a digital identity of an access user based on multiple models according to an embodiment of the present application.

[0012] Figure 3 This is a flowchart of sub-step S4 of the method for identifying the digital identity of an access user based on multiple models according to an embodiment of the present application.

[0013] Figure 4 The present invention is a block diagram of a system for identifying a digital identity of an access user based on multiple models according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0015] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0016] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0017] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0018] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0019] In the technical solution of the present application, a method for identifying the digital identity of an access user based on multiple models is proposed. Figure 1 The present invention is a flowchart of a method for identifying a digital identity of an access user based on multiple models according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the method for identifying the digital identity of the access user based on multiple models according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, a method for identifying a digital identity of an accessor based on multiple models includes the following steps: S1, obtaining a fingerprint image of the accessor collected by a fingerprint module; S2, obtaining a facial image of the accessor collected by a camera; S3, performing feature analysis on the fingerprint image of the accessor and the facial image of the accessor to obtain a fingerprint feature of the accessor and a facial feature of the accessor; S4, obtaining a joint fine-grained interactive perception feature of the accessor's face-fingerprint based on a fine-grained global interaction analysis between the fingerprint feature of the accessor and the facial feature of the accessor; S5, performing identity recognition based on the fingerprint feature of the accessor to determine a probability distribution of a first identity recognition class; performing identity recognition based on the facial feature of the accessor to determine a probability distribution of a second identity recognition class; performing identity recognition based on the joint fine-grained interactive perception feature of the face-fingerprint of the accessor to determine a probability distribution of a third identity recognition class; S6, determining whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution.

[0020] In particular, S1 and S2 obtain the fingerprint image of the accessor collected by the fingerprint module; and obtain the facial image of the accessor collected by the camera. It should be understood that fingerprints are highly personalized biometric features with high personal specificity; and each person's facial features are unique. These unique features make facial recognition highly personal specific and can effectively distinguish different individuals. By further extracting and classifying the features of the accessor's fingerprint image and facial image, the digital identity of the accessor can be identified and verified.

[0021] In particular, S3 performs feature analysis on the accessor's fingerprint image and the accessor's face image to obtain the accessor's fingerprint feature and the accessor's face feature. In a specific example of the present application, the accessor's fingerprint image is input into a fingerprint feature extractor based on a first convolutional neural network model to obtain an accessor's fingerprint feature map as the accessor's fingerprint feature; and the accessor's face image is input into a face feature extractor based on a second convolutional neural network model to obtain an accessor's face feature map as the accessor's face feature. Here, the accessor's fingerprint image is input into a fingerprint feature extractor based on a first convolutional neural network model for feature mining to extract implicit feature distribution information about the accessor's fingerprint in the accessor's fingerprint image, thereby obtaining an accessor's fingerprint feature map. At the same time, the accessor's face image is input into a face feature extractor based on a second convolutional neural network model for feature mining to extract implicit semantic feature information about the accessor's face in the accessor's face image, thereby obtaining an accessor's face feature map.

[0022] It is worth noting that convolutional neural network is a deep learning model that is particularly suitable for processing data with grid structures, such as images and videos. CNN is widely used in the field of computer vision for tasks such as image classification, object detection, and image segmentation. The core idea of ​​CNN is to extract features of input data through convolutional layers and pooling layers, and perform classification or regression through fully connected layers. The following are the main components of CNN: Convolutional layer: The convolutional layer is the core component of CNN, which extracts features from input data through convolution operations. The convolution operation uses a set of learnable filters (also called convolution kernels) to convolve the input data to obtain a feature map. Each filter can capture different features of the input data, such as edges, textures, etc. By stacking multiple filters, higher-level features can be extracted; Activation function: Applying activation functions on the output of the convolutional layer introduces nonlinear transformations and increases the expressive power of the network. Common activation functions include ReLU, Sigmoid, and Tanh; Pooling layer: The pooling layer is used to reduce the spatial dimension of the feature map, reduce the number of parameters, and extract more robust features. Commonly used pooling operations include maximum pooling and average pooling, which downsample by taking the maximum or average value of the features in the window; Fully connected layer: The fully connected layer flattens the feature map output by the pooling layer into a one-dimensional vector, and performs classification or regression through a series of fully connected neurons. Fully connected layers usually appear in the last few layers of the network; Dropout layer: The Dropout layer is a regularization technique used to reduce overfitting. During the training process, the Dropout layer randomly sets the output of some neurons to zero, thereby forcing the network to learn more robust features; Loss function: The loss function is used to measure the difference between the model prediction result and the true label. Commonly used loss functions include cross entropy loss and mean square error; Optimization algorithm: The optimization algorithm is used to update the parameters of the neural network so that it gradually optimizes the model performance. Commonly used optimization algorithms include stochastic gradient descent, Adam, and RMSprop.

[0023] In particular, the S4 obtains the accessor face-fingerprint joint fine-grained interactive perception features based on the fine-grained global interactive analysis between the accessor's fingerprint features and the accessor's facial features. It should be understood that since the accessor's fingerprint feature map and the accessor's facial feature map respectively contain information about the accessor's fingerprint features and facial features, both features are about the accessor's biometric features and have an implicit association relationship. This implicit interactive association information is crucial for subsequent more accurate identity identification and ensuring the security of verification. Therefore, in addition to using the accessor's fingerprint features and facial features to perform digital identity identification and verification respectively, it is also necessary to explore and analyze the fine-grained intrinsic connection and interactive semantics between fingerprint features and facial features to improve the accuracy and security of identity identification. Based on this, in the technical solution of the present application, the accessor face-fingerprint joint fine-grained interactive perception features are further obtained based on the fine-grained global interactive analysis between the accessor's fingerprint features and the accessor's facial features. In a specific example of the present application, such as Figure 3 As shown, the S4 includes: S41, respectively performing fine-grained deconstruction on the accessor fingerprint features and the accessor facial features, and performing one-way attention interaction analysis on the set of deconstructed accessor fingerprint local features and the set of accessor facial local features to obtain one-way global interaction optimized accessor fingerprint features and one-way global interaction optimized accessor facial features; S42, weighted fusion of the one-way global interaction optimized accessor fingerprint features and the one-way global interaction optimized accessor facial features to obtain the accessor face-fingerprint joint fine-grained interaction perception features.

[0024] Specifically, the S41 performs fine-grained deconstruction on the accessor fingerprint features and the accessor face features, respectively, and performs one-way attention interaction analysis on the deconstructed set of accessor fingerprint local features and the set of accessor face local features to obtain one-way global interaction optimized accessor fingerprint features and one-way global interaction optimized accessor face features. Specifically, first, the accessor fingerprint feature map and the accessor face feature map are subjected to feature fine-grained deconstruction to obtain a set of accessor fingerprint channel dimension local feature matrices and a set of accessor face channel dimension local feature matrices, and the set of accessor fingerprint channel dimension local feature matrices is used as the set of accessor fingerprint local features, and the set of accessor face channel dimension local feature matrices is used as the set of accessor face local features;; Through feature fine-grained deconstruction, that is, segmentation along the channel dimension of the feature map, a set of accessor fingerprint channel dimension local feature matrices and a set of accessor face channel dimension local feature matrices are obtained. These local feature matrices represent different parts or sub-features of the original feature map, that is, they reflect the local fine-grained feature information about the accessor's fingerprint and face, and provide rich input for the subsequent attention mechanism, so that the model can more accurately identify and process the key areas in the accessor's fingerprint feature map and the accessor's face feature map. Next, each accessor fingerprint channel dimension local feature matrix in the set of accessor fingerprint channel dimension local feature matrices is used as a query feature matrix, and the set of accessor face channel dimension local feature matrices is used as a set of key feature matrices. The query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on the first converter structure to obtain a set of one-way global attention optimized accessor fingerprint channel dimension local feature matrices; similarly, each accessor face channel dimension local feature matrix in the set of accessor face channel dimension local feature matrices is used as a query feature matrix, and the set of key feature matrices is ... The set of local feature matrices of the accessor fingerprint channel dimension is used as the set of key feature matrices, and the set of query feature matrices and the set of key feature matrices are input into a unidirectional global attention interaction module based on the second transformer structure to obtain a set of local feature matrices of the accessor face channel dimension optimized by unidirectional global attention; that is, after fine-grained deconstruction of the accessor fingerprint feature map and the accessor face feature map, the multi-head attention mechanism in the Transformer structure is used to measure the correlation between different feature matrices, and strengthen the relationship between features, so that the model can pay more attention to information-rich fingerprint and face feature areas.Specifically, each local feature matrix of the fingerprint feature map is used as a query feature matrix, and interacts with all local feature matrices of the face feature map as key feature matrices to obtain a set of local feature matrices of the accessor's fingerprint channel dimension that are optimized by one-way global attention; similarly, each local feature matrix of the face feature map is used as a query feature matrix, and interacts with all local feature matrices of the fingerprint feature map as key feature matrices to obtain a set of local feature matrices of the accessor's face channel dimension that are optimized by one-way global attention. This step enables fingerprint features and face features to perform one-way and comprehensive semantic interactions at a fine-grained level, capturing the global semantic association information between fingerprint features and face features. Specifically, in the technical solution of the present application, each accessor's fingerprint channel dimension local feature matrix of the accessor's fingerprint feature map can respectively perform one-way semantic interactions with the set of all accessor's face channel dimension local feature matrices of the accessor's face feature map to more comprehensively capture the global semantic association information between each accessor's fingerprint channel dimension local feature matrix of the accessor's fingerprint feature map and the accessor's face feature map. Similarly, each accessor face channel dimension local feature matrix of the accessor face feature map can perform unidirectional semantic interaction with all accessor fingerprint channel dimension local feature matrices of the accessor fingerprint feature map to more comprehensively capture the global semantic association information between each accessor face channel dimension local feature matrix of the accessor face feature map and the accessor fingerprint feature map. Subsequently, the set of the unidirectional global attention optimized accessor fingerprint channel dimension local feature matrix and the set of the unidirectional global attention optimized accessor face channel dimension local feature matrix are respectively coupled along the channel dimension to obtain a unidirectional global interaction optimized accessor fingerprint feature map and a unidirectional global interaction optimized accessor face feature map, and the unidirectional global interaction optimized accessor fingerprint feature map is used as the unidirectional global interaction optimized accessor fingerprint feature, and the unidirectional global interaction optimized accessor face feature map is used as the unidirectional global interaction optimized accessor face feature. Here, the effect of the one-way global attention interaction is to enhance the fine-grained channel relationship within the feature map, so that the model can pay more attention to the key information in the feature map, improve the quality and discrimination of the feature representation, and thus help improve the performance of the model in various visual tasks. The set of local feature matrices of the one-way global attention optimized accessor fingerprint channel dimension and the set of local feature matrices of the one-way global attention optimized accessor face channel dimension are coupled along the channel dimension to recombine the set of local feature matrices of the one-way global attention optimized accessor fingerprint channel dimension and the set of local feature matrices of the one-way global attention optimized accessor face channel dimension along the channel dimension to form a complete feature map, providing a complete feature representation for the final fusion.

[0025] The process of taking each accessor fingerprint channel dimension local feature matrix in the set of accessor fingerprint channel dimension local feature matrices as a query feature matrix, taking the set of accessor face channel dimension local feature matrices as a set of key feature matrices, and inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a first converter structure to obtain a set of one-way global attention optimized accessor fingerprint channel dimension local feature matrices includes: selecting a predetermined accessor fingerprint channel dimension local feature matrix from the set of accessor fingerprint channel dimension local feature matrices as a query feature matrix; calculating the product between the predetermined accessor fingerprint channel dimension local feature matrix and the transposed matrix of each accessor face channel dimension local feature matrix to obtain a set of accessor fingerprint-face channel dimension local semantic interaction feature matrices; determining the scale square root of the predetermined accessor fingerprint channel dimension local feature matrix, dividing each accessor fingerprint-face channel dimension local semantic interaction feature matrix by the scale square root of the predetermined accessor fingerprint channel dimension local feature matrix according to the position point, and then using The function performs soft maximum normalization processing on each feature matrix in the obtained set of feature matrices to obtain a set of local weight matrices of the accessor fingerprint channel dimension; using the set of local weight matrices of the accessor fingerprint channel dimension as weighted weights, calculates the position-weighted sum between each local feature matrix of the accessor fingerprint channel dimension in the set of local feature matrices of the accessor fingerprint channel dimension to obtain the unidirectional global attention optimized local feature matrix of the accessor fingerprint channel dimension.

[0026] More specifically, each accessor face channel dimension local feature matrix in the set of accessor face channel dimension local feature matrices is used as a query feature matrix, and the set of accessor fingerprint channel dimension local feature matrices is used as a set of key feature matrices. The query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on a second converter structure to obtain a set of one-way global attention optimized accessor face channel dimension local feature matrices. The process includes: selecting a predetermined accessor face channel dimension local feature matrix from the set of accessor face channel dimension local feature matrices as a query feature matrix; calculating the product between the predetermined accessor face channel dimension local feature matrix and the transposed matrix of each accessor fingerprint channel dimension local feature matrix to obtain a set of accessor face-fingerprint channel dimension local semantic interaction feature matrices; determining the scale square root of the predetermined accessor face channel dimension local feature matrix, and dividing each accessor face-fingerprint channel dimension local semantic interaction feature matrix by the scale square root of the predetermined accessor face channel dimension local feature matrix according to the position point, and then using The function performs soft maximum normalization processing on each feature matrix in the obtained set of feature matrices to obtain a set of local weight matrices of the accessor face channel dimension; using the set of local weight matrices of the accessor face channel dimension as weighted weights, calculates the position-weighted sum of the local feature matrices of the accessor face channel dimension in the set of local feature matrices of the accessor face channel dimension to obtain the unidirectional global attention optimized local feature matrix of the accessor face channel dimension.

[0027] Specifically, the S42 weightedly fuses the one-way global interactive optimization accessor fingerprint feature and the one-way global interactive optimization accessor face feature to obtain the accessor face-fingerprint joint fine-grained interactive perception feature. Specifically, the weighted sum by position between the one-way global interactive optimization accessor fingerprint feature map and the one-way global interactive optimization accessor face feature map is calculated to obtain the accessor face-fingerprint joint fine-grained perception feature map, and the accessor face-fingerprint joint fine-grained perception feature map is used as the accessor face-fingerprint joint fine-grained interactive perception feature. In the technical solution of the present application, a feature representation containing rich interactive information of fingerprint features and facial features is generated by calculating the weighted sum by position between the one-way global interactive optimization accessor fingerprint feature map and the one-way global interactive optimization accessor face feature map, that is, the accessor face-fingerprint joint fine-grained perception feature map is used as the accessor face-fingerprint joint fine-grained interactive perception feature.

[0028] In summary, in the above embodiment, obtaining the accessor face-fingerprint joint fine-grained interactive perception feature based on the fine-grained global interaction analysis between the accessor fingerprint feature and the accessor face feature includes: performing a fine-grained global interaction analysis between the accessor fingerprint feature and the accessor face feature using the following fine-grained feature interaction formula to obtain the accessor face-fingerprint joint fine-grained interactive perception feature; wherein the fine-grained feature interaction formula is: , in, and Respectively represent the access user's fingerprint feature map and the access user's face feature map, It is a feature fine-grained deconstruction operation. are respectively the first, second, and third and The local feature matrix of the access user fingerprint channel dimension, are respectively the first, second, and third and The local feature matrix of the face channel dimension of the access user, For the The scale of the local feature matrix of the fingerprint channel dimension of each access user, is matrix multiplication, for function, is the number of local feature matrices of the accessor fingerprint channel dimension in the accessor fingerprint feature graph, For the said The one-way global attention optimizes the local feature matrix of the accessor fingerprint channel dimension corresponding to the local feature matrix of the accessor fingerprint channel dimension, For the The one-way global attention optimizes the local feature matrix of the accessor’s face channel dimension corresponding to the local feature matrix of the accessor’s face channel dimension, Indicates that the features are coupled along the channel dimension, and They are the fingerprint feature map of the one-way global interaction optimization access user and the face feature map of the one-way global interaction optimization access user. and are weighted hyper parameters of the one-way global interaction optimization access user fingerprint feature map and the one-way global interaction optimization access user face feature map, respectively. It is the joint fine-grained perceptual feature map of the access user's face and fingerprint.

[0029] It is worth mentioning that in other specific examples of the present application, the accessor face-fingerprint joint fine-grained interactive perception features can also be obtained based on the fine-grained global interactive analysis between the accessor fingerprint features and the accessor facial features in other ways, for example: input the accessor fingerprint features and the accessor facial features; align the fingerprint features and the facial features to ensure that the two are interactively analyzed in the same space; use an attention mechanism (such as Self-Attention, Transformer) to capture the fine-grained interactive relationship between fingerprint features and facial features; use a cross-modal fusion method (such as Cross-Modal Attention, Cross-Modal Fusion) to interactively analyze fingerprint features and facial features; weightedly fuse fingerprint features and facial features according to the importance of the features; use a multi-layer perceptron to deeply fuse fingerprint features and facial features; and average pool the fused features to obtain the accessor face-fingerprint joint fine-grained interactive perception features.

[0030] In particular, S5 performs identity recognition based on the accessor's fingerprint features to determine the probability distribution of the first identity recognition class; performs identity recognition based on the accessor's facial features to determine the probability distribution of the second identity recognition class; and performs identity recognition based on the accessor's face-fingerprint combined fine-grained interactive perception features to determine the probability distribution of the third identity recognition class. In order to evaluate the accessor's identity from different perspectives and levels, and ultimately determine whether the accessor is a legitimate user more accurately through a fusion mechanism, in the technical solution of the present application, the accessor's fingerprint feature map is further input into a first identity recognizer based on a classifier to obtain a probability distribution of the first identity recognition class, the accessor's facial feature map is input into a second identity recognizer based on a classifier to obtain a probability distribution of the second identity recognition class, and the accessor's face-fingerprint combined fine-grained perception feature map is input into a third identity recognizer based on a classifier to obtain a probability distribution of the third identity recognition class. Among them, through fine-grained interactive perception technology, combining the two biometric features of face and fingerprint, the correlation between the two is captured through fine-grained interactive perception technology, and the interaction between different features is more deeply understood, thereby building a more robust identity recognition model, adapting to more diverse application scenarios, and further improving the accuracy and reliability of identity recognition.

[0031] In a preferred example, the present application takes into account that the accessor fingerprint feature map and the accessor face feature map represent the image semantic features of the accessor's fingerprint image and face image respectively. When they are passed through a bidirectional global attention joint perception module based on fine-grained deconstruction, the spatial fine-grained image semantic feature distribution difference based on the source image semantic distribution will also have the global attention difference of local spatial distribution deconstruction. Therefore, the accessor face-fingerprint joint fine-grained perception feature map will also have local-global bidirectional image semantic feature spatial distribution difference relative to the source image semantic feature representation of the accessor fingerprint feature map and the accessor face feature map, thereby affecting the probability distribution consistency of the third identity recognition class probability distribution relative to the first identity recognition class probability distribution and the second identity recognition class probability distribution, and affecting the accuracy of determining whether the accessor is a legitimate user.

[0032] Based on this, in the preferred example of the present application, the accessor fingerprint feature map, the accessor face feature map and the accessor face-fingerprint joint fine-grained perceptual feature map are jointly optimized, specifically including: first, the accessor fingerprint feature map, the accessor face feature map and the accessor face-fingerprint joint fine-grained perceptual feature map are cascaded along the channel to expand them into an accessor face-fingerprint source image semantic-fine-grained perceptual cascade feature vector; then, for the first eigenvalue of the accessor face-fingerprint source image semantic-fine-grained perceptual cascade feature vector and the second eigenvalue , calculate the absolute value of the difference divided by the sum of the absolute values ​​and then correct the relative probability value, wherein the correction makes the relative probability value greater than zero and less than one; calculate the first product of the first eigenvalue multiplied by the relative probability value, the second product of the second eigenvalue multiplied by the absolute value of the difference between the relative probability value and one, the first quotient of the first eigenvalue divided by the absolute value of the difference between the relative probability value and one, and the second quotient of the second eigenvalue divided by the relative probability value; after adding the first product and the second product, further subtract the difference between the first quotient and the second quotient to obtain the value equal to the first eigenvalue and the second eigenvalue The corresponding eigenvalues ​​of the corresponding optimization matrix ; and, multiplying the optimization matrix with the accessor face-fingerprint source image semantic-fine-grained perception cascade feature vector to obtain an optimized accessor face-fingerprint source image semantic-fine-grained perception cascade feature vector.

[0033] Finally, the optimized accessor face-fingerprint source image semantic-fine-grained perceptual cascade feature vector is restored to an optimized accessor face-fingerprint source image semantic-fine-grained perceptual cascade feature map, and the optimized accessor face-fingerprint source image semantic-fine-grained perceptual cascade feature map is split along the channel to obtain an optimized accessor fingerprint feature map, an optimized accessor face feature map and an optimized accessor face-fingerprint joint fine-grained perceptual feature map, which are input into a classifier-based identity identifier to obtain corresponding first identity recognition class probability distribution, second identity recognition class probability distribution and third identity recognition class probability distribution.

[0034] Specifically, the access user face-fingerprint source image semantic-fine-grained perception cascade feature vector The optimization expression is: ; in, represents the semantic-fine-grained perception cascade feature vector of the access user's face-fingerprint source image, and respectively represent the first eigenvalue and the second eigenvalue of the semantic-fine-grained perception cascade feature vector of the access user's face-fingerprint source image, represents the relative probability value, Represents the first eigenvalue and the second eigenvalue The corresponding eigenvalues ​​of the corresponding optimization matrix, Represents the first eigenvalue and the second eigenvalue The corresponding optimization matrix is, represents matrix multiplication, Represents the optimized accessor face-fingerprint source image semantic-fine-grained aware cascade feature vector.

[0035] Among them, the first eigenvalue and the second eigenvalue The corresponding relative probability value It is expressed as: ;in, and respectively represent the first eigenvalue and the second eigenvalue of the semantic-fine-grained perception cascade feature vector of the access user's face-fingerprint source image, represents the predetermined hyperparameters, Represents the relative probability value.

[0036] Here, for the region-boundary integral relationship of the high-dimensional feature manifold of the semantic-fine-grained perception cascade feature vector of the accessor's face-fingerprint source image, the probability distribution of the feature set of the semantic-fine-grained perception cascade feature vector of the accessor's face-fingerprint source image is defined as a manifold constraint boundary to approximate the high-dimensional manifold of the feature set of the semantic-fine-grained perception cascade feature vector of the accessor's face-fingerprint source image, which is composed of a single-connected region representation of eigenvalue pairs, so as to avoid the mapping ambiguity of the diversified feature representation of the semantic-fine-grained perception cascade feature vector of the accessor's face-fingerprint source image to the local manifold representation in the high-dimensional feature classification regression convergence space, and improve the execution iteration consistency of each local feature distribution in the mapping task, so as to improve the accuracy of determining whether the accessor is a legitimate user. In this way, multiple identity recognition models and feature interaction perception technology based on artificial intelligence can be used to more accurately identify and verify the digital identity of the accessor, thereby improving the intelligence level and reliability of the digital identity authentication of the accessor.

[0037] In particular, the S6 determines whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution. In a specific example of the present application, the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution are fused to obtain a multi-model identity recognition class probability distribution; here, in order to improve the accuracy and robustness of identity authentication, the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution are further fused to obtain a multi-model identity recognition class probability distribution. It should be understood that different biometric features (fingerprints and faces) have different characteristics. Fingerprints may not be as stable as facial features in some cases (for example, injured or dry fingers), while facial features may perform poorly under certain conditions (such as changes in lighting). Therefore, a single-feature recognition system may have a high false recognition rate and false rejection rate due to environmental or individual differences. By fusing the probability distributions of multiple features, these two indicators can be better balanced and the occurrence of errors can be reduced. In particular, the use of fine-grained interactive information between fingerprints and facial features can further reveal the implicit correlation between the biological features of the accessor, thereby making full use of the complementary information between them to further improve the accuracy of the digital identity recognition of the accessor. Furthermore, based on the multi-model identity recognition class probability distribution, it is determined whether the accessor is a legitimate user. In this way, multiple identity recognition models and feature interaction perception technology based on artificial intelligence can be used to identify and verify the digital identity of the accessor, rather than just using the accessor's single biological feature or separate different biological features for identity recognition. By exploring the intrinsic connection between fingerprint features and facial features, the intelligence level and reliability of the accessor's digital identity authentication are improved.

[0038] In summary, the method for identifying the digital identity of the accessor based on multiple models according to the embodiment of the present application is explained, which analyzes the fingerprint image and the face image by using an image processing and analysis algorithm based on artificial intelligence and machine learning, and performs two-way global attention joint perception based on fine-grained deconstruction on these fingerprint features and face features. Further, different identity identifier models are used to process these different features separately to generate identity recognition class probability distributions corresponding to fingerprints, faces, and face-fingerprint joint fine-grained perception semantics, and then a fusion mechanism is used to fuse these identity recognition class probability distributions to obtain the recognition result of the accessor's digital identity to determine whether the accessor is a legitimate user. In this way, multiple identity recognition models and feature interaction perception technology based on artificial intelligence can be used to identify and verify the digital identity of the accessor, thereby improving the intelligence level and reliability of the accessor's digital identity authentication.

[0039] Furthermore, a system for identifying the digital identity of an access user based on multiple models is also provided.

[0040] Figure 4 FIG. 1 is a block diagram of a system for identifying a digital identity of an access user based on multiple models according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the system 300 for identifying the digital identity of the accessor based on multiple models includes: a fingerprint image acquisition module 310, which is used to acquire the fingerprint image of the accessor acquired by the fingerprint module; a face image acquisition module 320, which is used to acquire the face image of the accessor acquired by the camera; an image feature analysis module 330, which is used to perform feature analysis on the fingerprint image of the accessor and the face image of the accessor to obtain the fingerprint feature and the face feature of the accessor; a fine-grained global interaction analysis module 340, which is used to analyze the fingerprint feature and the face feature of the accessor based on the fine-grained global interaction analysis between the fingerprint feature and the face feature of the accessor. The accessor's face-fingerprint joint fine-grained interactive perception feature is obtained by analysis; an identity recognition module 350 is used to perform identity recognition based on the accessor's fingerprint feature to determine a first identity recognition class probability distribution; perform identity recognition based on the accessor's face feature to determine a second identity recognition class probability distribution; perform identity recognition based on the accessor's face-fingerprint joint fine-grained interactive perception feature to determine a third identity recognition class probability distribution; an identity authentication module 360 ​​is used to determine whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution.

[0041] As described above, the system 300 for identifying the digital identity of an accessor based on multiple models according to the embodiment of the present application can be implemented in various wireless terminals, such as a server having an algorithm for identifying the digital identity of an accessor based on multiple models. In a possible implementation, the system 300 for identifying the digital identity of an accessor based on multiple models according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the system 300 for identifying the digital identity of an accessor based on multiple models can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the system 300 for identifying the digital identity of an accessor based on multiple models can also be one of the many hardware modules of the wireless terminal.

[0042] Alternatively, in another example, the system 300 for identifying the digital identity of an accessor based on multiple models and the wireless terminal may be separate devices, and the system 300 for identifying the digital identity of an accessor based on multiple models may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0043] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying a user's digital identity based on multiple models, characterized in that: include: Obtaining the fingerprint image of the access user collected by the fingerprint module; Acquire a facial image of the access user captured by a camera; Performing feature analysis on the fingerprint image of the access user and the face image of the access user to obtain the fingerprint feature and face feature of the access user; Based on the fine-grained global interaction analysis between the accessor fingerprint feature and the accessor face feature, the accessor face-fingerprint joint fine-grained interactive perception feature is obtained, including: respectively performing fine-grained deconstruction on the accessor fingerprint feature and the accessor face feature, and performing one-way attention interaction analysis on the set of deconstructed accessor fingerprint local features and the set of accessor face local features to obtain one-way global interaction optimization accessor fingerprint features and one-way global interaction optimization accessor face features; weighted fusion of the one-way global interaction optimization accessor fingerprint features and the one-way global interaction optimization accessor face features to obtain the accessor face-fingerprint joint fine-grained interactive perception feature; Performing identity recognition based on the accessor's fingerprint feature to determine a first identity recognition class probability distribution; performing identity recognition based on the accessor's face feature to determine a second identity recognition class probability distribution; performing identity recognition based on the accessor's face-fingerprint joint fine-grained interactive perception feature to determine a third identity recognition class probability distribution; Based on the first identity recognition class probability distribution, the second identity recognition class probability distribution, and the third identity recognition class probability distribution, it is determined whether the accessor is a legitimate user.

2. The method for identifying the digital identity of an access user based on multiple models according to claim 1, characterized in that: Performing feature analysis on the fingerprint image of the access user and the face image of the access user to obtain the fingerprint feature of the access user and the face feature of the access user, including: Inputting the fingerprint image of the access user into a fingerprint feature extractor based on a first convolutional neural network model to obtain an access user fingerprint feature map as the access user fingerprint feature; The facial image of the access user is input into a facial feature extractor based on a second convolutional neural network model to obtain an access user facial feature map as the access user facial feature.

3. The method for identifying the digital identity of an access user based on multiple models according to claim 2, characterized in that: The accessor fingerprint feature and the accessor face feature are respectively fine-grainedly deconstructed, and a one-way attention interaction analysis is performed on the set of the deconstructed accessor fingerprint local features and the set of the accessor face local features to obtain a one-way global interaction optimized accessor fingerprint feature and a one-way global interaction optimized accessor face feature, including: Performing feature fine-grained deconstruction on the accessor fingerprint feature graph and the accessor face feature graph to obtain a set of accessor fingerprint channel dimension local feature matrices and a set of accessor face channel dimension local feature matrices, and using the set of accessor fingerprint channel dimension local feature matrices as the set of accessor fingerprint local features, and using the set of accessor face channel dimension local feature matrices as the set of accessor face local features; Using each accessor fingerprint channel dimension local feature matrix in the set of accessor fingerprint channel dimension local feature matrices as a query feature matrix, using the set of accessor face channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a first converter structure to obtain a set of one-way global attention optimized accessor fingerprint channel dimension local feature matrices; Using each local feature matrix of the accessor face channel dimension in the set of local feature matrices of the accessor face channel dimension as a query feature matrix, using the set of local feature matrices of the accessor fingerprint channel dimension as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a second converter structure to obtain a set of local feature matrices of the accessor face channel dimension optimized by one-way global attention; The set of local feature matrices of the one-way global attention optimization accessor fingerprint channel dimension and the set of local feature matrices of the one-way global attention optimization accessor face channel dimension are respectively coupled along the channel dimension to obtain a one-way global interaction optimization accessor fingerprint feature map and a one-way global interaction optimization accessor face feature map, and the one-way global interaction optimization accessor fingerprint feature map is used as the one-way global interaction optimization accessor fingerprint feature, and the one-way global interaction optimization accessor face feature map is used as the one-way global interaction optimization accessor face feature.

4. The method for identifying the digital identity of an access user based on multiple models according to claim 3, characterized in that: Using each accessor fingerprint channel dimension local feature matrix in the set of accessor fingerprint channel dimension local feature matrices as a query feature matrix, using the set of accessor face channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a first converter structure to obtain a set of one-way global attention optimized accessor fingerprint channel dimension local feature matrices, including: Selecting a predetermined local feature matrix of the accessor fingerprint channel dimension from the set of local feature matrices of the accessor fingerprint channel dimension as a query feature matrix; Calculate the product of the predetermined accessor fingerprint channel dimension local feature matrix and the transposed matrix of each accessor face channel dimension local feature matrix to obtain a set of accessor fingerprint-face channel dimension local semantic interaction feature matrices; Determine the scale square root of the predetermined access user fingerprint channel dimension local feature matrix, and divide each access user fingerprint-face channel dimension local semantic interaction feature matrix by the scale square root of the predetermined access user fingerprint channel dimension local feature matrix according to the position point, and then use The function performs a soft maximum normalization process on each feature matrix in the obtained feature matrix set to obtain a set of local weight matrices of the access user's fingerprint channel dimension; Taking the set of local weight matrices of the accessor fingerprint channel dimension as weighted weights, the position-weighted sum between the local feature matrices of the accessor fingerprint channel dimension in the set of local feature matrices of the accessor fingerprint channel dimension is calculated to obtain the unidirectional global attention optimized local feature matrix of the accessor fingerprint channel dimension.

5. The method for identifying the digital identity of an access user based on multiple models according to claim 4, characterized in that: Using each local feature matrix of the accessor face channel dimension in the set of local feature matrices of the accessor face channel dimension as a query feature matrix, using the set of local feature matrices of the accessor fingerprint channel dimension as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a second converter structure to obtain a set of local feature matrices of the accessor face channel dimension optimized by one-way global attention, including: Selecting a predetermined local feature matrix of the accessor's face channel dimension from the set of local feature matrices of the accessor's face channel dimension as a query feature matrix; Calculate the product of the predetermined local feature matrix of the accessor face channel dimension and the transposed matrix of each local feature matrix of the accessor fingerprint channel dimension to obtain a set of local semantic interaction feature matrices of the accessor face-fingerprint channel dimension; Determine the scale square root of the predetermined accessor face channel dimension local feature matrix, and divide each accessor face-fingerprint channel dimension local semantic interaction feature matrix by the scale square root of the predetermined accessor face channel dimension local feature matrix according to the position point, and then use The function performs a soft maximum normalization process on each feature matrix in the obtained set of feature matrices to obtain a set of local weight matrices of the accessee's face channel dimension; Taking the set of local weight matrices of the accessor face channel dimension as weighted weights, the position-weighted sum of the local feature matrices of the accessor face channel dimension in the set of local feature matrices of the accessor face channel dimension is calculated to obtain the unidirectional global attention optimized local feature matrix of the accessor face channel dimension.

6. The method for identifying the digital identity of an access user based on multiple models according to claim 5, characterized in that: The one-way global interaction optimization accessor fingerprint feature and the one-way global interaction optimization accessor face feature are weightedly fused to obtain the accessor face-fingerprint joint fine-grained interactive perception feature, including: calculating the position-weighted sum between the one-way global interaction optimization accessor fingerprint feature map and the one-way global interaction optimization accessor face feature map to obtain the accessor face-fingerprint joint fine-grained perception feature map, and using the accessor face-fingerprint joint fine-grained perception feature map as the accessor face-fingerprint joint fine-grained interactive perception feature.

7. The method for identifying the digital identity of an access user based on multiple models according to claim 6, characterized in that: Performing identity recognition based on the accessor fingerprint feature to determine a first identity recognition class probability distribution includes: inputting the accessor fingerprint feature map into a first identity recognizer based on a classifier to obtain the first identity recognition class probability distribution.

8. The method for identifying the digital identity of an access user based on multiple models according to claim 7, characterized in that: Performing identity recognition based on the accessor's facial features to determine a second identity recognition class probability distribution includes: inputting the accessor's facial feature map into a second identity recognizer based on a classifier to obtain the second identity recognition class probability distribution.

9. The method for identifying the digital identity of an access user based on multiple models according to claim 8, characterized in that: Determining whether the access user is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution, and the third identity recognition class probability distribution includes: fusing the first identity recognition class probability distribution, the second identity recognition class probability distribution, and the third identity recognition class probability distribution to obtain a multi-model identity recognition class probability distribution; Based on the multi-model identity recognition class probability distribution, it is determined whether the accessor is a legitimate user.

10. A system for identifying the digital identity of an access user based on multiple models, characterized in that: include: A fingerprint image acquisition module is used to acquire the fingerprint image of the access user collected by the fingerprint module; A face image acquisition module, used to acquire the face image of the access user captured by a camera; An image feature analysis module, used for performing feature analysis on the fingerprint image of the access user and the face image of the access user to obtain the fingerprint features and face features of the access user; A fine-grained global interaction analysis module, used to obtain a joint fine-grained interaction perception feature of the accessor's face and fingerprint based on a fine-grained global interaction analysis between the accessor's fingerprint feature and the accessor's face feature; An identity recognition module, configured to perform identity recognition based on the access user's fingerprint feature to determine a first identity recognition class probability distribution; Performing identity recognition based on the facial features of the access user to determine a second identity recognition class probability distribution; Performing identity recognition based on the access user's face-fingerprint combined with fine-grained interactive perception features to determine a third identity recognition class probability distribution; The identity authentication module is used to determine whether the accessor is a legitimate user based on the first identity recognition class probability distribution, the second identity recognition class probability distribution and the third identity recognition class probability distribution.

Citation Information

Patent Citations

  • Village-network co-construction convenience service terminal based on multiple identity authentication

    CN117115928A

  • Fine-grained deep fake face identity detection method under remote identity authentication scene

    CN118172810A

  • Method for fingerprint recognization using a mobile and apparatus using the same

    KR1020240114676A

  • Multi-biometric enrollment kiosk including biometric enrollment and verification, face recognition and fingerprint matching systems

    US20120293642A1

  • Enhanced system for generation of facial models and animation

    US20220398796A1

Cited By

  • Method and system for carrying out joint verification on digital identity by using combined model

    CN119293772A