Offline User Identification Method and Device

Abstract

This specification provides an offline user identification method and apparatus. In this offline user identification method, a heterogeneous graph of user behavior relationships is constructed based on the user's historical location access data. During offline user identification, biometric data of the user to be identified is acquired at a first location to be visited; based on the acquired biometric data, a target user account is retrieved from a candidate user account database; and based on the heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, the matching degree between the target user account, the first location, and the biometric data of the user to be identified is determined to perform user identification.

Description

Technical Field

[0001] The embodiments in this specification generally relate to the field of user identification, and particularly to offline user identification methods and apparatus. Background Technology

[0002] Offline user identification refers to the identification of users using biometric features such as facial recognition in offline locations. It has been applied in many offline scenarios, such as bank verification, hotel check-in, and offline shopping. In these scenarios, most implementations use facial recognition and are primarily based on visual features and computer vision models. However, in offline scenarios, significant variations exist due to factors such as pose, lighting, and occlusion, making it impossible to achieve satisfactory accuracy using only computer vision algorithms in real-world situations. Furthermore, when the number of candidate users reaches billions, computer vision algorithms need to possess extremely high efficiency and accuracy. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide an offline user identification method and apparatus. Using this offline user identification method and apparatus, embedded representations of the target user's account, location, and biometrics are obtained by modeling graph information in a heterogeneous graph of the user's biometrics and user behavior relationships. User identification is then performed based on these embedded representations. This approach incorporates offline user habit / behavioral information mined from the heterogeneous graph of user behavior relationships during user identification, thereby improving the accuracy of offline user identification.

[0004] According to one aspect of the embodiments of this specification, an offline user identification method is provided, comprising: acquiring biometric data of a user to be identified at a first location; retrieving a target user account from a candidate user account database based on the acquired biometric data; and determining the matching degree between the target user account, the first location, and the biometric data of the user to be identified based on a user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account, for user identification, wherein the user behavior relationship heterogeneity graph is constructed based on the user account's location history access data.

[0005] Optionally, in one example of the above aspects, determining the matching between the target user account, the first location, and the biometric data of the user to be identified based on the user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account may include: using a machine learning model to predict the matching between the target user account, the first location, and the biometric data of the user to be identified based on the user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account.

[0006] Optionally, in one example of the above aspects, using a machine learning model to predict the matching between the target user account, the first location, and the biometric data of the user to be identified based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account may include: using a first machine learning model to model the target user account and the first location based on the heterogeneous graph of user behavior relationships to obtain embedding representations of the target user account and the first location under various heterogeneous relationships; using a second machine learning model to model the biometric data of the user to be identified and the biometric data of the target user account to obtain biometric embedding representations that reflect the similarity between the biometric data of the user to be identified and the biometric data of the target user account; and providing the embedding representations of the target user account and the first location under various heterogeneous relationships and the biometric embedding representations to an aggregation layer to predict the matching between the target user account and the biometric data of the first location and the user to be identified.

[0007] Optionally, in one example of the above aspects, the first machine learning model includes a graph neural network, and the second machine learning model includes a multilayer perceptron model.

[0008] Alternatively, in one example of the above aspects, the graph neural network includes an attention-based graph neural network.

[0009] Alternatively, in one example of the above aspects, the aggregation layer includes an attention-based aggregation layer.

[0010] Alternatively, in one example of the above aspects, the first machine learning model, the second machine learning model, and the aggregation layer are trained as a single model.

[0011] Optionally, in one example of the above aspects, the total loss function for training the model includes the cross-entropy loss function.

[0012] Optionally, in one example of the above aspects, the total loss function for model training includes a loss function with L2 regularization introduced on the model parameters.

[0013] Optionally, in one example of the above aspects, determining the matching between the target user account, the first location, and the biometric data of the user to be identified based on the user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account may include: determining the matching between the target user account and the first location based on the user behavior relationship heterogeneity graph; determining the matching between the target user account and the biometric data of the user to be identified based on the biometric data of the target user account and the biometric data of the user to be identified; and determining that user identification is successful in response to the target user account matching the biometric data of both the first location and the user to be identified.

[0014] Optionally, in one example of the above aspects, the candidate user account database includes a local candidate user account database and a global candidate user account database. Retrieving a target user account from the candidate user account database based on the acquired biometric data may include: retrieving the target user account from the corresponding local candidate user account database at the first location based on the acquired biometric data; and, in response to the failure to find a matching target user account in the local candidate user account database, retrieving the target user account from the global candidate user account database based on the acquired biometric data.

[0015] Optionally, in one example of the above aspects, each candidate user account local database includes a candidate user account database constructed based on historical access data at the corresponding location and historical access data at other locations within a given distance from the corresponding location.

[0016] Alternatively, in one example of the above aspects, the biometric data includes visual feature data of facial information.

[0017] According to embodiments of this specification, an offline user identification device is provided, comprising: a biometric data acquisition unit for acquiring biometric data of a user to be identified at a first location; a target user account retrieval unit for retrieving a target user account from a candidate user account database based on the acquired biometric data; and a user identification unit for determining the matching degree between the target user account, the first location, and the biometric data of the user to be identified based on a user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account, for user identification, wherein the user behavior relationship heterogeneity graph is constructed based on the user account's location history access data.

[0018] Optionally, in one example of the above aspects, the user identification unit uses a machine learning model to predict the matching degree between the target user account, the first location, and the biometric data of the user to be identified, based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account.

[0019] Optionally, in one example of the above aspects, the user identification unit includes: a first embedding representation generation module, which uses a first machine learning model to model the target user account and the first location based on a heterogeneous graph of user behavior relationships to obtain embedding representations of the target user account and the first location under various heterogeneous relationships; a second embedding representation generation module, which uses a second machine learning model to model the biometric data of the user to be identified and the biometric data of the target user account to obtain biometric embedding representations reflecting the similarity between the biometric data of the user to be identified and the biometric data of the target user account; and a prediction module, which provides the embedding representations of the target user account and the first location under various heterogeneous relationships and the biometric embedding representations to an aggregation layer to predict the matching between the target user account and the biometric data of the first location and the user to be identified.

[0020] Optionally, in one example of the above aspects, the user identification unit includes: a first matching module, which determines the matching between the target user account and the first location based on a user behavior relationship heterogeneity graph; a second matching module, which determines the matching between the target user account and the biometric data of the user to be identified based on the biometric data of the target user account and the biometric data of the user to be identified; and a user identification module, which determines that user identification is successful in response to the target user account matching both the biometric data of the first location and the biometric data of the user to be identified.

[0021] Optionally, in one example of the above aspects, the candidate user account database includes a local candidate user account database and a global candidate user account database. The target user account retrieval unit includes: a local retrieval module, which retrieves a target user account in the corresponding local candidate user account database of the first location based on the acquired biometric data; and a global retrieval module, which, in response to the failure to retrieve a matching target user account from the local candidate user account database, retrieves the target user account in the global candidate user account database based on the acquired biometric data.

[0022] According to another aspect of the embodiments of this specification, an offline user identification device is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the offline user identification method as described above.

[0023] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores executable instructions, which, when executed, cause a processor to perform the offline user identification method as described above.

[0024] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the offline user identification method as described above. Attached Figure Description

[0025] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0026] Figure 1 An example schematic diagram of the offline user identification process is shown.

[0027] Figure 2 An example flowchart of an offline user identification method according to an embodiment of this specification is shown.

[0028] Figure 3 An example flowchart of a target user account retrieval process according to an embodiment of this specification is shown.

[0029] Figure 4 An example schematic diagram of a heterogeneous graph of user behavior relationships according to an embodiment of this specification is shown.

[0030] Figure 5 An example flowchart of a user authentication process according to an embodiment of this specification is shown.

[0031] Figure 6 An example schematic diagram of a graph neural network according to an embodiment of this specification is shown.

[0032] Figure 7 An example schematic diagram of a user verification process according to an embodiment of this specification is shown.

[0033] Figure 8 Another example flowchart of a user verification process according to an embodiment of this specification is shown.

[0034] Figure 9 An example block diagram of an offline user identification device according to an embodiment of this specification is shown.

[0035] Figure 10 An example block diagram of a target user account retrieval unit according to an embodiment of this specification is shown.

[0036] Figure 11 An example block diagram of a user identification unit according to an embodiment of this specification is shown.

[0037] Figure 12 Another example block diagram of a user identification unit according to an embodiment of this specification is shown.

[0038] Figure 13 An example schematic diagram of an offline user identification device implemented based on a computer system according to an embodiment of this specification is shown. Detailed Implementation

[0039] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0040] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0041] The use of biometrics (especially facial information) for user identification is increasingly being applied in fields such as social security, finance, and commerce. User identification can be divided into two categories: online one-to-one identification and offline one-to-many identification. Online identification typically occurs on personal mobile devices, such as logging into applications or making online payments on a mobile phone. This scenario only requires one-to-one verification between the user's biometrics and the logged-in account. However, offline identification scenarios are entirely different and more complex.

[0042] Figure 1 An example schematic diagram of the offline identification process 100 is shown.

[0043] like Figure 1 As shown, at location 110, an offline recognition system performs facial scanning. These systems are deployed in various locations and are equipped with facial scanning devices using cameras. When a user requests a specific service at these locations, such as entering Universal Studios or purchasing products, the facial scanning device in the offline recognition system scans the user's face.

[0044] At 120, the offline recognition system retrieves an account from all accounts whose face resembles the scanned face.

[0045] After retrieving accounts with similar faces, at 130, computer vision (CV) technology is used to verify between the user and the retrieved account based on the visual features of the scanned user's face and the corresponding visual features of the retrieved account.

[0046] If the confidence level of the verification is high, i.e., the verification result in step 130 is yes, then the verification process is complete, and in step 150, the user is confirmed to have passed the identification process and thus receives the corresponding service. If the confidence level of the verification is not high enough, i.e., the verification result in step 130 is no, then in step 140, further verification is required, such as entering a complete or partial ID number or other personal information.

[0047] If the verification result at step 140 is yes, then at step 150, the user identification is confirmed to have passed. If the verification result at step 140 is no, then at step 160, the user identification is confirmed to have failed.

[0048] If the aforementioned visual feature-based user recognition scheme is applied to offline user recognition in industrial scenarios, the environment is extremely complex. Influenced by factors such as posture, makeup, dim lighting, or backlighting, almost everyone can find some similar users, making purely visual feature-based facial recognition very difficult. Furthermore, offline user recognition scenarios can involve billions of candidate users; such a massive data scale necessitates a highly efficient user recognition solution.

[0049] In view of the above, embodiments of this specification provide an offline user identification scheme. In offline user identification scenarios, users, locations, and their different types of interactions can form a heterogeneous graph. Mining this heterogeneous graph can yield a wealth of information about users' offline habits and behaviors. In the provided offline user identification scheme, historical location access information of users is mined to construct a heterogeneous relationship graph between users and locations. During offline user identification, in addition to biometric verification, the constructed heterogeneous relationship graph is used to determine whether the retrieved user account will visit the current location for supplementary verification. This allows for user identification based on users' offline behavior and habits, thereby improving the accuracy of offline user identification.

[0050] The offline user identification method and offline user identification apparatus according to embodiments of this specification will now be described with reference to the accompanying drawings.

[0051] Figure 2 An example flowchart of an offline user identification method 200 according to an embodiment of this specification is shown.

[0052] like Figure 2 As shown, at 210, biometric data of the user to be identified is acquired at the first location. Examples of the acquired biometrics may include, but are not limited to, facial features, voiceprint features, gesture features, and posture features. For example, corresponding data acquisition devices can be used to acquire the biometrics; for instance, facial scanning devices, voice capture devices, and camera devices can be used to acquire facial features, voiceprint features, gesture features, and posture features. The acquired data may include image data and audio data. Data preprocessing can also be performed on the acquired data to obtain the expected biometric data.

[0053] For example, when the collected data consists of facial images, preprocessing can be performed, including image quality checks, face and keypoint detection, alignment, and cropping. The preprocessed images are then input into a ResNet-based model to extract visual features from the facial images. These extracted visual features are represented as 256-dimensional floating-point vectors, which can describe, for example, both 2D and 3D facial features. All facial images undergo this data preprocessing to extract corresponding visual features, which serve as the user's biometric data.

[0054] In step 220, based on the acquired biometric data, the target user account is retrieved from the candidate user account database.

[0055] A candidate user account database can be created based on users' location history access data. A global candidate user account database can be created and maintained based on the location history access data of all users. In some embodiments, target user accounts can be retrieved from the global candidate user account database based on acquired biometric data.

[0056] In some embodiments, in addition to creating and maintaining a global database of candidate user accounts, a local database of candidate user accounts may also be created and maintained. The local database of candidate user accounts is a location-based database. In other words, a local database of candidate user accounts is created and maintained for each location. In the local database of candidate user accounts for each location, biometric data of users who visit that location is recorded. Optionally, in one example, the local database of candidate user accounts for each location may only record biometric data of users who visited that location within a specified time period, such as biometric data of users who visited that location within the past year. Furthermore, optionally, in another example, the local database of candidate user accounts for each location may include a database of candidate user accounts constructed based on historical access data at that location and historical access data at other locations within a given distance from that location. That is, the local database of candidate user accounts for each location may record biometric data of users who visit that location as well as biometric data of users who visit other locations within a given distance from that location.

[0057] Figure 3 An example flowchart of a target user account retrieval process 300 according to an embodiment of this specification is shown. Figure 3 In the example, the candidate user account database created and maintained includes a local candidate user account database and a global candidate user account database.

[0058] like Figure 3 As shown, at 310, a local database of corresponding candidate user accounts for the first location is determined. Next, at 320, based on the acquired biometric data, a target user account is retrieved from the local database of corresponding candidate user accounts for the first location, and at 330, it is determined whether a matching target user account has been retrieved.

[0059] If a matching target user account is found, then at step 360, the target user account is identified. If no matching target user account is found, then at step 340, based on the acquired biometric data, the target user account is searched in the global database of candidate user accounts, and at step 350, it is determined whether a matching target user account has been found.

[0060] If a matching target user account is found, the target user account is confirmed at step 360. If no matching target user account is found, the user identification process is considered unsuccessful at step 370, and the user identification process ends.

[0061] User behavior patterns are local and stable. For example, users typically visit locations near their homes or offices. Therefore, maintaining a local database and a global database of candidate user accounts, and first searching for target user accounts in the local database, only searching the global database if the target user account is not found in the local database—that is, using a local-global search mechanism—can retrieve target user accounts more quickly, thus improving search efficiency. Furthermore, by limiting the local database to biometric data of users who visited the location within a specified timeframe, the database is time-limited, reducing its size and further improving search efficiency. Additionally, by limiting the local database for each location to historical access data from that location and other locations within a given distance, the database exhibits expanded spatial locality, improving the hit rate of target user account searches. In addition, if the candidate user account local database for each location is constructed based on the historical access data of that location within a specified time period and the historical access data of other locations within a given distance from the corresponding location, then the created candidate user account local database has expanded spatiotemporal locality, thereby improving retrieval efficiency while ensuring the retrieval hit rate of the target user account.

[0062] Back Figure 2 After retrieving the target user account as described above, the matching between the target user account, the first location, and the biometric data of the user to be identified is determined based on the heterogeneous graph of user behavior relationship, the biometric data of the user to be identified, and the biometric data of the target user account, so as to perform user identification.

[0063] Specifically, at step 230, based on the heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, the matching ability among the biometric data of the target user account, the first location, and the user to be identified is determined. Next, at step 240, it is determined whether the biometric data of the target user account, the first location, and the user to be identified match. If they match, at step 250, user identification is confirmed as successful. If they do not match, at step 260, user identification is confirmed as unsuccessful.

[0064] Heterogeneous graphs of user behavior relationships can be pre-constructed based on a user account's location history access data. For example, heterogeneous relationships between a user account and the locations visited can be extracted from the location history access data, and a heterogeneous graph of user behavior relationships can be created based on the user account, location, and heterogeneous relationships. In this specification, heterogeneous relationships are used to reflect user behavior relationships of a user account at a location, such as a user making a QR code payment at a location, or a user making a facial recognition payment at a location.

[0065] In this specification, the heterogeneous graph of user behavior relationships can be represented as G = {V, E, X}. nd ,X eg}, where V represents the set of nodes and E represents the set of edges. Represents the node attribute matrix, and The edge attribute matrix is ​​represented. The heterogeneous graph G representing user behavior relationships can also be associated with a node type mapping function Φ:V→A and an edge type mapping function Ψ:E→R, where A represents the set of node types and R represents the set of edge types (a set of heterogeneous relationship types). In this specification, A may, for example, consist of users and locations, and R may, for example, include QR code payment and facial recognition payment.

[0066] Figure 4 An example schematic diagram of a heterogeneous graph of user behavior relationships according to this specification is shown. Figure 4 In the example, edge features represent the strength of the relationship between users and venues, such as payment frequency and payment amount. These are key indicators characterizing the relationship between users and venues.

[0067] In one example, a machine learning model can be used to predict the matching between the target user account, the first location, and the biometrics of the user to be identified, based on a heterogeneous graph of user behavior relationships, the biometrics of the user to be identified, and the biometrics of the target user account.

[0068] Figure 5 An example flowchart of a user authentication process 500 according to an embodiment of this specification is shown. Figure 5 In the example, the machine learning model may include a first machine learning model, a second machine learning model, and an aggregation layer.

[0069] In 510, the first machine learning model is used to model the target user account and the first place based on the heterogeneous graph of user behavior relationship, so as to obtain the embedded representation of the target user account and the first place under various heterogeneous relationships.

[0070] In one example, the first machine learning model may include a graph neural network. A graph neural network is a network used to extract node feature representations (embeddings) from graph-structured data. Figure 6 An example schematic diagram of a graph neural network according to an embodiment of this specification is shown.

[0071] like Figure 6 As shown, a graph neural network can include multiple hidden layers, such as... Figure 6 The graph neural network consists of hidden layers 1 through N. In each hidden layer, for each graph node, it receives and aggregates the node embedding representations output from the previous hidden layer of each of its neighboring nodes, generating a new node embedding representation for that hidden layer. This generated node embedding representation is then propagated to the neighboring nodes in the next hidden layer. This process continues layer by layer, with the node embedding representation obtained in the last layer being output as the final node embedding representation after processing by the graph neural network. Here, the term "hidden layer" can also be called a "message passing layer."

[0072] When using a first-order machine learning model to model target user accounts and primary locations based on a heterogeneous graph of user behavior relationships, information from the user account's first-order locations and higher-order neighbors can be propagated through heterogeneous relationships, thereby capturing the user account's behavioral preferences in offline scenarios. However, heterogeneous relationships have different characteristics. For example, facial recognition payment relationships are relatively scarce but more directly indicate the user's facial recognition payment trend. Conversely, QR code payment relationships are richer and can provide more information about the user's consumption habits. Considering their different statistical properties, information propagation under each type of heterogeneous relationship can be modeled separately, and then the embedded representations learned from different heterogeneous relationships can be aggregated.

[0073] For each heterogeneous relationship, user and location embedding representations can be constructed based on that heterogeneous relationship. Then, the information propagation concept of graph neural networks is used to aggregate neighbor information layer by layer to obtain the user or location embedding representation. The following example demonstrates learning user embedding representations based on QR code payment relationships. Let's take an example to illustrate.

[0074] In one example, the graph neural network can be an attention-based graph neural network. In this case, the target user account and the first location can be modeled according to the following formula (1):

[0075]

[0076] in, and These represent the embeddings of the l-th hidden layer nodes of the target user account u and the first venue v, respectively. and Define the neighbor nodes of the target user account u and the first venue v in the QR code payment relationship, respectively. n This represents the input features of node n. Representing edge e u,v In the context of QR code payment relationships, the edge feature g(·) represents a layer-level perceptual projection function with ReLU as the activation function. and It is a learnable matrix of node paths and edge paths. and These are node-level attention values ​​and edge-level attention values. Note that n represents user u or place v. To learn user embeddings, n represents user u if the layer number is even, and place v otherwise.

[0077] Considering that different neighbors of a node and an edge should have different importance for different target nodes, node-level attention and edge-level attention can be introduced. Node-level attention aims to measure the importance of neighboring nodes to the target node and aggregate information about neighboring nodes. Simultaneously, edge-level attention aims to measure the importance of neighboring edges to the target node and aggregate edge information. Specifically, the node (edge)-level attention score is calculated by matching the embedding representation of node n1 with the embedding representation of node n2 or an edge. The embedding characterization yields:

[0078]

[0079]

[0080] Where ·T is the transpose, W attn a is used to map node embedding representations or edge embedding representations to the attention space. attn (b attn ) is used to map uniform embedding representations to attention scores.

[0081] Attention weight The attention scores on all selected neighbors can be normalized using the softmax function.

[0082]

[0083]

[0084] According to the information propagation rules expressed in formula (1), the user embedding representation on the QR code payment path can be obtained. Similarly, user embedding representations along the facial recognition payment path can be obtained. and site embedding representation under these two heterogeneous relationships. and

[0085] In 520, a second machine learning model is used to model the biometric data of the user to be identified and the biometric data of the target user account to obtain a biometric embedding representation that reflects the similarity between the biometric data of the user to be identified and the biometric data of the target user account.

[0086] In one example, the second machine learning model can be implemented using a multilayer perceptron (e.g., the Wide model). When using a multilayer perceptron for modeling, the input features can include features strongly correlated with the current heterogeneous relationship (e.g., facial recognition payment), such as: historical facial recognition information of the user and the device, other correlation information, and real-time features of the current facial recognition payment, such as facial recognition score and location information. This modeling can enhance the memory of facial recognition behavior while giving the model a certain degree of interpretability.

[0087] For example, the Wide model can be used to model the biometric data of the user to be identified and the biometric data of the target user's account separately. The advantage of the Wide model is its good memory, effectively preserving the importance of biometrics for user matching. The Wide model can be characterized as a generalized linear model as follows:

[0088] h wide =W wd x wd +b wd ,

[0089] Among them, h wide Represents biological embedding features, x wd W represents the original visual features of the Wide model. wd Let b represent the weight matrix. wd This represents the bias vector of the Wide model.

[0090] In step 530, the embedding representations of the target user account and the first location under various heterogeneous relationships, as well as the biometric embedding representations, are provided to the aggregation layer to predict the matching between the target user account and the biometrics of the first location and the user to be identified.

[0091] The aggregation layer is used to aggregate user embedding representations, location embedding representations, and biometric embedding representations. Considering that different types of nodes, different types of heterogeneous relationships, and biometric data contribute differently to different offline recognition scenarios, the aggregation layer used can include a type-level attention-based aggregation layer. This aggregation layer can aggregate different types of embedding representations according to the following formula:

[0092]

[0093] in,

[0094] In addition, the softmax function can be used to obtain the attention weights for each type of embedding representation:

[0095]

[0096] Then, all types of embedded representations are cascaded and projected using MLP according to Equation (2) into predicted values ​​that reflect the matching between the target user account and the biometrics of the first location and the user to be identified.

[0097]

[0098] In some embodiments, the first machine learning model, the second machine learning model, and the aggregation layer can be trained as a single model. During model training, the cross-entropy loss function can be used as the overall loss function. For example, the overall loss function can be represented as follows:

[0099]

[0100] Among them, y uvf ∈{0,1} are the true labels of the triples (user u, location v, and face f), and Tr is the training set.

[0101] In some embodiments, the total loss function for model training may include a loss function with L2 regularization applied to the model parameters. This total loss function helps avoid overfitting and optimizes the overall model using mini-batch stochastic gradient descent.

[0102] Figure 7 An example schematic diagram of a user verification process according to an embodiment of this specification is shown.

[0103] like Figure 7 As shown, a first machine learning model, such as a graph neural network, is used to iteratively obtain user embedding representations on the QR code payment path and the facial recognition payment path. and place embedding representation Furthermore, a second machine learning model, such as a multilayer perceptron model, is used to obtain the facial visual feature embedding representation h. wide Then, the user is embedded in the representation. Place Embedding Representation and facial visual feature embedding representation h wide After cascading, the results are provided to the MLP in the aggregation layer to predict the matching scores (i.e., predictions) of the triples (user u, location v, and face f).

[0104] Figure 8 Another example flowchart of a user verification process 800 according to an embodiment of this specification is shown.

[0105] like Figure 8 As shown, at 810, based on the heterogeneous graph of user behavior relationships, the matching degree between the target user account and the first location is determined, and at 820, it is determined whether the target user account and the first location match. For example, a graph neural network can be used to predict the matching degree between the target user account and the first location based on the heterogeneous graph of user behavior relationships, that is, to predict the probability (likelihood) of the target user account accessing the first location. If the result at 820 is a mismatch, then at 860, it is determined that user identification has failed.

[0106] If the result at step 820 is a match, then at step 830, based on the biometric data of the target user account and the biometric data of the user to be identified, the matching degree between the biometric data of the target user account and the user to be identified is determined, and at step 840, it is determined whether the biometric data of the target user account and the user to be identified match. For example, the similarity between the facial visual feature data of the target user account and the facial visual feature data of the user to be identified can be determined based on the facial visual feature data of the target user account and the user to be identified. If the similarity is not less than a predetermined threshold, then the facial visual feature data of the target user account and the user to be identified are determined to match. If the similarity is less than the predetermined threshold, then the facial visual feature data of the target user account and the user to be identified are determined to not match.

[0107] If the result at 840 is a match, then at 850, the user identification is confirmed to have passed. If the result at 840 is a non-match, then at 860, the user identification is confirmed to have failed.

[0108] As per the above reference Figures 1 to 8 An offline user identification method according to embodiments of this specification is described.

[0109] Using the aforementioned offline user identification method, historical location access information of users is mined to construct a heterogeneous relationship graph between users and locations. During offline user identification, in addition to biometric verification, the constructed heterogeneous relationship graph is used to determine whether the retrieved user account will visit the current location for supplementary verification. That is, the matching degree between the input user's biometrics, the retrieved user account, and the location to be visited is determined. This allows for user identification to be assisted based on the user's offline behavior and habits, thereby improving the accuracy of offline user identification.

[0110] By utilizing the aforementioned offline user identification method, and through the creation and maintenance of local and global candidate user account databases, and employing a local-global retrieval mechanism, the retrieval efficiency of target user accounts can be improved. Furthermore, by limiting the local candidate user account database to a database based on the biometric data of users who visited the location within a specified timeframe, the created local candidate user account database has a time limitation, reducing its size and further improving the retrieval efficiency of target user accounts. Moreover, by limiting the local candidate user account database for each location to a database constructed based on historical access data from that location and historical access data from other locations within a given distance, the created local candidate user account database exhibits expanded spatial locality, thereby improving the retrieval hit rate of target user accounts.

[0111] By using the aforementioned offline user identification method, and by modeling graph information and biometric data separately, the embedding and representation efficiency of user, location, and biometric data can be improved.

[0112] By utilizing the aforementioned discrete user identification method, training the first machine learning model, the second machine learning model, and the aggregation layer together as a single model can improve the overall identification accuracy of these three models. Furthermore, introducing L2 regularization into the total loss function during model training avoids overfitting, and mini-batch stochastic gradient descent is used to optimize the overall model.

[0113] Figure 9 An example block diagram of a discrete user identification device 900 according to an embodiment of this specification is shown. Figure 9 As shown, the discrete user identification device 900 includes a biometric data acquisition unit 910, a target user account retrieval unit 920, and a user identification unit 930.

[0114] The biometric data acquisition unit 910 is configured to acquire the biometric data of the user to be identified at the first location. The operation of the biometric data acquisition unit 910 can be referenced above. Figure 2 The operation described in 210.

[0115] The target user account retrieval unit 920 is configured to retrieve target user accounts from the candidate user account database based on the acquired biometric data. The operation of the target user account retrieval unit 920 can be referenced above. Figure 2 220 and Figure 3 The described operation.

[0116] User identification unit 930 is configured to determine the matching between the target user account, the first location, and the biometric data of the user to be identified based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, in order to perform user identification. The operation of user identification unit 930 can be referenced above. Figure 2 230 and Figures 4-8 The described operation.

[0117] Figure 10 An example block diagram of a target user account retrieval unit 1000 according to an embodiment of this specification is shown. Figure 10 In the example, the candidate user account database includes a local candidate user account database and a global candidate user account database. For example... Figure 10 As shown, the target user account retrieval unit 1000 includes a local retrieval module 1010 and a global retrieval module 1020.

[0118] The local retrieval module 1010 is configured to retrieve the target user account from the corresponding candidate user account local database in the first location based on the acquired biometric data.

[0119] The global retrieval module 1020 is configured to retrieve the target user account from the global database of candidate user accounts based on the acquired biometric data in response to the failure to retrieve a matching target user account from the local database of candidate user accounts.

[0120] In one example, the user identification unit is configured to use a machine learning model to predict the matching between the target user account, the first location, and the biometric data of the user to be identified, based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account.

[0121] Figure 11 An example block diagram of a user identification unit 1100 according to an embodiment of this specification is shown. Figure 11As shown, the user identification unit 1100 includes a first embedded representation generation module 1110, a second embedded representation generation module 1120, and a prediction module 1130.

[0122] The first embedding representation generation module 1110 is configured to use a first machine learning model to model the target user account and the first location based on a heterogeneous graph of user behavior relationships, so as to obtain the embedding representations of the target user account and the first location under various heterogeneous relationships. The operation of the first embedding representation generation module 1110 can be referred to the above reference. Figure 5 The operation described in 510.

[0123] The second embedding representation generation module 1120 is configured to use a second machine learning model to model the biometric data of the user to be identified and the biometric data of the target user account, in order to obtain a biometric embedding representation that reflects the similarity between the biometric data of the user to be identified and the biometric data of the target user account. The operation of the second embedding representation generation module 1120 can be referenced above. Figure 5 The operation described in 520.

[0124] The prediction module 1130 is configured to provide the aggregation layer with the embedded representations of the target user account and the first location under various heterogeneous relationships, as well as the biometric embedding representations, to predict the matching degree between the target user account and the biometric data of the first location and the user to be identified. The operation of the prediction module 1130 can be referenced above. Figure 5 The operation described in 530.

[0125] Figure 12 Another example block diagram of a user identification unit 1200 according to an embodiment of this specification is shown. Figure 12 As shown, the user identification unit 1200 includes a first matching module 1210, a second matching module 1220, and a user identification module 1230.

[0126] The first matching module 1210 is configured to determine the matching between the target user account and the first location based on a heterogeneous graph of user behavior relationships. The operation of the first matching module 1210 can be referenced above. Figure 8 The operation described in 810.

[0127] The second matching module 1220 is configured to determine the matching degree between the biometric data of the target user account and the biometric data of the user to be identified, based on the biometric data of the target user account and the biometric data of the user to be identified. The operation of the second matching module 1220 can be referenced above. Figure 8 The operation described in 830.

[0128] The user identification module 1230 is configured to determine that user identification is successful in response to a match between the target user account and the biometric data of the first location and the user to be identified, and to determine that user identification is unsuccessful in response to a mismatch between the target user account and the biometric data of the first location and the user to be identified.

[0129] As per the above reference Figures 1 to 12 This specification describes a discrete user identification method and a discrete user identification apparatus according to embodiments thereof. The discrete user identification apparatus described above can be implemented in hardware, software, or a combination of hardware and software.

[0130] Figure 13 An example schematic diagram of a discrete user identification device 1300 implemented based on a computer system according to an embodiment of this specification is shown. Figure 13 As shown, the discrete user identification device 1300 may include at least one processor 1310, a memory (e.g., non-volatile memory) 1320, a main memory 1330, and a communication interface 1340, and the at least one processor 1310, memory 1320, main memory 1330, and communication interface 1340 are connected together via a bus 1360. At least one processor 1310 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0131] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 1310 to: acquire biometric data of a user to be identified at a first location; retrieve a target user account from a candidate user account database based on the acquired biometric data; and determine the matching between the target user account, the first location, and the biometric data of the user to be identified based on a user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account, for user identification, wherein the user behavior relationship heterogeneity graph is constructed based on the user account's location history access data.

[0132] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1310 to perform the above-described combinations in the various embodiments of this specification. Figures 2-12 The description includes various operations and functions.

[0133] According to one embodiment, a program product, such as a machine-readable medium (e.g., a non-transitory machine-readable medium), is provided. The machine-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a machine, cause the machine to perform the above-described combinations of the various embodiments of this specification. Figures 2-12The various operations and functions described. Specifically, a system or apparatus equipped with a readable storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer or processor of the system or apparatus to read and execute the instructions stored in the readable storage medium.

[0134] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0135] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0136] According to one embodiment, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, causes the processor to perform the above-described combinations of the various embodiments of this specification. Figures 2-12 The description includes various operations and functions.

[0137] Those skilled in the art will understand that the various embodiments disclosed above can be modified and varied without departing from the spirit of the invention. Therefore, the scope of protection of this invention should be defined by the appended claims.

[0138] It should be noted that not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above embodiments can be a physical structure or a logical structure; that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0139] In the above embodiments, the hardware units or modules can be implemented mechanically or electrically. For example, a hardware unit, module, or processor may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit or processor may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0140] The specific embodiments described above with reference to the accompanying drawings are exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of the claims. The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" compared to other embodiments. Specific details are included to provide an understanding of the described techniques. However, these techniques can be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0141] The foregoing description of this disclosure is provided to enable any person skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. An offline user identification method, comprising: Acquire biometric data of the user to be identified at the first location; Based on the acquired biometric data, the target user account is retrieved from the candidate user account database; as well as Based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, the matching degree between the target user account, the first location, and the biometric data of the user to be identified is determined for user identification. The heterogeneous graph of user behavior relationships is constructed based on the user account's location history access data. Specifically, determining the matching between the target user account, the first location, and the biometric data of the user to be identified, based on the heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, includes: The first machine learning model is used to model the target user account and the first location based on the heterogeneous graph of user behavior relationship, so as to obtain the embedding representation of the target user account and the first location under various heterogeneous relationships. A second machine learning model is used to model the biometric data of the user to be identified and the biometric data of the target user account to obtain a biometric embedding representation that reflects the similarity between the biometric data of the user to be identified and the biometric data of the target user account; and The embedding representations of the target user account and the first location under various heterogeneous relationships, as well as the biometric embedding representations, are provided to the aggregation layer to predict the matching degree between the target user account and the biometric data of the first location and the user to be identified.

2. The offline user identification method as described in claim 1, wherein, The first machine learning model includes a graph neural network, and the second machine learning model includes a multilayer perceptron model.

3. The offline user identification method as described in claim 2, wherein, The graph neural network includes graph neural networks based on attention mechanisms.

4. The offline user identification method as described in claim 1, wherein, The aggregation layer includes an attention-based aggregation layer.

5. The offline user identification method as described in claim 1, wherein, The first machine learning model, the second machine learning model, and the aggregation layer are trained as a single model.

6. The offline user identification method as described in claim 5, wherein, The total loss function for model training includes the cross-entropy loss function.

7. The offline user identification method as described in claim 5, wherein, The total loss function for model training includes a loss function with L2 regularization applied to the model parameters.

8. The offline user identification method as described in claim 1, wherein, The candidate user account database includes a local candidate user account database and a global candidate user account database. Based on the acquired biometric data, the target user account is retrieved from the candidate user account database, including: Based on the acquired biometric data, the target user account is retrieved from the corresponding local database of candidate user accounts at the first location; and In response to the failure to find a matching target user account in the local database of candidate user accounts, the target user account is retrieved in the global database of candidate user accounts based on the acquired biometric data.

9. The offline user identification method as described in claim 8, wherein, Each candidate user account local database consists of a candidate user account database constructed based on historical access data at the corresponding location and historical access data at other locations within a given distance from the corresponding location.

10. The offline user identification method as described in claim 1, wherein, The biometric data includes visual feature data of facial information.

11. An offline user identification method, comprising: Acquire biometric data of the user to be identified at the first location; Based on the acquired biometric data, the target user account is retrieved from the candidate user account database; as well as Based on a heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, the matching degree between the target user account, the first location, and the biometric data of the user to be identified is determined for user identification. The heterogeneous graph of user behavior relationships is constructed based on the user account's location history access data. Specifically, determining the matching between the target user account, the first location, and the biometric data of the user to be identified, based on the heterogeneous graph of user behavior relationships, the biometric data of the user to be identified, and the biometric data of the target user account, includes: Based on the heterogeneous graph of user behavior relationships, the matching between the target user account and the first location is determined; Based on the biometric data of the target user account and the biometric data of the user to be identified, determine the matching degree between the biometric data of the target user account and the biometric data of the user to be identified; and In response to the target user account matching the biometric data of both the first location and the user to be identified, it is determined that the user identification is successful.

12. An offline user identification device, comprising: The biometric data acquisition unit acquires the biometric data of the user to be identified at the first location. The target user account retrieval unit retrieves target user accounts from the candidate user account database based on the acquired biometric data. as well as The user identification unit determines the matching degree between the target user account, the first location, and the biometric data of the user to be identified, based on a user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account, in order to perform user identification. The user behavior relationship heterogeneity graph is constructed based on the user account's location history access data. The user identification unit includes: The first embedding representation generation module uses a first machine learning model to model the target user account and the first location based on a heterogeneous graph of user behavior relationships, so as to obtain the embedding representations of the target user account and the first location under various heterogeneous relationships. The second embedding representation generation module uses a second machine learning model to model the biometric data of the user to be identified and the biometric data of the target user account, to obtain a biometric embedding representation reflecting the similarity between the biometric data of the user to be identified and the biometric data of the target user account; and The prediction module provides the embedding representations of the target user account and the first location under various heterogeneous relationships, as well as the biometric embedding representations, to the aggregation layer to predict the matching degree between the target user account and the biometric data of the first location and the user to be identified.

13. The offline user identification device as claimed in claim 12, wherein, The candidate user account database includes a local candidate user account database and a global candidate user account database. The target user account retrieval unit includes: The local retrieval module, based on the acquired biometric data, retrieves the target user account from the corresponding candidate user account local database at the first location; and The global retrieval module, in response to the failure to find a matching target user account from the local database of candidate user accounts, retrieves the target user account from the global database of candidate user accounts based on the acquired biometric data.

14. An offline user identification device, comprising: The biometric data acquisition unit acquires the biometric data of the user to be identified at the first location. The target user account retrieval unit retrieves target user accounts from the candidate user account database based on the acquired biometric data. as well as The user identification unit determines the matching degree between the target user account, the first location, and the biometric data of the user to be identified, based on a user behavior relationship heterogeneity graph, the biometric data of the user to be identified, and the biometric data of the target user account, in order to perform user identification. The user behavior relationship heterogeneity graph is constructed based on the user account's location history access data. The user identification unit includes: The first matching module determines the matching degree between the target user account and the first location based on the heterogeneous graph of user behavior relationships; The second matching module determines the matching degree between the biometric data of the target user account and the biometric data of the user to be identified, based on the biometric data of the target user account and the biometric data of the user to be identified; and The user identification module determines that user identification is successful when the target user account matches the biometric data of both the first location and the user to be identified.

15. An offline user identification device, comprising: At least one processor, Memory coupled to the at least one processor, and A computer program stored in the memory, which is executed by the at least one processor to implement the offline user identification method as described in any one of claims 1 to 11.

16. A computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the offline user identification method as described in any one of claims 1 to 11.

17. A computer program product comprising a computer program executed by a processor to implement the offline user identification method as described in any one of claims 1 to 11.

Images