User biological feature vector clustering method and device
By combining approximate nearest neighbor search and community discovery algorithms, the topological map of biometric vectors is constructed, which solves the problem of insufficient recognition accuracy and calculation amount in biometric recognition technology, and achieves efficient and accurate clustering results, which are suitable for large-scale Large-scale users and massive data scenarios.
Patent Information
- Application Number
- CN202411104596.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-08
AI Technical Summary
The existing biometric recognition technology has shortcomings in recognition accuracy and calculation amount, especially in large-scale users and massive data scenarios, the calculation amount of conventional clustering processing is too large, making it difficult to achieve efficient identification.
The topology map of bioeigenvectors is constructed using the approximate nearest neighbor search ANNS algorithm, and combined with the community discovery algorithm, high discrimination and available clustering results are obtained through clustering processing, reducing the computational amount. The specific steps include using the hierarchable small world HNSW algorithm to build an ANNS graph, processing community discovery based on the Louvain algorithm, and performing similarity calculation and filtering.
While significantly reducing the calculation amount, it realizes high discrimination and strong usability of similar biometric data, improves the accuracy of biometric recognition, supports incremental updates, and is suitable for large-scale users and massive data scenarios. .
Smart Images

Figure CN118918348A_ABST
Abstract
Description
Technical Field
[0001] One or more embodiments of the present specification relate to the field of machine learning technology, and more particularly, to a method and device for clustering user biometric feature vectors. Background Art
[0002] Biometric Recognition Technology is a type of technology that uses human biological characteristics, such as fingerprints, irises, faces, and voices, to authenticate and identify identities. The main advantages of this type of technology are: 1) High security: biometric characteristics are difficult to copy or forge, which improves security; 2) Convenience: users do not need to remember passwords or carry additional equipment, making it easy to use; 3) Uniqueness: everyone's biometric characteristics are unique.
[0003] Therefore, the embodiments of this specification propose an improvement scheme that can meet higher requirements for biometric recognition technology in practical applications, for example, effectively improve recognition accuracy, etc. Summary of the invention
[0004] The embodiments of this specification describe a method and device for clustering user biometric feature vectors, which can effectively improve the accuracy of biometric feature recognition.
[0005] According to a first aspect, a clustering method for user biometric feature vectors is provided. The method comprises: using an approximate nearest neighbor search ANNS algorithm to construct an ANNS graph between biometric feature vectors, wherein the biometric feature vectors corresponding to some nodes carry a user ID; processing the ANNS graph based on a community discovery algorithm to obtain multiple communities, wherein each community includes a number of node identifiers; for each community, performing similarity calculation and filtering processing on other nodes based on the nodes having the user ID therein, thereby obtaining a clustering cluster corresponding to the user ID.
[0006] In one embodiment, the ANNS algorithm is a hierarchical navigable small-world HNSW algorithm, and the ANNS graph is a HNSW graph; the construction of the HNSW graph includes: grouping a plurality of biometric feature vectors according to the collection areas of the corresponding plurality of original biometric features; selecting a plurality of biometric feature vectors from the groups corresponding to the respective collection areas as entry points of the HNSW algorithm, thereby executing the HNSW algorithm to obtain the HNSW graph.
[0007] Furthermore, in a specific embodiment, it also includes: in the process of executing the HNSW algorithm, the similarity between the biometric feature vectors is used as the weight of the connection edge between the corresponding nodes.
[0008] In one embodiment, the community discovery algorithm is a Louvain algorithm; wherein, the ANNS graph is processed based on the community discovery algorithm to obtain multiple communities, including: based on the user IDs corresponding to the partial nodes, merging multiple nodes corresponding to the same user ID in the ANNS graph into aggregate nodes to obtain a simplified ANNS graph; using the Louvain algorithm to process the simplified ANNS graph to obtain multiple first communities; restoring the aggregate nodes in the multiple first communities to original node IDs to obtain multiple second communities as the multiple communities.
[0009] Furthermore, in a specific embodiment, obtaining a simplified ANNS graph further includes: reconstructing the connecting edges between the multiple nodes into self-loop edges of the aggregated node, the weight of the self-loop edge being the sum of the weights of the connecting edges between the multiple nodes.
[0010] In another specific embodiment, obtaining the simplified ANNS graph further includes: reconstructing the connection edges between the multiple nodes and other nodes into connection edges between the aggregation node and other nodes.
[0011] On the other hand, in a specific embodiment, after obtaining the clustering class corresponding to the target user ID, the method further includes: based on a batch of newly added biometric feature vectors, continuing to execute the ANNS algorithm on the basis of the ANNS graph to obtain a new ANNS graph; according to the multiple second communities, merging the nodes belonging to the same community in the new ANNS graph into aggregate nodes to obtain a simplified new ANNS graph; processing the simplified new ANNS graph using the Louvain algorithm to obtain multiple third communities; restoring the aggregate nodes in the multiple third communities to the original node IDs to obtain multiple fourth communities as multiple new communities; for each new community, performing similarity calculation and filtering processing on other nodes based on the nodes with the user ID therein, thereby obtaining the clustering class corresponding to the user ID.
[0012] According to a second aspect, a device for clustering user biometric feature vectors is provided. The device includes: a graph construction module configured to use an approximate nearest neighbor search ANNS algorithm to construct an ANNS graph between biometric feature vectors, wherein the biometric feature vectors corresponding to some nodes carry a user ID; a community discovery module configured to process the ANNS graph based on the community discovery algorithm to obtain multiple communities, wherein each community includes a number of node identifiers; and a filtering module configured to perform similarity calculation and filtering processing on other nodes based on the nodes with the user ID in each community, thereby obtaining a cluster corresponding to the user ID.
[0013] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method provided in the first aspect.
[0014] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method provided in the first aspect is implemented.
[0015] In summary, the above method and device disclosed in the embodiments of this specification are used to innovatively propose the establishment of a topological map of user biometric data and the application of a community discovery algorithm to the clustering scenario of biometrics. This can significantly reduce the amount of calculation while obtaining clustering results with high discrimination and high usability for similar biometric data. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0017] Figure 1 This is an implementation architecture diagram of the user biometric data clustering solution disclosed in the embodiments of this specification;
[0018] Figure 2 A schematic diagram of the process steps of the method for clustering user biometric feature vectors disclosed in an embodiment of this specification;
[0019] Figure 3 A simple example of a HNSW graph;
[0020] Figure 4 This is a functional structure diagram of the device for clustering user biometric feature vectors disclosed in an embodiment of this specification. DETAILED DESCRIPTION
[0021] The solution provided in this specification is described below in conjunction with the accompanying drawings.
[0022] As mentioned above, biometric identification technology is a type of technology that uses human biological features, such as fingerprints, faces, etc., to perform identity authentication and recognition. For ease of understanding, the following embodiments are mainly described by taking the biometric feature being a face image as an example.
[0023] Many payment platforms support users to pay by face recognition. Typically, the payment platform pre-associates and stores the platform account and facial feature vector of each user. The facial feature vector can be obtained by the payment platform using a machine learning model to process the facial image entered by the user when the user activates the facial recognition payment function; later, when the user initiates facial recognition payment based on his login account, the payment platform uses a machine learning model to process the real-time collected facial image to obtain the corresponding facial feature vector, and calls the pre-stored facial feature vector associated with the login account to compare it, and then initiates a deduction if the comparison is successful, otherwise the payment process is terminated.
[0024] However, if only a single facial feature vector corresponding to a single facial image is stored for a single user, the robustness of the comparison result will be poor. Ideally, for each user, it is hoped that multiple facial feature vectors corresponding to his or her facial images of different environments, angles, or expressions can be stored in advance, so as to improve the robustness and usability of the comparison result.
[0025] However, it is difficult to collect rich and varied facial images with known identities in reality. However, there are a large number of facial images with unknown identities (such as identity ID lost or low confidence). Therefore, it is hoped that these facial images can be used to enrich the facial feature vector set corresponding to each user. It can be understood that the identity ID may be replaced by the user ID in this article, where ID is the abbreviation of the English word for the Chinese word "identification". It should be emphasized that all user data used in this article, including biometrics, are collected and used after the user confirms the authorization.
[0026] Therefore, it is proposed to mix the face images with known identities and the face images with unknown identities together for clustering processing, so that the face images with the same identity (or corresponding to the same user) are classified into a cluster (or divided into a group or a collection). The face images with known identities in any cluster can identify the user identities corresponding to all the face images in this cluster, thereby realizing the effective utilization of the massive face images with unknown identities.
[0027] Obviously, the accuracy of face clustering determines the accuracy of subsequent face recognition. Generally, existing clustering algorithms, such as k-means clustering and spectral clustering, can be directly used to cluster the face feature vectors corresponding to the face image to obtain multiple clusters. However, in actual business scenarios, the number of users on the payment platform is large, such as hundreds of millions, and the magnitude of related face images is even greater, which will cause the conventional clustering processing to be too computationally intensive and difficult to implement.
[0028] Based on the above observations and analysis, the applicant proposed a newly designed clustering algorithm for user biometric data (such as face images, face feature vectors, etc.), which can significantly reduce the amount of calculation while obtaining clustering results with high discrimination and strong usability for similar biometric data (such as similar faces, etc.).
[0029] Figure 1 This is a diagram of the implementation architecture of the user biometric data clustering solution disclosed in the embodiments of this specification. Figure 1 As shown, first, a relationship network graph G is constructed based on multiple user biometric data. Different nodes in the graph G correspond to different user biometric data, and connection edges are established between different nodes because the corresponding biometric features have high similarities; then, the community discovery algorithm is used to process the relationship network graph G to obtain a clustering result table; finally, the clustering result table is filtered using biometric data with known identities to obtain the final clustering result.
[0030] In this way, the applicant pioneered the idea of establishing a topological map of user biometric data and applying the community discovery algorithm to the biometric clustering scenario, so that the user biometrics can be pre-clustered quickly and efficiently. After that, it is only necessary to calculate and filter the similarity within the cluster based on the pre-clustering results, thereby achieving high-quality clustering results with a lower amount of calculation.
[0031] Combine the following Figure 2 And more embodiments, introducing Figure 1 The specific implementation steps of the clustering scheme are shown in FIG. Figure 2 This is a schematic diagram of the process steps of the method for clustering user biometric feature vectors disclosed in an embodiment of this specification. The execution subject of the method can be any device, server or equipment cluster with computing and processing capabilities. Figure 2 The following method steps are shown:
[0032] Step S210, using the approximate nearest neighbor search ANNS algorithm to construct an ANNS graph between biometric feature vectors, wherein the biometric feature vectors corresponding to some nodes carry user IDs; step S220, processing the ANNS graph based on the community discovery algorithm to obtain multiple communities, wherein each community includes several node identifiers; step S230, for each community, performing similarity calculation and filtering processing on other nodes based on the nodes with user IDs therein, thereby obtaining a cluster corresponding to the user ID.
[0033] The above steps are expanded as follows:
[0034] First, in step S210, an approximate nearest neighbor search (ANNS) algorithm is used to construct an ANNS graph between biometric feature vectors, wherein the biometric feature vectors corresponding to some nodes carry user IDs.
[0035] To help understand, the following is a brief introduction to the ANNS (Approximate Nearest Neighbor Search) algorithm. Generally, the input of the ANNS algorithm includes a query vector and multiple vectors to be retrieved, where the multiple vectors to be retrieved constitute the search space. The main goal of the ANNS algorithm is to quickly find one or a group of vectors that are closest to the query vector from the search space. It should be understood that the algorithm allows a certain degree of error in the query results.
[0036] To achieve its goal, the ANNS algorithm first builds some kind of acceleration data structure, such as a graph, tree, or hash table, to increase the query speed during execution, and then uses the query vector to search in the constructed acceleration data structure to quickly find the approximate nearest neighbor vector.
[0037] In this step, only the ANNS algorithm is used to construct the graph, and no subsequent query in the constructed graph is involved. Specifically, the ANNS algorithm is used to process multiple biometric feature vectors to obtain a constructed ANNS graph.
[0038] Furthermore, the above-mentioned multiple biometric feature vectors can be obtained by processing the collected multiple original biometric feature data (such as multiple face images) using a trained machine learning model. In addition, some of the multiple original biometric feature data carry high-confidence identity IDs, while others do not have identity IDs (or the identity IDs have low confidence, which is equivalent to no identity IDs). For this, please refer to Figure 1 .
[0039] On the other hand, the above-constructed ANNS graph includes multiple nodes, each of which corresponds to a biological feature vector; the establishment of connecting edges between nodes in the ANNS algorithm is usually based on similarity calculations, and these similarity calculations can use various distance measurement methods, such as Euclidean distance, cosine similarity, Manhattan distance, etc.
[0040] It should be noted that the construction of the ANNS graph usually does not perform pairwise similarity calculations on all nodes, but instead uses some strategies to achieve efficient graph construction, such as local search and neighborhood selection, random projection and hashing, partitioning and stratification, nearest neighbor propagation, etc. Through these strategies, the ANNS algorithm can significantly reduce the amount of calculation in the graph construction stage and improve efficiency while maintaining high search accuracy.
[0041] According to one embodiment, in this step, the Hierarchical Navigable Small World (HNSW) algorithm can be selected as a specific implementation of the ANNS algorithm. It should be understood that the HNSW algorithm is highly scalable and very suitable for processing large-scale data sets. When constructing the HNSW graph, new nodes are connected to local neighboring nodes and locally optimized; each time a new node is inserted, the connection status of its neighboring nodes is updated to ensure that the structure of the graph remains efficient.
[0042] Figure 3 The constructed HNSW graph is briefly illustrated, where the HNSW graph is composed of multiple levels, each of which is an independent small-world graph. At the high level (top level), the graph is relatively sparse, with fewer nodes and fewer connected edges, which facilitates quick navigation. At the low level (bottom level), the graph gradually becomes denser, with more nodes and more connected edges, providing a more refined nearest neighbor search.
[0043] It is important to understand that Figure 3 The dotted line in connects the same nodes in two adjacent layers. For any node, its highest level in the HNSW graph is generally determined randomly, and once its highest level is determined, it will appear in that level and all levels below it. Accordingly, in the process of constructing the HNSW graph, the selection of the entry point in the highest level is generally random. In this regard, the applicant proposes that the selection of the entry point can be optimized, thereby further accelerating the construction of the HNSW graph and reducing the amount of calculation.
[0044] Specifically, it is proposed that before constructing the HNSW graph, multiple biometric feature vectors (such as face feature vectors) are grouped according to the collection areas (such as provinces) of the corresponding multiple original biometric features; then, several biometric feature vectors are selected from the groups corresponding to each collection area as the entry points of the HNSW algorithm. In this way, considering that user biometric features usually have certain regional similarities from a biological perspective, combined with the characteristic that the HNSW algorithm starts searching from the entry point when inserting a new node in the graph composition stage, it is proposed to select the entry point by region, which can effectively improve the execution efficiency of the HNSW algorithm and improve the usability of the constructed HNSW graph.
[0045] In addition, considering that in the process of constructing the HNSW graph, similarity is generally not explicitly used as the weight of the connecting edge (or edge weight for short), and if edge weight can be used in the subsequent execution of the community discovery algorithm, the quality of the detected community can be effectively improved. Therefore, it is also proposed that in the process of constructing the HNSW graph, the calculated similarity (such as cosine similarity) is retained as the edge weight.
[0046] The above describes the construction of the HNSW graph. It should be understood that the ANNS graph constructed in this step can be, but is not limited to, an HNSW graph, and can also be other ANNS graphs such as a k-nearest neighbor graph, a small-world random graph without a hierarchical structure, etc.
[0047] Based on the ANNS graph constructed above, step S220 can be executed to process the ANNS graph based on a community discovery algorithm to obtain multiple communities, where each community includes multiple node identifiers. It should be noted that the multiple in the text can refer to one or more, but in actual situations it is generally multiple.
[0048] It should be understood that the community detection algorithm is a technique in graph theory, which is used to divide the nodes in the graph into different communities (or clusters). The nodes in the same community are more closely connected in the graph, and the nodes in different communities are less connected. Furthermore, the connecting edges between ANNS graphs are established based on similarity. Therefore, in the multiple communities obtained by processing the ANNS graph based on the community detection algorithm, the nodes in the same community have high similarity, the nodes in different communities have low similarity, and the biometric feature vectors in different communities have high discrimination.
[0049] For the multiple communities processed, see Figure 1 The clustering result table shown in , includes the node IDs in the ANNS graph and the community IDs corresponding to each node ID.
[0050] Common community discovery algorithms include but are not limited to the following categories: 1) Algorithms based on modularity optimization: Girvan-Newman algorithm, Louvain algorithm; 2) Algorithms based on spectral clustering; 3) Random walk and propagation algorithms; 4) Algorithms based on statistical models.
[0051] According to one embodiment, the above-mentioned Louvain algorithm with relatively good effect can be selected as the community discovery algorithm in this step. Further, the applicant proposes to perform a hot start on the execution of the algorithm according to the characteristics of the Lovain algorithm to reduce the number of steps of Louvain operation and improve the quality of the detected community.
[0052] Specifically, the implementation of Louvain hot start includes: based on the user ID (or identity ID) carried by some nodes in the ANNS graph, multiple nodes corresponding to the same user ID in the ANNS graph are merged into an aggregate node, thereby obtaining a simplified ANNS graph. Exemplarily, assuming that the user IDs corresponding to nodes 1 and 2 are both user 1, and the user IDs corresponding to nodes 3 and 4 are both user 2, at this time, nodes 1 and 2 can be merged into aggregate node a, and nodes 3 and 4 can be merged into aggregate node b.
[0053] Furthermore, in a specific embodiment, Louvain hot start, or obtaining the simplified ANNS graph mentioned above, also includes: reconstructing the connecting edges between multiple nodes corresponding to the same user ID into self-loop edges of the aggregation node, and the weight of the self-loop edge is the sum of the weights of the connecting edges between the multiple nodes.
[0054] In another specific embodiment, the implementation of Louvain hot start may further include: rebuilding the connection edges between multiple nodes corresponding to the same user ID and other nodes into connection edges between the aggregation node and other nodes.
[0055] The above can complete the warm start of the Louvain algorithm, and then the Louvain algorithm can be used to process the simplified ANNS graph to obtain multiple first communities. It should be noted that the "first" in "first community" and the "second", "third" and other similar terms in the text are used to distinguish similar things and do not have other limiting functions such as sorting.
[0056] Based on the obtained multiple first communities, each aggregation node therein is restored to an original node ID to obtain multiple second communities, which are used as the multiple communities finally obtained in this step.
[0057] From the above, we can get multiple communities in the ANNS graph.
[0058] Then, in step S230, for each community, similarity calculation and filtering processing are performed on other nodes based on the nodes having the user ID therein, so as to obtain the cluster corresponding to the user ID.
[0059] It should be understood that the nodes carrying user IDs in the ANNS graph are dispersed into multiple communities, and these nodes are equivalent to seed nodes. Therefore, seed nodes can be used to filter non-seed nodes, thereby improving the accuracy of the final clustering results, and further improving the accuracy of subsequent biometric recognition.
[0060] The similarity calculation in this step can be performed based on the biometric feature vector corresponding to the node, and the filtering rules can be set according to actual needs. For example, for a certain community, the similarity between a seed node and each non-seed node can be calculated respectively, and when the calculated similarity is less than a predetermined threshold (such as 0.9), the corresponding non-seed node is removed from the community.
[0061] In addition, for each community, the user ID of the seed node therein can be used as the user ID corresponding to the community.
[0062] From the above, multiple communities can be obtained after filtering, which can be used as the final multiple clusters.
[0063] In summary, the clustering method of user biometric vectors disclosed in the embodiments of this specification is innovatively proposed to establish a topological map of user biometric data, and apply the community discovery algorithm to the clustering scenario of biometrics. This can greatly reduce the amount of calculation while obtaining clustering results with high discrimination and strong usability for similar biometric data.
[0064] According to another aspect of the embodiment, for the multiple clusters obtained by the above clustering method, incremental update support is also provided. When a new biometric feature vector is added, it is not necessary to re-execute the above clustering method from the beginning together with the original biometric feature vector, but dynamic clustering with less computational effort can be performed, thereby accurately and efficiently realizing incremental update. The incremental update includes the following steps S31-S33:
[0065] Step S31, based on a batch of newly added biometric feature vectors, the ANNS algorithm is continued to be executed on the basis of the ANNS graph obtained in the above step S210 to obtain a new ANNS graph.
[0066] In one embodiment, the ANNS algorithm is implemented as an HNSW algorithm, and the ANNS graph obtained in step S210 is an HNSW graph. It should be understood that the HNSW graph supports dynamic expansion, that is, a batch of newly added biometric feature vectors can be used as a batch of new nodes and inserted into the existing HNSW graph to obtain a new HNSW graph.
[0067] Furthermore, in a specific embodiment, a batch of newly added biometric feature vectors can be grouped according to the collection area of the corresponding original biometric features, and then a number of newly added biometric feature vectors are selected from the groups corresponding to each collection area as entry points, so as to continue to execute the HNSW algorithm on the basis of the HNSW graph to obtain a new HNSW graph.
[0068] From the above, we can quickly get the new ANNS graph where the node is newly added.
[0069] Step S32: Process the new ANNS graph based on a community discovery algorithm to obtain multiple new communities, where each new community includes a number of node identifiers.
[0070] In one implementation, step S32 may be performed with reference to the execution method of the aforementioned step S220.
[0071] In another implementation, assuming that the Louvain algorithm is used as the community discovery algorithm in step S220, the process of obtaining a new community can be further simplified in this step. Specifically, this implementation may include the following sub-steps:
[0072] Step S321: Based on the multiple second communities obtained in the above step S220, nodes belonging to the same community in the new ANNS graph are merged into aggregate nodes to obtain a simplified new ANNS graph.
[0073] It should be understood that the description of this step can also refer to the introduction of Louvain hot start in the above embodiment.
[0074] Step S322, using the Louvain algorithm to process the simplified new ANNS graph to obtain multiple third communities;
[0075] Step S323: restore the aggregation nodes in the plurality of third communities to the original node IDs to obtain a plurality of fourth communities as a plurality of new communities.
[0076] From the above, we can obtain multiple new communities that are quickly and accurately detected from the new ANNS graph.
[0077] Step S33: for each new community, similarity calculation and filtering are performed on other nodes based on the nodes having the user ID therein, so as to obtain the cluster corresponding to the user ID.
[0078] It should be understood that the introduction to this step can refer to the aforementioned description of step S230; in addition, for the similarity calculation in this step, the calculation result in the aforementioned step S230 can be partially reused.
[0079] From the above, the incremental update of clustering results can be achieved accurately and efficiently.
[0080] Corresponding to the above clustering method, the embodiments of this specification also disclose a clustering device. Figure 4 Schematic diagram of the functional structure of the device for clustering user biometric feature vectors disclosed in the embodiment of this specification. Figure 4 As shown, the clustering device 400 includes the following functional modules:
[0081] The graph construction module 410 is configured to use the approximate nearest neighbor search ANNS algorithm to construct an ANNS graph between biometric feature vectors, wherein the biometric feature vectors corresponding to some nodes carry user IDs. The community discovery module 420 is configured to process the ANNS graph based on the community discovery algorithm to obtain multiple communities, wherein each community includes a number of node identifiers. The filtering module 430 is configured to perform similarity calculation and filtering processing on other nodes based on the nodes with user IDs in each community, thereby obtaining clusters corresponding to the user IDs.
[0082] In some embodiments, the ANNS algorithm is a hierarchical navigable small-world HNSW algorithm, and the ANNS graph is a HNSW graph; the graph construction module 410 is specifically configured as follows: for multiple biometric feature vectors, grouping them according to the collection areas of the corresponding multiple original biometric features; selecting several biometric feature vectors from the groups corresponding to each collection area as entry points of the HNSW algorithm, thereby executing the HNSW algorithm to obtain the HNSW graph.
[0083] Furthermore, in some specific embodiments, the graph construction module 410 is further configured to: during the execution of the HNSW algorithm, use the similarity between the biometric feature vectors as the weight of the connection edge between the corresponding nodes.
[0084] In some embodiments, the community discovery algorithm is a Louvain algorithm; wherein the community discovery module 420 includes: a graph simplification unit 421, configured to merge multiple nodes corresponding to the same user ID in the ANNS graph into aggregate nodes based on the user IDs corresponding to the partial nodes, to obtain a simplified ANNS graph; a community discovery unit 422, configured to process the simplified ANNS graph using the Louvain algorithm to obtain multiple first communities; a node restoration unit 423, configured to restore the aggregate nodes in the multiple first communities to the original node IDs, to obtain multiple second communities as the multiple communities.
[0085] Furthermore, in some specific embodiments, the graph simplification unit 421 is also configured to: reconstruct the connecting edges between the multiple nodes into self-loop edges of the aggregated node, and the weight of the self-loop edge is the sum of the weights of the connecting edges between the multiple nodes.
[0086] In some other specific embodiments, the graph simplification unit 421 is further configured to: reconstruct the connection edges between the multiple nodes and other nodes into connection edges between the aggregation node and other nodes.
[0087] On the other hand, in some specific embodiments, the clustering device 400 also includes an incremental update module 440, which is configured to: based on a batch of newly added biometric feature vectors, continue to execute the ANNS algorithm on the basis of the ANNS graph to obtain a new ANNS graph; according to the multiple second communities, merge the nodes belonging to the same community in the new ANNS graph into aggregate nodes to obtain a simplified new ANNS graph; use the Louvain algorithm to process the simplified new ANNS graph to obtain multiple third communities; restore the aggregate nodes in the multiple third communities to the original node IDs to obtain multiple fourth communities as multiple new communities; for each new community, perform similarity calculation and filtering processing on other nodes based on the nodes with user IDs therein, so as to obtain the clustering cluster corresponding to the user ID.
[0088] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute Figure 2 The method described.
[0089] According to another embodiment, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, Figure 2 The method described.
[0090] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.
[0091] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.
Claims
1. A clustering method for user biometric feature vectors, comprising: The approximate nearest neighbor search ANNS algorithm is used to construct an ANNS graph between biometric vectors, where the biometric vectors corresponding to some nodes carry user IDs; Based on the community discovery algorithm, the ANNS graph is processed to obtain multiple communities, each of which includes several node identifiers; For each community, similarity calculation and filtering are performed on other nodes based on the nodes with user IDs, so as to obtain the cluster corresponding to the user ID.
2. The method according to claim 1, wherein: The ANNS algorithm is a hierarchical navigable small-world HNSW algorithm, and the ANNS graph is a HNSW graph; the construction of the HNSW graph includes: For multiple biometric feature vectors, grouping them according to the collection areas of the corresponding multiple original biometric features; A number of biometric feature vectors are respectively selected from the groups corresponding to the respective collection areas as entry points of the HNSW algorithm, so as to execute the HNSW algorithm and obtain the HNSW graph.
3. The method according to claim 2, further comprising: In the process of executing the HNSW algorithm, the similarity between the biometric feature vectors is used as the weight of the connection edge between the corresponding nodes.
4. The method according to claim 1, wherein: The community discovery algorithm is the Louvain algorithm; wherein, based on the community discovery algorithm, the ANNS graph is processed to obtain multiple communities, including: Based on the user IDs corresponding to the partial nodes, multiple nodes corresponding to the same user ID in the ANNS graph are merged into an aggregate node to obtain a simplified ANNS graph; Processing the simplified ANNS graph using the Louvain algorithm to obtain a plurality of first communities; The aggregation nodes in the multiple first communities are restored to original node IDs to obtain multiple second communities as the multiple communities.
5. The method according to claim 4, wherein: The simplified ANNS diagram also includes: The connecting edges between the multiple nodes are reconstructed into self-loop edges of the aggregation node, and the weight of the self-loop edge is the sum of the weights of the connecting edges between the multiple nodes.
6. The method according to claim 4, wherein: The simplified ANNS diagram also includes: The connection edges between the multiple nodes and other nodes are reconstructed into connection edges between the aggregation node and other nodes.
7. The method according to claim 4, wherein: After obtaining the clustering cluster corresponding to the target user ID, the method further includes: Based on a batch of newly added biometric feature vectors, the ANNS algorithm is continued to be executed on the basis of the ANNS graph to obtain a new ANNS graph; According to the multiple second communities, nodes in the new ANNS graph belonging to the same community are merged into aggregated nodes to obtain a simplified new ANNS graph; Processing the simplified new ANNS graph using the Louvain algorithm to obtain a plurality of third communities; Restoring the aggregation nodes in the plurality of third communities to original node IDs to obtain a plurality of fourth communities as a plurality of new communities; For each new community, similarity calculation and filtering are performed on other nodes based on the nodes with user IDs therein, so as to obtain the cluster corresponding to the user ID.
8. A device for clustering user biometric feature vectors, comprising: A graph construction module configured to construct an ANNS graph between biometric feature vectors using an approximate nearest neighbor search ANNS algorithm, wherein the biometric feature vectors corresponding to some nodes carry a user ID; A community discovery module, configured to process the ANNS graph based on a community discovery algorithm to obtain a plurality of communities, wherein each community includes a plurality of node identifiers; The filtering module is configured to perform similarity calculation and filtering processing on other nodes based on the nodes with user IDs in each community, so as to obtain the cluster corresponding to the user ID.
9. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.
10. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 7 is implemented.
