Face clustering method, related equipment and computer program product

Abstract

The invention discloses a face clustering method, related equipment and a computer program product, and relates to the technical field of face recognition. The method comprises the following steps: determining a query sequence from an initial hierarchy to a highest hierarchy corresponding to target equipment according to a multi-hierarchy equipment affiliation relationship; and determining a query result of the initial hierarchy according to a clustering result of the to-be-processed face image and an existing single-device figure cluster of the target device. And performing layer-by-layer upward query from the initial hierarchy of the query sequence, taking the query result of the previous hierarchy as the query target of the current hierarchy in the query process, and querying the identifier of the entity comprising the query target in the parallel query set of the current hierarchy as the query result of the current hierarchy. And determining a final clustering result of the to-be-processed face image based on all the character clusters associated with the entity identifier queried in the last hierarchy. According to the face clustering method and device, face clustering is completed through entity identifier query, the calculation amount in the face clustering process is remarkably reduced, and the face clustering efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of facial recognition technology, and more specifically, to a facial clustering method, related equipment, and computer program products. Background Technology

[0002] To effectively maintain urban safety and stability, city-level security systems have emerged. These systems connect to the city's video surveillance equipment (such as facial recognition cameras and webcams) to acquire facial images. Based on the structured data from these facial images, real-time continuous clustering of all individuals appearing under all surveillance equipment can promptly detect abnormal gatherings of people and suspicious activity trajectories, providing strong support for investigation and public security prevention work by higher-level departments.

[0003] However, the commonly used face clustering methods usually involve comparing features of all face images in a quantized manner. Although this method can achieve people clustering to a certain extent, it suffers from problems such as huge computational load, low clustering efficiency, and poor real-time clustering performance.

[0004] Therefore, there is an urgent need for a face clustering method to reduce the amount of computation in the clustering process, thereby improving clustering efficiency and real-time performance. Summary of the Invention

[0005] In view of the above problems, this application is proposed to provide a face clustering method, related equipment, and computer program product to reduce the computational load in the clustering process. The specific solution is as follows:

[0006] Firstly, this application provides a face clustering method, including:

[0007] Acquire the face image to be processed from the target device;

[0008] Based on the multi-level device affiliation relationship, determine the query sequence from the initial level corresponding to the target device to the highest level;

[0009] The features of the face image to be processed are incrementally clustered with the features of the face images in the existing single-device person clusters of the target device to obtain the cluster identifier of the single-device person cluster to which the face image to be processed belongs, and the cluster identifier is used as the query result of the initial level. The single-device person cluster is a set of face images collected by the same device and belonging to the same person, and the entity identifier set for each single-device person cluster is the cluster identifier.

[0010] Starting from the initial level of the query sequence, the query proceeds upwards layer by layer. During the query process, the query result of the previous level is used as the query target of the current level. The identifier of the entity containing the query target is queried in the union set of the current level and used as the query result of the current level. The union set includes at least one entity carrying an entity identifier. The single entity in the union set of the current level is the set of entity identifiers belonging to the same person in the previous level.

[0011] Based on all the person clusters associated with the entity identifiers queried at the last level, the final clustering result of the face image to be processed is determined.

[0012] In one possible design, in another implementation of the first aspect of the embodiments of this application, at each level, there is a stored person cluster corresponding to each of the entities in the disjoint-set data structure, wherein the person cluster is a set of face images in a single-device person cluster associated with each entity identifier contained in the entity.

[0013] The process of retrieving the identifiers of entities containing the query target from the current level's union set and using them as the query results for the current level includes:

[0014] If the query hit in the current level's union set contains the identifier of the entity that is the target of the query, then the identifier of the entity that is the target of the query hit will be used as the query result of the current level.

[0015] If the query in the current level's join set does not find the identifier of an entity containing the query target:

[0016] The face image to be processed is clustered with the face images in all the person clusters in the current level;

[0017] If the clustering result indicates that the face image to be processed belongs to any of the person clusters, then the identifier of the entity corresponding to the person cluster to which the face image to be processed belongs is used as the query result of the current level.

[0018] If the clustering result indicates that the face image to be processed does not belong to any of the person clusters, then in the current level's union-find set, a new person cluster containing the face image to be processed is created, and an entity containing the cluster identifier of the newly created person cluster is created, and the identifier of the newly created entity is used as the query result of the current level.

[0019] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of determining the final clustering result of the face image to be processed based on the entity identifier queried at the last level includes:

[0020] Get the latest update time of all person clusters associated with the entity identifier found at the last level, and the data acquisition device corresponding to the latest update time;

[0021] Based on the physical distance between each of the acquisition devices and the target device, the shortest movement time from each acquisition device to the target device is estimated, and the shortest movement time corresponding to each group of people is obtained;

[0022] The difference between the acquisition time of the face image to be processed and the latest update time of each person cluster is calculated as the actual motion duration corresponding to each person cluster;

[0023] The cluster of people whose actual movement duration is not less than the shortest movement duration is taken as the final clustering result of the face image to be processed.

[0024] In one possible design, in another implementation of the first aspect of this application, the process of clustering the face image to be processed with the face images in all the person clusters in the current level includes:

[0025] Obtain the latest update time and acquisition device for each single-device character cluster in all character clusters at the current level;

[0026] Based on the physical distance between each of the acquisition devices and the target device, the shortest movement time from each of the acquisition devices to the target device is estimated, and the shortest movement time corresponding to each single device group of people is obtained.

[0027] Based on the difference between the acquisition time of the face image to be processed and the shortest movement duration corresponding to each single device group of people, the latest departure time of each group of people is determined.

[0028] The face image to be processed is clustered with the face images in the single-device person cluster that meet the preset conditions, wherein the latest update time of the single-device person cluster is earlier than its corresponding latest departure time.

[0029] In one possible design, in another implementation of the first aspect of this application, the process of clustering the face image to be processed with the face images in all the person clusters in the current level includes:

[0030] The features of the face image to be processed are compared with the features of the face images in each of the person clusters to obtain the similarity corresponding to each person cluster;

[0031] If the similarity of all the person clusters is less than a preset threshold, the clustering result is determined to be that the face image to be processed does not belong to any of the person clusters.

[0032] If the similarity of at least one of the person clusters is not less than the preset threshold, the clustering result is determined to be that the face image to be processed belongs to the target person cluster, the target person cluster is the person cluster with the largest similarity value among all candidate person clusters, and the candidate person cluster is the person cluster with the similarity not less than the preset threshold;

[0033] If the clustering result indicates that the face image to be processed belongs to any one of the person clusters, the method further includes:

[0034] Detect the image quality of the face image to be processed;

[0035] If the image quality of the face image meets the preset high quality condition, determine whether all the candidate object clusters are associated with the same entity;

[0036] When all candidate clusters are associated with the same entity, the entity identifier of the entity is used as the clustering result of the face image to be processed at the current level.

[0037] When all candidate clusters are associated with different entities, all entities associated with all candidate clusters are merged to obtain a new entity. The entity identifier of the new entity is used as the clustering result of the face image to be processed at the current level. The merging relationship between entities in the new entity is passed to the level after the current level. In the level after the current level, the entity identifiers associated with the candidate clusters in the disjoint-setup are merged according to the merging relationship.

[0038] In one possible design, in another implementation of the first aspect of this application, after incrementally clustering the features of the face image to be processed with the features of face images in the existing single-device person clusters of the target device to obtain the cluster identifier of the single-device person cluster to which the face image to be processed belongs, the method further includes:

[0039] Detect the image quality of the face image to be processed;

[0040] If the image quality meets the preset quality conditions, calculate the similarity between the feature vector of the face image to be processed and the feature vector of each existing face image in the single device person cluster to which the face image to be processed belongs.

[0041] If the similarity of each feature vector is not greater than a preset similarity threshold, the face image to be processed is added to the single-device person cluster to which the face image to be processed belongs, and the feature vector of the face image to be processed is added to the feature index of the single-device person cluster to which the face image to be processed belongs.

[0042] In one possible design, in another implementation of the first aspect of the embodiments of this application, the method further includes:

[0043] The latest update time of the single-device person cluster to which the face image to be processed belongs is updated to the acquisition time of the face image to be processed;

[0044] Iterate through the latest update time of each of the existing single-device person clusters on the target device;

[0045] Calculate the difference between the latest update time and the current time to obtain the unupdated duration for each individual device cluster;

[0046] Delete the single-device person clusters that have not been updated for a period not less than the preset storage period.

[0047] Secondly, this application provides an electronic device, including: a memory and a processor;

[0048] The memory is used to store programs;

[0049] The processor is configured to execute the program to implement the face clustering method described in any of the first aspects of this application.

[0050] Thirdly, this application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the face clustering method described in any of the preceding first aspects of this application.

[0051] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the face clustering method described in any of the first aspects of this application.

[0052] The face clustering method provided in this application first determines a query sequence from the initial level to the highest level corresponding to the target device based on the multi-level device affiliation relationship. In the initial level, based on the existing single-device person clusters within a single device, the collected face images to be processed are clustered to obtain the clustering results (also known as query results) of the face images to be processed in the initial level.

[0053] Furthermore, the query proceeds upwards layer by layer from the initial level of the query sequence. Each level's union-find set pre-stores the clustering results of entities from the previous level; that is, the entities in the union-find set are a set of entity identifiers belonging to the same person from the previous level. Based on this, during the query process, the query results of the previous level are used as the query target for the current level. From the union-find set of the current level, the identifiers of entities containing the query target are retrieved, directly yielding the clustering results of the face image to be processed at the current level. This process is repeated until the entity identifiers are retrieved from the union-find set of the last level.

[0054] In this application, face clustering is completed through entity identifier query, eliminating the need for full-range feature comparison and significantly reducing the computational load in the face clustering process. Furthermore, since the query operation is faster than feature comparison, it greatly improves face clustering efficiency and is more suitable for real-time applications such as security and face recognition.

[0055] Finally, based on the entity identifiers retrieved at the last level and the clusters of all individuals belonging to the same person within the largest clustering range, the final clustering result of the face image to be processed is determined. Furthermore, since each level is configured with a unique entity identifier for the same person, the accuracy of face clustering is ensured.

[0056] In summary, in the multi-level face clustering method provided in this application, the disjoint set, person cluster, and feature index of each level are stored and updated independently. Therefore, the layers can be executed in a pipeline manner, and different clustering units within each level can be executed in parallel, thereby improving the operating efficiency of the entire clustering system. Attached Figure Description

[0057] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0058] Figure 1 This is a schematic diagram of an implementation system architecture for the face clustering method provided in an embodiment of this application;

[0059] Figure 2 This is a flowchart illustrating an implementation of a face clustering method provided in an embodiment of this application.

[0060] Figure 3 An example diagram illustrating a multi-level device attribution relationship provided in this application embodiment;

[0061] Figure 4A schematic diagram of the initial-level clustering process provided for embodiments of this application;

[0062] Figure 5 A flowchart illustrating the arbitrary-level query process provided in the embodiments of this application;

[0063] Figure 6 Example diagram of a common index provided in the embodiments of this application;

[0064] Figure 7 Example diagram of the active index provided in the embodiments of this application;

[0065] Figure 8 A three-level clustering example diagram of the face clustering method provided in this application embodiment;

[0066] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] Currently, various methods are used for face clustering in city-level security systems. For example, one approach is to extract the feature vector from each face image, digitize features such as facial organ shape, skin texture, hair, and skin color using this vector, and then perform face clustering based on this feature vector. Another approach involves comparing the features of a face image with the facial features of existing people on the same camera, depending on the distance between cameras. If a match is found, the face image is assigned to that person's cluster. If no match is found, the clustering scope is expanded by comparing the facial features of existing people on other cameras, until all cameras have no matching features. Finally, a new person cluster is created, completing the full-range clustering.

[0069] The aforementioned face clustering methods, whether full clustering or hierarchical clustering, all employ feature comparison, the core of which is calculating the similarity between facial feature vectors. However, as the scope of urban security deployment expands, the number of face images requiring clustering increases exponentially, leading to a massive load on computing hardware and resulting in time-consuming and inefficient clustering tasks. This severely impacts the real-time response and dynamic deployment capabilities of security systems.

[0070] To address the aforementioned problems, this application provides a face clustering method. The face clustering method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0071] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200, wherein the server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0072] Either terminal 100 or server 200 can be used independently to execute the face clustering method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the face clustering method provided in the embodiments of this application.

[0073] In this application embodiment, the terminal 100 can be a camera, mobile phone, tablet computer, teaching large screen, wearable device, vehicle device, conference terminal, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0074] The product form of server 200 in this application embodiment may include, but is not limited to, cloud servers (running as virtual server instances on physical servers), micro servers (suitable for small office or home environments), storage servers (focused on data storage and management), GPU servers (equipped with graphics processing units, suitable for high-performance computing and graphics processing tasks), and modular servers (allowing flexible configuration and expansion of hardware components as needed).

[0075] This application provides a face clustering method, illustrated by applying the method to a computer device. Specifically, the computer device may be... Figure 1 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 2 The present application provides a flowchart of a face clustering method, which may include steps S100 to S140. These steps are described in detail below.

[0076] Step S100: Obtain the face image to be processed collected by the target device.

[0077] It is understandable that security systems typically connect to a large number of monitoring devices, receiving face images captured by each device in real time, and performing steps S120-S140 on each face image to achieve clustering of each face image. In this embodiment, taking a monitoring device as the target device, the face clustering process of the face images to be processed captured by the target device is used as an example to introduce the face clustering method of this application.

[0078] Step S110: Based on the multi-level device affiliation relationship, determine the query sequence from the initial level to the highest level corresponding to the target device.

[0079] It is understandable that, starting with the target device, a complete clustering query sequence can be found from bottom to top, from specific devices to macro regions, based on the hierarchical relationship of the target device's superior, superior's superior, and so on. Clustering based on this query sequence can achieve clustering of the face images to be processed in single devices, across devices, and across regions.

[0080] Reference Figure 3 This application provides an example diagram of a multi-level device ownership relationship. For ease of understanding, a tree-like structure is used to interpret the multi-level device ownership relationship.

[0081] In a tree structure representing multi-level device affiliation, all devices are divided into multiple cluster units (nodes) at different levels according to different scope scales. The affiliation relationship between cluster units at adjacent levels is represented by lines (edges). For example, the device range corresponding to cluster unit L1 includes device 1, device 2, ..., device N; the device range corresponding to cluster unit M1 includes all devices within cluster units L1, L2, ..., Li, and so on. If two cluster units at adjacent levels have an affiliation relationship, the cluster unit at the later level contains several cluster units at the previous level.

[0082] In this context, the initial level refers to the level with the smallest clustering range. In this embodiment, the clustering unit of the initial level is only a single device. This level is the lowest and most specific level in the hierarchical structure (multi-level device affiliation). Based on this, the clustering unit to which the target device belongs in the initial level is located from the multi-level device affiliation. Starting from the initial level, the query sequence is obtained by tracing upwards level by level from the initial level corresponding to the target device to the highest level.

[0083] In one possible implementation, the clustering units at each level of the multi-level device affiliation relationship are divided according to geographical administrative regions. Then, tracing back from the initial level to the last level, the resulting query sequence can be: street monitoring (target device) → street → district → city → province.

[0084] In another possible implementation, all facial recognition devices are grouped according to their geographical proximity, resulting in a hierarchical relationship of "device → device group → global devices". For example, when grouping devices, a fully connected graph is constructed for all facial recognition devices globally. Each node in the graph represents a device, and the weight of each edge represents the distance between the two connected devices. The distance between devices can be calculated using the Haversine formula based on the latitude and longitude of the two devices, or it can be obtained by calling a map service. Then, a graph partitioning algorithm is used to obtain several subgraphs, with the devices in each subgraph forming a device group. The granularity of graph partitioning can be determined based on the number of devices and the performance of the processing device. Based on this, the hierarchical relationship of "device → device group → global devices" is obtained.

[0085] It should be noted that in the multi-level device affiliation relationship, the clustering unit of each level can be customized, such as according to the distance between each device, the jurisdiction of the region to which each device belongs, etc., and no unique limitation is made in the embodiments of this application.

[0086] Step S120: Incremental clustering is performed between the face image to be processed and the face images in the existing single-device person clusters of the target device to obtain the cluster identifier of the single-device person cluster to which the face image to be processed belongs, and the cluster identifier is used as the query result of the initial level.

[0087] Based on the query sequence of the target device from the corresponding initial level to the highest level determined in step S110, the face images to be processed are clustered level by level from bottom to top.

[0088] Reference Figure 4 The following is a schematic diagram of the initial-level clustering process provided in this application embodiment to illustrate the implementation process of step S120. First, in the initial level corresponding to the target device, the face image to be processed is clustered with the face images in the single-device person clusters that the target device has previously processed. Here, a single-device person cluster is a set of face images collected by the same device and belonging to the same person, and the entity identifier set for each single-device person cluster is a cluster identifier. The single-device person cluster also carries the latest update time of the face features.

[0089] If the clustering result indicates that the face image to be processed does not belong to any existing single-device person cluster, the quality of the face image to be processed is first checked, such as blurriness, occlusion, and face angle. If the quality of the face image to be processed meets the preset conditions, a new single-device person cluster can be created based on the face image to be processed. The cluster identifier of the newly generated single-device person cluster is used as the initial level query result. It is important to note that the newly generated single-device person cluster should be stored in the target device's directory as an existing single-device person cluster for incremental clustering when processing other face images, thus accumulating data for subsequent face clustering.

[0090] Conversely, if the quality of the face image to be processed does not meet the preset conditions, no new cluster is generated for that face image. The initial processing of the face image is skipped, and information indicating that the face image does not meet the clustering requirements or that clustering has failed is output to the front end. In another possible implementation, if the quality of the face image to be processed does not meet the preset conditions, the query result of the face image at the initial level can be set to empty before proceeding to subsequent steps such as S130.

[0091] If the clustering result indicates that the face image to be processed belongs to any existing single-device person cluster, then the cluster identifier of the existing single-device person cluster to which the face image to be processed belongs is used as the initial level query result, and the latest update time of the single-device person cluster to which the face image to be processed belongs is updated to the acquisition time of the face image to be processed, and the face image to be processed is added to the single-device person cluster to which it belongs.

[0092] Understandably, the core value of a single-device person cluster lies not in the number of facial features it contains, but in how these features collectively construct a high-quality, representative person model. Therefore, blindly adding all facial images belonging to a particular person into the cluster would, on the one hand, result in a large amount of redundant data occupying storage space, and on the other hand, a large number of highly similar features within the cluster would over-amplify the weight of those features, causing the feature center to deviate from reality.

[0093] Therefore, in the process of adding the face image to be processed to its respective single-device person cluster in this embodiment, firstly, if the image quality of the face image to be processed meets the preset quality conditions, the feature vector of the face image to be processed is calculated, and the feature similarity between it and the feature vector of each existing face image in the single-device person cluster to which the face image to be processed belongs is calculated; if each feature similarity is not greater than the preset similarity threshold, the face image to be processed is added to the single-device person cluster to which the face image to be processed belongs.

[0094] This embodiment actively rejects redundant information by filtering similar features, ensuring the diversity and balance of features within each individual device's person cluster. This makes these features more robust in representing the overall appearance of the person and improves the accuracy of subsequent comparisons.

[0095] Understandably, the aforementioned face clustering relies on the similarity comparison of features extracted from face images. If the quality of the face image is poor, the extracted features may lose key facial features, thus affecting the accuracy of the clustering results. The causes of poor image quality are random, such as blurring due to weather, masks obscuring the face, and differences in capture angle. This means that a poor-quality face may not be captured a second time. If a new cluster is created using the features of this poor-quality face, this cluster may never match any face images belonging to the same person, rendering it useless for tracking and instead consuming the security system's memory.

[0096] Similarly, if features of poor-quality facial images are added to existing single-device person clusters, the feature centers of the clusters will deviate from the true and accurate features. This will lead to the rejection of high-quality images that should be matched in subsequent clustering processes, and the incorrect clustering of images that are not the same person, affecting the accuracy of clustering at the initial level and even later levels.

[0097] Therefore, in this embodiment, before deciding whether to create a new single-device person cluster or whether to add the face image to be processed to an existing single-device person cluster, the image quality of the face image to be processed is first detected. Based on this, it is ensured that the facial features in each single-device person cluster are valid and accurate. It is understood that the value of constructing single-device person clusters based on quality detection results is not directly reflected in the immediate feedback of a single processing step, but rather in building a reliable underlying feature cluster library for the entire security system. This, through a cumulative effect, fundamentally improves the accuracy of clustering, ensuring a high degree of consistency and stability in the final clustering results.

[0098] Step S130: Starting from the initial level of the query sequence, query upwards layer by layer. During the query process, the query result of the previous level is used as the query target of the current level. In the union set of the current level, the identifier of the entity including the query target is queried and used as the query result of the current level.

[0099] Each clustering unit at each level corresponds to a disjoint-set data structure (DFS), which includes at least one entity carrying an entity identifier. A single entity in the DFS of the current level is the set of entity identifiers belonging to the same person in the previous level. The DFS for each level is illustrated using the query sequence "Equipment Layer → Street Layer → District Layer" as an example.

[0100] The street-level disjoint-set data structure manages the cluster identifiers of single-device person clusters for all devices within the street's scope. Specifically, a single entity in the street-level disjoint-set data structure contains a set of cluster identifiers for single-device person clusters belonging to the same person across all devices within the street's scope, and this entity's identifier is the street-level person ID. For example, Street1_100{A5, B2, C8} means that "person cluster #5 of device A", "person cluster #2 of device B", and "person cluster #8 of device C" within street 1 belong to the same person, and are uniformly represented by Street1_100 at the street level.

[0101] Similarly, the district-level disjoint-set data structure manages the street-level person IDs of all streets within the district. A single entity in the district-level disjoint-set data structure contains a set of street-level person IDs belonging to the same person across all streets within the district, such as district1_20{Street1_100, Street2_10, Street6_50}.

[0102] Starting from the initial level of the query sequence, the query proceeds upwards level by level. The query result of the previous level is used as the query target of the current level. In the union set of the current level, the identifier of the entity containing the query target is searched and used as the query result of the current level. This process is repeated until the identifier of the entity containing the query result of the previous level is found in the union set of the last level.

[0103] For example, starting from the device layer in the query sequence "device layer → street layer → district layer", the query results A5 of the target device A obtained in step S120 are used as the query target of the street layer. The identifier of the entity containing A5 is queried from the street layer to obtain Street1_100. This process is repeated until the last layer, the district layer, is found in the union set of the query results. The identifier of the entity containing Street1_100, namely district1_20, is found. This is the identity identifier of the person to whom the face image to be processed belongs within the highest level of the maximum cluster range of the current query sequence.

[0104] In the above implementation, each level of entity has an independent namespace, such as Device_A_* for the device level, Street_* for the street level, and District_* for the district level. This allows each level to independently and concurrently assign its own ID and reduces the complexity of queries.

[0105] In this embodiment, a multi-level device attribution relationship and a disjoint-set data structure at each level are constructed. This hierarchical and disjoint-set structure merges and associates the underlying, local person identifiers layer by layer, ultimately forming a globally unified identity view. This avoids the traditional full feature comparison; during a query, only a few efficient "lookup" operations are needed within the disjoint-set data structure, rather than calculating the similarity between massive feature vectors, significantly improving the efficiency of face clustering. Furthermore, when adding devices or data, only the disjoint-set relationship of the affected part needs to be updated, without reconstructing the entire clustering structure, resulting in strong scalability.

[0106] Step S140: Based on all the person clusters associated with the entity identifiers queried at the last level, determine the final clustering result of the face image to be processed.

[0107] It should be noted that in each level, there is a cluster of people corresponding to each entity in the disjoint-set data structure, and a feature index for clustering comparison corresponding to each cluster of people. The cluster of people is the set of face images in the cluster of people corresponding to each entity identifier contained in the entity, and the feature index is the set of feature vectors corresponding to all face images in the corresponding cluster of people.

[0108] Continuing with the query sequence "Device Layer → Street Layer → District Layer" above, each device in the device layer stores at least one single-device person cluster. The street layer stores a set of single-device person clusters belonging to the same person across all devices in the current street. This set is a person cluster, and the person cluster is associated with the entity corresponding to that person. For example, the entity Street1_100{A5, B2, C8} in the street layer stores a person cluster containing "person cluster #5 of device A", "person cluster #2 of device B", and "person cluster #8 of device C". Similarly, the entity district1_20{Street1_100, Street2_10, Street6_50} in the district layer stores a person cluster containing "person cluster of Street1_100", "person cluster of Street2_10", and "person cluster of Street6_50".

[0109] Based on this, each person cluster in the last level contains all person clusters belonging to the same person in the global scope. Therefore, this step can determine the final clustering result of the face image to be processed based on all person clusters associated with the entity identifiers queried in the last level.

[0110] In another possible implementation, the entity identifier retrieved at the last level is the global identity identifier (such as ID number, name, etc.) of the person to whom the face image to be processed belongs, and this global identity identifier can be directly used as the final clustering result.

[0111] In summary, the face clustering method provided in this application first determines the query sequence from the initial level to the highest level corresponding to the target device based on the multi-level device affiliation relationship. In the initial level, based on the existing single-device person clusters within a single device, the collected face images to be processed are clustered to obtain the clustering results (also known as query results) of the face images to be processed in the initial level.

[0112] Furthermore, the query proceeds upwards layer by layer from the initial level of the query sequence. Each level's union-find set pre-stores the clustering results of entities from the previous level; that is, the entities in the union-find set are a set of entity identifiers belonging to the same person from the previous level. Based on this, during the query process, the query results of the previous level are used as the query target for the current level. From the union-find set of the current level, the identifiers of entities containing the query target are retrieved, directly yielding the clustering results of the face image to be processed at the current level. This process is repeated until the entity identifiers are retrieved from the union-find set of the last level.

[0113] In this application, face clustering is completed through entity identifier query, eliminating the need for full-range feature comparison and significantly reducing the computational load in the face clustering process. Furthermore, since the query operation is faster than feature comparison, it greatly improves face clustering efficiency and is more suitable for real-time applications such as security and face recognition.

[0114] Finally, based on the entity identifiers retrieved at the last level and the clusters of all individuals belonging to the same person within the largest clustering range, the final clustering result of the face image to be processed is determined. Furthermore, since each level is configured with a unique entity identifier for the same person, the accuracy of face clustering is ensured.

[0115] In summary, in the multi-level face clustering method provided in this application, the disjoint set, person cluster, and feature index of each level are stored and updated independently. Therefore, the layers can be executed in a pipeline manner, and different clustering units within each level can be executed in parallel, thereby improving the operating efficiency of the entire clustering system.

[0116] Next, other possible implementations of the face clustering method provided in this application will be described through the following embodiments.

[0117] Reference Figure 5 The flowchart of the arbitrary level query process provided in this application embodiment describes the process of querying the identifier of the entity of the query target in the current level's union retrieval set as the query result of the current level.

[0118] When retrieving the identifier of an entity containing the query target from the current level's merged set, there are two scenarios: First, if the query finds an identifier of an entity containing the query target within the current level's merged set, then that identifier is used as the query result for the current level. Second, if the query does not find an identifier of an entity containing the query target within the current level's merged set.

[0119] It is understood that in the embodiments of this application, the disjoint-set data structure (DFS) of each level (such as the equipment level, street level, and district level) is maintained independently. This means that the DFS content of a certain level is generated only based on the clustering results of that level itself, and will not be automatically or in real time updated to follow data changes in other levels. That is, when a new entity is created at a certain level, the DFS of higher levels will not immediately respond to this change. Instead, only when a higher level is executing its own query or clustering process, and the newly created entity is included in the clustering calculation of the current level, will the entity identifier of the newly created entity be assigned to an entity in the current level, thus realizing the update of the current level.

[0120] Furthermore, if the query result at the previous level is empty, since the entities in each level of the union set are a set of entity identifiers, it is impossible to find an entity with an empty identifier in the query result of the union set.

[0121] If the query in the current level's union-find set does not find the identifier of an entity containing the query target: the face image to be processed is clustered with the face images in all person clusters in the current level; if the clustering result shows that the face image to be processed belongs to any person cluster, then the identifier of the entity corresponding to the person cluster to which the face image to be processed belongs is used as the query result of the current level; if the clustering result shows that the face image to be processed does not belong to any person cluster, then a new person cluster containing the face image to be processed is created in the current level's union-find set, and a new entity containing the cluster identifier of the new person cluster is created, and the identifier of the new entity is used as the query result of the current level.

[0122] This application embodiment does not exclude the possibility that the clustering of the face image to be processed in the previous level is incomplete. For example, if the person to be processed in the face image has not appeared in device A, then a single-device person cluster for the face image to be processed needs to be created in device A. However, device B already has a single-device person cluster for the person to be processed in the face image to be processed, but this is only at the single-device level and cannot achieve cross-device clustering. Therefore, it can be seen that not finding an entity containing the query target in the current level's union-setup does not mean that the current level does not contain an entity containing the person to be processed in the face image to be processed.

[0123] Therefore, if the query in the current level's joint lookup set does not find an identifier for the entity containing the query target, firstly, based on the person clusters corresponding to the entities stored at each level, the features of the face image to be processed are compared with the features of the face images in each person cluster (such as cluster center features) to obtain the similarity between the features of the face image to be processed and the features of each person cluster. It is then determined whether there exists a person cluster with a similarity not less than a preset threshold. If so, the face image to be processed is identified as belonging to that person cluster. If multiple similarity thresholds exist, and the face image to be processed meets the quality control requirements, all person clusters with similarity not less than the preset threshold are considered candidate person clusters. The candidate person cluster with the highest similarity value is selected as the target person cluster to which the face image to be processed belongs. If the similarity corresponding to all person clusters is less than the preset threshold, the face image to be processed is identified as not belonging to any person cluster.

[0124] If clustering is successful at the current level, meaning the face image to be processed belongs to any person cluster, then the entity associated with the face image to be processed and that person cluster also belong to the same person. The identifier of the entity corresponding to the person cluster can be used as the query result at the current level.

[0125] If clustering fails, meaning the face image to be processed does not belong to any person cluster, it proves that the face image to be processed does not belong to any entity in the current level's disjoint-set data structure (DFS), and is a new person independent of all entities. Therefore, in the current level's DFS, a new person cluster containing the face image to be processed is created, and a new entity containing the cluster identifier of the newly created person cluster is created. The identifier of the newly created entity is then used as the query result for the current level.

[0126] It should be noted that the quality of the face image to be processed used to create a new entity needs to meet the preset conditions. If the image quality of the face image to be processed still does not meet the preset conditions of the current level, the query results at the current level will still be empty.

[0127] Based on this, even if the current level query does not find a match, the query will not be interrupted. Instead, the face image to be processed will be pushed to the next level for feature comparison. This ensures that the entire processing flow (from the initial level to the highest level) can proceed smoothly and avoids process delays caused by data lag.

[0128] Furthermore, within each level, each entity corresponds to multiple person clusters, and each entity strictly corresponds to one person. When clustering is performed on the face image to be processed, and the face image to be processed simultaneously hits multiple candidate person clusters, the following two situations exist:

[0129] First, if all candidate clusters are associated with the same entity, this indicates that the person entity corresponding to the face image to be processed is clear, that is, the face image to be processed matches the person represented by this entity. In this case, the entity identifier of this entity can be used as the clustering result of the face image to be processed at the current level.

[0130] Secondly, if all candidate clusters are associated with different entities, it means that at the current clustering level, the entities associated with these candidate clusters may actually belong to the same person. The reason why clusters of people who are the same person are clustered into different entities is that when the camera captures facial images, interference factors such as shooting angle and light brightness affect the judgment of the clustering algorithm, causing different facial images that originally belong to the same person to be incorrectly classified into different entities in the previous clustering process.

[0131] Therefore, in order to correct this clustering error, this embodiment of the application merges all entities associated with all candidate clusters at the current level to obtain a new entity. The entity can be renamed or any entity identifier from the merged entity can be used as the entity identifier of the new entity. The entity identifier of the new entity is used as the clustering result of the face image to be processed at the current level.

[0132] Simultaneously, following the processing order of the face images to be processed, the merging relationships between entities associated with candidate clusters are passed to subsequent levels, allowing later levels to adjust the entities in the disjoint-setup according to these merging relationships. For example, if entities Street1_100{A5, B2, C8} and Street1_8{D5, E2, Q8} are merged in the street level, then in the district level, entities district1_20{Street1_100, Street2_10, Street6_50} containing Street1_100 and entities district1_50{Street1_8, Street8_50, Street7_54} containing Street1_8 will also be merged.

[0133] Based on this, not only is the disjoint set in the current level updated, but the relevant entity relationships in the disjoint set in subsequent levels are also updated synchronously, ensuring the consistency and integrity of entity information in the entire hierarchical structure.

[0134] The clustering mechanism of this application allows the system to continuously learn new data during operation, associate new people or new facial features with the correct entities at the current level, dynamically update the person clusters corresponding to each entity and the entity identifiers contained therein, and automatically expand the person database.

[0135] Understandably, in the process of cross-device clustering, if the above query fails to find the target image, the process of clustering the face image to be processed by feature comparison usually requires clustering the face image to be processed with almost all single-device person clusters in the clustering unit. The computational load of clustering is large, which leads to a decrease in clustering efficiency.

[0136] To address this issue, one possible implementation involves clustering the face image to be processed with face images in all person clusters at the current level. This process includes: obtaining the latest update time and acquisition device for each single-device person cluster in all person clusters at the current level; estimating the shortest motion time from each acquisition device to the target device based on the physical distance between each acquisition device and the target device, thus obtaining the shortest motion time corresponding to each single-device person cluster; determining the latest departure time of each acquisition device based on the difference between the acquisition time of the face image to be processed and the shortest motion time corresponding to each acquisition device; and clustering the features of the face image to be processed with the features of face images in single-device person clusters that meet a preset condition, where the latest update time of a single-device person cluster is earlier than its corresponding latest departure time.

[0137] From the current level, retrieve all person clusters associated with each entity. Each person cluster contains at least one single-device person cluster captured by a single device. Simultaneously, retrieve the capturing device and latest update time for each single-device person cluster. As mentioned above, each single-device person cluster stored in the capturing device directory is marked with its latest update time, which is the capture time of the face image to be processed when the single-device person cluster is clustered into a face image belonging to that cluster. Essentially, the latest update time represents the most recent appearance time of the person belonging to that cluster under that capturing device.

[0138] Furthermore, for each single-device person cluster, based on the physical distance between the acquisition device of the single-device person cluster and the target device acquiring the face image to be processed, the shortest movement time for a person to travel from the acquisition device of the single-device person cluster to the target device is estimated. The shortest movement time is usually calculated based on the maximum reasonable speed (such as urban road speed limits, average pedestrian speed, etc.), representing a theoretically fastest possible speed.

[0139] For each single-device person cluster, the following calculation is performed: Acquisition time of the face image to be processed - Shortest movement duration corresponding to the cluster = Latest departure time. The latest departure time means that if a person wants to have their face captured by the target device during the acquisition time of the face image to be processed, they must leave the location of the acquisition device at the latest by the "latest departure time". If they leave after this time, even if they move at the fastest speed, they will not be able to reach the target device on time.

[0140] Therefore, it can be deduced that for a single-device group of people, if the latest update time of the group is later than its corresponding latest departure time, then the person corresponding to the group could not have arrived at the target device on time to be captured, and the probability that the group and the face image to be processed belong to the same person is greatly reduced.

[0141] Based on this conclusion, single-device person clusters whose latest update time is later than their corresponding latest departure time are filtered out, and only single-device person clusters whose latest update time is earlier than their corresponding latest departure time are clustered with the face image to be processed at the current level.

[0142] In this embodiment, before clustering the face images to be processed using the face images in the single-device person cluster, a spatiotemporal filter is first performed on the single-device person cluster in each person cluster. This significantly reduces the number of clusters that need to be compared for features, greatly narrows the comparison range, and improves the efficiency of clustering.

[0143] In another possible implementation, to improve clustering efficiency, an activity-based feature comparison index is designed for each level in the system or device deploying this face clustering method. This feature comparison index is divided into two parts: a normal index and an activity index. Each person cluster in each level corresponds to one normal index and one activity index. (See reference...) Figure 6 and Figure 7 This application provides example diagrams of ordinary indexes and active indexes in its embodiments.

[0144] Let's take a person cluster stored at the street level above as an example. In the active index, the feature vector of each track (single-device person cluster) contained in the person cluster constitutes a separate index block, and the index blocks are arranged in order according to the latest update time of the track. When the latest update time of a track is updated, only the position of the index block corresponding to that track in the list needs to be updated, which reduces the overhead of feature vector movement while taking into account computational efficiency.

[0145] In a standard index, all tracks on a single device (single-device person cluster) are indexed in blocks based on their latest update time. Each index block (time block) stores the features of tracks whose latest update time falls within the corresponding time period. Index blocks are independent and ordered. The start and end times of an index block are the minimum and maximum values ​​of the latest update time of the tracks to which the features belong within that block, respectively. Index block partitioning considers both time segment length and the number of features. When a new feature is added to an index block, if the number of features in the block exceeds a preset limit, or if the time span covered by the block exceeds a preset time limit, a new index block is created. When the latest update time of a track to which a feature belongs in an index block is updated, if the new update time exceeds the time range of the index block, these features need to be moved to a new index block. If a track repeatedly appears on a single device within a certain period, its feature vectors may move multiple times between index blocks. To address this, tracks whose feature vector count in an index block exceeds a set threshold can be moved to the active index.

[0146] Based on this, when comparing the features of the face image to be processed with the existing face images in the single-device person cluster, it is only necessary to compare the features in the index block earlier than the latest departure time with the features of the face image to be processed. This can reduce the efficiency of single-device person cluster (tracking) screening and improve the computational performance of feature comparison by utilizing the continuity of feature storage in the index block.

[0147] It is understandable that storing large amounts of data over a long period, whether on local storage devices such as monitoring equipment or computers used to deploy security systems, or on cloud storage devices, significantly increases the difficulty and cost of data retrieval, backup, and maintenance. Therefore, in this embodiment, for each single-device person cluster stored in each device directory, the latest update time of each existing single-device person cluster on the target device is periodically traversed; the difference between the latest update time and the current time is calculated to obtain the unupdated duration of each single-device person cluster; and single-device person clusters with an unupdated duration not less than a preset storage period are deleted.

[0148] Specifically, in an active index, when the latest update time of a trace expires, the corresponding index block can be deleted immediately. In a normal index, when a trace expires, its feature vector in the index is not deleted immediately; instead, the entire index block is deleted after it expires. Index block expiration means that the difference between the end time of the index block and the current time is not less than a preset storage period.

[0149] Based on the above method, in the query sequence from the initial level to the highest level corresponding to the target device, starting from the initial level, the face images to be processed are clustered level by level. Finally, in the union set of the last level, the entity identifier is obtained by querying, and all the person clusters associated with the entity corresponding to the entity identifier are used as the clustering result of the face images to be processed.

[0150] This application takes into account that although high-level entities are obtained through feature similarity clustering, feature similarity is not the same as the same person. Mismatches may be caused by similar appearance or feature extraction errors. If all associated person clusters are directly used as the final clustering result without screening, the face image to be processed may be clustered into a group of people who are physically impossible to be associated, thereby reducing the accuracy and reliability of the clustering results.

[0151] In another possible implementation, the process of determining the final clustering result of the face image to be processed based on the entity identifier queried at the last level includes: obtaining the latest update time of all person clusters associated with the entity identifier queried at the last level, and the acquisition device corresponding to the latest update time; estimating the shortest motion time from each acquisition device to the target device based on the physical distance between each acquisition device and the target device, and obtaining the shortest motion time corresponding to each person cluster; calculating the difference between the acquisition time of the face image to be processed and the latest update time of each person cluster as the actual motion time corresponding to each person cluster; and taking the person clusters whose actual motion time is not less than the shortest motion time as the final clustering result of the face image to be processed.

[0152] Specifically, first, obtain the latest update time of all person clusters associated with the entity identifier queried from the last level, as well as the collection device corresponding to the latest update time.

[0153] Furthermore, the following calculation is performed for each object cluster: Acquisition time of the face image to be processed - latest update time of the object cluster = actual movement time. Actual movement time refers to the actual time spent by a person moving from the acquisition device corresponding to the object cluster to the target device if the object cluster and the face image to be processed belong to the same person.

[0154] For each single-device group of people, based on the physical distance between the acquisition device of the group of people and the target device for acquiring the face image to be processed, the shortest movement time for a person to travel from the acquisition device of the group of people to the target device is estimated.

[0155] Therefore, it can be deduced that for a cluster of people, if the actual movement time corresponding to the cluster is less than the shortest movement time, then the person corresponding to the cluster is unlikely to arrive at the target device on time to have their face captured, and the probability that the cluster and the face image to be processed belong to the same person is greatly reduced.

[0156] Based on this conclusion, the clusters of people whose actual motion duration is greater than or equal to the shortest motion duration are taken as the final clustering result of the face images to be processed, while the clusters of people whose actual motion duration is less than the shortest motion duration are ignored.

[0157] It is understandable that this embodiment introduces real-world physical constraints into the clustering results obtained based on visual features, and filters the clustering results again, so that the final output is a high-quality clustering result that not only has similar visual features, but also reasonable spatiotemporal trajectories of people, which greatly enhances the reliability and accuracy of the clustering results.

[0158] Next, refer to Figure 8 This application provides an example diagram of a three-level face clustering method, illustrating the three-level face clustering method implemented using the face clustering method proposed in this application. As shown in the diagram, this example uses the multi-level face clustering implementation method of this application, constructing a three-level clustering structure consisting of a single-device processing stage, a cross-device processing stage, and a global processing stage.

[0159] In the single-device processing stage, taking device A as an example, the newly acquired face image is clustered with the single-device person cluster (tracking) within the single device to obtain the cluster identifier of the newly acquired face image in the single-device processing stage, and the cluster identifier is used as the processing result of the single-device processing stage.

[0160] In the cross-device processing stage, first determine the device group 1 to which device A belongs, and obtain the local disjoint set of device group 1. The local disjoint set contains at least one entity carrying an entity identifier. Among them, the single entity in the local disjoint set includes: the cluster identifier of the single device person cluster belonging to the same person in the existing single device person cluster of all devices in the device group (such as device A, device B, device C, etc.).

[0161] The processing results of the single-device processing stage are used as the query targets of the cross-device processing stage. The identifiers of entities containing the query targets are retrieved from the local disjoint retrieval set and used as the processing results of the cross-device processing stage.

[0162] In the global processing phase, firstly, a global disjoint-set data structure is obtained. The global disjoint-set data structure contains the merging relationships between the cluster identifiers of all character clusters at the previous level. These merging relationships can be stored in a tree structure, or as mentioned above, the global disjoint-set data structure stores multiple entities carrying identifiers. Each entity is a set of entity identifiers (i.e., character cluster identifiers) at the previous level belonging to the same character, and these merging relationships are stored in this way.

[0163] Furthermore, the query operation in the cross-device processing stage is repeated, and the processing result of the cross-device processing stage is used as the query target of the global processing stage. The identifier of the entity containing the query target is queried from the global unjoint retrieval set and used as the processing result of the global processing stage, that is, the entity identifier obtained by the last level query in this example. This entity identifier is used as the final clustering result.

[0164] In summary, the face clustering method provided in this application significantly improves the efficiency and accuracy of face clustering through a hierarchical, progressive, and multi-dimensional optimized processing flow. Specifically:

[0165] First, a phased and hierarchical processing strategy, "from local to global," is adopted, dividing the clustering process into three or more levels: single-device, cross-device, and global processing. The processing results of the previous level serve as the basis for triggering the calculation of the next level, which effectively avoids unnecessary global feature comparisons and thus significantly improves clustering efficiency from a macroscopic perspective.

[0166] Secondly, a spatiotemporal filtering mechanism is introduced at the cross-device processing level / stage after single-device processing. Based on the spatiotemporal relationship between the target device and other devices of the face image to be processed, the optimal possible clustering target is dynamically selected, reducing the number of feature comparisons required from the source. This achieves the dual goals of improving feature comparison accuracy while significantly reducing computational overhead.

[0167] Furthermore, in order to efficiently manage massive features and accelerate comparison, a feature comparison index was designed. By storing features in blocks and marking their activity, the most relevant index blocks can be prioritized for high-speed search during feature comparison, thereby comprehensively optimizing the management efficiency and comparison speed of features.

[0168] Finally, in the last level of processing, the clustering results of visual features were not stopped. Instead, a strict spatiotemporal rationality check was applied again. By verifying the spatiotemporal rationality between the target device that collected the facial features to be processed and the device that collected all the person clusters obtained from the last level of clustering, false matches caused by similar appearances were effectively eliminated, further ensuring the accuracy and reliability of global clustering.

[0169] This application also provides an electronic device in its embodiments. (See reference...) Figure 9 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, tablets, large-screen teaching displays, wearable devices, etc. Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0170] like Figure 9 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the face clustering method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0171] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0172] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the face clustering methods provided in this application.

[0173] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the face clustering methods provided in this application.

[0174] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0175] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0176] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0177] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0178] It is understood that the data involved in the technical solution of this application (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0179] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0180] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0181] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0182] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0183] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A face clustering method, characterized in that, include: Acquire the face image to be processed from the target device; Based on the multi-level device affiliation relationship, determine the query sequence from the initial level to the highest level corresponding to the target device; The features of the face image to be processed are incrementally clustered with the features of the face images in the existing single-device person clusters of the target device to obtain the cluster identifier of the single-device person cluster to which the face image to be processed belongs, and the cluster identifier is used as the query result of the initial level. The single-device person cluster is a set of face images collected by the same device and belonging to the same person, and the entity identifier set for each single-device person cluster is the cluster identifier. Starting from the initial level of the query sequence, the query proceeds upwards layer by layer. During the query process, the query result of the previous level is used as the query target of the current level. The identifier of the entity containing the query target is queried in the union set of the current level and used as the query result of the current level. The union set includes at least one entity carrying an entity identifier. The single entity in the union set of the current level is the set of entity identifiers belonging to the same person in the previous level. Based on all the person clusters associated with the entity identifiers queried at the last level, the final clustering result of the face image to be processed is determined.

2. The face clustering method according to claim 1, characterized in that, At each level, there is stored: a person cluster corresponding to each entity in the disjoint-set data structure, wherein the person cluster is a set of face images in a single-device person cluster associated with each entity identifier contained in the entity; The process of retrieving the identifiers of entities containing the query target from the current level's union set and using them as the query results for the current level includes: If the query hit in the current level's union set contains the identifier of the entity that is the target of the query, then the identifier of the entity that is the target of the query hit will be used as the query result of the current level. If the query in the current level's join set does not find the identifier of an entity containing the query target: The face image to be processed is clustered with the face images in all the person clusters in the current level; If the clustering result indicates that the face image to be processed belongs to any of the person clusters, then the identifier of the entity corresponding to the person cluster to which the face image to be processed belongs is used as the query result of the current level. If the clustering result indicates that the face image to be processed does not belong to any of the person clusters, then in the current level's union-find set, a new person cluster containing the face image to be processed is created, and an entity containing the cluster identifier of the newly created person cluster is created, and the identifier of the newly created entity is used as the query result of the current level.

3. The face clustering method according to claim 2, characterized in that, The process of determining the final clustering result of the face image to be processed based on the entity identifier queried at the last level includes: Get the latest update time of all person clusters associated with the entity identifier found at the last level, and the data acquisition device corresponding to the latest update time; Based on the physical distance between each of the acquisition devices and the target device, the shortest movement time from each acquisition device to the target device is estimated, and the shortest movement time corresponding to each group of people is obtained; The difference between the acquisition time of the face image to be processed and the latest update time of each person cluster is calculated as the actual motion duration corresponding to each person cluster; The cluster of people whose actual movement duration is not less than the shortest movement duration is taken as the final clustering result of the face image to be processed.

4. The face clustering method according to claim 2, characterized in that, The process of clustering the face image to be processed with the face images in all the person clusters in the current level includes: Obtain the latest update time and acquisition device for each single-device character cluster in all character clusters at the current level; Based on the physical distance between each of the acquisition devices and the target device, the shortest movement time from each acquisition device to the target device is estimated, and the shortest movement time corresponding to each single device group of people is obtained. Based on the difference between the acquisition time of the face image to be processed and the shortest movement duration corresponding to each single device group of people, the latest departure time of each group of people is determined. The face image to be processed is clustered with the face images in the single-device person cluster that meet the preset conditions, wherein the latest update time of the single-device person cluster is earlier than its corresponding latest departure time.

5. The face clustering method according to claim 2, characterized in that, The process of clustering the face image to be processed with the face images in all the person clusters in the current level includes: The features of the face image to be processed are compared with the features of the face images in each of the person clusters to obtain the similarity corresponding to each person cluster; If the similarity of all the person clusters is less than a preset threshold, the clustering result is determined to be that the face image to be processed does not belong to any of the person clusters. If the similarity value corresponding to at least one of the person clusters is not less than the preset threshold, the clustering result is determined to be that the face image to be processed belongs to the target person cluster, the target person cluster is the person cluster with the largest similarity value among all candidate person clusters, and the candidate person cluster is the person cluster with the similarity value not less than the preset threshold; If the clustering result indicates that the face image to be processed belongs to any one of the person clusters, the method further includes: Detect the image quality of the face image to be processed; If the image quality of the face image meets the preset high quality condition, determine whether all the candidate object clusters are associated with the same entity; When all candidate clusters are associated with the same entity, the entity identifier of the entity is used as the clustering result of the face image to be processed at the current level. When all candidate clusters are associated with different entities, all entities associated with all candidate clusters are merged to obtain a new entity. The entity identifier of the new entity is used as the clustering result of the face image to be processed at the current level. The merging relationship between entities in the new entity is passed to the level after the current level. In the level after the current level, the entity identifiers associated with the candidate clusters in the disjoint-setup are merged according to the merging relationship.

6. The face clustering method according to claim 1, characterized in that, After performing incremental clustering on the features of the face image to be processed and the features of face images in the existing single-device person clusters of the target device to obtain the cluster identifier of the single-device person cluster to which the face image to be processed belongs, the method further includes: Detect the image quality of the face image to be processed; If the image quality meets the preset quality conditions, calculate the similarity between the feature vector of the face image to be processed and the feature vector of each existing face image in the single device person cluster to which the face image to be processed belongs. If the similarity of each feature vector is not greater than a preset similarity threshold, the face image to be processed is added to the single-device person cluster to which the face image to be processed belongs, and the feature vector of the face image to be processed is added to the feature index of the single-device person cluster to which the face image to be processed belongs.

7. The face clustering method according to claim 6, characterized in that, The method also includes: The latest update time of the single-device person cluster to which the face image to be processed belongs is updated to the acquisition time of the face image to be processed; Iterate through the latest update time of each of the existing single-device person clusters on the target device; Calculate the difference between the latest update time and the current time to obtain the unupdated duration for each individual device cluster; Delete the single-device person clusters that have not been updated for a period not less than the preset storage period.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the face clustering method as described in any one of claims 1-7.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the face clustering method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the face clustering method as described in any one of claims 1-7.

Images