Face image clustering method and device based on classification model, equipment and medium

Abstract

The present application relates to the technical field of artificial intelligence, and especially relates to a face image clustering method and device based on a classification model, equipment and medium. The method takes a face image as a node, constructs an edge between two nodes according to the similarity of the two nodes, forms node graph data, calculates the similarity between a human body image and other human body images for any human body image, filters out a target image according to the calculation result, inputs the similarity of the human body image and the target image and the collection information of the two into a classification model, updates the node graph data according to the classification result, clusters the update result, takes the collection information as the input of the classification model, makes the classification result more accurate, provides additional information for the node graph data through the similarity between the human body images, assists the node graph data in updating, improves the accuracy of the clustering analysis based on the updated node graph data, and further improves the accuracy of the face image clustering.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and medium for clustering facial images based on a classification model. Background Technology

[0002] Currently, with the development of artificial intelligence technology, face clustering technology has been widely used in smart communities, intelligent security and other application scenarios. It can provide accurate clustering information for application tasks such as face recognition and pedestrian re-identification. Existing technologies usually extract features of face images to calculate similarity, and then perform clustering analysis on face images based on similarity to obtain clustering results.

[0003] However, in actual face capture, factors such as image blurring and differences in capture angle may cause two facial images of the same person to have different features, resulting in "one person, multiple profiles" after cluster analysis. At the same time, factors such as facial occlusion may also cause two facial images of different people to have similar features, resulting in "multiple people, one profile" after cluster analysis. This greatly reduces the accuracy of face image clustering. Therefore, how to improve the accuracy of face image clustering has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, device and medium for face image clustering based on a classification model to solve the problem of low accuracy in face image clustering.

[0005] In a first aspect, embodiments of the present invention provide a face image clustering method based on a classification model, the face image clustering method comprising:

[0006] Obtain N face images and the facial features of each face image. Using each face image as a node, calculate the first similarity between the facial features of any two nodes. Based on the first similarity, construct an edge between the corresponding two nodes to form a node graph data. N is an integer greater than one.

[0007] Obtain the human body image to which each face image belongs; for any human body image, calculate the second similarity between the human body image and other human body images; and filter out target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold.

[0008] Acquire the acquisition information of the human body image and the target image, concatenate the second similarity between the human body image and the target image, as well as the acquisition information of the human body image and the target image, input the concatenation result into the trained classification model to obtain the classification result, and update the node graph data according to the classification result;

[0009] A preset clustering algorithm is used to perform clustering analysis on the updated node graph data to obtain the clustering results of the N face images.

[0010] Secondly, embodiments of the present invention provide a face image clustering device based on a classification model, the face image clustering device comprising:

[0011] The graph construction module is used to obtain N face images and the facial features of each face image. Each face image is used as a node. The first similarity between the facial features of any two nodes is calculated. Based on the first similarity, an edge is constructed between the corresponding two nodes to form node graph data. N is an integer greater than one.

[0012] The similarity calculation module is used to obtain the human body image to which each face image belongs, calculate the second similarity between the human body image and other human body images for any human body image, and filter out target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold.

[0013] The graph update module is used to acquire the acquisition information of the human body image and the target image, concatenate the second similarity between the human body image and the target image and the acquisition information of the human body image and the target image, input the concatenation result into the trained classification model to obtain the classification result, and update the node graph data according to the classification result;

[0014] The clustering analysis module is used to perform clustering analysis on the updated node graph data using a preset clustering algorithm to obtain the clustering results of the N face images.

[0015] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face image clustering method as described in the first aspect.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the face image clustering method as described in the first aspect.

[0017] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0018] Obtain N face images and their facial features. Using each face image as a node, calculate the first similarity between the facial features of any two nodes. Construct edges between the corresponding two nodes based on the first similarity to form a node graph. Obtain the human image to which each face image belongs. For any human image, calculate the second similarity between the human image and other human images. Filter out target images from all other human images that satisfy the second similarity being greater than a preset similarity threshold. Obtain the acquisition information of the human images and the target images. Combine the second similarity between the human images and the target images with the acquisition information of the human images and the target images. The data is stitched together, and the stitched result is input into a trained classification model to obtain the classification result. The node graph data is then updated based on the classification result. A preset clustering algorithm is used to perform cluster analysis on the updated node graph data to obtain the clustering results of N face images. Human images, target images, and collected information are used as input to the classification model to make the classification results more accurate. The similarity between the human images to which the face image belongs provides additional information to the node graph data, assisting in the update of the node graph data and improving the accuracy of the node graph data update. This, in turn, improves the accuracy of the clustering results obtained from the clustering analysis based on the updated node graph data, that is, improves the accuracy of face image clustering. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for a face image clustering method based on a classification model provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart illustrating a face image clustering method based on a classification model provided in Embodiment 1 of the present invention;

[0022] Figure 3 This is a schematic diagram of the node graph data update process in a face image clustering method based on a classification model provided in Embodiment 1 of the present invention;

[0023] Figure 4 This is a schematic diagram of a face image clustering device based on a classification model provided in Embodiment 2 of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0025] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0026] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0027] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0028] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0029] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0031] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0032] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0033] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0034] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0035] The first embodiment of this invention provides a face image clustering method based on a classification model, which can be applied to, for example... Figure 1In this application environment, the client communicates with the server. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The client can be deployed in application scenarios such as face recognition, image classification, intelligent security, and product recommendation, providing face image clustering information for tasks in these scenarios. Typically, face image clustering information can be used to improve task execution efficiency or as verification information to improve task accuracy. For example, in a face recognition application, if a face image is detected within a known face image cluster, the person corresponding to that face image can be directly identified based on the cluster information without further inference from the recognition model, thus improving the efficiency of the face recognition task. Alternatively, after a face image is recognized using a recognition model, it can be checked whether the face image is included in the corresponding face image cluster information of the identified person. This face image cluster information can then be used as verification information to avoid misjudgments in face recognition, thereby improving the accuracy of the face recognition task.

[0036] See Figure 2 This is a flowchart illustrating a face image clustering method based on a classification model provided in Embodiment 1 of the present invention. The aforementioned face image clustering method can be applied to... Figure 1 The client-side computer connects to the server to obtain N face images, along with the facial features of each image, the corresponding human body image, and acquisition information. The N face images refer to the face images that need to be clustered. The client-side computer has a pre-trained classification model deployed on it, which can be used to classify whether the human body images belong to the same person. Figure 2 As shown, this face image clustering method may include the following steps:

[0037] Step S201: Obtain N face images and the facial features of each face image. Using each face image as a node, calculate the first similarity between the facial features of any two nodes. Based on the first similarity, construct an edge between the corresponding two nodes to form a node graph data.

[0038] Among them, the face image can be obtained by face extraction from the captured image. The captured image can be obtained by image acquisition devices such as surveillance cameras and surveillance video recorders. The face features can be obtained by extracting features from the face image through a face feature extraction model. N is an integer greater than one.

[0039] A node can refer to a graph node, and the data contained in a graph node is a face image. The first similarity can be used to characterize the degree of feature similarity between the face features of two nodes. An edge can refer to the connection relationship between two nodes. Node graph data can include nodes and edges. Node graph data can be a graph structure representation of N face images and the connection relationship between the face images.

[0040] Specifically, image acquisition devices such as surveillance cameras and video recorders can be deployed in specific application scenarios, such as smart security scenarios, where image acquisition devices can capture images at a fixed sampling frequency.

[0041] For the captured images, faces can be extracted using a trained first object detection model. This model can be implemented using models such as YOLO or Mask-RCNN. The output of the trained first object detection model can be the bounding box of the face region. The bounding box can be represented by the coordinates of the upper left and lower right corners of the bounding box. Based on the coordinates of the upper left and lower right corners of the bounding box, the first cropping region can be determined in the captured image. The captured image is then cropped based on the first cropping region to obtain the face image. It should be noted that the cropped face image needs to be normalized in size so that different face images can be processed in the same way in the future.

[0042] The facial feature extraction model can use the encoder part of a pre-trained facial recognition model. The encoder part usually includes convolutional layers and pooling layers. Convolutional layers can be used for feature aggregation, and pooling layers can be used for feature dimensionality reduction.

[0043] The first similarity can be calculated using methods such as cosine similarity, Euclidean distance, and Manhattan distance. In this embodiment, cosine similarity is used to calculate the first similarity. The value range of the first similarity is [0,1]. The closer the first similarity is to 1, the more similar the facial features of the two nodes used to calculate the first similarity are. The closer the first similarity is to 0, the less similar the facial features of the two nodes used to calculate the first similarity are.

[0044] Optionally, constructing an edge between two corresponding nodes based on the first similarity includes:

[0045] The first similarity is compared with a preset connection threshold to obtain the comparison result;

[0046] If the comparison result is that the first similarity is greater than the connection threshold, then an edge is constructed between the two corresponding nodes;

[0047] If the comparison result is that the first similarity is less than or equal to the connection threshold, then no edge is constructed between the corresponding two nodes.

[0048] The connection threshold can be determined by the implementer based on the actual working conditions. For example, when the first similarity is calculated using cosine similarity, its value range is [0,1]. The closer the first similarity is to 1, the more similar the facial features of the two nodes used to calculate the first similarity are. Therefore, the connection threshold can be set to 0.7 so that there is an edge connection between two nodes whose facial features are sufficiently similar.

[0049] Specifically, if the comparison result is that the first similarity is greater than the connection threshold, it means that the two facial features used to calculate the first similarity are sufficiently similar. In this case, it can be considered that the facial images corresponding to the two facial features used to calculate the first similarity belong to the same person. Therefore, edges are constructed between the nodes corresponding to the two facial images to connect them, so as to reduce the situation of "one person with multiple profiles" in the subsequent clustering process and improve the accuracy of facial image clustering.

[0050] If the comparison result is that the first similarity is less than or equal to the connection threshold, it means that the two facial features used to calculate the first similarity are not similar. In this case, it can be considered that the facial images corresponding to the two facial features used to calculate the first similarity do not belong to the same person. Therefore, the nodes corresponding to the two facial images are not connected to reduce the occurrence of "multiple people in one file" in the subsequent clustering process and improve the accuracy of facial image clustering.

[0051] In this embodiment, the connection relationship between two nodes corresponding to two facial features for which the first similarity is calculated is determined based on the comparison result of the first similarity and the connection threshold. This improves the representational ability of the constructed node graph data, enabling it to contain similarity information between facial features corresponding to nodes. This facilitates more accurate clustering in subsequent facial image clustering processes, avoiding situations of "one person, multiple profiles" or "multiple people, one profile," and effectively improving the accuracy of facial image clustering.

[0052] The steps described above—obtaining N face images and the facial features of each image, using each face image as a node, calculating the first similarity between the facial features of any two nodes, and constructing edges between the corresponding two nodes based on the first similarity to form a node graph data—create a node graph data structure based on the face images and their facial features. This allows the N face images to be represented in a structured form, facilitating the rapid discovery of relationships between nodes and enabling optimization, updates, and modifications. It avoids the need for large storage spaces and computational resources for data storage and computation when there are a large number of face images with complex relationships, effectively improving the convenience of face image clustering.

[0053] Step S202: Obtain the human body image to which each face image belongs; for any human body image, calculate the second similarity between the human body image and other human body images; and select target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold.

[0054] Among them, human body image can refer to the human body image of the person to whom the face image belongs in the captured image. The second similarity can be used to characterize the degree of similarity between human body images. The similarity threshold can be used to determine whether human body images are sufficiently similar, that is, whether the human body images belong to the same person. The target image is a human body image among other human body images that may belong to the same person as the targeted human body image.

[0055] Specifically, the face image is obtained by extracting the face from the captured image. Thus, one face image corresponds to one captured image, which is the captured image to which the face image belongs. Human body extraction is performed on this captured image. Human body extraction can be performed using a trained second object detection model. The trained second object detection model can also adopt the same architecture as the trained first object detection model, such as the YOLO model, Mask-RCNN model, etc. The output of the trained second object detection model can be the bounding box of the human body region. The bounding box of the human body region can be represented by the coordinates of the upper left corner and the lower right corner of the bounding box. Based on the coordinates of the upper left corner and the lower right corner of the bounding box, the second cropping region can be determined in the captured image. The captured image is cropped according to the second cropping region to obtain the human body image. It should be noted that the cropped human body image also needs to be normalized in size so that different human body images can be processed in the same way in the future.

[0056] Since a captured image may contain multiple people, human body extraction from the captured image may result in more than one human body image. In this case, it is necessary to determine which human body image the face image belongs to. Specifically, the intersection-union ratio (IUR) is calculated by comparing the area of ​​the intersection of the face region's bounding box and the bounding box of each human body region with the area of ​​the union of the face region's bounding box and the bounding box of each human body region. The resulting ratio is the IUR.

[0057] Determine the bounding box of the human body region corresponding to the maximum cross-union ratio among all cross-union ratios. The human body image cropped from the captured image based on the bounding box of the human body region is the human body image to which the face image belongs.

[0058] Following the above method, we can obtain the human body image to which each face image belongs. Since we obtain N face images, the number of human body images is also N. For any human body image, the number of other human body images corresponding to it is N-1. We calculate the similarity between the N-1 other human body images and the corresponding human body image to obtain N-1 second similarity scores.

[0059] The N-1 second similarities are compared with a preset similarity threshold. The second similarities greater than the similarity threshold are selected from the N-1 second similarities. The human images corresponding to all second similarities greater than the similarity threshold are the target images. In this embodiment, the preset similarity threshold can be set to 0.6. Accordingly, the second similarities need to be normalized after calculation to ensure that the value range of the second similarities is [0,1].

[0060] If the comparison result is that the second similarity is less than or equal to the similarity threshold, it means that the two human images used to calculate the second similarity are not similar. In this case, it can be considered that the two human images used to calculate the second similarity do not belong to the same person.

[0061] If the second similarity is greater than the similarity threshold, it means that the two human images used to calculate the second similarity are sufficiently similar. In this case, it can be considered that the two human images used to calculate the second similarity may belong to the same person, and other human images that may belong to the same person as the target human image are identified as the target image.

[0062] Optionally, for any human body image, calculating the second similarity between the human body image and other human body images includes:

[0063] Each human body image is input into a trained feature extraction model for feature extraction, resulting in human body features for each corresponding human body image;

[0064] For any human body image, the similarity between the human body features of the human body image and the human body features of other human body images is calculated to obtain the second similarity between the human body image and other human body images.

[0065] Among them, the trained feature extraction model can be used to extract features of human images. The trained feature extraction model can adopt the encoder structure of the trained target detection model. The input of the trained feature extraction model is the human image, and the output is the human features corresponding to the input human image. Human features can be used to represent the information of the human image.

[0066] Specifically, when directly calculating similarity using human images, the low-level information in the human images may differ due to factors such as image noise, leading to deviations in the similarity calculation results. However, by using a trained feature extraction model, high-level feature information is obtained. This high-level feature information focuses more on the information of the human body itself and can ignore irrelevant noise information, thereby improving the representational ability of the similarity calculation results.

[0067] In this embodiment, the similarity between human features can also be calculated using cosine similarity, Euclidean distance, Manhattan distance, etc.

[0068] In this embodiment, the similarity calculation is performed using the human features corresponding to the human image, thereby ignoring the noise information of the image, improving the accuracy of the similarity calculation, and also improving the representational ability of the similarity calculation results.

[0069] Optionally, selecting target images from all other human images that satisfy a second similarity greater than a preset similarity threshold includes:

[0070] Based on all the second similarities, calculate the similarity threshold, and compare each second similarity with the similarity threshold respectively;

[0071] Other human images with a similarity score greater than the second similarity threshold are identified as target images.

[0072] The similarity threshold is determined based on all actual second similarities obtained, and the calculation method can be determined by the mean, median, etc.

[0073] Specifically, in this embodiment, all second similarities are sorted in descending order to obtain a second similarity sequence. The (K+1)th element in the second similarity sequence is determined, and the similarity corresponding to the (K+1)th element is used as the similarity threshold. In this embodiment, the value of K is set to 5.

[0074] At this point, the number of second similarities greater than the similarity threshold is K, which means the number of target images is K.

[0075] In this embodiment, a dynamic adjustment method is used to determine the similarity threshold, so that the number of target images is stabilized at K. While reducing the amount of calculation, it ensures that there are not too few target images due to a fixed similarity threshold, which would result in incomplete extraction of target images of the same person and omissions. This ensures the accuracy of personnel target image extraction.

[0076] The steps described above involve obtaining the human body image to which each face image belongs, calculating the second similarity between the human body image and other human body images for any given human body image, and selecting target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold. By comparing the second similarity between the human body images to which the face image belongs with the preset similarity threshold, a target image corresponding to a human body image is determined, that is, a target image that may belong to the same person as the human body image. This facilitates subsequent judgment based on the target image and the human body image, updates the node graph data, and improves the accuracy of the information representation of the node graph data.

[0077] Step S203: Obtain the acquisition information of the human body image and the target image, concatenate the second similarity between the human body image and the target image, as well as the acquisition information of the human body image and the target image, input the concatenation result into the trained classification model, obtain the classification result, and update the node graph data according to the classification result.

[0078] The information collected may include collection time information and collection location information, namely, the image acquisition device that acquires human or target images, the time point when the image was acquired, and the world coordinate system location of the image acquisition device.

[0079] The splicing method can be concatenation. The trained classification model includes an encoder layer and a fully connected layer. The encoder layer can be used to extract the input features, and the fully connected layer can be used to map the input features to the classification space. In this embodiment, the classification result includes two categories. The first category is the same person category, which is used to represent that the input human image and the target image belong to the same person. The second category is the different person category, which is used to represent that the input human image and the target image do not belong to the same person.

[0080] The way to update node graph data based on the classification results can be by adding or deleting edges in the node graph data.

[0081] Specifically, first, the time difference T between the acquisition time of the human image and the acquisition time of the target image is calculated, and the distance D between the acquisition location of the human image and the acquisition location of the target image is calculated. Then, the second similarity α between the human image and the target image, the time difference T between the acquisition time of the human image and the acquisition time of the target image, and the distance D between the acquisition location of the human image and the acquisition location of the target image are concatenated. The concatenated result can be represented as [α,T,D], which is the input data of the trained classification model. By inputting the input data into the trained classification model, the classification result can be obtained.

[0082] In one embodiment, the second similarity α between the human image and the target image, the acquisition time information t1 of the human image, the acquisition location information d1 of the human image, the acquisition time information t2 of the target image, and the acquisition location information d2 of the target image can be directly stitched together, and the stitched result can be represented as [α, t1, d1, t2, d2].

[0083] Optionally, updating the node graph data based on the classification results includes:

[0084] When the classification result is detected as the preset first category, in the node graph data, it is detected whether there is an edge between the node corresponding to the human image and the node corresponding to the target image;

[0085] If no edge is detected between the node corresponding to the human image and the node corresponding to the target image, an edge is constructed between the node corresponding to the human image and the node corresponding to the target image to obtain the updated node graph data.

[0086] The preset first category can refer to the same person category, that is, the input human image and the target image belong to the same person, and the updated node graph data can refer to the node graph data after the edge addition process.

[0087] Specifically, when the classification result is detected as the preset first category, it means that the human image and the target image are considered to belong to the same person. Then, it is detected whether there is an edge between the node corresponding to the human image and the node corresponding to the target image. If there is no edge, it means that when forming node graph data based on the similarity between face images, the face images are considered not to belong to the same person.

[0088] Due to various factors such as the pose and lighting conditions when the image acquisition device captures images, facial images may be unclear. Since facial features are mainly concentrated in details such as eyebrows, eyes, nose, and lips, in facial recognition applications, it is usually not possible to ensure that the person being photographed is in the same fixed pose for the image acquisition device to capture. Instead, the image acquisition device captures images while the person is in continuous motion. Therefore, these details may be occluded or blurred, leading to errors in facial image similarity calculation.

[0089] At the same time, since the subjects being photographed may have their faces obscured, such as by wearing masks or glasses, it may be impossible to identify detailed facial features, which may also lead to errors in the calculation of facial image similarity.

[0090] Therefore, due to objective environmental factors in the application scenario and subjective factors of the people being photographed, the reliability of similarity calculated solely based on facial images is low, affecting the accuracy of the representation of the node graph data. The features of human images mainly focus on obvious features, such as the type and color of clothing, the distribution of key points on the human body, and information about the items carried by the person. These obvious features are generally considered to remain unchanged within a certain period of time. Therefore, the classification results based on human images and target images can help update the node graph data.

[0091] If the classification result is detected as the preset first category and there is no edge between the node corresponding to the human image and the node corresponding to the target image, it indicates that the calculation result of the face image similarity may be unreliable. An edge is constructed between the node corresponding to the human image and the node corresponding to the target image. That is, the node graph data is updated by adding new edges to obtain the updated node graph data.

[0092] In this embodiment, when the classification result is detected as the preset first category and there is no edge between the node corresponding to the human image and the node corresponding to the target image, an edge is constructed between the node corresponding to the human image and the node corresponding to the target image. This avoids the influence of objective factors of the scene and subjective factors of the person being photographed, such as large deviations in the shooting angle, which would result in low similarity calculated based on the face image and missing edges in the node graph data, leading to the situation of "one person, multiple profiles". This improves the representation accuracy of the node graph data.

[0093] Optionally, updating the node graph data based on the classification results includes:

[0094] When the classification result is detected as the preset second category, in the node graph data, it is detected whether there is an edge between the node corresponding to the human image and the node corresponding to the target image;

[0095] If an edge is detected between a node corresponding to the human image and a node corresponding to the target image, the edge between the node corresponding to the human image and the node corresponding to the target image is deleted, resulting in updated node graph data.

[0096] The preset second category can refer to different personnel categories, that is, the input human image and the target image do not belong to the same person, and the updated node graph data can refer to node graph data that has been processed by removing edges.

[0097] Specifically, when the classification result is detected as the preset second category, it means that the human image and the target image are considered to belong to different people. Then, it is detected whether there is an edge between the node corresponding to the human image and the node corresponding to the target image. If there is an edge, it means that when the node graph data is formed based on the similarity between face images, the face images are considered to belong to the same person.

[0098] If the classification result is detected as the preset second category and there is an edge between the node corresponding to the human image and the node corresponding to the target image, it also indicates that the calculation result of the face image similarity may be unreliable. The edge between the node corresponding to the human image and the node corresponding to the target image is deleted. That is, the node graph data is updated by deleting edges to obtain the updated node graph data.

[0099] In this embodiment, when the classification result is detected as the preset second category and there is an edge between the node corresponding to the human image and the node corresponding to the target image, an edge is constructed between the node corresponding to the human image and the node corresponding to the target image. This avoids the influence of objective factors of the scene and subjective factors of the person being photographed, such as image blurring or wearing a mask, which may result in a high similarity calculated based on the face image and erroneous edges in the node graph data, leading to the situation of "multiple people in one file". This improves the representation accuracy of the node graph data.

[0100] Optionally, the concatenation result of sample similarity and sample collection information is used as the training sample of the classification model, the sample category is used as the training label of the classification model, and the binary cross-entropy loss is used as the loss function when training the classification model. The sample similarity is used to characterize the similarity between two human body images. The sample collection information includes the sample collection time and sample collection location.

[0101] The training process for a classification model includes:

[0102] The sample interval is obtained by calculating the difference in the sample acquisition time corresponding to two sample human body images;

[0103] The location of the sample collection corresponding to the two sample human images is compared to obtain the location comparison result;

[0104] The sample similarity, sample interval, and location comparison results are concatenated and then input into the classification model to obtain the sample classification results.

[0105] Based on the sample classification results, sample categories, and binary cross-entropy loss, the classification loss is calculated. The classification model is then trained using the classification loss to obtain a well-trained classification model.

[0106] Among them, the sample human body image can refer to the human body image used for training the classification model, the sample collection information is the time and location information when the sample human body image was collected, and the sample category is the pre-labeled category, which includes the same person category and different people categories.

[0107] Specifically, in this embodiment, 2500 pairs of positive samples (two human images of the same person category) and 2500 pairs of negative samples (one pair of human images of different people categories) are labeled. A test set is also constructed, containing at least 1000 pairs of positive and negative samples, which helps to quantitatively assess the model's generalization performance.

[0108] Let the classification model be denoted as f, then the following mathematical function expression relationship exists: y = f(X), where X represents the concatenation result and y represents the sample classification result. In this embodiment, the classification model f can also adopt machine learning models such as random forest and support vector machine.

[0109] In this embodiment, multiple sets of positive and negative samples are used to train the classification model, which improves the model fitting accuracy and thus ensures the generalization ability and accuracy of the trained classification model.

[0110] The steps described above—acquiring the acquisition information of the human body image and the target image, concatenating the second similarity between the human body image and the target image with the acquisition information of the human body image and the target image, inputting the concatenated result into the trained classification model to obtain the classification result, and updating the node graph data based on the classification result—add the acquisition time and location information of the human body image and the target image to the input data when using the trained classification model to classify whether the human body image and the target image belong to the same person. This provides temporal and spatial information to assist in classification judgment and improves the accuracy of the classification results output by the classification model.

[0111] Step S204: Use a preset clustering algorithm to perform clustering analysis on the updated node graph data to obtain the clustering results of N face images.

[0112] The pre-defined clustering algorithm can employ community detection algorithms, such as the InfoMap algorithm. The clustering results after clustering analysis include multiple cluster sets, where each cluster set is a collection of face images of the same person.

[0113] Specifically, the results of facial image clustering can be applied to application scenarios such as facial recognition, image classification, intelligent security, and product recommendation. Taking intelligent security as an example, images of the entrance to a community within a target time period are captured to obtain several facial images. After processing these facial images, several cluster sets can be obtained. The number of facial images contained in each cluster set is different. The personnel corresponding to the cluster set with fewer facial images are likely to be non-residents or outsiders, thus providing initial screening information for regulatory personnel.

[0114] The above steps involve using a preset clustering algorithm to perform clustering analysis on the updated node graph data to obtain the clustering results of N face images. By performing clustering analysis on the updated node graph data, it is possible to effectively cluster face images of the same person into one group and face images of different people into multiple groups, thereby improving the accuracy of face image clustering.

[0115] This embodiment provides additional information to the node graph data by using the similarity between the human body images to which the face image belongs, assisting in updating the node graph data and improving the accuracy of the node graph data update. This, in turn, improves the accuracy of the clustering results obtained from the clustering analysis based on the updated node graph data, that is, improves the accuracy of face image clustering.

[0116] Corresponding to the face image clustering method based on the classification model in the above embodiment, Figure 4 The diagram illustrates the structural block diagram of a face image clustering device based on a classification model, as provided in Embodiment 2 of the present invention. This face image clustering device is applied to a client-side computer device connected to a server to obtain N face images, the facial features of each face image, the human body image to which each face image belongs, and the acquisition information of each human body image from the server. The N face images refer to the face images that need to be clustered. A trained classification model is deployed on the client-side computer device, which can be used to classify whether human body images belong to the same person. For ease of explanation, only the parts relevant to the embodiments of the present invention are shown.

[0117] See Figure 4 The face image clustering device includes:

[0118] Graph construction module 41 is used to obtain N face images and the face features of each face image. Each face image is used as a node. The first similarity between the face features of any two nodes is calculated. An edge is constructed between the corresponding two nodes based on the first similarity to form node graph data. N is an integer greater than one.

[0119] The similarity calculation module 42 is used to obtain the human body image to which each face image belongs, calculate the second similarity between the human body image and other human body images for any human body image, and filter out target images from all other human body images that meet the condition that the second similarity is greater than a preset similarity threshold.

[0120] The graph update module 43 is used to acquire the acquisition information of the human body image and the target image, and to concatenate the second similarity between the human body image and the target image as well as the acquisition information of the human body image and the target image. The concatenation result is input into the trained classification model to obtain the classification result, and the node graph data is updated according to the classification result.

[0121] The clustering analysis module 44 is used to perform clustering analysis on the updated node graph data using a preset clustering algorithm to obtain the clustering results of N face images.

[0122] Optionally, the graph construction module 41 mentioned above includes:

[0123] The first comparison unit is used to compare the first similarity with a preset connection threshold to obtain a comparison result.

[0124] The first construction unit is used to construct an edge between the two corresponding nodes if the comparison result is that the first similarity is greater than the connection threshold.

[0125] The second construction unit, if the comparison result is that the first similarity is less than or equal to the connection threshold, does not construct an edge between the corresponding two nodes.

[0126] Optionally, the similarity calculation module 42 mentioned above includes:

[0127] The feature extraction unit is used to input each human body image into the trained feature extraction model for feature extraction, and obtain the human body features of each corresponding human body image;

[0128] The feature calculation unit is used to calculate the similarity between the human features of any human image and the human features of other human images to obtain a second similarity between the human image and other human images.

[0129] Optionally, the similarity calculation module 42 mentioned above includes:

[0130] The second comparison unit is used to compare each second similarity with a similarity threshold respectively;

[0131] The target image determination unit is used to determine other human images with a second similarity greater than the similarity threshold as target images.

[0132] Optionally, the above graph update module 43 includes:

[0133] The first detection unit is used to detect whether there is an edge between the node corresponding to the human image and the node corresponding to the target image in the node graph data when the classification result is detected as a preset first category.

[0134] The first update unit is used to construct an edge between the node corresponding to the human image and the node corresponding to the target image if it is detected that there is no edge between the node corresponding to the human image and the node corresponding to the target image, so as to obtain the updated node graph data.

[0135] Optionally, the above graph update module 43 includes:

[0136] The second detection unit is used to detect, when the classification result is detected as a preset second category, whether there is an edge between the node corresponding to the human image and the node corresponding to the target image in the node graph data.

[0137] The second update unit is used to delete the edge between the node corresponding to the human image and the node corresponding to the target image if an edge is detected between the node corresponding to the human image and the node corresponding to the target image, so as to obtain the updated node graph data.

[0138] Optionally, the concatenation result of sample similarity and sample collection information is used as the training sample of the classification model, the sample category is used as the training label of the classification model, and the binary cross-entropy loss is used as the loss function when training the classification model. The sample similarity is used to characterize the similarity between two human body images. The sample collection information includes the sample collection time and sample collection location.

[0139] The aforementioned face image clustering device also includes:

[0140] The interval calculation module is used to calculate the difference in sample acquisition time between two sample human body images to obtain the sample interval.

[0141] The location comparison module is used to compare the sample collection locations corresponding to two sample human body images and obtain the location comparison results;

[0142] The sample classification module is used to concatenate the results of sample similarity, sample interval, and location comparison, and then input them into the classification model to obtain the sample classification results.

[0143] The model training module is used to calculate the classification loss based on the sample classification results, sample categories, and binary cross-entropy loss. Based on the classification loss, the classification model is trained to obtain a trained classification model.

[0144] It should be noted that the information interaction and execution process between the above modules and units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0145] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above-described embodiments of the face image clustering methods.

[0146] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0147] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0149] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0150] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0151] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0152] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0153] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A face image clustering method based on a classification model, characterized in that, The face image clustering method includes: Obtaining The face feature of each face image is obtained, and the first similarity between the face features of any two nodes is calculated, and an edge is constructed between the corresponding two nodes according to the first similarity, to form node graph data, is an integer greater than one; Obtain the human body image to which each face image belongs; for any human body image, calculate the second similarity between the human body image and other human body images; and filter out target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold. Acquire the acquisition information of the human body image and the target image, concatenate the second similarity between the human body image and the target image, as well as the acquisition information of the human body image and the target image, input the concatenation result into the trained classification model to obtain the classification result, and update the node graph data according to the classification result; The collected information includes collection time information and collection location information. A concatenation method is used to combine the second similarity between the human image and the target image, the time difference between the collection time information of the human image and the collection time information of the target image, and the distance between the collection location information of the human image and the collection location information of the target image to obtain a concatenation result. Alternatively, a concatenation method can be used to combine the second similarity between the human image and the target image, the collection time information of the human image, the collection location information of the human image, the collection time information of the target image, and the collection location information of the target image to obtain a concatenation result. The preset clustering algorithm is used to perform clustering analysis on the updated node graph data, and the clustering result of the face image is obtained. The clustering result of the face image.

2. The face image clustering method according to claim 1, characterized in that, The step of constructing an edge between two corresponding nodes based on the first similarity includes: The first similarity is compared with a preset connection threshold to obtain the comparison result; If the comparison result is that the first similarity is greater than the connection threshold, then an edge is constructed between the two corresponding nodes; If the comparison result is that the first similarity is less than or equal to the connection threshold, then no edge is constructed between the corresponding two nodes.

3. The face image clustering method according to claim 1, characterized in that, The calculation of the second similarity between any human body image and other human body images includes: Each human body image is input into a trained feature extraction model for feature extraction, resulting in human body features for each corresponding human body image; For any human body image, the similarity between the human body features of the human body image and the human body features of other human body images is calculated to obtain a second similarity between the human body image and other human body images.

4. The face image clustering method according to claim 1, characterized in that, The step of selecting target images from all other human images that satisfy the second similarity threshold being greater than the preset similarity threshold includes: Each second similarity is compared with the aforementioned similarity threshold; Other human images with a second similarity greater than the aforementioned similarity threshold are identified as target images.

5. The face image clustering method according to claim 1, characterized in that, The step of updating the node graph data based on the classification result includes: When the classification result is detected to be a preset first category, the system detects whether there is an edge between the node corresponding to the human body image and the node corresponding to the target image in the node graph data. If no edge is detected between the node corresponding to the human body image and the node corresponding to the target image, an edge is constructed between the node corresponding to the human body image and the node corresponding to the target image to obtain the updated node graph data.

6. The face image clustering method according to claim 1, characterized in that, The step of updating the node graph data based on the classification result includes: When the classification result is detected to be a preset second category, the system detects whether there is an edge between the node corresponding to the human body image and the node corresponding to the target image in the node graph data. If an edge is detected between the node corresponding to the human body image and the node corresponding to the target image, the edge between the node corresponding to the human body image and the node corresponding to the target image is deleted to obtain the updated node graph data.

7. The face image clustering method according to any one of claims 1 to 6, characterized in that, The concatenation result of sample similarity and sample collection information is used as the training sample of the classification model, the sample category is used as the training label of the classification model, and the binary cross-entropy loss is used as the loss function during the training of the classification model. The sample similarity is used to characterize the similarity between two human body images. The sample collection information includes the sample collection time and sample collection location. The training process of the classification model includes: The sample interval is obtained by calculating the difference in the sample acquisition time corresponding to the two sample human body images; The sample collection locations corresponding to the two sample human images are compared to obtain the location comparison results; The sample similarity, sample interval, and location comparison results are concatenated and then input into the classification model to obtain the sample classification results. Based on the sample classification results, the sample categories, and the binary cross-entropy loss, the classification loss is calculated. The classification model is then trained using the classification loss to obtain the trained classification model.

8. A face image clustering device based on a classification model, characterized in that, The face image clustering device includes: The graph building module is used to obtain... Given a set of face images and their facial features, each face image is used as a node. The first similarity between the facial features of any two nodes is calculated. Based on this first similarity, edges are constructed between the corresponding two nodes to form a node graph. It is an integer greater than one; The similarity calculation module is used to obtain the human body image to which each face image belongs, calculate the second similarity between the human body image and other human body images for any human body image, and filter out target images from all other human body images that satisfy the second similarity being greater than a preset similarity threshold. The graph update module is used to acquire the acquisition information of the human body image and the target image, concatenate the second similarity between the human body image and the target image and the acquisition information of the human body image and the target image, input the concatenation result into the trained classification model to obtain the classification result, and update the node graph data according to the classification result; The collected information includes collection time information and collection location information. A concatenation method is used to combine the second similarity between the human image and the target image, the time difference between the collection time information of the human image and the collection time information of the target image, and the distance between the collection location information of the human image and the collection location information of the target image to obtain a concatenation result. Alternatively, a concatenation method can be used to combine the second similarity between the human image and the target image, the collection time information of the human image, the collection location information of the human image, the collection time information of the target image, and the collection location information of the target image to obtain a concatenation result. The clustering analysis module is used to perform clustering analysis on the updated node graph data using a preset clustering algorithm to obtain the clustering results. Clustering results of Zhang's face images.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face image clustering method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the face image clustering method as described in any one of claims 1 to 7.

Images