A portrait archiving method, device, terminal and computer-readable storage medium

Abstract

The present application provides a portrait archiving method, device, terminal and computer-readable storage medium. The portrait archiving method includes: obtaining a first cluster and a second cluster; determining a first candidate set according to a first target object, and determining a second candidate set according to a second target object; determining that a first identity image and a second identity image having a preset relationship form an identity image pair; based on the first identity image in the first target object and the identity image pair, and the second identity image in the second target object and the identity image pair, determining whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair. According to the first target object and the second target object having a concomitant relationship and a preset relationship, the first identity image and the second identity image having a preset relationship, the similarity between the first target object and the first identity image, and the similarity between the second target object and the second identity image, the accuracy of cluster archiving is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a portrait archiving method, device, terminal and computer-readable storage medium. Background Art

[0002] With the widespread popularity of intelligent video surveillance equipment, a huge amount of portrait images are accumulated every day. Using portrait clustering methods to archive portrait images by person is a common method. Faced with a large number of portrait images, the actual effect of portrait clustering based only on the similarity between image features is often greatly restricted. In particular, when the facial features of children have not yet been formally formed, there is often a low degree of facial recognition; due to wrinkles and other reasons, the corresponding facial features of the elderly are also greatly reduced in recognizability, resulting in a low accuracy rate in the identity filing of the portrait image set. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a portrait archiving method, device, terminal and computer-readable storage medium to solve the problem of low accuracy of identity archiving of portrait image sets in the prior art.

[0004] To solve the above technical problems, the first technical solution adopted in the present application is: to provide a portrait archiving method, the portrait archiving method comprising: obtaining a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; determining a first candidate set according to the first target object, and determining a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; determining that the first identity image and the second identity image having a preset relationship constitute an identity image pair; based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair.

[0005] Wherein, a first candidate set is determined according to a first target object, and a second candidate set is determined according to a second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image, including: calculating a first similarity between the first target object and each identity image in an identity image library; comparing the first similarity with a first similarity threshold; in response to the first similarity being greater than the first similarity threshold, clustering the identity images corresponding to the first similarity to obtain a first candidate set; the identity images in the first candidate set are used as first identity images; calculating a second similarity between the second target object and each identity image in the identity image library; comparing the second similarity with a second similarity threshold; in response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the second similarity to obtain a second candidate set; the identity images in the second candidate set are used as second identity images.

[0006] Among them, in response to the first similarity being greater than the first similarity threshold, the identity images corresponding to the first similarity are clustered to obtain a first candidate set, including: in response to the first similarity being greater than the first similarity threshold, the identity images corresponding to the first similarities of the top preset number are selected to cluster to obtain a first candidate set, and the first candidate set includes the first identity images of the preset first number; in response to the second similarity being greater than the second similarity threshold, the identity images corresponding to the second similarity are clustered to obtain a second candidate set, including: in response to the second similarity being greater than the second similarity threshold, the identity images corresponding to the second similarities of the top preset number are selected to cluster to obtain a second candidate set, and the second candidate set includes the second identity images of the preset second number.

[0007] The first cluster includes multiple images containing a first target object, and the second cluster includes multiple images containing a second target object; calculating the first similarity between the first target object and each identity image in the identity image library includes: determining the average feature vector of the first target object corresponding to the first cluster according to the feature vector of the first target object contained in each image in the first cluster; calculating the first similarity between the average feature vector of the first target object and the feature vector of each identity image in the identity image library; in response to the first similarity being greater than a first similarity threshold, clustering the identity images corresponding to the first similarity, including: in response to the first similarity being greater than the first similarity threshold, clustering the calculated identity images The identity images corresponding to the feature vectors of the first similarity are clustered; the second similarity between the second target object and each identity image in the identity image library is calculated, including: determining the average feature vector of the second target object corresponding to the second cluster according to the feature vector of the second target object contained in each image in the second cluster; calculating the second similarity between the average feature vector of the second target object and the feature vector of each identity image in the identity image library; in response to the second similarity being greater than a second similarity threshold, the identity images corresponding to the second similarity are clustered, including: in response to the second similarity being greater than the second similarity threshold, the identity images corresponding to the feature vectors calculated to obtain the second similarity are clustered.

[0008] Among them, the preset relationship is a kinship relationship; determining that a first identity image and a second identity image with the preset relationship form an identity image pair, including: judging whether the first identity image in the first candidate set and the second identity image in the second candidate set have a kinship relationship; if the first identity image and the second identity image have a kinship relationship, the first identity image and the second identity image with the kinship relationship form an identity image pair, wherein there is at least one identity image pair.

[0009] Wherein, based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is filed in the first identity image in the identity image pair, and whether the second cluster is filed in the second identity image in the same identity image pair, also includes: obtaining the weighted similarity corresponding to each identity image pair according to the weighted sum of the similarity between the first identity image and the first target object and the similarity between the second identity image and the second target object in each identity image pair; judging whether the weighted similarity corresponding to each identity image pair is greater than a similarity threshold; if the weighted similarity is greater than the similarity threshold, filing the first cluster in the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster in the second identity image in the identity image pair corresponding to the similarity.

[0010] Among them, the age grade of the first target object is a child, and the age grade of the second target object is an old man; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is a third similarity threshold; judging whether the weighted similarities corresponding to each identity image pair are greater than the similarity threshold, including: judging whether the weighted similarities corresponding to each identity image pair are greater than the third similarity threshold; if the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity, including: if the weighted similarity is greater than the third similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity.

[0011] Among them, the age grade of the first target object is children, and the age grade of the second target object is youth; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is a fourth similarity threshold; judging whether the weighted similarities corresponding to each identity image pair are greater than the similarity threshold, including: judging whether the weighted similarities corresponding to each identity image pair are greater than the fourth similarity threshold; if the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity, including: if the weighted similarity is greater than the fourth similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity.

[0012] Among them, the age grade of the first target object is the elderly, and the age grade of the second target object is the young; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is the fifth similarity threshold; judging whether the weighted similarities corresponding to each identity image pair are greater than the similarity threshold, including: judging whether the weighted similarities corresponding to each identity image pair are greater than the fifth similarity threshold; if the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity, including: if the weighted similarity is greater than the fifth similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the identity image pair corresponding to the similarity.

[0013] Wherein, obtaining the first cluster and the second cluster includes: obtaining multiple image sets, where the image sets include multiple images of the same target object; determining the trajectory of the target object corresponding to each image set based on the shooting time of the image; and determining at least two image sets with similar trajectories based on the overlap between the trajectories of the target objects.

[0014] Among them, the shooting time and shooting location of the image constitute the space-time point of the image, and according to the overlap between the trajectories of each target object, at least two image sets with similar trajectories are determined, including: counting the number of space-time points that overlap with each other between the trajectories of each target object; judging whether the number of overlapping space-time points is greater than a preset number; if the number of overlapping space-time points is greater than the preset number, determining that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0015] Among them, the shooting time and shooting location of the image constitute the space-time point of the image, and according to the overlap between the trajectories of each target object, at least two image sets with similar trajectories are determined, including: identifying and selecting a preset number of shooting locations that frequently appear in the trajectory of each target object; judging whether the number of space-time points that overlap between the space-time points corresponding to the shooting locations that frequently appear in the trajectory of each target object is greater than the preset number, if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0016] Among them, the shooting time and shooting location of the image constitute the time-space point of the image, and according to the overlap between the trajectories of each target object, at least two image sets with similar trajectories are determined, including: determining the shooting location corresponding to each image in the trajectory of each target object as the spatial trajectory of the target object; counting the number of shooting locations that overlap with each other between the spatial trajectories of each target object; judging whether the number of the overlapping shooting locations is greater than a preset number; if the number of the overlapping shooting locations is greater than the preset number, it is determined that the two image sets corresponding to the spatial trajectory of the target object have similar trajectories.

[0017] Wherein, according to the overlap between the trajectories of the target objects, at least two image sets with similar trajectories are determined, and the similarity between the trajectories of the target objects is calculated by formula 1:

[0018]

[0019] In Formula 1: is the trajectory similarity between trajectory A and trajectory B; represents the distance between two spatial points; Head(X) is the first shooting location of trajectory X; Rest(X) is the set of elements remaining in trajectory X except the first element.

[0020] Among them, obtaining the first cluster and the second cluster includes: calculating the average age corresponding to each image set in at least two image sets with similar trajectories; determining the age level of the target object corresponding to the image set based on the average age corresponding to the image set; and taking one image set of two image sets with different age levels of corresponding target objects and similar trajectories as the first cluster and the other as the second cluster.

[0021] Wherein, based on the average age corresponding to the image set, determining the age grade of the target object corresponding to the image set includes: in response to the average age corresponding to the image set being in a first preset age group, determining the age grade of the target object corresponding to the image set is a child.

[0022] Wherein, based on the average age corresponding to the image set, determining the age grade of the target object corresponding to the image set includes: in response to the average age corresponding to the image set being in a second preset age group, determining the age grade of the target object corresponding to the image set is youth.

[0023] Wherein, based on the average age corresponding to the image set, determining the age grade of the target object corresponding to the image set includes: in response to the average age corresponding to the image set being in a third preset age group, determining the age grade of the target object corresponding to the image set is elderly.

[0024] In order to solve the above technical problems, the second technical solution adopted in the present application is: to provide a portrait archiving device, the portrait archiving device comprising: an acquisition module, used to acquire a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; a comparison module, used to determine a first candidate set according to the first target object, and to determine a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; an analysis module, used to determine that the first identity image and the second identity image with a preset relationship constitute an identity image pair; and a archiving module, used to determine whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair.

[0025] In order to solve the above technical problems, the third technical solution adopted in the present application is: to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is used to execute program data to implement the steps in the above portrait archiving method.

[0026] In order to solve the above technical problems, the fourth technical solution adopted in the present application is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above portrait archiving method are implemented.

[0027] The beneficial effects of the present application are: different from the prior art, a portrait archiving method, device, terminal and computer-readable storage medium are provided, and the portrait archiving method includes: obtaining a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; determining a first candidate set according to the first target object, and determining a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; determining that the first identity image and the second identity image with a preset relationship constitute an identity image pair; based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair. The present application determines whether a first cluster corresponding to the first target object and a second cluster corresponding to the second target object are respectively filed in the first identity image and the second identity image of the same identity image pair based on a first target object and a second target object having a companion relationship and a preset relationship, a first identity image and a second identity image having a preset relationship, the similarity between the first target object and the first identity image, and the similarity between the second target object and the second identity image, thereby improving the accuracy of identity filing of a portrait image set. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 It is a flowchart of the portrait archiving method provided by this application;

[0030] Figure 2 It is a flowchart of an embodiment of a portrait archiving method provided by the present application;

[0031] Figure 3 It is a flowchart of a specific embodiment of the portrait archiving method provided by the present application;

[0032] Figure 4It is a flowchart of another embodiment of the portrait archiving method provided by the present application;

[0033] Figure 5 yes Figure 3 A flowchart of a specific embodiment of step S207 in the image capture method provided;

[0034] Figure 6 yes Figure 3 A flowchart of another specific embodiment of step S207 in the image capture method provided;

[0035] Figure 7 yes Figure 3 A schematic diagram of a first candidate set and a second candidate set obtained in step S207 of the provided image capture method;

[0036] Figure 8 yes Figure 3 The identity image pair obtained in step S209 of the provided image capture method;

[0037] Fig. 9 is a schematic block diagram of an embodiment of a portrait archiving device provided by the present application;

[0038] Fig.10 is a schematic block diagram of another embodiment of the portrait archiving device provided by the present application;

[0039] Fig.11 is a schematic block diagram of an implementation scheme of a terminal provided by the present application;

[0040] Fig.12 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0041] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0042] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0043] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0044] In order to enable those skilled in the art to better understand the technical solution of the present application, a portrait archiving method provided by the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] The inventor of this application considers that it may not be possible to correctly complete the archiving of portrait clustering from the perspective of a single facial feature. It is observed that in real life, children are often accompanied by their parents or grandparents. Elderly people with limited mobility are often accompanied by their children. Therefore, it is considered to further optimize the accuracy of archiving after portrait clustering from the perspective of personnel accompaniment and kinship generation relationship.

[0046] See also Figure 1 and Figure 2 , Figure 1 It is a flowchart of the portrait archiving method provided by this application; Figure 2 1 is a flow chart of an embodiment of a portrait archiving method provided by the present application. In this embodiment, a portrait archiving method is provided, and the portrait archiving method includes the following steps.

[0047] S11: Obtain the first type of clusters and the second type of clusters.

[0048] Specifically, the first cluster is an image including a first target object, the second cluster is an image including a second target object, and the first target object and the second target object have a companion relationship and a preset relationship. The preset relationship includes a kinship generation relationship, and the kinship generation relationship is at least any one of a grandparent-grandchild relationship, a father-son relationship, a father-daughter relationship, a mother-son relationship, and a mother-daughter relationship. The companion relationship indicates that the first target object and the second target object are at the same shooting location at the same shooting time, and may also indicate that the first target object and the second target object have similar trajectory information. In other words, the first target object and the second target object appear in the same image, at the same shooting time and shooting location, that is, it is determined that the first target object and the second target object have a companion relationship. On the basis that the first target and the second target object have a companion relationship, the age level of the first target object and the second target object is different, that is, it is determined that the first target object and the second target object have a preset relationship.

[0049] In a specific embodiment, the first cluster may be an image containing the first target object, and the second cluster may be an image containing the second target object. In a specific embodiment, the first cluster may be multiple images containing the first target object, and the second cluster may be multiple images containing the second target object.

[0050] Prior to this, the method further includes acquiring a first type of cluster and a second type of cluster having an accompanying relationship and a preset relationship from the acquired multiple image sets.

[0051] In one embodiment, multiple image sets are acquired, each image set including multiple images of the same target object; based on the shooting time of the image, the trajectory of the target object corresponding to each image set is determined; and based on the overlap between the trajectories of the target objects, at least two image sets with similar trajectories are determined.

[0052] In a specific embodiment, the shooting time and shooting location of the image constitute the space-time points of the image, and the number of space-time points that overlap with each other between the trajectories of each target object is counted; it is determined whether the number of overlapping space-time points is greater than a preset number; if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0053] In a specific embodiment, a preset number of shooting locations that frequently appear in the trajectory of each target object is identified and selected; it is determined whether the number of overlapping space-time points corresponding to the shooting locations that frequently appear in the trajectory of each target object is greater than the preset number; if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0054] In a specific embodiment, the shooting location corresponding to each image in the trajectory of each target object is determined as the spatial trajectory of the target object; the number of shooting locations that overlap between the spatial trajectories of each target object is counted; and it is determined whether the number of overlapping shooting locations is greater than a preset number; if the number of overlapping shooting locations is greater than the preset number, it is determined that the two image sets corresponding to the spatial trajectory of the target object have similar trajectories.

[0055] Specifically, the similarity between the trajectories of the target objects is calculated by formula 1:

[0056]

[0057] In formula 1: S DWT is the trajectory similarity between trajectory A and trajectory B; dist(·) represents the distance between two spatial points; Head(X) is the first shooting location of trajectory X; Rest(X) is the set of elements remaining in trajectory X except the first element.

[0058] In one embodiment, the average age corresponding to each of at least two image sets with similar trajectories is calculated; based on the average age corresponding to the image sets, the age level of the target object corresponding to the image sets is determined; one of the two image sets with different age levels of the corresponding target objects and similar trajectories is used as the first cluster, and the other is used as the second cluster. The age levels include the elderly, children, and young people.

[0059] In a specific embodiment, in response to the average age corresponding to the image set being in a first preset age group, the age level of the target object corresponding to the image set is determined to be a child. In response to the average age corresponding to the image set being in a second preset age group, the age level of the target object corresponding to the image set is determined to be a young person. In response to the average age corresponding to the image set being in a third preset age group, the age level of the target object corresponding to the image set is determined to be an elderly person.

[0060] S12: Determine a first candidate set according to the first target object, and determine a second candidate set according to the second target object.

[0061] Specifically, the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image.

[0062] In one embodiment, a first similarity between a first target object and each identity image in an identity image library is calculated; the first similarity is compared with a first similarity threshold; in response to the first similarity being greater than the first similarity threshold, the identity images corresponding to the first similarity are clustered to obtain a first candidate set; the identity images in the first candidate set are used as first identity images;

[0063] In a specific embodiment, the first similarity between the feature vector of the first target object and the feature vectors of each identity image in the identity image library is calculated. In response to the first similarity being greater than a first similarity threshold, the identity images corresponding to the feature vectors having the first similarity calculated are clustered.

[0064] In a specific embodiment, in response to the first similarity being greater than a first similarity threshold, a preset first number of identity images corresponding to the first similarities ranked high are selected for clustering to obtain a first candidate set, which includes a preset first number of first identity images.

[0065] In one embodiment, a second similarity between the second target object and each identity image in the identity image library is calculated; the second similarity is compared with a second similarity threshold; in response to the second similarity being greater than the second similarity threshold, the identity images corresponding to the second similarity are clustered to obtain a second candidate set; the identity images in the second candidate set are used as second identity images.

[0066] In a specific embodiment, the second similarity between the feature vector of the second target object and the feature vectors of each identity image in the identity image library is calculated. In response to the second similarity being greater than a second similarity threshold, the identity images corresponding to the feature vectors having the second similarity calculated are clustered.

[0067] In a specific embodiment, in response to the second similarity being greater than a second similarity threshold, a preset second number of top-ranked identity images corresponding to the second similarities are selected for clustering to obtain a second candidate set, which includes a preset second number of second identity images.

[0068] S13: Determine that a first identity image and a second identity image having a preset relationship form an identity image pair.

[0069] Specifically, the preset relationship is a kinship relationship. It is determined whether the first identity image in the first candidate set and the second identity image in the second candidate set have a kinship relationship; if the first identity image and the second identity image have a kinship relationship, the first identity image and the second identity image with the kinship relationship form an identity image pair, wherein the identity image pair is at least one.

[0070] S14: Based on the first target object and the first identity image in the identity image pair and the second target object and the second identity image in the identity image pair, determine whether the first cluster is filed in the first identity image in the identity image pair and whether the second cluster is filed in the second identity image in the same identity image pair.

[0071] Specifically, the weighted similarity corresponding to each identity image pair is obtained according to the weighted sum of the similarity between the first identity image and the first target object and the similarity between the second identity image and the second target object in each identity image pair; it is determined whether the weighted similarity corresponding to each identity image pair is greater than a similarity threshold; if the weighted similarity is greater than the similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0072] In a specific embodiment, the age grade of the first target object is a child, and the age grade of the second target object is an elderly person; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is a third similarity threshold; it is determined whether the weighted similarities corresponding to each identity image pair are greater than the third similarity threshold; if the weighted similarity is greater than the third similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0073] In another specific embodiment, the age grade of the first target object is children, and the age grade of the second target object is youth; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is a fourth similarity threshold; it is determined whether the weighted similarities corresponding to each identity image pair are greater than the fourth similarity threshold; if the weighted similarity is greater than the fourth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0074] In another specific embodiment, the age grade of the first target object is elderly, and the age grade of the second target object is young; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is the fifth similarity threshold; it is determined whether the weighted similarities corresponding to each identity image pair are greater than the fifth similarity threshold; if the weighted similarity is greater than the fifth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0075] The portrait archiving method provided by the present embodiment includes: obtaining a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; determining a first candidate set according to the first target object, and determining a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; determining that the first identity image and the second identity image having a preset relationship constitute an identity image pair; based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair. The present application determines whether a first cluster corresponding to the first target object and a second cluster corresponding to the second target object are respectively filed in the first identity image and the second identity image of the same identity image pair based on a first target object and a second target object having a companion relationship and a preset relationship, a first identity image and a second identity image having a preset relationship, the similarity between the first target object and the first identity image, and the similarity between the second target object and the second identity image, thereby improving the accuracy of identity filing of a portrait image set.

[0076] See also Figure 3 and Figure 4 , Figure 3It is a flowchart of a specific embodiment of the portrait archiving method provided by the present application; Figure 4 1 is a flow chart of another embodiment of the portrait archiving method provided by the present application. In this embodiment, a portrait archiving method is provided, and the portrait archiving method includes the following steps.

[0077] S201: Acquire multiple image sets.

[0078] Specifically, the image set includes multiple images containing the same target object; the shooting time and shooting location of the image constitute the time-space point of the image. In this embodiment, an image set includes multiple portrait images containing the same target object. In another embodiment, an image set includes multiple face images containing the same target object, and may also include multiple head and shoulder images containing the same target object. Among them, one image set corresponds to one target object. In other words, the image set is multiple images of the same target object that are accurately clustered.

[0079] S202: Determine the trajectory of the target object corresponding to each image set based on the shooting time of the image.

[0080] Specifically, in order to determine the trajectory of the target object corresponding to the image set, all images in the image set corresponding to the target object can be sorted according to the shooting time of each image, and the shooting location of the image is used as the location of the target object at the shooting time to form the trajectory information of the target object.

[0081] S203: Determine at least two image sets having similar trajectories according to the overlap between the trajectories of the target objects.

[0082] Specifically, the overlap between the trajectories of the corresponding target objects is measured according to the number of overlaps of the spatiotemporal points between the trajectories of the target objects. The more the number of overlaps of the spatiotemporal points between the trajectories of the target objects, the higher the overlap between the trajectories of the corresponding target objects. If the overlap between the trajectories of the target objects exceeds the preset overlap, it indicates that the two target objects corresponding to the overlap have a companion relationship.

[0083] In one embodiment, image sets with similar trajectories can be determined based on a method of trajectory overlap points. Specifically, the number of space-time points that overlap between the trajectories of each target object is counted; it is determined whether the number of overlapping space-time points is greater than a preset number; if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories. If the number of overlapping space-time points is less than the preset number, it is determined that the trajectories of the target objects corresponding to the two image sets corresponding to the overlapping space-time points are not similar.

[0084] In one embodiment, in order to reduce the amount of calculation and improve the calculation efficiency, the image set with similar trajectories can be determined based on the frequent item mining method. Specifically, a preset number of shooting locations that frequently appear in the trajectory of each target object is identified and selected. For example, the frequency of each shooting location in the trajectory information of the target object is identified by the location recognition model, and a preset number of shooting locations with the highest frequency of shooting locations are selected, and the space-time points corresponding to the selected shooting locations are used as the important space-time points corresponding to the trajectory of the target object. In a specific embodiment, the frequency of each shooting location in the trajectory information of the target object can be identified by using Apriori, FP-Growth, PrefixSpan and other association mining methods. Then, it is determined whether the number of space-time points that overlap between the space-time points corresponding to the shooting locations that frequently appear in the trajectory of each target object is greater than the preset number. For example, it is determined whether the number of space-time points that overlap between the important space-time points corresponding to the trajectory of each target object is greater than the preset number. If the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories. If the number of the overlapping space-time points is less than a preset number, it is determined that the trajectories of the target objects respectively corresponding to the two image sets corresponding to the overlapping space-time points are not similar.

[0085] In one embodiment, the trajectory similarity is mined based on the DWT (Dynamic Time Warping Distance) method. The shooting location corresponding to each image in the trajectory of each target object is determined as the spatial trajectory of the target object; the number of shooting locations that overlap between the spatial trajectories of each target object is counted; it is determined whether the number of overlapping shooting locations is greater than a preset number; if the number of overlapping shooting locations is greater than a preset number, it is determined that the two image sets corresponding to the spatial trajectory of the target object have similar trajectories.

[0086] In a specific embodiment, the image set includes three images, and the three space-time points corresponding to the three images are sorted in time order, namely (8 o'clock, location 1), (11 o'clock, location 2) and (12 o'clock, location 3). In order to more conveniently measure the similarity of the trajectories corresponding to two target objects, only the similarity of the shooting locations can be considered, and the trajectory corresponding to the image set can be composed of (location 1, location 2, location 3). For example, the shooting locations of trajectory A corresponding to a target object are sorted according to the order of shooting time, and the resulting sequence is (a1, a2, ..., a3). n ); After the shooting locations of the trajectory B corresponding to a target object are sorted according to the order of shooting time, the resulting sequence is (b1, b2, ..., b n ).

[0087] Specifically, the trajectory similarity between trajectory A and trajectory B is calculated by formula 1:

[0088]

[0089] In formula 1: S DWT is the trajectory similarity between trajectory A and trajectory B; dist(·) represents the distance between two spatial points; Head(X) is the first shooting location of trajectory X; Rest(X) is the set of elements remaining in trajectory X except the first element.

[0090] S204: Calculate and obtain the average age corresponding to each of at least two image sets having similar trajectories.

[0091] Specifically, in order to determine the age level of the target object corresponding to the image set, the target objects contained in all the images in the image set are identified, and the ages of the target objects contained in all the images are obtained. In order to obtain a more accurate age of the target object, the ages of the target objects corresponding to all the images in the image set are added up, and the average age is calculated, and the obtained average age is determined as the average age of the target object corresponding to the image set.

[0092] In a specific embodiment, the age of the target object in each image in the image set is respectively identified based on the portrait age recognition module to obtain the age corresponding to the target object in each image. The age of the target object in each image in the image set can also be obtained by other methods.

[0093] In order to better utilize kinship, a first cluster and a second cluster that meet preset conditions are determined in at least two image sets with similar trajectories, and the average age of the target objects corresponding to the at least two image sets with similar trajectories is calculated, thereby determining the average age corresponding to the image sets.

[0094] S205: Determine the age level of the target object corresponding to the image set based on the average age corresponding to the image set.

[0095] Specifically, the age levels include children, youth, elderly and other four age levels. For example, the first preset age group is pre-set to 0-7 years old, and the corresponding age level of the target object is children; the second preset age group is pre-set to 18-45 years old, and the corresponding age level of the target object is youth; the third preset age group is pre-set to be greater than 60 years old, and the corresponding age level of the target object is elderly; the age level of target objects in other age ranges is others. In other embodiments, the preset age groups corresponding to the age levels can also be limited according to actual conditions. In this embodiment, image sets with other age levels are not considered, and image sets with age levels of children, youth and elderly are screened from at least two image sets with similar trajectories.

[0096] In one embodiment, in response to the average age corresponding to the image set being in a first preset age range, the age grade of the target object corresponding to the image set is determined to be a child.

[0097] In another embodiment, in response to the average age corresponding to the image set being in a second preset age range, the age grade of the target object corresponding to the image set is determined to be youth.

[0098] In another embodiment, in response to the average age corresponding to the image set being in a third preset age range, the age grade of the target object corresponding to the image set is determined to be elderly.

[0099] S206: One of the two image sets with different age levels and similar trajectories of the corresponding target objects is taken as a first cluster, and the other is taken as a second cluster.

[0100] Specifically, the target objects corresponding to the image sets are divided into age levels and two image sets with accompanying relationships containing different age levels are screened out, one of which is used as the first cluster and the other as the second cluster.

[0101] In a specific embodiment, in order to improve the accuracy of filing image sets containing pictures of the elderly or children, and based on three special scenarios of the elderly with children, parents with children, and children accompanying the elderly, the preset conditions are set as the elderly and children with a relative generation relationship, the elderly and young people with a relative generation relationship, or the young people and children with a relative generation relationship. Based on the preset conditions, each image set in at least two image sets with similar trajectories is combined in pairs.

[0102] Two image sets meeting preset conditions are selected from at least two image sets with similar trajectories, one of which is used as a first cluster and the other as a second cluster. The first cluster includes multiple images containing a first target object, and the second cluster includes multiple images containing a second target object.

[0103] In a specific embodiment, the three image sets with similar trajectories are screened and determined to be A, B, and C, and the age levels corresponding to the three image sets A, B, and C are different. For example, the age level of image set A is children, the age level of image set B is the elderly, and the age level of image set C is children. Then, according to the preset relationship, two pairs of image sets with kinship relationships are determined, specifically (A, B) and (B, C). For example, the age level of image set A is children, the age level of image set B is the elderly, and the age level of image set C is youth. Then, according to the preset relationship, three pairs of image sets with kinship relationships are determined, specifically (A, B), (B, C), and (C, A). For example, in the image set pair (A, B), one of image set A and image set B is used as the first cluster, and the other is used as the second cluster.

[0104] S207: Determine a first candidate set according to the first target object, and determine a second candidate set according to the second target object.

[0105] See also Figure 5 , Figure 6 and Figure 7 , Figure 5 yes Figure 3 A flowchart of a specific embodiment of step S207 in the image capture method provided; Figure 6 yes Figure 3 A flowchart of another specific embodiment of step S207 in the image capture method provided; Figure 7 yes Figure 3 A schematic diagram of a first candidate set and a second candidate set obtained in step S207 in the image capture method provided.

[0106] Specifically, in order to preliminarily determine the target identities to which the first cluster and the second cluster belong, the target objects corresponding to each cluster are compared with each preset target in the identity image library.

[0107] See also Figure 5 In one embodiment, the specific steps of determining the first candidate set according to the first target object are as follows.

[0108] S2071a: Calculate and obtain a first similarity between the first target object and each identity image in the identity image library.

[0109] Specifically, according to the feature vector of the first target object contained in each image in the first cluster, the average feature vector of the first target object corresponding to the first cluster is determined. The average feature vector of the first target object is the centroid of the first cluster. The first similarity corresponding to the average feature vector of the first target object and the feature vectors of each identity image in the identity image library is calculated. Specifically, the first similarity is cosine similarity.

[0110] S2071b: Compare the first similarity with a first similarity threshold.

[0111] Specifically, in order to reduce the amount of calculation, only identity images with higher similarity in the identity image library need to be recalled, and then the calculated first similarity needs to be compared with the first similarity threshold.

[0112] S2071c: In response to the first similarity being greater than a first similarity threshold, clustering the identity images corresponding to the first similarity to obtain a first candidate set.

[0113] Specifically, if the calculated first similarity is greater than the first similarity threshold, the identity image corresponding to the first similarity is recalled, and the recalled identity images are clustered to obtain a first candidate set. The identity images in the first candidate set are used as the first identity images. In a specific embodiment, in response to the first similarity being greater than the first similarity threshold, a preset first number of identity images corresponding to the first similarities that are ranked top are selected for clustering to obtain a first candidate set, and the first candidate set includes a preset first number of first identity images.

[0114] In a specific embodiment, in response to the first similarity being greater than a first similarity threshold, the identity images corresponding to the feature vectors for which the first similarity is calculated are clustered.

[0115] In this embodiment, the first similarity threshold may be set to 0.8-0.85, or may be set according to actual conditions, and is not limited here.

[0116] In this embodiment, the first candidate set includes N first identity images, wherein the value range of N is 5-20, and can also be set according to actual conditions, and is not limited here.

[0117] See also Figure 7 In a specific embodiment, the first target object or the first cluster centroid is represented by a square 1, and the corresponding age level is the elderly. The first similarity threshold is set to 0.85, and the preset first number N is set to 5. Then, in the identity image library, a maximum of 5 identity images whose similarity with the first target object or the first cluster centroid is greater than the first similarity threshold are recalled, such as A, B, C, D, and E; wherein the similarities between the first target object or the first cluster centroid and identity images A, B, C, D, and E are 0.93, 0.92, 0.89, 0.86, and 0.85, respectively.

[0118] See also Figure 6 In one embodiment, the specific steps of determining the second candidate set according to the second target object are as follows.

[0119] S2072a: Calculate and obtain a second similarity between the second target object and each identity image in the identity image library.

[0120] Specifically, according to the feature vector of the second target object contained in each image in the second cluster, the average feature vector of the second target object corresponding to the second cluster is determined. The average feature vector of the second target object is the centroid of the second cluster. The second similarity between the average feature vector of the second target object and the feature vectors of each identity image in the identity image library is calculated. Specifically, the second similarity is cosine similarity.

[0121] S2072b: Compare the second similarity with a second similarity threshold.

[0122] Specifically, in order to reduce the amount of calculation, only identity images with higher similarity in the identity image library need to be recalled, and then the calculated second similarity needs to be compared with the second similarity threshold.

[0123] S2072c: In response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the second similarity to obtain a second candidate set.

[0124] Specifically, if the calculated second similarity is greater than the second similarity threshold, the identity image corresponding to the second similarity is recalled, and the recalled identity images are clustered to obtain a second candidate set. The identity images in the second candidate set are used as second identity images. In a specific embodiment, in response to the second similarity being greater than the second similarity threshold, a preset second number of identity images corresponding to the second similarities that are ranked top are selected for clustering to obtain a second candidate set, and the second candidate set includes a preset second number of second identity images.

[0125] In a specific embodiment, in response to the second similarity being greater than a second similarity threshold, the identity images corresponding to the feature vectors for which the second similarity is calculated are clustered.

[0126] In this embodiment, the second similarity threshold may be set to 0.8-0.85, or may be set according to actual conditions, and is not limited here.

[0127] In this embodiment, the second candidate set includes N second identity images, wherein the value range of N is 5 to 20, and can also be set according to actual conditions, and is not limited here.

[0128] See also Figure 7 In a specific embodiment, the second target object or the second cluster centroid is represented by a circle 2, and the corresponding age level is a child. The second similarity threshold is set to 0.85, and the preset second number N is also set to 5, then a maximum of 5 identity images corresponding to the second target object or the second cluster centroid are recalled in the identity image library, such as f, b, g, e, and h, whose similarity with the second target object or the second cluster centroid is greater than the second similarity threshold; wherein the similarity between the second target object or the second cluster centroid and the identity images f, b, g, e, and h are 0.91, 0.90, 0.89, 0.87, and 0.86, respectively.

[0129] S208: Determine whether the first identity image in the first candidate set and the second identity image in the second candidate set have a kinship relationship.

[0130] Specifically, the identity image library is searched to see whether there is a kinship relationship between the first identity image in the first candidate set and the second identity image in the second candidate set. The kinship relationship specifically includes a kinship generation relationship. The kinship generation relationship is any one of a grandparent-grandchild relationship, a father-son relationship, a father-daughter relationship, a mother-son relationship, and a mother-daughter relationship.

[0131] In a specific embodiment, the first candidate set includes first identity images A, B, C, D, E, and the second candidate set includes second identity images f, b, g, e, h. The identity image library is further searched to see whether any one of the first identity images A, B, C, D, E has a kinship relationship with any one of the second identity images f, b, g, e, h.

[0132] If there is a kinship relationship between the first identity image and the second identity image, the process directly jumps to step S209 ; if there is no kinship relationship between the first identity image and the second identity image, the process directly jumps to step S210 .

[0133] S209: Combining the first identity image and the second identity image having a kinship relationship into an identity image pair.

[0134] Specifically, if the first identity image and the second identity image have a kinship relationship, the first identity image and the second identity image having the kinship relationship are combined into an identity image pair, wherein there is at least one identity image pair.

[0135] See also Figure 8 , Figure 8 yes Figure 3 The identity image pair obtained in step S209 of the provided image capture method.

[0136] In a specific embodiment, the first identity image and the second identity image that meet the kinship generation relationship are selected by searching the identity image library for the relationship between any one of the first identity images A, B, C, D, E and each of the second identity images f, b, g, e, h. After searching, it is determined that the first identity image B and the second identity image b have a grandfather-grandson relationship, and the first identity image E and the second identity image e have a grandfather-grandson relationship. It is then determined that the first target object and the second target object with the accompanying relationship and the kinship generation relationship may belong to the first identity image B and the second identity image b with the grandfather-grandson relationship, or belong to the first identity image E and the second identity image e with the grandfather-grandson relationship. The first identity image B and the second identity image b form a first identity image pair; the first identity image E and the second identity image e form a second identity image pair.

[0137] S210: Delete the first identity image and the second identity image that have no kinship relationship.

[0138] Specifically, if there is no kinship relationship between the first identity image and the second identity image, the first identity image and the second identity image that do not have a preset relationship with any identity image in another image set are deleted. In other words, the first identity image and the second identity image that do not meet the grandparent-grandchild relationship are deleted.

[0139] In a specific embodiment, the first identity image and the second identity image that do not conform to the kinship generation relationship are deleted by searching the identity image library for the relationship between any one of the first identity images A, B, C, D, and E and each of the second identity images f, b, g, e, and h. After searching, it is determined that any one of the first identity images A, C, and D does not have a grandparent-grandchild relationship with each of the second identity images f, b, g, e, and h; and any one of the second identity images f, g, and h does not have a kinship generation relationship with each of the first identity images A, B, C, D, and E. Then, it is determined that the first target object and the second target object that have a companion relationship and a kinship generation relationship do not belong to the first identity image and the second identity image that do not have a kinship generation relationship, and the first identity images A, C, D and the second identity images f, g, and h are deleted. That is, the first identity images B and E are retained in the first candidate set, and the second identity images b and e are retained in the second candidate set.

[0140] S211: Obtaining a weighted similarity corresponding to each identity image pair according to a weighted sum of a similarity between a first identity image and a first target object and a similarity between a second identity image and a second target object in each identity image pair.

[0141] Specifically, the similarity of the identity image pair is determined according to the sum of the similarity between the first identity image and the first target object and the similarity between the second identity image and the second target object in each identity image pair.

[0142] In a specific embodiment, a weighted sum of the similarity between the first identity image and the first target object and the similarity between the second identity image and the second target object in each identity image pair is calculated. The first identity image and the second identity image have different kinship generations, and the weight of the similarity between the first identity image and the first target object is different from the weight of the similarity between the second identity image and the second target object.

[0143] Specifically, the weighted similarity between the first identity image and the first target object and the weighted similarity between the second identity image and the second target object are calculated by Formula 2.

[0144] S weight =aS age-level1 +bS age-level2 (Formula 2)

[0145] In formula 2, S weightis the weighted similarity; a and b are weighting coefficients.

[0146] In one embodiment, when one of the first target object and the second target object is an old person and the other is a child, the weighting coefficient of the old person is 0.7 and the weighting coefficient of the child is 0.3; when one of the first target object and the second target object is a young person and the other is a child, the weighting coefficient of the young person is 0.75 and the weighting coefficient of the child is 0.25; when one of the first target object and the second target object is an old person and the other is a young person, the weighting coefficient of the old person is 0.35 and the weighting coefficient of the child is 0.65. The weighting coefficients a and b can also be set according to actual conditions.

[0147] In a specific embodiment, the age level of the first target object or the first cluster is an old person, and the age level of the second target object or the second cluster is a child, then the weighted coefficient a of the similarity between the first target object or the first cluster centroid and the first identity image is 0.7; the weighted coefficient b of the similarity between the second target object or the second cluster centroid and the second identity image is 0.3. The weighted similarity of the first identity image pair calculated by formula 2 is 0.914, and the weighted similarity of the second identity image pair is 0.856.

[0148] S212: Determine whether the weighted similarities corresponding to each identity image pair are greater than a similarity threshold.

[0149] Specifically, the age grade of the first target object is a child, and the age grade of the second target object is an old person; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is the third similarity threshold; it is determined whether the weighted similarities corresponding to each identity image pair are greater than the third similarity threshold. Among them, the third similarity threshold can be set to 0.91, or it can be set by itself. In a specific embodiment, the age grade of the first target object or the first cluster is an old person, and the age grade of the second target object or the second cluster is a child, then the weighted similarity of the first identity image pair is compared with the third similarity threshold.

[0150] In another embodiment, the age grade of the first target object is a child, and the age grade of the second target object is a youth; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is a fourth similarity threshold. Determine whether the weighted similarities corresponding to each identity image pair are greater than the fourth similarity threshold. Among them, the fourth similarity threshold can be set to 0.90, or it can be set by itself. In a specific embodiment, the age grade of the first target object or the first cluster is a child, and the age grade of the second target object or the second cluster is a youth, then the weighted similarity of the first identity image pair is compared with the fourth similarity threshold.

[0151] In another embodiment, the age grade of the first target object is old, and the age grade of the second target object is young; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; the similarity threshold is the fifth similarity threshold. Determine whether the weighted similarities corresponding to each identity image pair are greater than the fifth similarity threshold. Among them, the third similarity threshold can be set to 0.88, or it can be set by itself. In a specific embodiment, the age grade of the first target object or the first cluster is old, and the age grade of the second target object or the second cluster is young, then the weighted similarity of the first identity image pair is compared with the fifth similarity threshold.

[0152] In a specific embodiment, the weighted similarity of the first identity image pair calculated by formula 2 is 0.914, and the weighted similarity of the second identity image pair is 0.856. Since the age level of the first target object or the first cluster is the elderly, and the age level of the second target object or the second cluster is a child, the weighted similarity of each identity image pair is compared with the third similarity threshold. In other words, the weighted similarity of the first identity image pair, 0.914, and the weighted similarity of the second identity image pair, 0.856, are compared with the third similarity threshold of 0.91, respectively.

[0153] If the weighted similarity corresponding to the identity image pair is greater than the corresponding similarity threshold, then jump directly to step S213; if the weighted similarity corresponding to the identity image pair is less than the corresponding similarity threshold, then jump directly to step S214.

[0154] S213: File the first cluster with the first identity image in the identity image pair corresponding to the similarity, and file the second cluster with the second identity image in the identity image pair corresponding to the similarity.

[0155] Specifically, if the weighted similarity is greater than the similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0156] In one embodiment, if the weighted similarity is greater than a third similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0157] In another embodiment, if the weighted similarity is greater than a fourth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0158] In another embodiment, if the weighted similarity is greater than a fifth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the identity image pair corresponding to the similarity.

[0159] In a specific embodiment, the weighted similarity of the first identity image pair calculated by formula 2 is 0.914. By comparison, it can be obtained that the weighted similarity of the first identity image pair 0.914 is greater than the third similarity threshold 0.91, so the first cluster is filed in the first identity image in the first identity image pair, indicating that the first target object corresponding to the first cluster is consistent with the target in the first identity image; the second cluster is filed in the second identity image in the second identity image pair, indicating that the second target object corresponding to the second cluster is consistent with the target in the second identity image in the first identity image pair, thereby improving the accuracy of filing the first cluster and the second cluster.

[0160] In another specific embodiment, in order to further improve the accuracy of filing the first cluster and the second cluster, when the weighted similarity corresponding to the identity image pair is greater than the corresponding similarity threshold, it is further determined whether the similarity between the first cluster centroid or the first target object and the first identity image, and the similarity between the second cluster centroid or the second target object and the second identity image both exceed the similarity threshold. If it is determined that the similarity between the first cluster centroid or the first target object and the first identity image, and the similarity between the second cluster centroid or the second target object and the second identity image both exceed the similarity threshold, it is determined that the first cluster is filed with the first identity image in the identity image pair whose weighted similarity exceeds the similarity threshold, and the second cluster is filed with the second identity image in the same identity image pair.

[0161] In another optional embodiment, if the first cluster and the second cluster must be filed in the first identity image and the second identity image of the same identity image set, respectively, the filing accuracy is improved to a certain extent. When it is determined that one of the similarity between the centroid of the first cluster or the first target object and the first identity image, and the similarity between the centroid of the second cluster or the second target object and the second identity image exceeds the similarity threshold, and the other does not exceed the similarity threshold, the first cluster can be filed in the first identity image of the identity image pair whose weighted similarity exceeds the similarity threshold, and the second cluster can be filed in the second identity image of the same identity image pair.

[0162] In another optional embodiment, in order to file the first cluster and the second cluster in the accurate first identity image and the second identity image respectively, and ensure the accuracy of the filing of the first cluster and the second cluster, when it is determined that one of the similarity between the first cluster centroid or the first target object and the first identity image, and the similarity between the second cluster centroid or the second target object and the second identity image exceeds the similarity threshold, and the other does not exceed the similarity threshold, the first cluster may not be filed in the first identity image in the identity image pair whose weighted similarity exceeds the similarity threshold, and the second cluster may not be filed in the second identity image in the same identity image pair. Continue to screen accurate first identity images and second identity images for the first cluster and the second cluster for filing.

[0163] S214: Delete the first identity image and the second identity image corresponding to the weighted similarities that do not exceed the similarity threshold.

[0164] Specifically, if the weighted similarity corresponding to the identity image pair is less than the corresponding similarity threshold, the first identity image and the second identity image corresponding to the weighted similarity that does not exceed the similarity threshold are deleted. When the weighted similarity corresponding to the identity image pair is less than the corresponding similarity threshold, it indicates that the first target object corresponding to the first cluster is inconsistent with the target in the first identity image, and the second target object corresponding to the second cluster is inconsistent with the target in the second identity image.

[0165] In a specific embodiment, the weighted similarity of the second identity image pair is 0.856 calculated by formula 2. By comparison, it can be obtained that the weighted similarity of the second identity image pair 0.856 is less than the third similarity threshold 0.91, so the first identity image E in the first candidate set is deleted, and the second identity image e in the second candidate set is deleted.

[0166] The portrait archiving method provided by the present embodiment includes: obtaining a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have different age levels and have a companion relationship; determining a first candidate set according to the first target object, and determining a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; determining that a first identity image and a second identity image having a preset relationship form an identity image pair; based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair. In this embodiment, based on the first target object and the second target object with an accompanying relationship, the first identity image and the second identity image with a preset relationship, the similarity between the first target object and the first identity image, and the similarity between the second target object and the second identity image, it is determined whether the first cluster corresponding to the first target object and the second cluster corresponding to the second target object are respectively filed in the first identity image and the second identity image of the same identity image pair, thereby improving the accuracy of identity filing of the portrait image set.

[0167] See also Fig. 9 , Fig. 9 It is a schematic block diagram of an embodiment of a portrait archiving device provided by the present application.

[0168] This embodiment provides a portrait archiving device 60 , which includes an acquisition module 61 , a comparison module 62 , an analysis module 63 and an archiving module 64 .

[0169] The acquisition module 61 is used to acquire a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have different age levels and have a companion relationship;

[0170] The comparison module 62 is used to determine a first candidate set according to the first target object, and to determine a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image;

[0171] The analysis module 63 is used to determine that a first identity image and a second identity image having a preset relationship constitute an identity image pair;

[0172] The archiving module 64 is used to determine whether the first cluster is archived in the first identity image in the identity image pair and whether the second cluster is archived in the second identity image in the same identity image pair based on the first target object and the first identity image in the identity image pair and the second target object and the second identity image in the identity image pair.

[0173] See also Fig.10 , Fig.10 It is a schematic block diagram of another embodiment of the portrait archiving device provided by the present application.

[0174] In an optional embodiment, the portrait archiving device also includes a trajectory screening module 65 .

[0175] In an optional embodiment, the trajectory screening module 65 is used to count the number of space-time points that overlap with each other between the trajectories of each target object; and to determine whether the number of overlapping space-time points is greater than a preset number; if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0176] In an optional embodiment, the trajectory screening module 65 is used to identify and select a preset number of shooting locations that frequently appear in the trajectory of each target object; and determine whether the number of overlapping space-time points corresponding to the shooting locations that frequently appear in the trajectory of each target object is greater than the preset number; if the number of overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

[0177] In an optional embodiment, the trajectory screening module 65 is used to determine the shooting location corresponding to each image in the trajectory of each target object as the spatial trajectory of the target object; count the number of shooting locations that overlap between the spatial trajectories of each target object; determine whether the number of overlapping shooting locations is greater than a preset number; if the number of overlapping shooting locations is greater than a preset number, it is determined that the two image sets corresponding to the spatial trajectory of the target object have similar trajectories.

[0178] In an optional embodiment, the portrait archiving device further includes a matching module 66 .

[0179] In one embodiment, the matching module 66 is used to calculate the average age corresponding to each image set in at least two image sets with similar trajectories; based on the average age corresponding to the image set, determine the age level of the target object corresponding to the image set; and use one of the two image sets with different age levels of the corresponding target objects and similar trajectories as the first cluster and the other as the second cluster.

[0180] In a specific embodiment, the matching module 66 is configured to determine that the age grade of the target object corresponding to the image set is a child in response to the average age corresponding to the image set being in a first preset age range.

[0181] In a specific embodiment, the matching module 66 is configured to determine that the age grade of the target object corresponding to the image set is youth in response to the average age corresponding to the image set being in a second preset age range.

[0182] In a specific embodiment, the matching module 66 is configured to determine that the age grade of the target object corresponding to the image set is elderly in response to the average age corresponding to the image set being in a third preset age range.

[0183] The portrait archiving device provided in this embodiment includes: an acquisition module, used to acquire a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; a comparison module, used to determine a first candidate set according to the first target object, and to determine a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; an analysis module, used to determine that the first identity image and the second identity image with a preset relationship constitute an identity image pair; and an archiving module, used to determine whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair. Based on the first target object and the second target object with an accompanying relationship, the first identity image and the second identity image with a preset relationship, the similarity between the first target object and the first identity image, and the similarity between the second target object and the second identity image, it is determined whether the first cluster corresponding to the first target object and the second cluster corresponding to the second target object are respectively filed in the first identity image and the second identity image in the same identity image pair, thereby improving the accuracy of identity filing of the portrait image set.

[0184] See also Fig.11 , Fig.11 It is a schematic block diagram of a terminal implementation example provided by the present application. The terminal 70 in this implementation example includes: a processor 71, a memory 72, and a computer program stored in the memory 72 and executable on the processor 71. When the computer program is executed by the processor 71, the above-mentioned portrait archiving method is implemented. To avoid repetition, it is not described one by one here.

[0185] See also Fig.12 , Fig.12 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present application.

[0186] A computer-readable storage medium 90 is also provided in an embodiment of the present application. The computer-readable storage medium 90 stores a computer program 901. The computer program 901 includes program instructions. The processor executes the program instructions to implement the portrait archiving method provided in the embodiment of the present application.

[0187] The computer-readable storage medium 90 may be an internal storage unit of the computer device of the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium 90 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0188] The above are only implementation methods of the present application, and are not intended to limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A portrait archiving method, characterized in that: The portrait archiving method comprises: Obtain a first cluster and a second cluster; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have an accompanying relationship and a preset relationship; the accompanying relationship indicates that the first target object and the second target object are at the same shooting location at the same shooting time; and / or indicates that the first target object and the second target object have similar trajectory information; the preset relationship indicates that the first target object and the second target object have different age levels And have a kinship relationship ; Determine a first candidate set according to the first target object, and determine a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; Determining that the first identity image and the second identity image having the preset relationship form an identity image pair; Based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, determining whether the first cluster is filed in the first identity image in the identity image pair, and whether the second cluster is filed in the second identity image in the same identity image pair; The determining, based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair, whether the first cluster is filed in the first identity image in the identity image pair, and whether the second cluster is filed in the second identity image in the same identity image pair, comprises: According to the weighted sum of the similarities between the first identity image and the first target object and the similarities between the second identity image and the second target object in each identity image pair, the weighted similarities corresponding to each identity image pair are obtained; it is determined whether the weighted similarities corresponding to each identity image pair are greater than a similarity threshold; if the weighted similarity is greater than the similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the same identity image pair.

2. The portrait archiving method according to claim 1, characterized in that: The step of determining a first candidate set according to the first target object and determining a second candidate set according to the second target object, wherein the first candidate set includes at least one first identity image and the second candidate set includes at least one second identity image, comprises: Calculating and obtaining a first similarity between the first target object and each identity image in the identity image library; comparing the first similarity with a first similarity threshold; In response to the first similarity being greater than the first similarity threshold, clustering the identity images corresponding to the first similarity to obtain a first candidate set; the identity images in the first candidate set are used as the first identity images; Calculating and obtaining a second similarity between the second target object and each of the identity images in the identity image library; comparing the second similarity with a second similarity threshold; In response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the second similarity to obtain a second candidate set; the identity images in the second candidate set are used as the second identity images.

3. The portrait archiving method according to claim 2, characterized in that: In response to the first similarity being greater than the first similarity threshold, clustering the identity images corresponding to the first similarity to obtain a first candidate set, including: In response to the first similarity being greater than the first similarity threshold, selecting a first number of the identity images that are ranked top and correspond to the first similarities to perform clustering to obtain the first candidate set, wherein the first candidate set includes the first number of the first identity images; In response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the second similarity to obtain a second candidate set, including: In response to the second similarity being greater than the second similarity threshold, a preset second number of the identity images corresponding to the second similarities ranked top are selected for clustering to obtain the second candidate set, which includes the preset second number of the second identity images.

4. The portrait archiving method according to claim 2, characterized in that: The first cluster includes a plurality of images containing the first target object, and the second cluster includes a plurality of images containing the second target object; Calculating a first similarity between the first target object and each identity image in the identity image library includes: Determining an average feature vector of the first target object corresponding to the first cluster according to the feature vector of the first target object contained in each of the images in the first cluster; Calculating a first similarity between an average feature vector of the first target object and feature vectors of each identity image in the identity image library; In response to the first similarity being greater than the first similarity threshold, clustering the identity images corresponding to the first similarity includes: In response to the first similarity being greater than the first similarity threshold, clustering the identity images corresponding to the feature vectors for which the first similarity is calculated; The calculating and obtaining a second similarity between the second target object and each of the identity images in the identity image library includes: Determining an average feature vector of the second target object corresponding to the second cluster according to the feature vector of the second target object contained in each of the images in the second cluster; Calculating a second similarity between an average feature vector of the second target object and a feature vector of each identity image in the identity image library; In response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the second similarity includes: In response to the second similarity being greater than the second similarity threshold, clustering the identity images corresponding to the feature vectors for which the second similarity is calculated.

5. The portrait archiving method according to claim 1, characterized in that: The preset relationship is a kinship relationship; The determining that a first identity image and a second identity image having the preset relationship form an identity image pair includes: determining whether the first identity image in the first candidate set and the second identity image in the second candidate set have a kinship relationship; If there is a kinship relationship between the first identity image and the second identity image, the first identity image and the second identity image having the kinship relationship constitute the identity image pair, wherein the identity image pair is at least one.

6. The portrait archiving method according to claim 1, characterized in that: The age grade of the first target object is a child, and the age grade of the second target object is an elderly person; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is less than the weighted coefficient of the similarity between the second identity image and the second target object; The similarity threshold is a third similarity threshold; The determining whether the weighted similarities corresponding to each of the identity image pairs are greater than a similarity threshold comprises: Determining whether the weighted similarities corresponding to each of the identity image pairs are greater than the third similarity threshold; If the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the same identity image pair, comprises: If the weighted similarity is greater than the third similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the same identity image pair.

7. The portrait archiving method according to claim 1, characterized in that: The age grade of the first target object is a child, and the age grade of the second target object is a youth; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is smaller than the weighted coefficient of the similarity between the second identity image and the second target object; The similarity threshold is a fourth similarity threshold; The determining whether the weighted similarities corresponding to each of the identity image pairs are greater than a similarity threshold comprises: Determining whether the weighted similarities corresponding to each of the identity image pairs are greater than the fourth similarity threshold; If the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the same identity image pair, comprises: If the weighted similarity is greater than the fourth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the same identity image pair.

8. The portrait archiving method according to claim 1, characterized in that: The age grade of the first target object is old, and the age grade of the second target object is young; the weighted coefficient of the similarity between the first identity image and the first target object in the identity image pair is smaller than the weighted coefficient of the similarity between the second identity image and the second target object; The similarity threshold is a fifth similarity threshold; The determining whether the weighted similarities corresponding to each of the identity image pairs are greater than a similarity threshold comprises: Determining whether the weighted similarities corresponding to each of the identity image pairs are greater than the fifth similarity threshold; If the weighted similarity is greater than the similarity threshold, filing the first cluster with the first identity image in the identity image pair corresponding to the similarity, and filing the second cluster with the second identity image in the same identity image pair, comprises: If the weighted similarity is greater than the fifth similarity threshold, the first cluster is filed with the first identity image in the identity image pair corresponding to the similarity, and the second cluster is filed with the second identity image in the same identity image pair.

9. The portrait archiving method according to claim 1, characterized in that: The obtaining of the first type of cluster and the second type of cluster comprises: Acquire a plurality of image sets, wherein the image sets include a plurality of images containing the same target object; Determining the trajectory of the target object corresponding to each of the image sets based on the shooting time of the image; At least two image sets having similar trajectories are determined according to the overlap between the trajectories of the target objects.

10. The portrait archiving method according to claim 9, characterized in that: The shooting time and shooting location of the image constitute the time-space point of the image. The step of determining at least two image sets having similar trajectories according to the overlap between the trajectories of the target objects comprises: Counting the number of space-time points where the trajectories of the target objects overlap with each other; Determining whether the number of the overlapping space-time points is greater than a preset number; If the number of the overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

11. The portrait archiving method according to claim 9, characterized in that: The shooting time and shooting location of the image constitute the time-space point of the image. The step of determining at least two image sets having similar trajectories according to the overlap between the trajectories of the target objects comprises: Identifying and selecting a preset number of shooting locations that frequently appear in the trajectory of each of the target objects; Determine whether the number of mutually overlapping spatiotemporal points between the spatiotemporal points corresponding to the shooting locations that frequently appear in the trajectory of each target object is greater than a preset number; If the number of the overlapping space-time points is greater than the preset number, it is determined that the two image sets corresponding to the overlapping space-time points have similar trajectories.

12. The portrait archiving method according to claim 9, characterized in that: The shooting time and shooting location of the image constitute the time-space point of the image. The step of determining at least two image sets having similar trajectories according to the overlap between the trajectories of the target objects comprises: Determine a shooting location corresponding to each image in the trajectory of each target object as the spatial trajectory of the target object; Counting the number of the shooting locations where the spatial trajectories of the target objects overlap with each other; Determining whether the number of the mutually overlapping shooting locations is greater than a preset number; If the number of the overlapping shooting locations is greater than the preset number, it is determined that the two image sets corresponding to the spatial trajectory of the target object have similar trajectories.

13. The portrait archiving method according to claim 12, characterized in that: The step of determining at least two image sets having similar trajectories according to the overlap between the trajectories of the target objects further comprises: The similarity between the trajectories of the target objects is calculated by formula 1: In formula 1: S DWT is the trajectory similarity between trajectory A and trajectory B; dist(·) represents the distance between two spatial points; Head(X) is the first shooting location of trajectory X; Rest(X) is the set of elements remaining in trajectory X except the first element.

14. The portrait archiving method according to claim 9, characterized in that: The obtaining of the first type of cluster and the second type of cluster comprises: Calculate and obtain the average age corresponding to each of the at least two image sets having similar trajectories; Determining the age grade of the target object corresponding to the image set based on the average age corresponding to the image set; One of the two image sets corresponding to the target objects having different age levels and similar trajectories is used as the first cluster, and the other is used as the second cluster.

15. The portrait archiving method according to claim 14, characterized in that: The determining the age grade of the target object corresponding to the image set based on the average age corresponding to the image set includes: In response to the average age corresponding to the image set being in a first preset age range, it is determined that the age grade of the target object corresponding to the image set is a child.

16. The portrait archiving method according to claim 14, characterized in that: The determining the age grade of the target object corresponding to the image set based on the average age corresponding to the image set includes: In response to the average age corresponding to the image set being in a second preset age group, it is determined that the age grade of the target object corresponding to the image set is youth.

17. The portrait archiving method according to claim 14, characterized in that: The determining the age grade of the target object corresponding to the image set based on the average age corresponding to the image set includes: In response to the average age corresponding to the image set being in a third preset age group, it is determined that the age grade of the target object corresponding to the image set is elderly.

18. A portrait archiving device, characterized in that: The portrait archiving device comprises: An acquisition module is used to acquire a first cluster and a second cluster; wherein the first cluster and the second cluster are acquired; wherein the first cluster corresponds to a first target object, the second cluster corresponds to a second target object, and the first target object and the second target object have a companion relationship and a preset relationship; the companion relationship indicates that the first target object and the second target object are at the same shooting location at the same shooting time; and / or indicates that the first target object and the second target object have similar trajectory information; the preset relationship indicates that the first target object and the second target object have different age levels And has Relatives' Generation Relationship ; A comparison module, configured to determine a first candidate set according to the first target object, and to determine a second candidate set according to the second target object; wherein the first candidate set includes at least one first identity image, and the second candidate set includes at least one second identity image; an analysis module, configured to determine that the first identity image and the second identity image having a preset relationship constitute an identity image pair; An archiving module is used to determine whether the first cluster is archived in the first identity image in the identity image pair, and whether the second cluster is archived in the second identity image in the same identity image pair based on the first target object and the first identity image in the identity image pair, and the second target object and the second identity image in the identity image pair; it is also used to obtain the weighted similarity corresponding to each identity image pair according to the weighted sum of the similarity between the first identity image and the first target object and the similarity between the second identity image and the second target object in each identity image pair; it is judged whether the weighted similarity corresponding to each identity image pair is greater than a similarity threshold; if the weighted similarity is greater than the similarity threshold, the first cluster is archived in the first identity image in the identity image pair corresponding to the similarity, and the second cluster is archived in the second identity image in the same identity image pair.

19. A terminal, characterized in that: The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor is used to execute program data to implement the steps in the portrait archiving method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the portrait archiving method according to any one of claims 1 to 17 are implemented.

Images