Portrait image clustering method, device, system, electronic device and storage medium

Abstract

The present application relates to a portrait image clustering method, device, system, electronic device, and storage medium, wherein the portrait image clustering method includes: obtaining portrait images of multiple targets to be measured at each image acquisition point; obtaining tag information for the targets to be measured identified by each reader device, calculating the motion trajectory of each target to be measured based on the tag information, and generating a spatiotemporal correlation result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed at each image acquisition point; and performing portrait clustering on the portrait images based on the spatiotemporal correlation result and the tag information to generate a clustering result for the portrait images. Through this application, the problems of low accuracy and efficiency of portrait image clustering are solved, and a portrait image clustering method based on radio frequency technology is realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, system, electronic device and storage medium for clustering portrait images. Background Art

[0002] With the continuous development of social security construction, the use of portrait bayonet cameras is increasing, and the demand for identity verification of captured portrait images is becoming increasingly urgent. Identity verification through portrait clustering technology has become a new hot spot in the security field. In related technologies, face recognition technology is usually used to compare the feature values ​​of collected face images with those of standard database images. If the similarity is above a certain threshold range, the identity of the collected image is marked as the identity information of the standard database, and the identity verification of the collected image is completed. If the similarity is below a certain threshold, the collected image cannot be recognized and the identity verification cannot be completed. However, due to differences in the image shooting environment, such as weather, occlusion or angle, there will be a large number of blurred portrait images in the images captured by the bayonet camera. When clustering the portrait images, these blurred portrait images are difficult to be clustered with the portrait images of the person through similarity comparison, which affects the clustering effect and leads to low accuracy and efficiency of portrait image clustering.

[0003] Currently, no effective solution has been proposed to address the problems of low accuracy and efficiency in portrait image clustering in related technologies. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, system, electronic device, and storage medium for clustering portrait images to at least solve the problems of low accuracy and efficiency in portrait image clustering in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for clustering portrait images, the method comprising:

[0006] Acquire portrait images of multiple targets at each image acquisition point;

[0007] Obtaining tag information for the target to be measured identified by each reader device, calculating a motion trajectory of each target to be measured based on the tag information, and generating a spatiotemporal correlation result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed under each image acquisition point;

[0008] Performing portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image.

[0009] In some embodiments, calculating the motion trajectory of each target to be detected based on the tag information includes:

[0010] Obtaining the number information of each image acquisition point;

[0011] performing grouping processing on the number information according to the label information to obtain a grouping result, and performing deduplication processing on the grouping result according to the number information to obtain a deduplication result;

[0012] The motion trajectory is generated based on the numbering information and the deduplication result.

[0013] In some embodiments, generating a spatiotemporal correlation result between all the image acquisition points according to the motion trajectory includes:

[0014] Using an association rule mining algorithm, a first-level candidate item set of the image acquisition point is calculated according to the motion trajectory, and a second-level candidate item set is obtained by iterative calculation based on the first-level candidate item set;

[0015] Calculating the lift of the image acquisition points using the association rule mining algorithm, obtaining association results between the secondary candidate item sets based on the lift, as well as preset support and confidence, and obtaining a target set based on the association results and the secondary candidate item sets;

[0016] The spatiotemporal association result is generated according to the target set.

[0017] In some embodiments, clustering the portrait images based on the spatiotemporal correlation result and the label information to generate clustering results of the portrait images includes:

[0018] Obtain image features from a portrait image;

[0019] Performing initial portrait clustering according to the spatiotemporal correlation result and the image features to obtain initial clusters, and calculating a first average centroid of each of the initial clusters;

[0020] A first similarity result between all the first average centroids is calculated, the initial clusters are merged based on the first similarity result to obtain an initial merged cluster set, and the clustering result is generated based on the initial merged cluster set and the label information.

[0021] In some embodiments, generating the clustering result based on the initial merged cluster set and the label information includes:

[0022] Filtering the initial merged cluster set based on the identity information in the tag information to obtain a target cluster set including the identity information and a second average centroid of each target cluster set;

[0023] Calculating second similarity results between all the second average centroids, and sorting the target cluster set based on the second similarity results to obtain a sorted cluster set;

[0024] Calculating corresponding trajectory missing value results according to the portrait images in the sorted cluster sets, and obtaining association information between the sorted cluster sets based on the trajectory missing value results;

[0025] A third similarity result between the sorted cluster sets is calculated based on the association information and a preset similarity threshold, and the sorted cluster sets are merged according to the third similarity result to obtain the cluster result.

[0026] In some embodiments, the method further comprises:

[0027] The portrait image and the tag information are sent to a remote server for storage.

[0028] In a second aspect, an embodiment of the present application provides a portrait image clustering device, the device comprising: an acquisition module, an association module, and a clustering module;

[0029] The acquisition module is used to acquire portrait images of multiple targets to be measured at each image acquisition point;

[0030] The association module is configured to obtain tag information for the target to be measured identified by each reader device, calculate a motion trajectory of each target to be measured based on the tag information, and generate a spatiotemporal association result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed under each image acquisition point;

[0031] The clustering module is used to perform portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image.

[0032] In a third aspect, an embodiment of the present application provides a portrait image clustering system, the system comprising: a main control device and various reader devices; wherein each reader device is correspondingly installed under each image acquisition point;

[0033] The main control device is used to execute the portrait image clustering method as described in the first aspect above.

[0034] In a fourth aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the portrait image clustering method as described in the first aspect above is implemented.

[0035] In a fifth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the program is executed by a processor, the portrait image clustering method described in the first aspect above is implemented.

[0036] Compared with the related art, the portrait image clustering method, device, system, electronic device and storage medium provided in the embodiments of the present application obtain portrait images of multiple targets to be measured at each image acquisition point; obtain label information for the target to be measured identified by each reader device, calculate the motion trajectory of each target to be measured based on the label information, and generate the spatiotemporal correlation results between all the image acquisition points based on the motion trajectory; wherein, each reader device is correspondingly installed under each image acquisition point; based on the spatiotemporal correlation results and the label information, the portrait images are clustered to generate the clustering results of the portrait images, thereby solving the problems of low accuracy and efficiency of portrait image clustering and realizing a portrait image clustering method based on radio frequency technology.

[0037] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 This is an application environment diagram of a portrait image clustering method according to an embodiment of the present application;

[0040] Figure 2 is a flowchart of a portrait image clustering method according to an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a motion trajectory according to an embodiment of the present application;

[0042] Figure 4 is a flow chart of a portrait image clustering method according to a preferred embodiment of the present application;

[0043] Figure 5 is a structural block diagram of a portrait image clustering device according to an embodiment of the present application;

[0044] Figure 6 This is a structural diagram of the interior of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means and should not be understood as the contents disclosed in the present application being insufficient.

[0046] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0047] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in this application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The portrait image clustering method provided in this application can be applied to Figure 1In the application environment shown, the image acquisition device 12 and the reader device 14 communicate with the main control device 16 via a network. Each image acquisition device 12 and each reader device 14 is installed at a corresponding image acquisition point. The main control device 16 obtains the portrait images captured by each image acquisition device 12 at each image acquisition point for multiple targets to be measured. The main control device 16 obtains the tag information for each target to be measured identified by each reader device 14, calculates the motion trajectory of each target to be measured based on the tag information, and generates a spatiotemporal correlation result between all image acquisition points based on the motion trajectory. Finally, the portrait images are clustered based on the spatiotemporal correlation result and the tag information to generate a clustering result of the portrait images. The image acquisition device 12 can be, but is not limited to, various binocular cameras, dome cameras, video recording devices, or other devices for capturing images. The main control device 16 can be, but is not limited to, various server devices, terminal devices such as personal computers or laptops, processing chips, or other devices for control. The server device can be implemented as an independent server or a server cluster consisting of multiple servers.

[0049] This embodiment provides a method for clustering portrait images. Figure 2 is a flow chart of a portrait image clustering method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0050] Step S220 , obtaining portrait images of multiple targets to be measured at each image acquisition point.

[0051] The aforementioned image acquisition points refer to checkpoints distributed throughout plazas, parks, and even cities, i.e., hardware devices installed on roadsides and intersections. Each checkpoint is used to capture portrait images of drivers of sports vehicles such as electric bicycles, electric vehicles, or motorcycles through the installed image acquisition equipment. The drivers are then the aforementioned targets to be measured. Portrait images include facial images, body images, and gait sequence images. It should be noted that each image may contain facial images, body images, both facial and body images, or neither.

[0052] Step S240: Obtain the tag information for the target to be measured identified by each reader device, calculate the motion trajectory of each target to be measured based on the tag information, and generate the spatiotemporal correlation results between all the image acquisition points based on the motion trajectory; wherein, each reader device is correspondingly installed under each image acquisition point.

[0053] Among them, the above-mentioned image acquisition point is also installed with a corresponding reader device, which is a radio frequency reader; the reader device can read the electric bicycles and other moving vehicles driven by the above-mentioned target to be measured that pass through the corresponding image acquisition point and are equipped with radio frequency electronic tags, thereby obtaining the tag information corresponding to the target to be measured, so that the above-mentioned main control device can parse the electronic tag information to obtain the registration information and location information of the moving vehicle. It should be noted that the registration information also includes the vehicle owner, contact information, vehicle model, license plate number, filing registration date, whether it is a stolen vehicle, and other related information. The above-mentioned main control device then draws the movement trajectory of the driver of the corresponding moving vehicle based on the above-mentioned tag information; for example, an electric bicycle passes through three image acquisition points in sequence within a day, and the three image acquisition points are abstractly labeled A, B, and C, then the movement trajectory of the electric bicycle can be obtained as [A, B, C].

[0054] It is understandable that when clustering portrait images of a large area such as a city, if the massive portrait image data is directly clustered, it will require huge computing power and high cost; and in a large area, there is a high possibility that portrait images of different people but with high similarity will appear, so it is necessary to divide the image acquisition points in the large area and divide the related image acquisition points into a time-space region as much as possible. For example, based on experience, if there is a high probability of passing through image acquisition point A and image acquisition point B, it means that image acquisition point A and image acquisition point B are related in time and space, and they can be divided into a time-space region. Specifically, the above-mentioned main control device can use the Apriori algorithm or FP-growth algorithm and other data mining algorithms to mine the correlation between image acquisition points based on the calculated motion trajectory to generate a time-space correlation result between all image acquisition points, and finally divide the time-space related image acquisition points into a time-space region.

[0055] Step S260 : performing portrait clustering on the portrait image based on the spatiotemporal correlation result and the tag information to generate a clustering result of the portrait image.

[0056] Among them, the above-mentioned main control device can separately cluster the portrait images in each spatiotemporal region according to the above-mentioned spatiotemporal correlation results to obtain cluster clusters in the same spatiotemporal region, and assist in merging the cluster clusters based on the above-mentioned label information; for example, different targets to be measured are distinguished based on label information, or cluster clusters corresponding to the same targets to be measured in different spatiotemporal regions are merged, so as to avoid portrait image clustering errors caused by different targets but similar portrait images in a large area, thereby effectively improving the accuracy of portrait image clustering.

[0057] Through the above steps S220 to S260, by obtaining the portrait image and the label information of each target to be measured identified by each reader device, the motion trajectory of each target to be measured is calculated according to the label information to mine the image acquisition points with temporal and spatial correlation, and finally the portrait image is clustered based on the temporal and spatial correlation results and the label information to assist, thereby realizing the division of the associated image acquisition points into a temporal and spatial region. Moreover, since the data acquisition accuracy of the reader device is less affected by environmental factors, the clustering recall rate caused by the presence of a large number of blurred portrait images in the captured images can be avoided, thereby improving the clustering efficiency and accuracy of the captured portrait images in a large area, solving the problems of low accuracy and efficiency of portrait image clustering, and realizing a portrait image clustering method based on radio frequency technology.

[0058] In some embodiments, the above-mentioned calculation of the motion trajectory of each target to be measured based on the label information also includes the following steps: obtaining the numbering information of each image acquisition point; grouping the numbering information according to the label information to obtain a grouping result, and deduplicating the grouping result according to the numbering information to obtain a deduplicated result; generating the motion trajectory based on the numbering information and the deduplication result.

[0059] In this case, each image acquisition point can be pre-numbered with a checkpoint, and the checkpoint number can be abstractly labeled. Since the checkpoint number is a national standard code and the string it represents is long, an abstract label can be used to refer to each checkpoint one by one; such as, but not limited to: checkpoint A, checkpoint B, etc., and summarized as the above number information. Specifically, data is deduplicated based on the checkpoint number information. When a moving vehicle passes through each image acquisition point, each image acquisition point will upload the vehicle data multiple times to a cloud platform deployed by a remote server. The main control device then groups the label information based on the license plate number in the label information. This data is grouped into one group, and adjacent data with duplicate checkpoint numbers is deduplicated. Ultimately, the driving trajectory of each moving vehicle is saved as a list. For example, an electric bicycle passes through three checkpoints in sequence in one day. The three checkpoints are abstractly labeled A, B, and C. The resulting trajectory of the electric bicycle is [A, B, C]. Through the above embodiment, preprocessing of image acquisition point data is achieved, which facilitates the subsequent calculation of portrait image clustering and helps improve the accuracy of portrait image clustering.

[0060] In some embodiments, generating the spatiotemporal correlation results between all the image acquisition points according to the motion trajectory further includes the following steps:

[0061] Step S241 , using an association rule mining algorithm, a first-level candidate item set of the image acquisition point is calculated according to the motion trajectory, and an iterative calculation is performed based on the first-level candidate item set to obtain a second-level candidate item set.

[0062] Specifically, taking the Apriori algorithm as an example of the association rule mining algorithm, the main control device can first use the Apriori algorithm to scan the set of input motion trajectories to find frequent k-item sets; in this embodiment, the data set composed of the motion trajectories can be: [[A, B, C], [A, B], [A, C], [D, E, F], [D, F], [E]], and then the first-level candidate item set is statistically obtained. The first-level candidate item set obtained by statistics for the data set is shown in the following table:

[0063] Table 1. First-level candidate item set

[0064] Itemset frequency A 3 B 2 C 2 D 2 E 2 F 2

[0065] Table 1 lists the frequent itemsets scanned for the above dataset, along with the corresponding number of occurrences for each item set. For example, item set A appears 3 times. After counting the number of occurrences of the first candidate item set, the master control device does not perform a removal operation to retain all items, and continues to generate secondary candidate item sets based on the previous round of candidate item sets until the iteration stops. In this embodiment, iteration stops after generating the secondary candidate item sets for the above dataset. The generated secondary candidate item sets are shown in the following table:

[0066] Table 2 Secondary candidate item set

[0067] Itemset frequency Itemset frequency {A, B} 2 {B, F} 0 {A, C} 2 {C, D} 0 {A, D} 0 {C, E} 0 {A, E} 0 {C, F} 0 {A, F} 0 {D, E} 1 {B, C} 1 {D, F} 2 {B, D} 0 {E, F} 1 {B, E} 0

[0068] Step S242: Calculate the lift of the image acquisition point using the association rule mining algorithm, obtain the association results between the secondary candidate item sets based on the lift, as well as the preset support and confidence, and obtain the target set based on the association results and the secondary candidate item sets.

[0069] By calculating the association rules between each image acquisition point using an association rule mining algorithm such as the Apriori algorithm, a column of lift values ​​can be obtained. Based on the lift, support, and confidence values, the association results between the aforementioned secondary candidate item sets can be obtained. Taking Table 2 as an example, the association results indicate that {A, B}, {A, C}, and {D, F} satisfy strong association rules with positive correlation. It can be understood that a strong association rule refers to an association rule that satisfies both the minimum confidence threshold and the minimum support threshold. Furthermore, a lift greater than 1 indicates a strong association rule with positive correlation, while a lift less than 1 indicates a strong association rule with negative correlation. It should be noted that the support and confidence values ​​can be pre-set. For example, the support value can be set to a fixed value of 0.5 and the confidence value to 0.7 based on experience. During the subsequent portrait image clustering process, the support and confidence parameters can be manually adjusted based on the clustering effect of the portrait images to continuously optimize the algorithm accuracy. Then, the main control device can construct a graph structure for the image acquisition points that meet the strong correlation rule of positive correlation, merge all image acquisition points with spatiotemporal correlation, and take the data in Table 2 as an example to form the set results {A, B, C} and {D, F}; and filter out the image acquisition points that do not meet the strong correlation rule of positive correlation, calculate the cumulative sum of the number of times the image acquisition point appears together with the image acquisition points in each spatiotemporal correlation set, that is, filter out the checkpoint E, and calculate the cumulative sum of the statistical values ​​of the checkpoint E and the secondary candidate item set {A, E}, {B, E} and {C, E} in the {A, B, C} set to be 0; similarly, the cumulative sum of the statistical values ​​of the checkpoint E and the secondary candidate item set {D, F} is 2; then the checkpoint E is divided into the {D, F} set to obtain the set {D, F, E}, and finally obtain the target set {A, B, C} and {D, F, E}.

[0070] Step S243: Generate the spatiotemporal correlation result according to the target set.

[0071] Specifically, the main control device may generate a spatiotemporal correlation result based on the target set to indicate which image acquisition points belong to spatiotemporal correlation, so as to divide the spatiotemporally correlated image acquisition points in the target set into a spatiotemporal region according to the spatiotemporal correlation result.

[0072] In related technologies, when using algorithms such as Apriori for association rule data mining, some noise is removed during the algorithm operation, which cannot meet the requirement of retaining all image acquisition points in the embodiment of the present application. However, the embodiment of the present application avoids the data screening of the first-level candidate item set through the above steps S241 to S243, providing a mining method for scenarios where it is necessary to mine the association relationship between data units and retain each data unit, thereby optimizing the association algorithm.

[0073] In some embodiments, clustering the portrait image based on the spatiotemporal correlation result and the tag information to generate a clustering result of the portrait image further includes the following steps:

[0074] Step S261: Acquire image features in the portrait image.

[0075] Among them, the above-mentioned main control device can perform feature analysis on each frame of the portrait image to obtain a multi-dimensional image feature value. For example, the image features in each portrait image can be obtained through a pre-trained neural network model, a fuzzy mathematical model, a residual neural network (Residual Neural Network, referred to as ResNet) model, etc. The image features may include feature information of the portrait displayed in the portrait image, such as facial contour, eyes, nose, ears, mouth, shoulders, limbs, body shape and other feature information. The specific content of the image features can be pre-set according to needs, and this embodiment of the present invention is not limited to this.

[0076] Step S262 , performing initial portrait clustering according to the spatiotemporal correlation result and the image features to obtain initial clusters, and calculating a first average centroid of each of the initial clusters.

[0077] Step S263 , calculating a first similarity result between all the first average centroids, merging the initial clusters based on the first similarity result to obtain an initial merged cluster set, and generating the clustering result based on the initial merged cluster set and the label information.

[0078] Specifically, the portrait image clustering process is divided into two stages. The first stage is implemented through the above-mentioned step S262. First, the image features of the portrait images in the same spatiotemporal region determined based on the above-mentioned spatiotemporal correlation results are compared for similarity. When the similarity meets the set threshold, they are clustered into a cluster to achieve separate clustering of the portrait images in each spatiotemporal region. After the initial clusters are clustered in the spatiotemporal region, the image feature values ​​of each portrait image in each initial cluster are summed and averaged to calculate the first average centroid of each cluster. The second stage is implemented through the above-mentioned step S263. The first average centroids of the team members of each cluster in the spatiotemporal region are compared for similarity to obtain a first similarity result. When the first similarity result reaches the preset similarity threshold, it indicates that the similarity of the corresponding clusters is high. Then, the corresponding initial clusters can be merged to obtain an initial merged cluster set, and the average centroid of the new cluster is calculated until the iteration ends. It should be noted that the first stage of clustering in the above-mentioned portrait clustering process can be performed every six hours, and the second stage of merging clusters generated by clustering in different spatiotemporal regions can be performed once a day to improve the efficiency of portrait clustering. The above-mentioned main control device can also perform auxiliary detection on the initial merged cluster set based on the above-mentioned tag information to improve the accuracy of portrait clustering, and ultimately generate the above-mentioned clustering results.

[0079] Through the above steps S261 to S263, all portrait images are clustered in multiple stages according to the spatiotemporal correlation results, which can improve the clustering efficiency of portrait images captured in a large area, thereby effectively improving the processing efficiency of portrait images.

[0080] In some embodiments, the above-mentioned step of generating the clustering result based on the initial merged cluster set and the label information further includes the following steps:

[0081] Step S264 : screening the initial merged cluster set based on the identity information in the tag information to obtain a target cluster set including the identity information and a second average centroid of each target cluster set.

[0082] Among them, the above-mentioned identity information refers to the unique identification information corresponding to the moving vehicles driven by the above-mentioned targets to be measured, such as license plate information or vehicle feature information. Specifically, when a moving vehicle passes through an image acquisition point, the image acquisition device installed at the image acquisition point can capture the portrait image corresponding to the target to be measured. When performing a similarity comparison on the portrait pictures, the facial and body features of the target to be measured as well as the features of the vehicle can be parsed from the portrait pictures; when the capture angle is appropriate, the license plate number of the moving vehicle can be parsed from some portrait pictures. In the clustered portrait image clusters, that is, the above-mentioned initial merged cluster set, the cluster containing the license plate number is screened out as the above-mentioned target cluster set, and the image feature values ​​of each portrait image in each initial cluster are summed and averaged to calculate the first average centroid of each cluster.

[0083] Step S265 : calculating the second similarity results between all the second average centroids, and sorting the target cluster set based on the second similarity results to obtain a sorted cluster set.

[0084] Among them, the similarity between the average centroid of the cluster and other clusters is compared, the clusters with higher similarity are screened, and the clusters are sorted according to the similarity; for example, the top three clusters with the highest similarity can be selected as the above-mentioned sorted cluster set.

[0085] Step S266 , calculating the corresponding trajectory missing value result according to the portrait image in the sorted cluster set, and obtaining the association information between the sorted cluster sets based on the trajectory missing value result.

[0086] Specifically, Figure 3 is a schematic diagram of a motion trajectory according to an embodiment of the present application. Figure 3 For example, the black triangle symbol in the figure represents the full set of image collection points where a vehicle with license plate number XX passed through, as indicated by the tag information read by the reader device. The main control device can depict the vehicle's trajectory through each image collection point based on the time of information acquisition. The black circle symbol in the figure represents cluster A, containing license plate XX, obtained through portrait clustering through steps S261 to S263. This refers to the image collection points where portrait images of the target cluster set were captured. Some of these image collection points have missing values ​​for identifier 2, meaning that some of these collection points did not capture the portrait images of cluster A. The main control device can then select clusters with a high similarity to cluster A from the sorted cluster set to obtain cluster B. The image collection points where portrait images of cluster B were captured are represented by the black rectangle symbol 3. If the capture time and camera number of the portrait images in cluster B meet the missing value for the trajectory of the image collection points where the vehicle with license plate number XX passed through, the temporal and spatial association information between clusters A and B is determined.

[0087] Step S267 : calculating a third similarity result between the sorted cluster sets based on the association information and a preset similarity threshold, and merging the sorted cluster sets according to the third similarity result to obtain the cluster result.

[0088] Specifically, the similarity between the average centroids of the clusters that satisfy the spatiotemporal association indicated by the association information is calculated. A similarity threshold is adjusted, and based on the calculated similarity, the clusters that satisfy the spatiotemporal association are compared for similarity. When the threshold is met, the two clusters are merged, ultimately generating the aforementioned clustering result.

[0089] In related technologies, portrait image clustering is typically achieved by comparing the similarity of features. This clustering only considers the features of the person. This can lead to poor portrait image clustering when a large number of blurred portrait images are captured due to weather, occlusion, or angle. However, the present embodiment, through steps S264 to S267, utilizes radio frequency technology to obtain relevant information about electric bicycles, determines the spatiotemporal relationship between portrait image clusters, adjusts the similarity threshold, and merges clusters that meet the similarity threshold. This assists in portrait image clustering, improves the clustering effect of portrait images captured in a large area, and further improves the accuracy of portrait image clustering.

[0090] In some embodiments, the portrait image clustering method further includes the following steps: sending the portrait image and the tag information to a remote server for storage. The image acquisition device sends the captured portrait image, capture time, device number, device latitude and longitude coordinates and other capture information to the main control device, and the main control device uploads the capture information to the cloud platform deployed by the remote server for storage. The reader device reads the electronic tag information and sends it to the main control device. The main control device obtains the registration information and location information of the moving vehicle based on the electronic tag information, and uploads this information to the cloud platform for storage. Alternatively, the image acquisition device and the reader device can directly upload each piece of information captured or read to the cloud platform separately. Through the above embodiments, uploading information such as portrait images and tag information to a remote server for storage can facilitate data storage and analysis, and effectively improve the capacity and accuracy of data storage.

[0091] The following describes the embodiments of this application in detail in conjunction with actual application scenarios. Figure 4 is a flow chart of a portrait image clustering method according to a preferred embodiment of the present application, such as Figure 4 As shown, the process includes the following steps:

[0092] Step S401: Storing card-point data. The card-point refers to the hardware device installed at the roadside or intersection, typically including a snapshot camera and a radio frequency reader, which are used to capture portraits of the target passing through the corresponding area and obtain relevant data about the moving vehicle driven by the target.

[0093] Step S402 divides the checkpoints into time and space. First, checkpoint data preprocessing is performed. Vehicle passing data is sorted chronologically by day, and the data is grouped and deduplicated by license plate number. The vehicle trajectories are then plotted to generate a trajectory list, which is then used to identify correlations between checkpoints. If most trajectory lists contain both checkpoint A and checkpoint B, checkpoints A and B are considered to be spatiotemporally correlated. Finally, the spatiotemporally correlated checkpoints are grouped into a spatiotemporal region.

[0094] Step S403: Portrait image clustering. The portrait image clustering process is divided into two stages: In the first stage, the portrait images captured by the camera in each spatiotemporal region are clustered separately, and the average centroid of each cluster is calculated. In the second stage, the clusters between spatiotemporal regions are compared for similarity. When the similarity meets the set threshold, the two clusters are merged, and the average centroid of the new cluster is calculated. Specifically, the clustering in the first stage is performed every six hours, and the merging of clusters generated by clustering in different spatiotemporal regions in the second stage is performed once a day.

[0095] Step S404: RF-assisted cluster merging. First, within the portrait image clusters clustered in step S403, clusters containing license plates are filtered out based on the tag information retrieved using RF technology. Then, for clusters containing license plates, the average centroid of each cluster is compared with that of other clusters to identify clusters with high similarity. Finally, the spatiotemporal relationships between clusters with high similarity are determined based on the tag information. Clusters that satisfy spatiotemporal correlations are then compared for similarity to achieve cluster merging.

[0096] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here. This embodiment also provides a portrait image clustering device, which is used to implement the above embodiments and preferred implementations, and those that have been described will not be repeated. As used below, the terms "module", "unit", "sub-unit", etc. can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0097] Figure 5 is a structural block diagram of a portrait image clustering device according to an embodiment of the present application, such as Figure 5 As shown, the device includes: an acquisition module 52, an association module 54 and a clustering module 56; the acquisition module 52 is used to acquire portrait images of multiple targets to be measured at each image acquisition point; the association module 54 is used to acquire label information of the target to be measured identified by each reader device, calculate the motion trajectory of each target to be measured based on the label information, and generate a spatiotemporal association result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed under each image acquisition point; the clustering module 56 is used to perform portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image.

[0098] Through the above embodiment, the acquisition module 52 acquires the portrait image and the label information of each target to be measured identified by each reader device. The association module 54 calculates the motion trajectory of each target to be measured based on the label information to mine the image acquisition points with temporal and spatial correlation. Finally, the clustering module 56 assists in clustering the portrait image based on the temporal and spatial correlation results and the label information, thereby realizing the division of the associated image acquisition points into a temporal and spatial region. Since the data acquisition accuracy of the reader device is less affected by environmental factors, the clustering recall rate caused by the presence of a large number of blurred portrait images in the captured images can be avoided, thereby improving the clustering efficiency and accuracy of the captured portrait images in a large area, solving the problems of low accuracy and efficiency of portrait image clustering, and realizing a portrait image clustering device based on radio frequency technology.

[0099] In some embodiments, the association module 54 is further used to obtain the numbering information of each image acquisition point; the association module 54 groups the numbering information according to the label information to obtain a grouping result, and deduplicates the grouping result according to the numbering information to obtain a deduplication result; the association module 54 generates the motion trajectory based on the numbering information and the deduplication result.

[0100] In some embodiments, the association module 54 is further configured to utilize an association rule mining algorithm to calculate a first-level candidate item set of the image acquisition point based on the motion trajectory, and perform iterative calculations based on the first-level candidate item set to obtain a second-level candidate item set; the association module 54 utilizes the association rule mining algorithm to calculate a lift of the image acquisition point, obtains association results between the second-level candidate item sets based on the lift, as well as preset support and confidence, and obtains a target set based on the association results and the second-level candidate item set; the association module 54 generates the spatiotemporal association results based on the target set.

[0101] In some embodiments, the clustering module 56 is also used to obtain image features in the portrait image; the clustering module 56 performs initial portrait clustering based on the spatiotemporal correlation result and the image features to obtain initial cluster clusters, and calculates the first average centroid of each initial cluster cluster; the clustering module 56 calculates the first similarity result between all the first average centroids, merges the initial cluster clusters based on the first similarity result to obtain an initial merged cluster set, and generates the clustering result based on the initial merged cluster set and the label information.

[0102] In some embodiments, the clustering module 56 is further configured to filter the initial merged cluster set based on the identity information in the tag information to obtain a target cluster set including the identity information and a second average centroid of each target cluster set; the clustering module 56 calculates a second similarity result between all the second average centroids, and sorts the target cluster set based on the second similarity result to obtain a sorted cluster set; the clustering module 56 calculates the corresponding trajectory missing value result based on the portrait image in the sorted cluster set, and obtains association information between the sorted cluster sets based on the trajectory missing value result; the clustering module 56 calculates a third similarity result between the sorted cluster sets based on the association information and a preset similarity threshold, and merges the sorted cluster sets based on the third similarity result to obtain the clustering result.

[0103] In some embodiments, the above-mentioned portrait image clustering device further includes a sending module; the sending module is used to send the portrait image and the label information to a remote server for storage.

[0104] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0105] This embodiment also provides a portrait image clustering system, which includes: a main control device and various reader devices; wherein each reader device is correspondingly installed under each image acquisition point; and the main control device is used to execute the steps of any of the above method embodiments. Through the above embodiment, the main control device obtains portrait images and tag information of each target to be measured identified by each reader device, calculates the motion trajectory of each target to be measured based on the tag information, and mines the image acquisition points associated with time and space. Finally, based on the time and space association results and the tag information, the portrait images are assisted in portrait clustering, and the associated image acquisition points are divided into a time and space region. Since the data acquisition accuracy of the reader device is less affected by environmental factors, the clustering recall rate caused by the presence of a large number of blurred portrait images in the captured images can be avoided, thereby improving the clustering efficiency and accuracy of captured portrait images in a large area, solving the problem of low accuracy and efficiency of portrait image clustering, and realizing a portrait image clustering system based on radio frequency technology.

[0106] In some embodiments, a computer device is provided, which may be a server. Figure 6 This is a structural diagram of the internal structure of a computer device according to an embodiment of the present application. Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store clustering results. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for clustering portrait images is implemented.

[0107] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0108] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0109] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0110] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0111] S1, obtaining portrait images of multiple targets to be measured at each image acquisition point.

[0112] S2, obtaining the tag information of the target to be measured identified by each reader device, calculating the motion trajectory of each target to be measured based on the tag information, and generating the spatiotemporal correlation results between all the image acquisition points based on the motion trajectory; wherein, each reader device is correspondingly installed under each image acquisition point.

[0113] S3: Performing portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image.

[0114] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0115] In addition, in conjunction with the portrait image clustering method in the above embodiment, the present application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, it implements any of the portrait image clustering methods in the above embodiment.

[0116] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0117] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A portrait image clustering method, characterized in that: The method comprises: Acquire portrait images of multiple targets at each image acquisition point; Obtaining tag information for the target to be measured identified by each reader device, calculating a motion trajectory of each target to be measured based on the tag information, and generating a spatiotemporal correlation result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed under each image acquisition point; Performing portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image includes: Obtain image features from a portrait image; Performing initial portrait clustering according to the spatiotemporal correlation result and the image features to obtain initial clusters, and calculating a first average centroid of each of the initial clusters; Calculating first similarity results between all the first average centroids, and merging the initial clusters based on the first similarity results to obtain an initial merged cluster set; Filtering the initial merged cluster set based on the identity information in the tag information to obtain a target cluster set including the identity information and a second average centroid of each target cluster set; Calculating second similarity results between all the second average centroids, and sorting the target cluster set based on the second similarity results to obtain a sorted cluster set; Calculating corresponding trajectory missing value results according to the portrait images in the sorted cluster sets, and obtaining association information between the sorted cluster sets based on the trajectory missing value results; A third similarity result between the sorted cluster sets is calculated based on the association information and a preset similarity threshold, and the sorted cluster sets are merged according to the third similarity result to obtain the cluster result.

2. The portrait image clustering method according to claim 1, characterized in that: The step of calculating the motion trajectory of each target to be detected according to the tag information includes: Obtaining the number information of each image acquisition point; performing grouping processing on the number information according to the tag information to obtain a grouping result, and performing deduplication processing on the grouping result according to the number information to obtain a deduplication result; The motion trajectory is generated based on the numbering information and the deduplication result.

3. The portrait image clustering method according to claim 1, characterized in that: Generating the spatiotemporal correlation results between all the image acquisition points according to the motion trajectory includes: Using an association rule mining algorithm, a first-level candidate item set of the image acquisition point is calculated according to the motion trajectory, and a second-level candidate item set is obtained by iterative calculation based on the first-level candidate item set; Calculating the lift of the image acquisition points using the association rule mining algorithm, obtaining association results between the secondary candidate item sets based on the lift, as well as preset support and confidence, and obtaining a target set based on the association results and the secondary candidate item sets; The spatiotemporal association result is generated according to the target set.

4. The portrait image clustering method according to any one of claims 1 to 3, characterized in that: The method further comprises: The portrait image and the tag information are sent to a remote server for storage.

5. A portrait image clustering device, characterized in that: The device includes: an acquisition module, an association module and a clustering module; The acquisition module is used to acquire portrait images of multiple targets to be measured at each image acquisition point; The association module is configured to obtain tag information for the target to be measured identified by each reader device, calculate a motion trajectory of each target to be measured based on the tag information, and generate a spatiotemporal association result between all the image acquisition points based on the motion trajectory; wherein each reader device is correspondingly installed under each image acquisition point; The clustering module is configured to perform portrait clustering on the portrait image based on the spatiotemporal association result and the label information to generate a clustering result of the portrait image; The clustering module is further configured to obtain image features in the portrait image; perform initial portrait clustering based on the spatiotemporal correlation result and the image features to obtain initial cluster clusters, and calculate a first average centroid of each of the initial cluster clusters; calculate a first similarity result between all the first average centroids, and merge the initial cluster clusters based on the first similarity result to obtain an initial merged cluster set; filter the initial merged cluster set based on the identity information in the tag information to obtain a target cluster set including the identity information, and a second average centroid of each target cluster set; calculate a second similarity result between all the second average centroids, and sort the target cluster set based on the second similarity result to obtain a sorted cluster set; calculate a corresponding trajectory missing value result based on the portrait image in the sorted cluster set, and obtain association information between the sorted cluster sets based on the trajectory missing value result; calculate a third similarity result between the sorted cluster sets based on the association information and a preset similarity threshold, and merge the sorted cluster sets based on the third similarity result to obtain the clustering result.

6. A portrait image clustering system, characterized in that: The system includes: a main control device and various reader devices; wherein, each reader device is correspondingly installed under each image acquisition point; The main control device is used to execute the portrait image clustering method according to any one of claims 1 to 4.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the portrait image clustering method according to any one of claims 1 to 4.

8. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the portrait image clustering method according to any one of claims 1 to 4 when running.

Images