Clustering method, device, electronic device, and computer-readable storage medium

Abstract

The present invention provides a clustering method, device, electronic device, and computer-readable storage medium, wherein the clustering method includes: clustering a feature set in a database based on a first similarity threshold, thereby obtaining multiple first-class clusters; merging at least two first-class clusters in the multiple first-class clusters based on a second similarity threshold and a constraint condition; wherein the second similarity threshold is less than the first similarity threshold. This can solve the problem of multiple levels of one target appearing at a high threshold and improve clustering accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image clustering, and in particular to a clustering method, device, electronic device, and computer-readable storage medium. Background Art

[0002] In the clustering process, the features corresponding to the target are generally extracted, the similarity is calculated through feature matching, and compared with the set threshold to determine whether to merge into one category or one cluster. Therefore, the discrimination of features and the setting of thresholds play a vital role in the clustering results. The high threshold strategy can ensure accuracy, but due to factors such as incomplete targets (truncation, object occlusion, etc.), the distinguishability of different parts of the same target is very different. Insufficient feature distinguishability often leads to the clustering of target features of the same identity to generate multiple clusters, resulting in multiple clusters for one target, which seriously affects the clustering accuracy. Summary of the invention

[0003] The present invention provides a clustering method, device, electronic device, and computer-readable storage medium for improving clustering accuracy.

[0004] To solve the above technical problems, the first technical solution provided by the present invention is: providing a clustering method, comprising: clustering a feature set in a database based on a first similarity threshold, thereby obtaining multiple first-class clusters; based on a second similarity threshold and a constraint condition, merging at least two first-class clusters in the multiple first-class clusters; wherein the second similarity threshold is less than the first similarity threshold; and the constraint condition is used to reduce the number of first-class clusters.

[0005] Among them, the constraints include spatiotemporal constraints. Based on the second similarity threshold and the constraints, the step of merging at least two first-category clusters from multiple first-category clusters includes: based on the second similarity threshold and the spatiotemporal constraints, merging at least two first-category clusters from multiple first-category clusters to obtain multiple second-category clusters; the spatiotemporal constraints utilize conditions in two different dimensions, time and space, to constrain the merged first-category clusters.

[0006] Among them, the constraints also include a central feature. Based on the second similarity threshold and the spatiotemporal constraints, at least two first clusters out of the multiple first clusters are merged to obtain multiple second clusters. After that, the step also includes: based on the second similarity threshold and the central feature of each second cluster, at least two second clusters out of the multiple second clusters are merged; wherein the central feature represents the average feature of the features in each second cluster.

[0007] Among them, based on the second similarity threshold and the time-space constraints, the step of merging at least two first clusters out of the multiple first clusters includes: selecting a first feature from some of the multiple first clusters, and selecting at least one second feature from each of the remaining first clusters; respectively determining the distance between a capture device corresponding to each first feature and a capture device corresponding to each second feature, and respectively determining the time difference between the capture time of each first feature and the capture time of each second feature; the remaining first clusters include the first clusters other than some of the multiple first clusters; based on the second similarity threshold, distance and time difference, at least two of the multiple first clusters are merged.

[0008] Among them, the step of merging at least two first clusters among multiple first clusters based on the second similarity threshold, the distance and the time difference includes: in response to the distance being not greater than the distance threshold, in response to the time difference being not greater than the time threshold, and in response to the similarity being not less than the second similarity threshold, merging the first cluster to which the first feature belongs and the first cluster to which the second feature belongs.

[0009] Among them, the clustering method also includes: setting a second similarity threshold set, a distance threshold set, and a time threshold set, wherein the time threshold in the time threshold set and the second similarity threshold in the second similarity threshold set are in a linear relationship; in response to the distance being greater than the distance threshold, adjusting the distance threshold based on the distance threshold set; or in response to the time difference being greater than the time threshold, adjusting the time threshold based on the time threshold set; or in response to the similarity being less than the second similarity threshold, adjusting the second similarity threshold based on the second similarity set.

[0010] Among them, the step of merging the corresponding at least two second clusters based on the second similarity threshold and the central feature includes: performing weighted average calculation on the features in each second cluster to obtain the central feature of each second cluster; and merging at least two second clusters using the central feature and the second similarity threshold.

[0011] The step of merging at least two second clusters using the central feature and the second similarity threshold comprises: calculating the similarity of the central features of each two second clusters respectively; and merging the second clusters corresponding to the central features in response to the similarity being not less than the second similarity threshold.

[0012] Wherein, the constraint condition includes a central feature, and based on the second similarity threshold and the constraint condition, the step of merging at least two first-category clusters from multiple first-category clusters includes: merging at least two first-category clusters from multiple first-category clusters based on the second similarity threshold and the central feature of each first-category cluster; wherein the central feature represents the average feature of the features in each first-category cluster.

[0013] Among them, the step of merging at least two first-class clusters among multiple first-class clusters based on the second similarity threshold and the central feature includes: performing weighted average calculation on the features in each first-class cluster to obtain the central feature of each first-class cluster; and merging at least two first-class clusters among multiple first-class clusters using the central feature and the second similarity threshold.

[0014] Among them, the step of merging at least two first clusters among multiple first clusters using the central feature and the second similarity threshold includes: calculating the similarity of the central features of each two first clusters respectively; in response to the similarity being not less than the second similarity threshold, merging the first clusters corresponding to the central features.

[0015] The first similarity threshold is greater than 90%.

[0016] The feature set includes features corresponding to the image, or the feature set is determined based on features of the image.

[0017] To solve the above technical problems, the second technical solution provided by the present invention is: to provide a clustering device, including: a clustering module, used to cluster the feature set in the database based on a first similarity threshold, and thereby obtain multiple first-class clusters; a merging module, used to merge at least two first-class clusters in the multiple first-class clusters based on a second similarity threshold and constraints; wherein the second similarity threshold is less than the first similarity threshold; and the constraints are used to reduce the number of first-class clusters.

[0018] To solve the above technical problems, the third technical solution provided by the present invention is to provide an electronic device, comprising: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute any of the above methods.

[0019] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, and the program file can be executed to implement any of the above methods.

[0020] The beneficial effect of the present invention is different from the prior art. The present invention clusters the feature set in the database based on the first similarity threshold, thereby obtaining multiple first-class clusters; based on the second similarity threshold less than the first similarity threshold and the constraint condition, at least two first-class clusters in the multiple first-class clusters are merged. In this way, the situation of one target with multiple levels when high-threshold clustering is performed can be solved, and the clustering accuracy can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. Among them:

[0022] Figure 1 It is a schematic diagram of the flow chart of the first embodiment of the clustering method of the present invention;

[0023] Figure 2 for Figure 1 A schematic flow chart of a first embodiment of step S12;

[0024] Figure 3 for Figure 1 A schematic flow chart of a second embodiment of step S12;

[0025] Figure 4 It is a structural schematic diagram of an embodiment of a clustering device of the present invention;

[0026] Figure 5 It is a structural schematic diagram of an embodiment of an electronic device of the present invention;

[0027] Figure 6 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0028] In recent years, with the development of artificial intelligence technology, a large amount of video image data will be generated in the fields of intelligent security, new retail and the Internet. Intelligent data analysis of video image content has important application value. Compared with traditional video image retrieval technology, image clustering technology can be used to establish files for pedestrians and vehicles. For example, the image and motion data features of people with the same identity are clustered into clusters, and file indexes are established to build a personnel file database. This can achieve higher-level file retrieval functions, thereby providing users with a more intelligent product experience. In addition, image clustering technology also plays an important role in spatiotemporal behavior trajectory analysis, user portrait analysis, abnormal behavior detection, criminal control and urban traffic flow analysis, and has broad application prospects.

[0029] In large-scale video data analysis, conventional clustering algorithms (such as K-means clustering, spectral clustering, and density clustering) can no longer meet actual business needs. Especially when clustering human data (such as pedestrians and cyclists), changes in perspective and posture make it difficult for traditional clustering methods to obtain good clustering accuracy. Clustering algorithms based on facial features can realize pedestrian identity landing and file building. Effective facial feature extraction often requires the camera to capture a clear frontal image of the pedestrian at close range, but in natural monitoring scenes, the perspective and posture of pedestrians are uncontrolled, and pedestrian behaviors such as lowering their heads or wearing masks will affect the effect of face clustering. Therefore, extracting pedestrians' body appearance features for human body clustering can serve as an effective supplement to face clustering, thereby improving the clustering recall rate and identity landing rate under natural monitoring conditions.

[0030] In the process of human clustering, human image features are generally extracted, and similarity is calculated by feature matching and compared with the set threshold to determine whether to merge into one class or one cluster. Therefore, the discriminability of features and threshold setting play a vital role in the clustering results. The high threshold strategy can ensure accuracy. Due to factors such as incomplete human images (truncation, object occlusion, etc.), resolution of captured images and differences in viewing angles, human features are much less distinguishable than facial features. Insufficient feature distinguishability often leads to multiple clusters of human features of the same identity, resulting in multiple files, and the low threshold strategy will cause clustering to have a greater risk of mis-clustering. Therefore, how to design an algorithm to improve the clustering recall rate while ensuring clustering accuracy is a technical problem that needs to be solved urgently. The present invention provides a clustering method, which can improve the clustering recall rate and improve the clustering accuracy. The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of this application.

[0031] See also Figure 1 , is a schematic diagram of a flow chart of a first embodiment of a clustering method of the present invention, which specifically includes:

[0032] Step S11: clustering the feature set in the database based on a first similarity threshold, thereby obtaining a plurality of first clusters.

[0033] Specifically, the feature set in the database is clustered using the first similarity threshold. The feature set may be a feature set corresponding to an image in the database or a feature set corresponding to text in the database. In another embodiment, the feature set may also be determined based on image features, which is not specifically limited.

[0034] In one embodiment, image data is taken as an example, and the K-means clustering algorithm is taken as an example for explanation. Specifically, a feature extraction algorithm such as a convolutional neural network algorithm is used to extract features from images in a database, thereby obtaining a feature set. The feature set is divided into K groups, and K objects are randomly selected as initial cluster centers, and then the distance between each object and each cluster center, that is, the similarity, is calculated, and the similarity is compared with a first similarity threshold, and each object whose similarity is greater than the first similarity threshold is assigned to a corresponding cluster center, thereby obtaining multiple first-class clusters.

[0035] Step S12: Based on the second similarity threshold and the constraint condition, at least two first-type clusters among the plurality of first-type clusters are merged.

[0036] Specifically, when the first similarity threshold is large, although the clustering accuracy can be guaranteed, due to factors such as incomplete target images (truncation, object occlusion, etc.), the distinguishability of human body features is much worse than that of facial features. Taking people as an example, if the captured image of a person is incomplete, the resolution of the captured image is different, or there are factors such as differences in viewing angles, insufficient feature distinguishability will often lead to multiple clusters of human body features of the same identity, resulting in multiple files for one person. Therefore, the present application uses the second similarity threshold and constraints to merge the first type of clusters, so that multiple files corresponding to the same person can be merged, which can not only ensure clustering accuracy, but also solve the problem of multiple files. In a specific embodiment, the second similarity threshold is less than the first similarity threshold.

[0037] In the prior art, only the second similarity threshold is introduced to merge the first type of clusters, but inaccurate or insufficient merging may occur. The present application introduces a constraint condition, which is used to reduce the number of first type clusters, so that the first type clusters are fully merged and the accuracy of the merging is improved.

[0038] In one embodiment, the constraint condition is a spatiotemporal constraint condition, which uses the conditions of two different dimensions, time and space, to constrain the merged first cluster. For example, at the same time, the same target appears in different spaces, which is impossible; or, at a similar time, the same target appears in a farther space, which is unreasonable. Based on this idea, the first cluster is merged in combination with the second similarity threshold to obtain multiple second clusters.

[0039] Specifically, in a specific embodiment, based on the second similarity threshold and the spatiotemporal constraint condition, at least two first-class clusters among the plurality of first-class clusters are merged to obtain a plurality of second-class clusters. Figure 2 , Figure 2 This is a flow chart of a first embodiment of step S12, which specifically includes:

[0040] Step S21: Select a first feature from some of the first clusters among the multiple first clusters, and select at least one second feature from each of the remaining first clusters; respectively determine the distance between the capture device corresponding to each first feature and the capture device corresponding to each second feature, and respectively determine the time difference between the capture time of each first feature and the capture time of each second feature; the remaining first clusters include the first clusters other than some of the first clusters among the multiple first clusters.

[0041] In a specific embodiment, a first feature is selected from some of the first clusters in the plurality of first clusters, and at least one second feature is selected from each of the remaining first clusters, and the remaining first clusters are first clusters other than some of the first clusters in the plurality of first clusters. The distance between the capture device corresponding to each first feature and the capture device corresponding to each second feature is determined, and the time difference between the capture time of each first feature and the capture time of each second feature is determined. The distance of the capture device is the GPS position of the capture device.

[0042] For example, the first cluster includes the first cluster A, the first cluster B and the first cluster C, wherein the first cluster A, the first cluster B and the first cluster C all contain multiple images. An image is selected from the first cluster A as the first feature, for example, the image with the highest quality score in the first cluster A can be selected as the first feature. At least one second feature can be selected from each of the first cluster B and the first cluster C, and similarly, the images with higher quality scores in the first cluster B and the first cluster C can be selected as the second feature. The distance between the capture device corresponding to the first feature in the first cluster A and the capture device corresponding to the second feature in the first cluster B is determined, and the time difference between the capture time of the first feature in the first cluster A and the capture time of the second feature in the first cluster B is determined, the distance between the capture device corresponding to the first feature in the first cluster A and the capture device corresponding to the second feature in the first cluster C is determined, and the time difference between the capture time of the first feature in the first cluster A and the capture time of the second feature in the first cluster C is determined.

[0043] Step S22: merging at least two first-type clusters from the plurality of first-type clusters based on the second similarity threshold, the distance and the time difference.

[0044] Specifically, the distance is compared with a distance threshold, the time difference is compared with a time threshold, and the similarity between the first feature and the second feature is compared with a second similarity threshold. In response to the distance being not greater than the distance threshold, in response to the time difference being not greater than the time threshold, and in response to the similarity being not less than the second similarity threshold, the first cluster of the corresponding first feature and the first cluster to which the second feature belongs are merged.

[0045] In a specific embodiment, a first feature may be selected from the first cluster A, and at least one second feature may be selected from each of the first cluster B and the first cluster C, and the distance between the capture devices corresponding to the first feature and the second feature and the time difference between the capture times corresponding to the first feature and the second feature may be determined, and the similarity between the first feature and the second feature may be calculated. If the distance between one of the second features and the first feature is not greater than a distance threshold, and the time difference is not greater than a time threshold, and the similarity is not less than a second similarity threshold, then the first cluster B or C corresponding to the second feature may be merged with the first cluster A corresponding to the first feature. For example, if a second feature in the first cluster B and a first feature in the first cluster A meet the following conditions:

[0046]

[0047] Among them, d n-k represents the distance between the capture device of the first feature n and the capture device of the second feature k, t n- t k represents the time difference between the capture time of the first feature n and the capture time of the second feature k, sim(n,k) represents the similarity between the first feature n and the second feature k, Δd represents the distance threshold, Δt represents the time threshold, and s1 represents the second similarity threshold.

[0048] If a second feature in the first cluster B and a first feature in the first cluster A satisfy the condition of formula (1), the first cluster A and the first cluster B are merged.

[0049] In another implementation, to ensure the accuracy of the merging, the number of second features can be further increased. For example, if multiple second features in the first cluster B and one first feature in the first cluster A meet the conditions of formula (1), the first cluster A and the first cluster B are merged.

[0050] Specifically, setting a distance threshold and a time threshold can reduce the impact of interference data, that is, under the constraints of the distance threshold and the time difference, the differences between different human bodies are large, and further setting a second similarity threshold can merge clusters with greater similarity under the constraints of the distance threshold and the time difference.

[0051] In a specific embodiment, a first-class cluster is merged at most once. Specifically, when the first-class cluster B is merged into the first-class cluster A, in the subsequent cluster-to-cluster merging judgment process, the feature data in the first-class cluster B will not be compared again as the first feature. This can further improve the clustering accuracy.

[0052] In another embodiment, a second similarity threshold set, a distance threshold set, and a time threshold set are set, wherein the time threshold in the time threshold set and the second similarity threshold in the second similarity threshold set are in a linear relationship. In response to the distance being greater than the distance threshold, the distance threshold is adjusted based on the distance threshold set; or in response to the time difference being greater than the time threshold, the time threshold is adjusted based on the time threshold set; or in response to the similarity being less than the second similarity threshold, the second similarity threshold is adjusted based on the second similarity set.

[0053] Specifically, an adaptive second similarity threshold can be set for different first features according to the spatiotemporal constraints from strong to weak. Specifically, the distance threshold or the time threshold is set to be adjustable. For example, the distance threshold Δd=500m is set, and the time threshold Δt is set to be adjustable. It can be understood that the larger the time threshold Δt is, the weaker the spatiotemporal constraint condition is. In order to ensure the clustering accuracy, a higher second similarity threshold needs to be set. That is, the time threshold is positively correlated with the second similarity threshold.

[0054] In a specific embodiment, when the distance threshold Δd is set to a fixed value, for example, Δd=500m, a discrete second similarity threshold set S1=(84, 85, 86) can be set, and the corresponding time threshold is calculated according to the linear calculation formula (2):

[0055]

[0056] The value range of the second similarity threshold s1 is [S min ,S max ], the value range of the time threshold Δt [Δt min ,Δt max ].

[0057] For example, the calculated time threshold Δt set is Δt = (150, 200, 300). You can traverse from large to small according to the second similarity threshold s1 and the time threshold Δt, that is, s1 = 86 and Δt = 300. At this time, if there is no second feature matching the first feature, that is, all second features do not satisfy formula (1), then try s1 = 85 and Δt = 200 until a second feature that satisfies the above formula (1) is matched. If the second feature that satisfies the above formula (1) is not matched when s1 = 84 and Δt = 150 are traversed, it means that there is no first class cluster that can be merged.

[0058] In one embodiment, the first similarity threshold is greater than 90%.

[0059] Through the spatiotemporal constraints and the relaxation threshold (i.e., the second similarity threshold is less than the first similarity threshold) of this embodiment, the efficiency and accuracy of cluster merging can be effectively improved, and the probability of difficult examples forming clusters alone can be reduced. In addition, in practical applications, the spatiotemporal relationship between adjacent cameras at adjacent times can be modeled based on the spatiotemporal constraints and the relaxation threshold strategy, and relay tracking across cameras can be achieved through human feature comparison while reducing interference.

[0060] In another embodiment, the constraint condition includes a central feature. For example, in a specific embodiment, at least two first-class clusters are merged based on the second similarity threshold and the central feature of each first-class cluster. Figure 3 , Figure 3 This is a flow chart of a second embodiment of step S12, which specifically includes:

[0061] Step S31: performing weighted average calculation on the features in each first cluster, and thereby obtaining the central feature of each first cluster.

[0062] Specifically, the features in each first cluster may be weighted averaged to obtain the central features of each first cluster. In a specific embodiment, the features in each first cluster may be weighted averaged based on the quality score of each image in the first cluster to obtain the central features of each first cluster. In another embodiment, the images in the first cluster may be screened to select images with better quality scores, and the features in each first cluster may be weighted averaged using the quality scores corresponding to the images to obtain the central features of each first cluster.

[0063] Furthermore, in one embodiment, a weighted average calculation is performed on the features in each first cluster, and the result after the weighted average calculation is normalized to obtain the central feature of each first cluster. The weighted average calculation method for the features in each first cluster is:

[0064] f=(q1f1+q2f2+…+q k f k ) / K;

[0065] Among them, q k is the quality score of the kth feature, f k is the kth feature.

[0066] The result of weighted average calculation is normalized to obtain the central feature:

[0067]

[0068] Step S32: merging at least two first clusters from the plurality of first clusters by using the central feature and the second similarity threshold.

[0069] Specifically, the similarity of the central features of the two first clusters is calculated, and the similarity is compared with the second similarity threshold, and in response to the similarity being not less than the second similarity threshold, the first clusters corresponding to the central features are merged. For example, if the similarity of the central feature of the first cluster A and the central feature of the first cluster B is not less than the second similarity threshold, the first cluster A is merged with the first cluster B.

[0070] In this embodiment, the central features of the first type of clusters are used to merge clusters, which can improve the distinction between clusters, reduce the amount of calculation, and improve the accuracy of clustering.

[0071] In another embodiment, the constraint condition includes a spatiotemporal constraint condition and a central feature constraint condition. That is, in a specific embodiment, based on the second similarity threshold and the spatiotemporal constraint condition, at least two first clusters in the plurality of first clusters are merged to obtain a plurality of second clusters; then, based on the second similarity threshold and the central feature of each second cluster, at least two second clusters in the plurality of second clusters are merged.

[0072] In this embodiment, based on the second similarity threshold and the spatiotemporal constraint conditions, at least two of the multiple first clusters are merged to obtain multiple second clusters in the same manner as in the above-described embodiment, which is not described in detail here. Based on the second similarity threshold and the central feature of each second cluster, at least two of the multiple second clusters are merged, specifically including: performing weighted average calculation on the features in each second cluster to obtain the central feature of each second cluster, and the weighted average calculation method is:

[0073] f=(q1f1+q2f2+…+q k f k ) / K;

[0074] Among them, q k is the quality score of the kth feature, f k is the kth feature.

[0075] The results of weighted average calculation are normalized to obtain the second type of cluster center features:

[0076]

[0077] The similarity of the central features of the two second clusters is calculated, and the similarity is compared with a second similarity threshold. In response to the similarity being not less than the second similarity threshold, the second clusters corresponding to the central features are merged.

[0078] In this embodiment, through spatiotemporal constraints and relaxation thresholds (i.e., the second similarity threshold is less than the first similarity threshold), the efficiency and accuracy of cluster merging can be effectively improved, and the probability of difficult examples forming clusters alone can be reduced. In addition, in practical applications, based on the strategy of spatiotemporal constraints and relaxation thresholds, the spatiotemporal relationship between adjacent cameras at adjacent times can be modeled, and cross-camera relay tracking can be achieved through human feature comparison while reducing interference. In addition, under the strategy of spatiotemporal constraints and relaxation thresholds, the second type of clusters already contain rich posture and visual feature data, and the central features of the second type of clusters are used to merge clusters, which can increase the merging of clusters in the spatiotemporal domain, further improve the distinguishability between clusters, reduce the amount of calculation, and improve the accuracy of clustering. The clustering method of this embodiment can improve the target clustering ability in complex environments, enhance the ability to converge difficult examples, and improve the clustering recall rate.

[0079] See also Figure 4 , is a schematic structural diagram of an embodiment of a clustering device of the present invention. The clustering device includes: a clustering module 51 and a merging module 52.

[0080] The clustering module 51 is used to cluster the feature set in the database based on a first similarity threshold, thereby obtaining a plurality of first clusters.

[0081] The merging module 52 is used to merge at least two first clusters among the plurality of first clusters based on a second similarity threshold and a constraint condition, wherein the second similarity threshold is smaller than the first similarity threshold; and the constraint condition is used to reduce the number of first clusters.

[0082] In one embodiment, the merging module 52 is used to merge at least two first-class clusters among multiple first-class clusters based on a second similarity threshold and a spatiotemporal constraint condition to obtain multiple second-class clusters; the spatiotemporal constraint condition utilizes conditions in two different dimensions of time and space to constrain the merged first-class clusters.

[0083] In one embodiment, the merging module 52 is used to merge at least two first clusters among the plurality of first clusters based on the second similarity threshold and the spatiotemporal constraint condition to obtain a plurality of second clusters; the spatiotemporal constraint condition uses conditions in two different dimensions of time and space to constrain the merged first clusters. Based on the second similarity threshold and the central feature of each second cluster, at least two second clusters among the plurality of second clusters are merged; wherein the central feature represents the average feature of the features in each second cluster.

[0084] Specifically, the merging module 52 selects first features from some of the first clusters among the multiple first clusters, and selects at least one second feature from each of the remaining first clusters; respectively determines the distance between the capture device corresponding to each first feature and the capture device corresponding to each second feature, and respectively determines the time difference between the capture time of each first feature and the capture time of each second feature; the remaining first clusters include the first clusters other than some of the first clusters among the multiple first clusters; based on the second similarity threshold, distance and time difference, at least two of the multiple first clusters are merged.

[0085] Specifically, in response to the distance being not greater than the distance threshold, the time difference being not greater than the time threshold, and the similarity being not less than the second similarity threshold, the merging module 52 merges the first cluster to which the first feature belongs with the first cluster to which the second feature belongs.

[0086] In one embodiment, the merging module 52 may also set a second similarity threshold set, a distance threshold set, and a time threshold set, wherein the time threshold in the time threshold set and the second similarity threshold in the second similarity threshold set are in a linear relationship; in response to the distance being greater than the distance threshold, the distance threshold is adjusted based on the distance threshold set; or in response to the time difference being greater than the time threshold, the time threshold is adjusted based on the time threshold set; or in response to the similarity being less than the second similarity threshold, the second similarity threshold is adjusted based on the second similarity set.

[0087] In one embodiment, the merging module 52 may further perform weighted average calculation on the features in each second cluster to obtain the central feature of each second cluster; and merge at least two second clusters using the central feature and the second similarity threshold.

[0088] Specifically, the merging module 52 calculates the similarity of the central features of every two second clusters respectively; in response to the similarity being not less than the second similarity threshold, the second clusters corresponding to the central features are merged.

[0089] In another embodiment, the merging module 52 may also merge at least two first clusters among the plurality of first clusters based on the second similarity threshold and the central feature of each first cluster; wherein the central feature represents the average feature of the features in each first cluster.

[0090] Specifically, the merging module 52 performs weighted average calculation on the features in each first cluster to obtain the central feature of each first cluster; and merges at least two first clusters among the multiple first clusters using the central feature and the second similarity threshold.

[0091] Specifically, the merging module 52 calculates the similarity of the central features of every two first clusters respectively; in response to the similarity being not less than the second similarity threshold, the first clusters corresponding to the central features are merged.

[0092] The clustering device of this embodiment can effectively improve the efficiency and accuracy of cluster merging through spatiotemporal constraints and relaxation thresholds (i.e., the second similarity threshold is less than the first similarity threshold), and reduce the probability of difficult examples forming clusters alone. In addition, in practical applications, based on the strategy of spatiotemporal constraints and relaxation thresholds, the spatiotemporal relationship between adjacent cameras at adjacent times can be modeled, and relay tracking across cameras can be achieved through human feature comparison while reducing interference. In addition, under the strategy of spatiotemporal constraints and relaxation thresholds, the second type of clusters already contain rich posture and visual feature data, and the central features of the second type of clusters are used to merge clusters, which can increase the merging of clusters in the spatiotemporal domain, further improve the distinguishability between clusters, reduce the amount of calculation, and improve the accuracy of clustering. The clustering method of this embodiment can improve the target clustering ability in complex environments, enhance the ability to converge difficult examples, and improve the clustering recall rate.

[0093] See also Figure 5 , is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, the electronic device includes a memory 202 and a processor 201 connected to each other.

[0094] The memory 202 is used to store program instructions for implementing any one of the above methods.

[0095] The processor 201 is used to execute program instructions stored in the memory 202 .

[0096] The processor 201 may also be referred to as a CPU (Central Processing Unit). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0097] The memory 202 can be a memory stick, a TF card, etc., which can store all the information in the electronic device of the device, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory. It stores and retrieves information according to the location specified by the controller. With the memory, the electronic device has a memory function to ensure normal operation. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0098] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0100] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a system server, or a network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application.

[0102] See also Figure 6 , which is a schematic diagram of the structure of the computer-readable storage medium of the present invention. The storage medium of the present application stores a program file 203 that can implement all the above methods, wherein, The program file 203 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage device includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, etc., which can store program codes, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0103] The above are only implementation modes of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A clustering method, characterized in that: The method comprises: Clustering a feature set in a database based on a first similarity threshold to obtain a plurality of first clusters, wherein the feature set includes features corresponding to the image, or the feature set is determined based on features of the image; Based on a second similarity threshold and a constraint condition, merging at least two of the plurality of first clusters; wherein the second similarity threshold is less than the first similarity threshold; and the constraint condition is used to reduce the number of the first clusters; The constraint condition includes a spatiotemporal constraint condition and a central feature; and merging at least two of the plurality of first clusters based on the second similarity threshold and the constraint condition includes: Selecting a first feature from part of the first clusters among the plurality of first clusters, and selecting at least one second feature from each of the remaining first clusters; respectively determining the distance between a capture device corresponding to each of the first features and a capture device corresponding to each of the second features, and respectively determining the time difference between the capture time of each of the first features and the capture time of each of the second features; the remaining first clusters include first clusters among the plurality of first clusters except the part of the first clusters; A second similarity threshold set, a distance threshold set, and a time threshold set are set, wherein the time threshold in the time threshold set and the second similarity threshold in the second similarity threshold set are in a linear relationship; the second similarity threshold is adaptively set for different first features from strong to weak according to the spatiotemporal constraint, and the time threshold is positively correlated with the second similarity threshold; In response to the distance being greater than a distance threshold, adjusting the distance threshold based on the distance threshold set; in response to the time difference being greater than a time threshold, adjusting the time threshold based on the time threshold set; in response to the similarity being less than the second similarity threshold, adjusting the second similarity threshold based on the second similarity threshold set; Based on the second similarity threshold, the distance and the time difference, merging at least two of the multiple first clusters to obtain multiple second clusters, including: in response to the distance being not greater than the distance threshold, the time difference being not greater than the time threshold, and the similarity being not less than the second similarity threshold, merging the first cluster to which the first feature belongs and the first cluster to which the second feature belongs to obtain multiple second clusters; Based on the second similarity threshold and the central feature of each second cluster, at least two of the multiple second clusters are merged; wherein the central feature represents the average feature of the features in each second cluster, and the central feature of each second cluster is obtained by performing weighted averaging calculation on the features in the second cluster based on the quality score of each image in the second cluster, and normalizing the result of the weighted averaging calculation.

2. The method according to claim 1, characterized in that The spatiotemporal constraint condition utilizes conditions in two different dimensions, time and space, to constrain the merged first clusters.

3. The method according to claim 1, characterized in that The step of merging the corresponding at least two second clusters based on the second similarity threshold and the central feature includes: Based on the quality score of each image in each of the first clusters, performing weighted average calculation on the features in each of the second clusters, and then normalizing the results of the weighted average calculation to obtain the central feature of each of the second clusters; At least two of the second clusters are merged using the central feature and the second similarity threshold.

4. The method according to claim 3, characterized in that The step of merging at least two of the second clusters by using the central feature and the second similarity threshold comprises: respectively calculating the similarity of the central features of every two of the second-type clusters; In response to the similarity being not less than the second similarity threshold, the second clusters corresponding to the central features are merged.

5. The method according to claim 1, characterized in that The first similarity threshold is greater than 90%.

6. A clustering device, characterized in that: The device comprises: A clustering module, used for clustering a feature set in a database based on a first similarity threshold, thereby obtaining a plurality of first clusters, wherein the feature set includes features corresponding to the image, or the feature set is determined based on features of the image; a merging module, configured to merge at least two of the plurality of first clusters based on a second similarity threshold and a constraint condition; wherein the second similarity threshold is less than the first similarity threshold; and the constraint condition is configured to reduce the number of the first clusters; The constraint condition includes a spatiotemporal constraint condition and a central feature; and merging at least two of the plurality of first clusters based on the second similarity threshold and the constraint condition includes: Selecting a first feature from part of the first clusters among the plurality of first clusters, and selecting at least one second feature from each of the remaining first clusters; respectively determining the distance between a capture device corresponding to each of the first features and a capture device corresponding to each of the second features, and respectively determining the time difference between the capture time of each of the first features and the capture time of each of the second features; the remaining first clusters include first clusters among the plurality of first clusters except the part of the first clusters; A second similarity threshold set, a distance threshold set, and a time threshold set are set, wherein a time threshold in the time threshold set and a second similarity threshold in the second similarity threshold set are in a linear relationship; the second similarity threshold is adaptively set for different first features from strong to weak according to the spatiotemporal constraint, and the time threshold is positively correlated with the second similarity threshold; In response to the distance being greater than a distance threshold, adjusting the distance threshold based on the distance threshold set; in response to the time difference being greater than a time threshold, adjusting the time threshold based on the time threshold set; in response to the similarity being less than the second similarity threshold, adjusting the second similarity threshold based on the second similarity threshold set; Based on the second similarity threshold, the distance and the time difference, merging at least two of the multiple first clusters to obtain multiple second clusters, including: in response to the distance being not greater than the distance threshold, the time difference being not greater than the time threshold, and the similarity being not less than the second similarity threshold, merging the first cluster to which the first feature belongs and the first cluster to which the second feature belongs to obtain multiple second clusters; Based on the second similarity threshold and the central feature of each second cluster, at least two of the multiple second clusters are merged; wherein the central feature represents the average feature of the features in each second cluster, and the central feature of each second cluster is obtained by performing weighted averaging calculation on the features in the second cluster based on the quality score of each image in the second cluster, and normalizing the result of the weighted averaging calculation.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the clustering method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A program file is stored, and the program file can be executed by a processor to implement the clustering method according to any one of claims 1 to 5.

Images