A clustering method, system, control device, and readable storage medium

Abstract

This invention relates to the field of clustering technology, specifically providing a clustering method, system, control device, and readable storage medium, aiming to solve the problem of how to effectively cluster low-quality images. To this end, the clustering method of this invention includes: acquiring vector feature information and spatiotemporal trajectory information of the targets to be clustered; obtaining the similarity between two targets in the target group based on the vector feature information and spatiotemporal trajectory information; and obtaining the clustering result based on the similarity between the two targets. By incorporating spatiotemporal trajectory as a criterion for clustering, and utilizing the spatiotemporal trajectory information within low-quality images, these images can participate in clustering, increasing image utilization and improving clustering effectiveness. The determination is based on similarity. Simultaneously, the original comparison between images is changed to determination in the form of trajectories, i.e., data groups, reducing the computational load of one-to-one image comparison and enabling more efficient and faster clustering.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of clustering, and specifically provides a clustering method, a clustering system, a control device and a readable storage medium. BACKGROUND

[0002] Clustering is to divide a set of physical or abstract objects into multiple classes composed of similar objects. The cluster generated by clustering is a set of data objects, which are similar to each other in the same cluster and different from the objects in other clusters. In the scenes of security monitoring, cross-border tracking, auxiliary annotation, and co-person retrieval, a method of face and body clustering is usually adopted. Face and body clustering refers to classifying unlabeled face and body data, so that face and body data belonging to the same person are as possible as in the same class, and data of different persons are in different classes.

[0003] At present, face and body clustering mostly classifies unlabeled face and body data according to face feature information and body feature information. However, due to the influence of environmental factors such as light, angle, and distance, the face and body data captured by the camera often contains a large number of low-quality pictures, which are difficult to cluster and greatly reduce the clustering effect.

[0004] Correspondingly, there is a need in the art for a new clustering scheme to solve the above problems. SUMMARY

[0005] In order to overcome the above defects, the present application is proposed to provide a clustering method for solving or at least partially solving the technical problem of how to effectively cluster data containing low-quality pictures.

[0006] In a first aspect, the present application provides a clustering method, comprising:

[0007] Obtaining vector feature information and space-time trajectory information of a target to be clustered;

[0008] According to the vector feature information and the space-time trajectory information, obtaining the similarity of two targets to be clustered in a to-be-clustered group;

[0009] According to the similarity of the two targets to be clustered, obtaining a clustering result.

[0010] In one embodiment, the vector feature information includes face feature information and body feature information, and the space-time trajectory information includes multiple time information in a trajectory of the target to be clustered and spatial position information corresponding to the time information.

[0011] In one embodiment, the method further comprises:

[0012] Obtaining face and body data of multiple targets to be clustered by an image acquisition device;

[0013] obtaining vector feature information and space-time trajectory information of the target to be clustered according to multiple face human body data of the same target to be clustered;

[0014] obtaining a space-time target to be clustered group according to the space-time trajectory information and the vector feature information;

[0015] obtaining a feature target to be clustered group according to the vector feature information.

[0016] In an embodiment, the method further comprises:

[0017] obtaining a neighbor feature vector set of the human body feature vector of the target to be clustered by a feature retrieval algorithm, wherein the neighbor feature vector in the neighbor feature vector set is a human body feature vector of another target to be clustered;

[0018] traversing the human body feature vector to obtain a neighbor feature vector set corresponding to the human body feature vector of each target to be clustered.

[0019] In an embodiment, the obtaining a space-time target to be clustered group according to the space-time trajectory information and the vector feature information comprises:

[0020] filtering a combination of any two targets to be clustered according to a preset time interval to obtain a first space-time target to be clustered group;

[0021] if the trajectory time of the two targets to be clustered in the first space-time target to be clustered group overlaps, filtering a second space-time target to be clustered group according to a preset distance threshold;

[0022] if the trajectory time of the two targets to be clustered in the first space-time target to be clustered group does not overlap, filtering a second space-time target to be clustered group according to a preset speed threshold.

[0023] In an embodiment, the obtaining a space-time target to be clustered group according to the space-time trajectory information and the vector feature information further comprises:

[0024] judging whether the two targets to be clustered in the second space-time target to be clustered group are neighbor feature vectors of each other according to the vector feature information;

[0025] if not, removing the second space-time target to be clustered group;

[0026] if yes, retaining the two targets to be clustered to obtain a space-time target to be clustered group.

[0027] In an embodiment, the obtaining a feature target to be clustered group according to the vector feature information comprises:

[0028] According to the human feature vector and the corresponding set of nearest neighbor feature vectors, a feature to-be-clustered group is obtained by a Prim algorithm.

[0029] In one embodiment, the filtering of combinations of any two to-be-clustered targets according to a preset time interval threshold to obtain a first spatiotemporal to-be-clustered group comprises:

[0030] determining whether the trajectory time interval of the two to-be-clustered targets is less than the preset time interval threshold;

[0031] if less than, retaining the two to-be-clustered targets to obtain the first spatiotemporal to-be-clustered group;

[0032] if not less than, removing the combination of the two to-be-clustered targets.

[0033] In one embodiment, if the trajectory time of the two to-be-clustered targets in the first spatiotemporal to-be-clustered group overlaps, a spatiotemporal to-be-clustered group is obtained by filtering according to a preset distance threshold, comprising:

[0034] determining whether the average distance of the trajectory of the time overlap part of the two to-be-clustered targets is greater than the preset distance threshold;

[0035] if greater than, removing the first spatiotemporal to-be-clustered group;

[0036] if not greater than, retaining the two to-be-clustered targets to obtain a second spatiotemporal to-be-clustered group.

[0037] In one embodiment, if the trajectory time of the two to-be-clustered targets in the first spatiotemporal to-be-clustered group does not overlap, a spatiotemporal to-be-clustered group is obtained by filtering according to a preset speed threshold, comprising:

[0038] determining, according to the time occurrence order, whether the speed required for a to-be-clustered target in the first spatiotemporal to-be-clustered group to be converted from a current trajectory to the trajectory of another to-be-clustered target is greater than a preset speed threshold;

[0039] if greater than, removing the first spatiotemporal to-be-clustered group;

[0040] if not greater than, retaining the two to-be-clustered targets to obtain a second spatiotemporal to-be-clustered group.

[0041] In one embodiment, the similarity of the two to-be-clustered targets in the to-be-clustered group is obtained according to the vector feature information and the spatiotemporal trajectory information, comprising:

[0042] obtaining the face feature similarity of the two to-be-clustered targets according to the face feature information;

[0043] obtaining the human feature similarity of the two to-be-clustered targets according to the human feature information;

[0044] According to the spatio-temporal trajectory information, a spatio-temporal trajectory similarity of the two clustering targets is obtained.

[0045] According to the face feature similarity, the body feature similarity and the spatio-temporal trajectory similarity, a similarity of the two clustering targets in the clustering group is obtained.

[0046] In one embodiment, the face feature similarity of the two clustering targets is obtained according to the face feature information, comprising:

[0047] According to the average face feature information of the clusters to which the two clustering targets respectively belong, the face feature similarity of the two clustering targets is obtained.

[0048] In one embodiment, the body feature similarity of the two clustering targets is obtained according to the body feature information, comprising:

[0049] According to the body feature information of the clusters to which the two clustering targets respectively belong, the face feature similarity of the two clustering targets is obtained.

[0050] In one embodiment, the spatio-temporal trajectory similarity of the two clustering targets is obtained according to the spatio-temporal trajectory information, comprising:

[0051] A first information pair is obtained by combining any one spatio-temporal trajectory information from the clusters to which the two clustering targets respectively belong;

[0052] The clusters to which the two clustering targets respectively belong are traversed to obtain a first set composed of a plurality of first information pairs;

[0053] The first set is traversed, and a second set is obtained by screening a first information pair with a trajectory time interval less than a preset time interval threshold according to the spatio-temporal trajectory information;

[0054] According to whether the two trajectory times of the first information pair in the second set overlap, an initial similarity corresponding to the first information pair is obtained.

[0055] The second set is traversed to obtain a plurality of initial similarities of a plurality of first information pairs in the second set;

[0056] According to the plurality of initial similarities, an average similarity of the second set is obtained, wherein the average similarity is the spatio-temporal trajectory similarity of the two clustering targets.

[0057] In one embodiment, the initial similarity corresponding to the first information pair is obtained according to whether the two trajectory times of the first information pair in the second set overlap, comprising:

[0058] determine whether two trajectory times of a first information pair in the second set overlap;

[0059] If the two trajectory times overlap, an initial similarity calculation formula is

[0060]

[0061] wherein, d represents an average distance of the two trajectory times;

[0062] If the two trajectory times do not overlap, an initial similarity calculation formula is

[0063]

[0064] wherein, d represents an average distance of the two trajectory times, d error is a preset distance threshold, and v represents a preset speed threshold. ij is a time interval of the two trajectory times of the two clustering targets.

[0065] In one embodiment, the clustering result is obtained according to the similarity of the two clustering targets, and the method comprises the following steps:

[0066] determining whether the similarity is greater than a preset similarity threshold;

[0067] If the similarity is greater than the preset similarity threshold, the two clustering targets are the same target, and the labels of the two clustering targets are updated.

[0068] If the similarity is not greater than the preset similarity threshold, the two clustering targets are not the same target.

[0069] In a second aspect, the present application provides a clustering system, comprising:

[0070] a preprocessing module, which converts the face and body data into the vector feature information and the space-time trajectory information;

[0071] a feature clustering group generation module, which obtains a feature clustering group according to the vector feature information;

[0072] a space-time clustering group generation module, which obtains a space-time clustering group according to the space-time trajectory information and the vector feature information;

[0073] a clustering prediction module, which calculates the similarity of two clustering targets in the feature clustering group and the space-time clustering group;

[0074] a result output module, which obtains a clustering result according to the similarity and outputs the clustering result.

[0075] In a third aspect, a control device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to execute the method according to any one of the technical solutions of the clustering method described above.

[0076] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored, the program codes being adapted to be loaded and run by a processor to execute the method according to any one of the technical solutions of the clustering method described above.

[0077] The one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0078] In the implementation of the technical solutions of the present application, by adding the spatio-temporal trajectory as the basis for judgment in the clustering judgment, by utilizing the spatio-temporal trajectory information inside the low-quality image, the image utilization rate is increased and the clustering effect is improved. The judgment is made according to the similarity during the judgment. At the same time, the original single image comparison is changed to the form of trajectory, i.e. the form of data group, so as to reduce the calculation amount of image-to-image comparison and complete the clustering more efficiently and quickly.

[0079] Further, in order to solve the problem that the blind area does not conform to the spatio-temporal locality, the feature to-be-clustered group is generated according to the vector feature information, and the data blank in the blind area is supplemented. The correctness of clustering is improved by further judgment according to the face, human body and spatio-temporal three dimensions, and the multi-dimensional consideration ensures that the clusters that do not conform to the spatio-temporal locality will not be clustered together in the subsequent connection process, so as to ensure the improvement of the clustering effect. BRIEF DESCRIPTION OF DRAWINGS

[0080] The disclosure of the present application will become more apparent from the following description in conjunction with the accompanying drawings. It is easily understood by those skilled in the art that the drawings are only for the purpose of illustration and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, wherein:

[0081] Figure 1 is a main step flow diagram of a clustering method according to an embodiment of the present application;

[0082] Figure 2 is a main step flow diagram of a method for obtaining a to-be-clustered group from vector feature information and spatio-temporal trajectory information according to an embodiment of the present application;

[0083] Figure 3 is a main step flow diagram of a method for obtaining a near neighbor feature vector set of a human body feature vector according to an embodiment of the present application;

[0084] Figure 4 is a main step flowchart diagram of a method of obtaining a spatio-temporal clustering group according to the spatio-temporal trajectory information and the vector feature information according to an embodiment of the present application;

[0085] Figure 5 is a main step flowchart diagram of a method of obtaining a first spatio-temporal clustering group according to a preset time interval threshold according to an embodiment of the present application;

[0086] Figure 6 is a main step flowchart diagram of a method of obtaining a second spatio-temporal clustering group according to a preset distance threshold according to an embodiment of the present application

[0087] Figure 7 is a main step flowchart diagram of a method of obtaining a second spatio-temporal clustering group according to a preset speed threshold according to an embodiment of the present application;

[0088] Figure 8 is a main step flowchart diagram of a method of obtaining a similarity between two clustering targets in a clustering group according to the vector feature information and the spatio-temporal trajectory information according to an embodiment of the present application;

[0089] Figure 9 is a main step flowchart diagram of a method of calculating a spatio-temporal trajectory similarity according to an embodiment of the present application;

[0090] Figure 10 is a main step flowchart diagram of a method of obtaining a clustering result according to a similarity between two clustering targets according to an embodiment of the present application;

[0091] Figure 11 is a main structure block diagram of a clustering system according to an embodiment of the present application. DETAILED DESCRIPTION

[0092] Some embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the protection scope of the present application.

[0093] In the description of the present application, "module" and "processor" can include hardware, software or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and can also include a software portion such as program code, and can be a combination of software and hardware. The processor can be a central processor, microprocessor, image processor, digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readable storage medium includes any suitable medium that can store program code, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a similar meaning as "A and / or B", and can include only A, only B, or A and B. The singular form of the term "one", "this" can also include the plural form.

[0094] Some terms related to the present application are explained first.

[0095] K-Nearest Neighbor Search: refers to searching K features most similar to the query feature in the feature library.

[0096] Maximum Spanning Tree: refers to a tree that connects each node and has the maximum total edge weight.

[0097] Connected graph: refers to a graph in which any two nodes are connected.

[0098] Refer to the accompanying Figure 1 , Figure 1 is a schematic diagram of the main steps of a clustering method according to an embodiment of the present application. It mainly includes the following steps S10-S30:

[0099] S10, obtaining the vector feature information and the space-time trajectory information of the target to be clustered.

[0100] In this embodiment, for the target to be clustered, in addition to obtaining its basic vector feature information, the space-time trajectory information of the target to be clustered is also needed. Traditional face and body clustering mostly makes a judgment on whether two targets to be clustered are the same class according to the vector feature information. For some images with poor image quality, it is difficult to make a judgment on whether two targets to be clustered are the same target according to the extracted vector feature information. Therefore, on the basis of retaining the original vector feature information reference, the space-time trajectory information reference of the target to be clustered is added.

[0101] In one embodiment, as Figure 2 shown, the method further includes steps S01-S04:

[0102] S01, obtaining, by an image acquisition device, face and body data of a plurality of clustering targets;

[0103] In the embodiment, the image acquisition device is mostly a camera or a video camera. Take the camera capture as an example for further description. The face and body data of a plurality of clustering targets are obtained from the video captured by the camera.

[0104] S02, obtaining vector feature information and space-time trajectory information of the clustering target according to the face and body data of the clustering target.

[0105] In the embodiment, the face and body data obtained by the camera is converted into a data format required for clustering. The images of different frames of the video are obtained, and the face and body of the clustering target in the images are respectively identified and framed. The clustering target appearing continuously in different frames of images under the same camera is associated by a body tracking technology to form an image data chain. That is, one image data chain corresponds to one clustering target.

[0106] In one specific embodiment, the vector feature information includes face feature information and body feature information, and the space-time trajectory information includes a plurality of time information in the trajectory of the clustering target and space position information corresponding to the time information.

[0107] On this basis, the vector feature information and space-time trajectory information of the clustering target obtained in step S02 specifically include face feature information, body feature information, and a plurality of time information in the trajectory of the clustering target and space position information corresponding to the time information, wherein the space information is three-dimensional coordinate information of the clustering target.

[0108] Further, since the face and body in the image data chain belong to the same person, only one face feature information and one body feature information are needed for the same person. One image is selected from the image data chain, and the face feature information of the clustering target is obtained by face feature extraction technology. Similarly, another image is selected from the image data chain, and the body feature information of the clustering target is obtained by body feature extraction technology. The selection criteria of the image include but are not limited to image clarity, face or body exposure ratio.

[0109] Further, the three-dimensional scene graph is combined with the camera position in the scene for calibration, and the camera parameters are taken as data references to realize a camera calibration algorithm capable of calculating the three-dimensional coordinates of the target appearing in the camera shooting range. The camera parameters include but are not limited to camera angle, focal length, and correction parameters. The coordinates of the human body frame are obtained through the recognition and labeling of the human body in the image, and the three-dimensional coordinates of the target to be clustered in the image are obtained through the camera calibration algorithm according to the human body frame coordinates. Specifically, for example, in a shopping mall, the specific position and floor height of the target to be clustered in the shopping mall can be obtained. The image data chain has time information corresponding to the image obtained by the camera. Therefore, the time information of the target to be clustered in the image and the spatial position information corresponding to the time information can be obtained.

[0110] On this basis, in order to facilitate description, the data information obtained by one target to be clustered is denoted as track, each track contains a plurality of dense data and sparse data, and the i-th dense data of a given track contains a timestamp ts i , a human body frame (x i , y i , w i , h i ), and human body three-dimensional coordinates (X i , Y i ). The sparse data of the given track contains face feature information, i.e., a face feature vector f, and body feature information, i.e., a body feature vector b. All the obtained face and body data are converted into a format required for clustering, i.e., n tracks are obtained.

[0111] In one embodiment, as shown in Figure 3 , the method further includes steps S11-S12

[0112] S11, obtaining a set of neighbor feature vectors of the body feature vector of the target to be clustered through a feature retrieval algorithm, wherein the neighbor feature vectors in the set of neighbor feature vectors are body feature vectors of other targets to be clustered.

[0113] In this embodiment, the neighbor feature vectors of the body feature vector of the given data, i.e., the target to be clustered, are obtained through a feature retrieval algorithm (i.e., K-nearest neighbor retrieval), and a plurality of neighbor feature vectors form a set of neighbor feature vectors. The proportion of human bodies appearing in an image is much higher than that of faces, and compared with faces, human bodies are more difficult to capture. Therefore, the use of body feature vectors for feature retrieval can increase the utilization rate of images, so that some images with occluded faces but clear body data can be fully utilized, and the clustering effect is further improved.

[0114] Specifically, the obtained tracks are placed into a set, resulting in a set of n tracks. Based on the human feature vectors in the tracks, a set B = {b1, b2, ..., bn} is obtained, consisting of human feature vectors. n For any human feature vector b i ∈B, with b i For the query vector, B i =Bb i The base vector is the vector after removing b. i The remaining human feature vectors. Calculate the inner product between the query vector and the base database vector, and take the k results with the largest inner integrals to obtain b. i The K nearest neighbors are obtained, which gives the human feature vector b. i The nearest neighbor feature vector set. Feature retrieval algorithms include, but are not limited to, brute-force retrieval, IVFPQ retrieval, and HASH retrieval.

[0115] S12, traverse all the human feature vectors to obtain the nearest neighbor feature vector set corresponding to the human feature vector of each target to be clustered.

[0116] In this embodiment, since multiple targets to be clustered are obtained, there will be multiple human feature vectors corresponding to these targets. Therefore, all human feature vectors are traversed to obtain the nearest neighbor feature vector set corresponding to each human feature vector.

[0117] Specifically, iterate through all human feature vectors in B to obtain the nearest neighbor feature vector set corresponding to each human feature vector. Integrate the obtained sets in matrix form to obtain the K nearest neighbor matrix S of the human feature vectors. n×k .

[0118] Furthermore, for ease of understanding, the clustering targets are described based on a connected graph. Two clustering targets in a group are considered as nodes to be connected, and the connection between them is called an edge. After establishing the connection between two nodes, it's necessary to determine if the edge is valid. That is, whether the two clustering targets can be clustered into one group. If it is valid (the two clustering targets belong to the same group), then a connection can be established between the two nodes, and an edge can be generated. If it is not valid (the two clustering targets are not in the same group), then a connection cannot be established between the two nodes, and an edge cannot be generated.

[0119] S03. Based on the spatiotemporal trajectory information and vector feature information, obtain the spatiotemporal clustering group to be clustered.

[0120] In one embodiment, such as Figure 4 As shown, the step of obtaining the spatiotemporal clustering group based on the spatiotemporal trajectory information and vector feature information includes steps S031-S033:

[0121] S031, filter the combination of any two targets to be clustered according to a preset time interval to obtain the first spatiotemporal clustering group.

[0122] In this embodiment, targets that need to be grouped into pairs are first preliminarily screened through a preset time interval. The spatiotemporal trajectory information contains the trajectory duration of the targets to be clustered; the shorter the time interval between two trajectories, the more likely they are to be the same person.

[0123] In one specific embodiment, such as Figure 5 As shown, the step of filtering any combination of two targets to be clustered according to a preset time interval threshold to obtain the first spatiotemporal clustering group includes steps S0311-S0313:

[0124] S0311, determine whether the time interval between the trajectories of the two targets to be clustered is less than a preset time interval threshold;

[0125] S0312, if it is less than, then retain the two targets to be clustered to obtain the first spatiotemporal clustering group;

[0126] S0313, if it is not less than, then remove the combination of the two clustering targets.

[0127] Specifically, for the track set T = {t1, t2, ..., t...} n Let t be an example of this. i The start timestamp is beg i The end timestamp is end. i Choose any t i and t j Record the time interval itv ij =max(beg i beg j )-min(end i end j Iterate through set T and retrieve sets that match itv. ij <tw's set of track pairs TP = {(t i ,t j )|itv ij <tw,t i ∈T,t j ∈T,i≠j}, where tw is a preset time interval threshold, i.e., a time window given by the user, used to limit the time range.

[0128] S032, if the trajectories of two targets to be clustered in the first spatiotemporal clustering group overlap in time, then a second spatiotemporal clustering group is obtained by filtering according to a preset distance threshold.

[0129] In this embodiment, for two trajectory time intervals less than a preset time interval threshold, there are two scenarios: either the two trajectory time intervals overlap, or they do not overlap. For two targets to be clustered that have overlapping trajectory time intervals, the first spatiotemporal clustering group is further filtered based on a preset distance threshold.

[0130] In one specific embodiment, such as Figure 6 As shown, if the trajectories of two targets to be clustered in the first spatiotemporal clustering group overlap in time, then a second spatiotemporal clustering group is obtained by filtering according to a preset distance threshold, including steps S0321-S0323:

[0131] S0321, determine whether the average distance between the trajectories of the overlapping time portion of the two targets to be clustered is greater than a preset distance threshold;

[0132] S0322, if it is greater than, then remove the first spatiotemporal group to be clustered;

[0133] S0323, if it is not greater than, then retain the two targets to be clustered to obtain the second spatiotemporal clustering group.

[0134] Specifically, iterate through the set TP, for a given pair of two clustering targets, i.e., track pairs (t... i ,t j ), if itv ij If the distance is less than 0, then the time ranges of the track pair overlap, and the average distance of the overlapping portion is calculated. Here, the Euclidean distance is used to calculate the average distance, i.e., the formula is used. Among them I ij For t i and t j The set of indices (serial numbers corresponding to each data point) formed by the overlapping parts of dense data, if Then delete (t) from TP i ,t j ), where d error The preset distance threshold is the error distance given by the user.

[0135] S033, if the trajectory times of two targets to be clustered in the first spatiotemporal clustering group do not overlap, then a second spatiotemporal clustering group is obtained by filtering according to a preset speed threshold.

[0136] In this embodiment, the trajectories of the two targets to be clustered do not overlap, which means that there is a time interval between the two trajectories. That is, it takes time for the target to be clustered to transform from one time trajectory to another time trajectory. At the same time, there is also a limiting factor, which is the moving speed.

[0137] In one specific embodiment, such as Figure 7As shown, if the trajectories of two targets to be clustered in the first spatiotemporal clustering group do not overlap in time, then a second spatiotemporal clustering group is obtained by filtering according to a preset velocity threshold, including steps S0331-S0333:

[0138] S0331, Based on the time sequence, determine whether the speed required for one target to be clustered in the first spatiotemporal clustering group to change from the current trajectory to the trajectory of another target to be clustered is greater than a preset speed threshold.

[0139] S0332, if it is greater than, then remove the first spatiotemporal group to be clustered;

[0140] If S0333 is not greater than, then the two targets to be clustered are retained to obtain the second spatiotemporal clustering group.

[0141] Specifically, if the time ranges of the two targets to be clustered (i.e., the track pair) do not overlap, the distance between the ends of the track pair is calculated, that is, the ending position of the earlier occurring track and the starting position of the later occurring track, according to the chronological order, using the formula... Calculation, where it is assumed that beg i <beg j And -1 represents t i The last index of dense data, if Then delete (t) from TP i ,t j ), where v is the preset speed threshold, i.e., the normal speed given by the user. The specific setting of the speed threshold can be flexibly adjusted according to the scenario. For example, when shopping in a mall, a normal walking speed is usually used; for scenarios with transportation such as subway stations, the preset speed threshold needs to be set with reference to the speed of the transportation.

[0142] In one embodiment, such as Figure 4 As shown, the method for obtaining spatiotemporal clustering groups based on the spatiotemporal trajectory information and vector feature information further includes steps S034-S036:

[0143] S034, Based on the vector feature information, determine whether two clustering targets in the second spatiotemporal clustering group are neighboring feature vectors;

[0144] S035, if not, then remove the second spatiotemporal clustering group;

[0145] S036, if so, then retain the two targets to be clustered to obtain the spatiotemporal clustering group.

[0146] In the embodiment, after screening any two target combinations to be clustered by the spatio-temporal trajectory information, the vector feature information is screened according to the vector feature information. At this time, the two target combinations to be clustered are limited by both the spatio-temporal trajectory data and the vector feature information.

[0147] Specifically, if the two target combinations to be clustered, i.e., the track pair, are not deleted by the above step, the K-neighbor matrix is searched to determine whether the track pair is the K-neighbor. If not, the track pair is deleted in the TP (t i ,t j ). Finally, the spatio-temporal target combinations to be clustered obtained by screening the spatio-temporal trajectory information and the vector feature information, in which the connection relationship between the two target combinations to be clustered is the spatio-temporal edge, if the spatio-temporal edge of the connection (t i ,t j ) is passed, t i and t j belong to the same target.

[0148] S04, obtaining a feature target combination to be clustered according to the vector feature information.

[0149] In one embodiment, the step of obtaining the feature target combination to be clustered according to the vector feature information includes the step S041:

[0150] S041, obtaining the feature target combination to be clustered by the Prim algorithm according to the human feature vector and the corresponding neighbor feature vector set.

[0151] In the embodiment, in addition to the spatio-temporal target combination to be clustered, the feature target combination to be clustered is obtained according to the vector feature information, which is mainly used to process the case that the target is in the blind area for a long time and thus the spatio-temporal edge cannot be generated. The maximum spanning tree can be obtained by the kruskal algorithm or the Prim algorithm. The Prim algorithm is used herein. The Prim algorithm is an algorithm for finding the optimal solution in a large space with the least search. The method of sorting by the weight of the edge added adopts the greedy idea. Each step selects the edge with the largest weight and adds it to the maximum spanning tree. Finally, the selected edge constitutes a maximum spanning tree.

[0152] Specifically, the K-neighbor matrix S n×k, the graph G=(T, S) is constructed, where the point set is the set of tracks, and the edge set is the set of K-neighbor matrices, the feature clustering group is obtained by the Prime algorithm, and the connection relationship between two clustering targets in the feature clustering group is established by the human feature vector to become the maximum spanning tree, i.e. the maximum spanning tree EP of the graph G. The judgment of whether the two clustering targets in the feature clustering group can be clustered is the judgment of whether the edge set provided by the maximum spanning tree EP can be established. The feature clustering group obtained by the human feature vector is called the feature edge provided by the maximum spanning tree EP. If the feature edge passes the judgment, the nodes (two clustering targets in the feature clustering group) at both ends of the feature edge belong to the same target.

[0153] In step S20, the similarity between the two clustering targets in the clustering group is obtained according to the vector feature information and the space-time trajectory information.

[0154] In this embodiment, the clustering group includes a space-time clustering group and a feature clustering group. That is, the edges that need to be judged for similarity calculation include space-time edges and feature edges, that is, the edge set obtained in the above steps is E=TP∪EP, the edges are judged, and the tracks are merged (connected) if the judgment passes.

[0155] In one embodiment, as shown in Figure 8 , the similarity between the two clustering targets in the clustering group is obtained according to the vector feature information and the space-time trajectory information, including steps S21-S24:

[0156] In step S21, the face feature similarity between the two clustering targets is obtained according to the face feature information.

[0157] Further, the face feature similarity between the two clustering targets is obtained according to the average face feature information of the clusters to which the two clustering targets belong respectively.

[0158] Specifically, the face similarity calculation method is: wherein is the face average feature of the cluster (connected graph) to which t i belongs, is the face average feature of the cluster to which t j belongs. For an unknown or initial t i , the cluster thereof only has itself, and in the process of continuously connecting nodes to form edges, the cluster gradually expands, and therefore the average feature is updated after each successful node merging, and the calculation formula is wherein C is the set of tracks constituting the cluster to which t i belongs, is the updated average feature.

[0159] S22, Based on the human body feature information, obtain the human body feature similarity between the two targets to be clustered.

[0160] Furthermore, based on the human feature information of the respective clusters to which the two targets to be clustered belong, the facial feature similarity of the two targets to be clustered is obtained.

[0161] Specifically, similar to the method for calculating facial similarity, the formula used is: in, For t i The average human characteristics of the cluster For t j The average human body features of the cluster. The average human body features are updated after each successful node merge. The update formula is the same as the face update formula, and will not be repeated here.

[0162] S23, Based on the spatiotemporal trajectory information, obtain the spatiotemporal trajectory similarity of the two targets to be clustered.

[0163] In one specific embodiment, such as Figure 9 As shown, step S23 includes steps S231-S236.

[0164] S231, Select one spatiotemporal trajectory information from each of the two clusters to which the two targets to be clustered belong and combine them to obtain the first information pair;

[0165] S232, traverse the clusters to which the two clustering targets belong respectively to obtain a first set composed of multiple first information pairs;

[0166] S233, traverse the first set, and filter out the first information pairs whose trajectory time interval is less than a preset time interval threshold according to the spatiotemporal trajectory information to obtain the second set;

[0167] S234, Based on whether the two trajectories of the first information pair in the second set overlap in time, the corresponding initial similarity is obtained;

[0168] S235, Traverse the second set to obtain multiple initial similarities for multiple first information pairs in the second set;

[0169] S236, the average similarity of the second set is obtained based on the multiple initial similarities, wherein the average similarity is the spatiotemporal trajectory similarity of the two targets to be clustered.

[0170] In this embodiment, in order to determine the two clustering targets t i and t jWhether to belong to the same target, can only be calculated two clustering target similarity, should also increase the clustering target reference of the cluster to which it belongs. In two clustering target judgment is also related to its cluster, for example, in the above face feature similarity and body feature similarity calculation, with the average feature of the cluster to which it belongs as the basis. Similarly in the space-time trajectory similarity calculation, also join the other data in the cluster as a reference.

[0171] Further, respectively from two clustering target cluster each optional and combination, form the first set. With the above steps in time interval threshold screening similar, preliminary screening of the combination to get the second set. According to the second set calculated from each combination of similarity, namely the initial similarity, according to the initial similarity, the average similarity is calculated as the space-time trajectory similarity of two clustering target.

[0172] Specifically, let t i The track of the cluster constitutes a set T i , t j The track of the cluster constitutes a set T j , T i T j The track pair set is formed, from which the set TP ij = {(t m ,t n )|itv mn <tw,t m ∈T i ,t n ∈T j} is obtained.

[0173] In one embodiment, the initial similarity corresponding to the first information pair in the second set is obtained according to whether the two trajectories overlap in time, comprising steps S2341-S2343:

[0174] S2341, determine whether the two trajectories of the first information pair in the second set overlap in time;

[0175] S2342, if overlap, the initial similarity calculation formula is

[0176]

[0177] Wherein, Indicates the average distance of the two trajectories overlap;

[0178] S2343, if not overlap, the initial similarity calculation formula is

[0179]

[0180] Wherein, denotes the average distance between two trajectories, d error is a preset distance threshold, v denotes a preset speed threshold, itv ij is the time interval of the time of the trajectories of two clustering targets.

[0181] Specifically, the set TP ij is traversed to determine whether the time trajectories of the information pairs in the set overlap. If the time trajectories of the track pairs overlap, the track pairs are recorded as If the time trajectories of the track pairs do not overlap, the track pairs are recorded as The average space-time score is denoted as

[0182] S24, according to the face feature similarity, the body feature similarity and the space-time trajectory similarity, the similarity of two clustering targets in the clustering group is obtained.

[0183] In this embodiment, for any given (t i ,t j ), the face similarity, the body similarity and the space-time similarity are calculated respectively, and the similarity of two clustering targets in the clustering group is calculated according to the face similarity, the body similarity and the space-time similarity. That is, the total score of the two clustering target edges is s = w f sf+w b sb+w st st, wherein w f , w b , w st are the face weight, the body weight and the space-time weight respectively. The selection and setting of the weights can be flexibly adjusted according to different scenes and specific conditions of the images.

[0184] S30, according to the similarity of the two clustering targets, a clustering result is obtained.

[0185] In one embodiment, as shown in Figure 10 , step S30 includes steps S31-S33

[0186] S31, it is determined whether the similarity is greater than a preset similarity threshold;

[0187] S32, if greater, the two clustering targets are the same target, and the labels of the two clustering targets are updated;

[0188] S33, if not greater, the two clustering targets are not the same target.

[0189] Specifically, s is compared with a given similarity threshold, if greater than the given threshold, it is determined to pass, and the edge is connected. The nodes belonging to the same connected graph are the same class. After the edge is connected, the average face feature and the average body feature are updated.

[0190] The graph obtained after the above is traversed to obtain all connected subgraphs, and all tracks belonging to the same connected subgraph are the same target. The face and body snapshots of all tracks in the connected subgraph have the same label, and the label result is output.

[0191] In the technical scheme of the present application, by adding the space-time track as the basis for judgment in the clustering judgment, the low-quality image internal space-time track information is utilized to enable it to participate in clustering, thereby increasing the utilization rate of the obtained images and improving the clustering effect. The judgment is made according to the similarity. At the same time, the original single image comparison is changed to track form, i.e. data group form, to reduce the calculation amount of image-to-image comparison, and the clustering can be completed more efficiently and quickly.

[0192] Further, in order to solve the problem of blind area not meeting the space-time locality, the feature to be clustered group is generated according to the vector feature information, and the data blank in the blind area is supplemented. According to the face, body and space-time three dimensions, the correctness of clustering is improved, and the multi-dimensional consideration ensures that the clusters not meeting the space-time locality will not be clustered together in the subsequent connection process, thereby ensuring the improvement of the clustering effect.

[0193] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, the different steps do not have to be executed in such an order, they can be executed simultaneously (in parallel) or in other order, and these changes are within the protection scope of the present application.

[0194] Further, the present application also provides a clustering system.

[0195] Referring to the accompanying drawings Figure 11 , Figure 11is the main structural block diagram of the clustering system according to an embodiment of the present application. Mainly includes preprocessing module 111, feature to be clustered group generation module 112, space-time to be clustered group generation module 113, clustering prediction module 114 and result output module 115. The preprocessing module 111 converts the human face body data into the vector feature information and the space-time trajectory information; the feature to be clustered group generation module 112 obtains the feature to be clustered group according to the vector feature information; the space-time to be clustered group generation module 113 obtains the space-time to be clustered group according to the space-time trajectory information and the vector feature information; the clustering prediction module 114 calculates the similarity of two to-be-clustered targets in the feature to be clustered group and the space-time to be clustered group; the result output module 115 obtains the clustering result according to the similarity and outputs. In some embodiments, one or more of the preprocessing module 111, the feature to be clustered group generation module 112, the space-time to be clustered group generation module 113, the clustering prediction module 114 and the result output module 115 can be combined together to become a module. The description of the specific implementation function can be referred to the description of the above steps.

[0196] The clustering system described above is used for the above-mentioned clustering method embodiment, and the technical principles, technical problems solved and technical effects generated are similar. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the clustering system can be referred to the description of the above-mentioned method embodiments, which will not be described here.

[0197] Those skilled in the art can understand that the present application can realize all or part of the processes in the above-mentioned embodiment method, and can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable storage medium can include any entity or device, medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0198] Further, the present application also provides a control device. In an embodiment of the control device according to the present application, the control device comprises a processor and a storage device, the storage device can be configured to store a program of the clustering method of the above-mentioned method embodiments, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program of the method of the above-mentioned method embodiments. For the convenience of illustration, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The control device can be a control device device formed by various electronic devices.

[0199] Further, the present application also provides a computer readable storage medium. In an embodiment of the computer readable storage medium according to the present application, the computer readable storage medium can be configured to store a program of the clustering method of the above-mentioned method embodiments, which can be loaded and run by a processor to realize the above-mentioned method. For the convenience of illustration, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The computer readable storage medium can be a storage device device formed by various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.

[0200] Further, it should be understood that, since the setting of each module is only for illustrating the functional units of the device of the present application, the corresponding physical device of the module can be the processor itself, or a part of software, hardware or the combination of software and hardware in the processor. Therefore, the number of each module in the figure is only illustrative.

[0201] Those skilled in the art can understand that each module in the device can be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principles of the present application, and therefore, the technical solution after splitting or combining will fall within the protection scope of the present application.

[0202] So far, the technical solution of the present application has been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without deviating from the principles of the present application, and the technical solution after the changes or replacements will fall within the protection scope of the present application.

Claims

1. A clustering method characterized by, The method comprises: obtaining vector feature information and space-time trajectory information of a target to be clustered; obtaining similarity of two targets to be clustered in a group to be clustered according to the vector feature information and the space-time trajectory information; obtaining a clustering result according to the similarity of the two targets to be clustered; the vector feature information comprises face feature information and body feature information, and the space-time trajectory information comprises multiple time information in a trajectory of the target to be clustered and space position information corresponding to the time information; the obtaining of the similarity of the two targets to be clustered in the group to be clustered according to the vector feature information and the space-time trajectory information comprises: obtaining face feature similarity of the two targets to be clustered according to the face feature information; obtaining body feature similarity of the two targets to be clustered according to the body feature information; obtaining space-time trajectory similarity of the two targets to be clustered according to the space-time trajectory information; and obtaining the similarity of the two targets to be clustered in the group to be clustered according to the face feature similarity, the body feature similarity and the space-time trajectory similarity; the obtaining of the space-time trajectory similarity of the two targets to be clustered according to the space-time trajectory information comprises: combining any one of space-time trajectory information in clusters to which the two targets to be clustered belong respectively to obtain a first information pair; traversing the clusters to which the two targets to be clustered belong respectively to obtain a first set composed of multiple first information pairs; traversing the first set to filter out a second set of first information pairs with a trajectory time interval less than a preset time interval threshold according to the space-time trajectory information; obtaining an initial similarity corresponding to the first information pair in the second set according to whether two trajectory times of the first information pair overlap; traversing the second set to obtain multiple initial similarities of multiple first information pairs in the second set; and obtaining an average similarity of the second set according to the multiple initial similarities, wherein the average similarity is the space-time trajectory similarity of the two targets to be clustered.

2. The clustering method of claim 1, wherein, The method further comprises: obtaining face and body data of multiple targets to be clustered by an image acquisition device; obtaining vector feature information and space-time trajectory information of a target to be clustered according to multiple face and body data of the same target to be clustered; obtaining a space-time group to be clustered according to the space-time trajectory information and the vector feature information; obtaining a feature group to be clustered according to the vector feature information.

3. The clustering method of claim 1, wherein, The method further comprises: obtaining a neighbor feature vector set of a body feature vector of the target to be clustered by a feature retrieval algorithm, wherein a neighbor feature vector in the neighbor feature vector set is a body feature vector of another target to be clustered; traversing the body feature vector to obtain a neighbor feature vector set corresponding to the body feature vector of each target to be clustered.

4. The clustering method of claim 2, wherein, The obtaining of the space-time group to be clustered according to the space-time trajectory information and the vector feature information comprises: filtering a combination of any two targets to be clustered according to a preset time interval to obtain a first space-time group to be clustered; if trajectory times of two targets to be clustered in the first space-time group to be clustered overlap, filtering a second space-time group to be clustered according to a preset distance threshold; If the track time of the two clustering targets in the first spatio-temporal clustering group does not overlap, a second spatio-temporal clustering group is obtained by screening according to a preset speed threshold.

5. The clustering method of claim 4, wherein, The obtaining of the spatio-temporal clustering group according to the spatio-temporal track information and the vector feature information further includes: According to the vector feature information, it is determined whether the two clustering targets in the second spatio-temporal clustering group are mutual neighbor feature vectors; If not, the second spatio-temporal clustering group is removed; If yes, the two clustering targets are retained to obtain a spatio-temporal clustering group.

6. The clustering method of claim 3, wherein, The obtaining of the feature clustering group according to the vector feature information includes: According to the human body feature vector and the corresponding neighbor feature vector set, the feature clustering group is obtained by using a Prim algorithm.

7. The clustering method of claim 4, wherein, The first spatio-temporal clustering group is obtained by screening a combination of any two clustering targets according to a preset time interval threshold, which includes: It is determined whether the track time interval of the two clustering targets is less than a preset time interval threshold; If yes, the two clustering targets are retained to obtain the first spatio-temporal clustering group; If not, the combination of the two clustering targets is removed.

8. The clustering method of claim 4, wherein, If the track time of the two clustering targets in the first spatio-temporal clustering group does not overlap, a second spatio-temporal clustering group is obtained by screening according to a preset speed threshold, which includes: It is determined whether the track average distance of the time overlapping part of the two clustering targets is greater than a preset distance threshold; If yes, the first spatio-temporal clustering group is removed; If not, the two clustering targets are retained to obtain a second spatio-temporal clustering group.

9. The clustering method of claim 4, wherein, If the track time of the two clustering targets in the first spatio-temporal clustering group does not overlap, a second spatio-temporal clustering group is obtained by screening according to a preset speed threshold, which includes: According to the time occurrence order, it is determined whether the speed required for one clustering target to be converted from a current track to another track of the other clustering target is greater than a preset speed threshold; If yes, the first spatio-temporal clustering group is removed; If not, the two clustering targets are retained to obtain a second spatio-temporal clustering group.

10. The clustering method of claim 1, wherein, The face feature similarity of the two clustering targets is obtained according to the face feature information, which includes: According to the average face feature information of the clusters to which the two clustering targets belong respectively, the face feature similarity of the two clustering targets is obtained.

11. The clustering method of claim 1, wherein, The body feature similarity of the two clustering targets is obtained according to the body feature information of the clusters to which the two clustering targets belong respectively. According to whether the track time of the two track time information pairs in the second set overlaps, the corresponding initial similarity is obtained, which includes:

12. The clustering method of claim 1, wherein, It is determined whether the track time of the two track time information pairs in the second set overlaps; If yes, the initial similarity calculation formula is If not, the initial similarity calculation formula is wherein, represents the average distance of the two trajectories overlap; The clustering result is obtained according to the similarity of the two clustering targets, which includes: wherein, denotes the average distance between two trajectories, is a preset distance threshold, v denotes a preset speed threshold, is the time interval of the trajectory time of two targets to be clustered.

13. The clustering method of claim 1, wherein, It is determined whether the similarity is greater than a preset similarity threshold; ​ If greater, the two clustering targets are the same target, and the two clustering target labels are updated. If not greater, the two clustering targets are not the same target.

14. A clustering system, characterized by The system is used to implement the clustering method in any one of claims 1-13, and the system comprises: a preprocessing module, which converts the face and human body data into vector feature information and space-time trajectory information; a feature clustering group generation module, which obtains a feature clustering group according to the vector feature information; a space-time clustering group generation module, which obtains a space-time clustering group according to the space-time trajectory information and the vector feature information; a clustering prediction module, which calculates the similarity of two clustering targets in the feature clustering group and the space-time clustering group; a result output module, which obtains a clustering result according to the similarity and outputs the clustering result.

15. A control device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the clustering method in any one of claims 1-13.

16. A computer readable storage medium having stored therein a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the clustering method in any one of claims 1-13.

Images