A clustering method, device, electronic device, computer readable storage medium

Abstract

The application provides a clustering method and device, electronic equipment and computer readable storage medium, wherein the clustering method comprises: obtaining at least two first clusters and space-time domain information corresponding to each first cluster, wherein the space-time domain information represents a space-time position relationship of an object corresponding to the first cluster; determining whether there is a space-time association between at least two first clusters based on the space-time domain information corresponding to each first cluster; and in response to the existence of the space-time association between at least two first clusters, clustering at least two first clusters based on the similarity between at least two first clusters. The problem of one person with multiple files is solved, and the accuracy of face clustering is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a clustering method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the large popularity of intelligent video monitoring devices, a large amount of portrait images will be accumulated every day. Using a portrait clustering method to realize archiving portrait images by person is a common means. Directly clustering portraits from a large amount of portrait snapshot data often has poor effect. For example, the phenomenon of one person having multiple archives (i.e., the same person has multiple clusters) after clustering may occur, and the clustering effect is poor. SUMMARY

[0003] The present application provides a clustering method, device, electronic equipment and computer readable storage medium, which can improve the accuracy of clustering.

[0004] To solve the above technical problems, the first technical solution provided by the present application is to provide a clustering method, comprising: obtaining at least two first clusters and time-space domain information corresponding to each first cluster, the time-space domain information representing the time-space position relationship of the object corresponding to the first cluster; determining whether there is a time-space association between at least two first clusters based on the time-space domain information corresponding to each first cluster; and in response to the existence of a time-space association between at least two first clusters, clustering at least two first clusters based on the similarity between at least two first clusters.

[0005] The step of determining whether there is a time-space association between at least two first clusters based on the time-space domain information corresponding to each first cluster comprises: determining whether the time-space domain information corresponding to at least two first clusters has the same time-space point; and in response to having the same time-space point, determining that there is a time-space association between at least two first clusters.

[0006] The step of clustering at least two first clusters based on the similarity between at least two first clusters comprises: determining a time-space correlation coefficient between at least two first clusters; determining a similarity threshold based on the time-space correlation coefficient; and in response to the similarity between at least two first clusters being greater than the similarity threshold, clustering at least two first clusters.

[0007] The step of determining a time-space correlation coefficient between at least two first clusters comprises: calculating the intersection of the time-space domain information corresponding to at least two first clusters; and calculating the union of the time-space domain information corresponding to at least two first clusters; and determining the time-space correlation coefficient based on the intersection and the union.

[0008] The step of determining the similarity threshold based on the spatio-temporal correlation coefficient comprises: determining the similarity threshold based on the preset highest similarity threshold, the preset lowest similarity threshold and the spatio-temporal correlation coefficient.

[0009] Before the step of clustering the at least two first-type clusters in response to the similarity between the at least two first-type clusters being greater than the similarity threshold, the method further comprises: calculating the average centroid of each of the at least two first-type clusters; and calculating the similarity between the at least two first-type clusters based on the average centroid of each of the at least two first-type clusters.

[0010] In response to the number of images in at least one of the at least two first-type clusters being less than a preset number, the step of determining the similarity threshold based on the spatio-temporal correlation coefficient comprises: determining a first similarity threshold based on the spatio-temporal correlation coefficient; and in response to the similarity between the at least two first-type clusters being greater than the first similarity threshold, the step of clustering the at least two first-type clusters comprises: clustering the at least two first-type clusters; in response to the number of images in at least one of the at least two first-type clusters being greater than the preset number, the step of determining the similarity threshold based on the spatio-temporal correlation coefficient comprises: determining a second similarity threshold based on the spatio-temporal correlation coefficient; and in response to the similarity between the at least two first-type clusters being greater than the second similarity threshold, the step of clustering the at least two first-type clusters comprises: clustering the at least two first-type clusters; and the first similarity threshold is greater than the second similarity threshold.

[0011] Before the step of determining whether there is a spatio-temporal correlation between the at least two first-type clusters based on the spatio-temporal domain information corresponding to each of the first-type clusters, the method further comprises: determining whether there is the same time information and / or spatial information in the spatio-temporal domain information corresponding to each of the first-type clusters; in response to there being the same time information and / or spatial information, determining an abnormal image based on the same time information and / or spatial information; and removing the abnormal image from the first-type cluster.

[0012] To address the aforementioned technical problems, the second technical solution provided in this application is: a clustering device comprising: an acquisition module, configured to acquire at least two first clusters and spatiotemporal information corresponding to each first cluster, wherein the spatiotemporal information characterizes the spatiotemporal positional relationship of objects corresponding to the first clusters; a determination module, configured to determine whether there is a spatiotemporal association between at least two first clusters based on the spatiotemporal information corresponding to each first cluster; and a clustering module, configured to cluster at least two first clusters based on the similarity between at least two first clusters in response to the existence of a spatiotemporal association between them.

[0013] To solve the above-mentioned technical problems, the third technical solution provided in this application is: to provide an electronic device, including: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute any of the above methods.

[0014] To solve the above-mentioned technical problems, the fourth technical solution provided in this application is: to provide a computer-readable storage medium storing a program file, the program file being executable to implement the clustering method described in any of the above-mentioned claims.

[0015] The beneficial effect of this application, unlike the prior art, is that it obtains at least two first-category clusters and the spatiotemporal domain information corresponding to each first-category cluster, and determines whether there is a spatiotemporal correlation between the at least two first-category clusters by combining the spatiotemporal domain information corresponding to each first-category cluster. If a spatiotemporal correlation exists, the at least two first-category clusters are clustered based on the similarity between them. By using the spatiotemporal domain information of each cluster, it determines whether different clusters have a spatiotemporal correlation, thereby determining whether at least two clusters can be clustered, solving the problem of multiple files for one person, and improving the accuracy of clustering. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the clustering method of this application;

[0018] Figure 2 for Figure 1 A flowchart illustrating an embodiment of step S2;

[0019] Figure 3A diagram for illustrating whether the spatiotemporal information corresponding to at least two first clusters has the same time-space point relationship by using an inverted index;

[0020] Figure 4 For Figure 1 A flow diagram of an embodiment of step S3 in the method;

[0021] Figure 5 A flow diagram of an embodiment of the clustering method of the present application;

[0022] Figure 6 A structural diagram of an embodiment of the clustering device of the present application;

[0023] Figure 7 For Figure 1 A flow diagram of an embodiment of step S1 in the method for obtaining the spatiotemporal information corresponding to the first clusters;

[0024] Figure 8 For Figure 7 A flow diagram of an embodiment of step S12 in the method;

[0025] Figure 9 For Figure 8 A flow diagram of an embodiment of step S121 in the method;

[0026] Figure 10 A spatiotemporal information encoding diagram in an embodiment of the present application;

[0027] Figure 11 For Figure 8 A flow diagram of an embodiment of step S122 in the method;

[0028] Figure 12 A structural diagram of an embodiment of the spatiotemporal information generation device of the clusters of the present application;

[0029] Figure 13 A structural diagram of an embodiment of the electronic device of the present application;

[0030] Figure 14 A structural diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0031] The prior art has a one-to-one profile face clustering method and system based on a dynamic algorithm. The design points are to acquire face photos in real time, record the snapshot time and corresponding location of the face photos, extract long features and short features from the snapshot face photos, wherein the long features are features with more key positions relative to the short features, and the short features are features with fewer key positions relative to the long features, compare and cluster the long features and short features of the acquired face photos, obtain one-to-one profile data for analysis, and acquire the activity track and activity frequency of the face. The one-to-one profile data is analyzed to obtain the activity track of the face and the activity frequency of different time periods in different locations. However, the face clustering method proposed by the method is essentially only using key information points of an image to construct long and short features based on weighting, and using a dynamic algorithm to optimize face clustering. Finally, the results after using face clustering are combined with the information of snapshot time and snapshot location for track display. The face clustering method does not fully utilize time and space information.

[0032] The prior art also has a face clustering method and device. Although the design considers feature similarity matching, space-time distance constraints, and assigns different weights according to the order of image snapshot time to attempt to optimize face clustering. However, the setting of the weight is only constrained by the time dimension, and cannot fully consider the influence of time and space information on face clustering.

[0033] The present application provides a clustering method that can fully consider the influence of time and space on clustering at the same time, and improve the accuracy of clustering. The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0034] Please refer to Figure 1 , Figure 1 The flowchart of the first embodiment of the clustering method of the present application is shown in the figure, which specifically includes:

[0035] Step S1: Acquire at least two first clusters and the corresponding space-time domain information of each first cluster, and the space-time domain information represents the space-time position relationship of the object corresponding to the first cluster.

[0036] This embodiment takes face as an example for illustration. The clustering algorithm is used to cluster the images in the database according to the face features to obtain at least two first clusters. The clustering algorithm includes one or any combination of k-means and DBSCAN.

[0037] Further, the application further obtains the spatio-temporal domain information corresponding to the first type of cluster after obtaining the first type of cluster, and the spatio-temporal domain information represents the spatio-temporal position relationship of the object corresponding to the first type of cluster. For example, the first type of cluster is the cluster of object A, and then the spatio-temporal domain information represents the time and place relationship of the travel of object A. One spatio-temporal domain information is composed of multiple fixed time and space places, and has the order of time and space. Specifically, there is a time, space geographical position corresponding to each image when the image is taken, and the spatio-temporal domain information of the image can also be obtained at the same time when the face feature information of the image is obtained through the clustering algorithm.

[0038] Specifically, please refer to Figure 7 , Figure 7 for Figure 1 an embodiment of the step S1 of obtaining the spatio-temporal domain information corresponding to the first type of cluster, and the embodiment specifically includes the following steps.

[0039] Step S11: obtaining the first type of cluster.

[0040] Specifically, the feature sets corresponding to the images in the database are clustered based on the similarity of the images in the database, so as to obtain multiple first type of clusters. It can be understood that the clustering algorithm includes but is not limited to k-means, DBSCAN.

[0041] In an embodiment, all portrait clustering information in a specified time period is obtained. In practice, the selection of the time period mainly depends on the length of stable operation of the portrait system, and generally all portrait clustering information in the recent three months can be selected. The portrait clustering information includes the first type of cluster ID and all portrait information contained in the first type of cluster. The portrait information mainly includes: snapshot time, snapshot place, picture type, picture ID and the like. The picture type can be a face picture or a body picture, that is, the cluster can contain only face pictures or body pictures, or can contain both face and body pictures.

[0042] Specifically, a feature extraction algorithm such as a convolutional neural network algorithm is used to extract features from the images in the database, and then a feature set is obtained. The feature set is divided into K groups, then K objects are randomly selected as initial cluster centers, then the distance between each object and each cluster center, that is, the similarity, is calculated, the similarity is compared with the similarity threshold, and each object with a similarity greater than the similarity threshold is assigned to the corresponding cluster center, and then multiple first type of clusters are obtained.

[0043] Step S12: obtaining the spatio-temporal association information corresponding to the first type of cluster based on the snapshot time and snapshot place of the images in the first type of cluster.

[0044] In an embodiment, spatio-temporal correlation mining is performed based on the snapshot time and the snapshot location of the images in the first type of cluster to obtain spatio-temporal correlation information corresponding to the first type of cluster.

[0045] In order to fully utilize the spatio-temporal correlation mining, the snapshot time and the snapshot location of each type of cluster need to be quantitatively calibrated. In an embodiment, a typical quantitative calibration method of the time dimension, i.e., the snapshot time, can be: dividing the continuous snapshot time into 24 different time periods according to hours, i.e., quantifying the snapshot time into 24 different time periods according to hours. The snapshot location can be an independent latitude and longitude information or a unique spatial domain ID information formed by multiple front-end snapshot devices.

[0046] Please refer to Figure 8 , Figure 8 for Figure 7 an embodiment of step S12.

[0047] Step S121: sorting the snapshot time and the snapshot location corresponding to the images according to the time sequence to generate sequence information.

[0048] Specifically, all the images in the first type of cluster obtained are sorted according to the snapshot time sequence. The sorted images have a certain continuity in time and can form a certain order in the snapshot location, thereby forming a time and space sequence information of a real and natural person trip.

[0049] In an embodiment, in order to efficiently utilize the existing correlation, when sorting the snapshot time and the snapshot location corresponding to the images according to the time sequence, spatio-temporal information coding can be performed on the snapshot time and the snapshot location corresponding to the images to obtain identifiers representing the spatio-temporal relationship of the images, and the identifiers corresponding to the images are sorted according to the time sequence to generate sequence information.

[0050] Specifically, please refer to Figures 9-10 , Figure 9 for Figure 8 an embodiment of step S121. Figure 10 is a spatio-temporal information coding schematic diagram in an embodiment of the present application.

[0051] As Figure 9 shown, step S121 includes:

[0052] Step S1211: performing spatio-temporal information coding on the snapshot time and the snapshot location corresponding to the images to obtain identifiers representing the spatio-temporal relationship of the images.

[0053] Specifically, the capture time and location of each image in the first cluster are combined and encoded. For example, the encoding method is: [Time(X), Area(Y)] => U(Z). Here, Time(X) represents the capture time converted to hours, Area(Y) represents the capture location, and U(Z) represents the unique identifier for this encoding method. Using this method, a spatiotemporal information encoding dictionary is constructed, where the key is [Time(X), Area(Y)] and the value is U(Z), facilitating spatiotemporal information conversion and efficiently utilizing existing association methods.

[0054] In one embodiment, such as Figure 10 As shown, assume there are three first-class clusters, represented by circles, triangles, and squares respectively. The first first-class cluster contains 5 images, encoded using spatiotemporal information as [Time(7), Area(A)], [Time(8), Area(A)], [Time(9), Area(B)], [Time(16), Area(B)], and [Time(17), Area(A)]. The corresponding 5 different identifiers are U(1), U(2), U(3), U(4), and U(5). Similarly, the second first-class cluster contains 5 images, encoded using spatiotemporal information as [Time(8), Area(B)], [Time(9), Area(C)], [Time(10), Area(D)], [Time(18), Area(D)], and [Time(17), Area(A)]. The five corresponding different identifiers are U(6), U(7), U(8), U(9), and U(10). The third first-class cluster contains six images, which are represented by spatiotemporal information encoding as [Time(8), Area(E)], [Time(8), Area(F)], [Time(9), Area(G)], [Time(17), Area(G)], [Time(17), Area(F)], and [Time(18), Area(E)]. The six corresponding different identifiers are U(11), U(12), U(13), U(14), U(15), and U(16).

[0055] Step S1212: Sort the identifiers corresponding to the images in chronological order to generate sequence information.

[0056] Specifically, sequence information is a unique identifier derived from dividing clustering information into daily segments based on each primary cluster and encoding them chronologically. Therefore, after encoding the time and spatial information of the three primary clusters for a given day, the corresponding sequence information can be obtained as follows:

[0057] The sequence information of the data in the first first-class cluster is: U(1), U(2), U(3), U(4), U(5).

[0058] The sequence information of the data in the second first cluster is: U(6), U(7), U(8), U(9), U(10).

[0059] The sequence information of the data in the third first cluster is: U(11), U(12), U(13), U(14), U(15), U(16).

[0060] If information from multiple days is aggregated for each first-class cluster, then each first-class cluster will contain multiple useful sequence information.

[0061] Step S122: Use the association mining algorithm to mine spatiotemporal associations based on sequence information to obtain spatiotemporal association information.

[0062] Specifically, by setting minimum support and minimum confidence for each first-class cluster, and then mining their association relationships, unique identifier sequence combinations with strong spatiotemporal information encoding can be obtained, improving the overall system utilization. The association relationship mining algorithm used in step S122 can be Apriori, FP-Growth, or other association relationship mining methods.

[0063] Please see Figure 11 , Figure 11 for Figure 8 A flowchart illustrating an embodiment of step S122. The step of using an association mining algorithm to mine spatiotemporal associations based on sequence information to obtain spatiotemporal association information includes:

[0064] Step S1221: Use the association mining algorithm to mine spatiotemporal associations based on sequence information to obtain multiple strongly related identifiers.

[0065] Specifically, in one embodiment, such as Figure 10 As shown, assuming that through association mining, we can obtain: the first strongly correlated spatiotemporal encoded unique identifier sequence combination in the first type of cluster is U(2)->U(3) and U(4)->U(5). The second strongly correlated spatiotemporal information encoded unique identifier sequence combination in the first type of cluster is U(7)->U(8) and U(9)->U(10). The third strongly correlated spatiotemporal information encoded unique identifier sequence combination in the first type of cluster is U(11), U(12)->U(13) and U(14), U(15)->U(16).

[0066] Step S1222: decoding the multiple identifiers with strong correlation based on the mapping relationship of the spatiotemporal information coding to obtain spatiotemporal correlation information.

[0067] Specifically, the result directly obtained by the association relationship mining is not intuitive and is not conducive to the later use. Therefore, the inverse spatiotemporal information coding dictionary can be obtained through the mapping relationship of the spatiotemporal information coding dictionary, and the obtained result is spatiotemporally decoded. Further, the inverse spatiotemporal information coding dictionary is generated by using the mapping relationship of the spatiotemporal information coding dictionary, and the key value and the value of the dictionary are swapped.

[0068] In an embodiment, the spatiotemporal strong correlation rule obtained by the association relationship mining is decoded by the inverse spatiotemporal information coding dictionary as follows:

[0069] The first first cluster: [Time(8), Area(A)]—>[Time(9), Area(B)] and [Time(16), Area(B)]—>[Time(17), Area(A)]. That is, the real natural person represented by the first first cluster often appears at A point at 8 o'clock and moves to B point at 9 o'clock. In addition, the real natural person often appears at B point at 16 o'clock and moves to A point at 17 o'clock.

[0070] Similarly, the second first cluster: [Time(9), Area(C)]—>[Time(10), Area(D)] and [Time(18), Area(D)]—>[Time(19), Area(C)].

[0071] The third first cluster: [Time(8), Area(E)], [Time(8), Area(F)]—>[Time(9), Area(G)] and [Time(17), Area(G)], [Time(17), Area(F)]—>[Time(18), Area(E)]. That is, the real natural person represented by the third first cluster often appears at E point after 8 o'clock and then passes through F point, and moves to G point after 9 o'clock. In addition, it often appears at G point after 17 o'clock and then passes through F point, and moves to E point at 18 o'clock.

[0072] Therefore, the inverse spatiotemporal information coding dictionary is obtained through the mapping relationship of the spatiotemporal information coding dictionary, the obtained result is spatiotemporally decoded, the intuitive time and space travel information of the real natural person can be obtained, and the effect of the later face clustering algorithm is improved.

[0073] Step S13: obtaining the spatiotemporal domain information corresponding to the first cluster based on the spatiotemporal correlation information.

[0074] Specifically, the strong spatio-temporal association information obtained in step S1222 for each first-type cluster is used to generate exclusive spatio-temporal area information for each first-type cluster. The spatio-temporal area at this time can be defined as Time Area_N, such as: [Time(X1), Area(Y1); ……].

[0075] For example, in an embodiment, the first first-type cluster contains two exclusive spatio-temporal area information, which are TimeArea_1: [Time(8), Area(A); Time(9), Area(B)] and TimeArea_2: [Time(16), Area(B); Time(17), Area(A)], respectively.

[0076] The second first-type cluster contains two exclusive spatio-temporal area information, which are TimeArea_1: [Time(9), Area(C); Time(10), Area(D)] and TimeArea_2: [Time(18), Area(D); Time(19), Area(C)], respectively.

[0077] The third first-type cluster contains two exclusive spatio-temporal area information, which are TimeArea_1: [Time(8), Area(E); Time(8), Area(F); Time(9), Area(G)] and TimeArea_2: [Time(17), Area(G); Time(17), Area(F); Time(18), Area(E)], respectively.

[0078] Obviously, the TimeArea_1 and TimeArea_2 of the three first-type clusters are not the same. Therefore, it can be concluded that each first-type cluster has exclusive spatio-temporal area information.

[0079] Before the step of obtaining the spatio-temporal association information corresponding to the first-type cluster based on the snapshot time and the snapshot location of the image corresponding to the first-type cluster, it includes: in response to the snapshot time and the snapshot location of the plurality of images being the same, only retaining the snapshot time and the snapshot location of one of the images.

[0080] Specifically, if the images in a first-type cluster all involve the same snapshot time and the same snapshot location, then this first-type cluster only contains one spatio-temporal area, i.e., TimeArea_1 is [Time(X1), Area(Y1)], where X1 is the only time point in the first-type cluster and Y1 is the only snapshot location in the first-type cluster.

[0081] In another embodiment, one possible case is that the first type of cluster contains one image, and the image's snapshot time and snapshot location are taken as the first type of cluster's corresponding time-space domain information.

[0082] Specifically, if a first type of cluster contains only one image, the snapshot time information and snapshot location information of the image are the only time and space information in the first type of cluster, and the snapshot time information and snapshot location information of the image are taken as the first type of cluster's time-space domain information. That is, the first type of cluster's time-space domain information Time Area_1 is [Time(X1), Area(Y1)], where X1 is the only time point in the first type of cluster, and Y1 is the only snapshot location in the first type of cluster.

[0083] Specifically, in an embodiment, when determining the first type of cluster's time-space domain information, the first type of cluster is screened, and the first type of cluster containing at least two images and having different snapshot times or snapshot locations is screened for time-space domain information determination, which can further improve the system's resource utilization. For a first type of cluster containing only one image, the snapshot time and snapshot location of the image can be directly determined as the time-space domain information.

[0084] The screened first type of cluster contains at least two images, which can facilitate later time-space correlation relationship mining. If the first type of cluster contains only one image, there is no need to mine the time-space correlation relationship. Therefore, step S12 requires the first type of cluster to contain at least two images, which can also improve the system's resource utilization to some extent.

[0085] In addition, before the step of obtaining the first type of cluster's corresponding time-space correlation information based on the first type of cluster's corresponding image's snapshot time and snapshot location, the step includes:

[0086] The snapshot time and snapshot location are divided by week to obtain the first time range's snapshot time and corresponding snapshot location, and the second time range's snapshot time and corresponding snapshot location.

[0087] Specifically, dividing the snapshot time and snapshot location by week means that a week is divided into two time ranges: the first time range is five days in a week, and the second time range is two days of the weekend.

[0088] In an embodiment, taking a week as an example, the travel trajectory of a same real natural person in five days of a week can be different from the travel trajectory in two days of a weekend. After the required clustering information is filtered out in step S12, the data of each first-class cluster can be divided into a week and a weekend, so as to obtain a week space-time domain and a weekend space-time domain, so as to better reflect the daily travel habits of the real natural person corresponding to the first-class cluster.

[0089] Please refer to Figure 12 , Figure 12 The flowchart of an embodiment of the space-time domain information generation device of the application is shown in the figure. Specifically, it includes a cluster acquisition module 11, a mining module 12, and a space-time domain information acquisition module 13.

[0090] The cluster acquisition module 11 is configured to acquire the first-class cluster.

[0091] The mining module 12 is configured to obtain the space-time correlation information corresponding to the first-class cluster based on the snapshot time and the snapshot location of the images in the first-class cluster.

[0092] Specifically, the mining module 12 uses the association relationship mining algorithm to mine the space-time correlation relationship based on the sequence information, obtains a plurality of identifiers with strong correlation, and decodes the plurality of identifiers with strong correlation based on the mapping relationship of the space-time information coding to obtain the space-time correlation information. Through the mapping relationship of the space-time information coding dictionary, an inverse space-time information coding dictionary is obtained, and the obtained result is decoded in the space-time relationship, so that the intuitive time and space travel information of the real natural person can be obtained, which facilitates the improvement of the effect of the face clustering algorithm in the later stage.

[0093] In an embodiment, the mining module 12 is configured to sort the snapshot time and the snapshot location corresponding to the images in chronological order to generate sequence information; use the association relationship mining algorithm to mine the space-time correlation relationship based on the sequence information to obtain the space-time correlation information.

[0094] In another embodiment, the mining module 12 is configured to code the snapshot time and the snapshot location corresponding to the images in the space-time information to obtain identifiers representing the space-time relationship of the images; sort the identifiers corresponding to the images in chronological order to generate sequence information.

[0095] In an embodiment, the mining module 12 uses the association relationship mining algorithm to mine the space-time correlation relationship based on the sequence information to obtain a plurality of identifiers with strong correlation; decodes the plurality of identifiers with strong correlation based on the mapping relationship of the space-time information coding to obtain the space-time correlation information.

[0096] In an embodiment, in response to the snapshot time and the snapshot location of the plurality of images being the same, only the snapshot time and the snapshot location of one of the images are retained.

[0097] In an embodiment, the snapshot time and the snapshot location are divided by week to obtain the snapshot time and the corresponding snapshot location in a first time range and the snapshot time and the corresponding snapshot location in a second time range.

[0098] The space-time domain information acquisition module 13 is configured to obtain the space-time domain information corresponding to the first type of cluster based on the space-time association information.

[0099] In an embodiment, when the first type of cluster contains one image, the space-time domain information acquisition module 13 takes the snapshot time and the snapshot location corresponding to the image as the space-time domain information corresponding to the first type of cluster.

[0100] For example, the space-time domain information corresponding to each first type of cluster is represented as Time Area_N: [Time(X1), Area(Y1); ……], where N is the serial number of the space-time domain, X represents time, and Y represents location. Specifically, each space-time domain Time Area is composed of multiple time-space points [Time(X1), Area(Y1)].

[0101] For example, assuming that the space-time domain information of a first type of cluster is Time Area_1: [Time(8), Area(A); Time(9), Area(B)]. The specific meaning is that the object corresponding to the first type of cluster, i.e., the real natural person, is most likely to start from location A at 8 am and arrive at location B at 9 am. Therefore, it can be seen that the space-time domain information of the first type of cluster in step S1 actually reflects the daily travel habits of the real natural person corresponding to the first type of cluster. The purpose of portrait clustering is to have one person in one category, i.e., to associate and map all snapshot pictures of each person with the ID picture of the person. The real natural person here refers to that, after portrait clustering, each cluster is correctly associated and mapped with the corresponding ID picture, and the person corresponding to the ID picture is the real natural person of the cluster. One person in one category means that a large number of portrait pictures are grouped by person, and all snapshot pictures of each person are grouped into a set, and the images in the set can reflect the travel relationship of the corresponding real natural person. However, after image clustering, the problem of one person in multiple categories often occurs, which will make the corresponding travel relationship disorder and not conducive to subsequent implementation of target tracking and other applications. In view of this, the present application combines multiple clusters according to the space-time domain information of the multiple clusters after clustering, thereby solving the problem of one person in multiple categories and improving the accuracy of clustering.

[0102] Step S2: determining whether there is a space-time association between at least two first type of clusters based on the space-time domain information corresponding to each first type of cluster.

[0103] Specifically, after obtaining the spatio-temporal information corresponding to each first cluster, the relevance between each first cluster and the time-space point is arranged by using the inverted index. The inverted index is derived from the actual application, which needs to find records according to the value of the attribute. Each item in this index table includes an attribute value and the address of each record with the attribute value. Since the attribute value is not determined by the record, but the position of the record is determined by the attribute value, it is called inverted index. That is, all the time-space point identifiers involved are used as indexes, and the information connected by each time-space point index is the cluster information of the corresponding spatio-temporal domain containing the time-space point. By using different spatio-temporal domains as inverted indexes, the first cluster information corresponding to the spatio-temporal domain covering the spatio-temporal domain is connected after each index. The first cluster information includes the cluster ID and the number of images in the cluster.

[0104] For example, assume that the first cluster 1 (ID: Cluster1) contains 5 images, and the corresponding spatio-temporal information is [Time (T1), Area (A); Time (T2), Area (B); Time (T3), Area (C)]. The first cluster 2 (ID: Cluster2) contains 6 images, and the corresponding spatio-temporal information is [Time (T2), Area (A), Time (T2), Area (B); Time (T3), Area (C)]. The first cluster 3 (ID: Cluster3) contains 1 image, and the corresponding exclusive spatio-temporal domain is [Time (T1), Area (A)].

[0105] By using the inverted index to organize the relevance between each cluster and the time-space point, the cluster information in the similar spatio-temporal domain can be quickly found, and the system running efficiency is improved. In an embodiment, step S2 only considers the spatio-temporal information corresponding to the first cluster, not the spatio-temporal information corresponding to each image in the cluster, the main reason is to focus on the main spatio-temporal travel relationship of the cluster, avoid the interference of noise signals, and improve the robustness of the system.

[0106] Specifically, please refer to Figure 2 , Figure 2 for Figure 1 the flowchart of an embodiment of step S2, which specifically includes:

[0107] Step S21: Determine whether the spatio-temporal information corresponding to at least two first clusters has the same time-space point.

[0108] Specifically, please refer to Figure 3 , Figure 3Fig. 1 is a schematic diagram for illustrating whether the spatiotemporal information corresponding to at least two first clusters has the same time-space point by using inverted index. The inverted index is used to arrange the relationship between the first cluster 1 (ID: Cluster1), the first cluster 2 (ID: Cluster2), the first cluster 3 (ID: Cluster3) and the time-space point. The first cluster 1 (ID: Cluster1) and the first cluster 3 (ID: Cluster3) are related to the time T1 and the space A point; the first cluster 2 (ID: Cluster2) is related to the time T2 and the space A point; the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) are related to the time T2 and the space B; the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) are related to the time T3 and the space C. That is, the first cluster 1 (ID: Cluster1) and the first cluster 3 (ID: Cluster3) have the same time-space point (T1, A); the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) have the same time-space point (T2, B); the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) have the same time-space point (T3, C).

[0109] Step S22: in response to having the same time-space point, it is determined that there is a spatiotemporal association between the at least two first clusters.

[0110] Specifically, the first cluster 1 (ID: Cluster1) and the first cluster 3 (ID: Cluster3) have the same time-space point (T1, A), and there is a spatiotemporal association between the first cluster 1 (ID: Cluster1) and the first cluster 3 (ID: Cluster3); the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) have the same time-space point (T2, B), and there is a spatiotemporal association between the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2); the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2) have the same time-space point (T3, C), and there is a spatiotemporal association between the first cluster 1 (ID: Cluster1) and the first cluster 2 (ID: Cluster2).

[0111] Step S3: in response to there being a spatiotemporal association between the at least two first clusters, the at least two first clusters are clustered based on the similarity between the at least two first clusters.

[0112] Specifically, the space-time field information is generally composed of multiple time-space points. If the first type of cluster only contains one image, the space-time field information of the first type of cluster can only be composed of one time-space point. Such a cluster is difficult to reflect the daily travel rule of the real person corresponding to the cluster. That is, in the above embodiment, the first type of cluster (ID: Cluster3) with only one image cannot reflect the daily travel rule of the real person corresponding to the cluster, that is, the first type of cluster (ID: Cluster3) is temporarily not considered to be merged.

[0113] That is, in the embodiment, the first type of cluster that needs to be merged based on the similarity is the first type of cluster 1 (ID: Cluster1) and the first type of cluster 2 (ID: Cluster2), and the time-space intersection of the first type of cluster 1 (ID: Cluster1) and the first type of cluster 2 (ID: Cluster2) is (Time: T2, Area: B) (Time: T3, Area: C). Based on the similarity between the first type of cluster 1 and the first type of cluster 2, it is determined whether to merge the first type of cluster 1 and the first type of cluster 2.

[0114] Specifically, please refer to Figure 4 , Figure 4 for Figure 1 an embodiment of step S3 in the flowchart, which specifically includes:

[0115] Step S31: determining the space-time correlation coefficient between the at least two first type of clusters.

[0116] Specifically, in the embodiment, the Jaccard coefficient is used to measure the space-time correlation coefficient between the clusters.

[0117] Specifically, the intersection of the space-time field information corresponding to the at least two first type of clusters is calculated, and the union of the space-time field information corresponding to the at least two first type of clusters is calculated. Assuming that the set of time-space points contained in the space-time field information of the first type of cluster 1 is M, and the set of time-space points contained in the space-time field information of the first type of cluster 2 is N. The intersection of the space-time field information corresponding to the at least two first type of clusters can be represented as: |M∩N|. The union of the space-time field information corresponding to the at least two first type of clusters can be represented as: |M∪N|. The space-time correlation coefficient is determined based on the intersection and the union. In an embodiment, the space-time correlation coefficient is denoted as a, a = |M∩N| / |M∪N|. By setting the space-time correlation coefficient, the dynamic threshold-based portrait clustering optimization is realized, and compared with the fixed threshold-based portrait clustering in the prior art, the robustness is stronger.

[0118] Step S32: determining the similarity threshold based on the space-time correlation coefficient.

[0119] Specifically, based on the determination of the spatiotemporal correlation coefficient a, the spatiotemporal correlation coefficient a can be used to dynamically adjust the similarity threshold of the at least two first clusters.

[0120] Specifically, the step of determining the similarity threshold based on the spatiotemporal correlation coefficient includes:

[0121] The similarity threshold is determined by using the preset highest similarity threshold, the preset lowest similarity threshold, and the spatiotemporal correlation coefficient.

[0122] For example, in an embodiment, the highest similarity threshold is defined as Sim_max, the lowest similarity threshold is defined as Sim_min, and the similarity threshold is:

[0123] Sim = Sim_max - (Sim_max - Sim_min) · a

[0124] Step S33: In response to the similarity between the at least two first clusters being greater than the similarity threshold, the at least two first clusters are clustered.

[0125] Specifically, after obtaining the similarity threshold Sim, the similarity between the at least two first clusters needs to be compared with the similarity threshold Sim to determine whether the at least two first clusters can be clustered. Therefore, before the step of determining whether the similarity between the at least two first clusters is greater than the similarity threshold, the similarity between the two clusters needs to be calculated.

[0126] Specifically, the average centroids of the at least two first clusters are calculated respectively. For example, the average centroid of the first cluster 1 is calculated by using all the centroids in the first cluster 1, and the average centroid of the first cluster 2 is calculated by using all the centroids in the first cluster 2. The similarity between the at least two first clusters is calculated based on the average centroids of the at least two first clusters. Specifically, after the average centroids of the first cluster 1 and the first cluster 2 are calculated, the similarity between the first cluster 1 and the first cluster 2 is calculated based on the average centroids of the first cluster 1 and the first cluster 2. The similarity between the first cluster 1 and the first cluster 2 is the distance between the average centroids of the first cluster 1 and the first cluster 2, and the distance measurement method includes but is not limited to Euclidean distance and cosine distance. After the similarity between the first cluster 1 and the first cluster 2 is determined, the size relationship between the similarity and the similarity threshold is determined. When the similarity is greater than the similarity threshold, the at least two first clusters are clustered, i.e., the first cluster 1 and the first cluster 2 are clustered together; when the similarity is less than the similarity threshold, the first cluster 1 and the first cluster 2 are not clustered.

[0127] In an embodiment of the present application, in order to further improve the accuracy of cluster clustering, it is necessary to determine the number of images in the first cluster. If the number of images in at least one of the at least two first clusters is less than a preset number, a first similarity threshold is determined based on the spatiotemporal correlation coefficient, and in response to the similarity between the at least two first clusters being greater than the first similarity threshold, the at least two first clusters are clustered. In an embodiment, the preset number is 2. If the first cluster contains only a few 2, i.e., only one image, the spatiotemporal information of the first cluster can only be composed of one time-space point. Such a cluster is difficult to reflect the daily travel rule of the real person corresponding to the cluster. That is, in the above embodiment, the first cluster (ID Cluster3) with only one image cannot reflect the daily travel rule of the real person corresponding to the cluster. At this time, if the first cluster (ID Cluster3) is to be merged, in order to improve the accuracy, a larger first similarity threshold is set.

[0128] In response to the number of images in the at least two first clusters being greater than the preset number, a second similarity threshold is determined based on the spatiotemporal correlation coefficient; in response to the similarity between the at least two first clusters being greater than the second similarity threshold, the at least two first clusters are clustered; and the first similarity threshold is greater than the second similarity threshold.

[0129] In the above embodiment, the first similarity threshold is greater than the second similarity threshold. That is, when the number of images to be processed is less than the preset number of images in the method, such as only one image, a higher similarity standard needs to be set when judging the size of the similarity threshold. This can ensure the accuracy of the clustered images.

[0130] In addition, the determination method of the first similarity threshold and the second similarity threshold is the same as the similarity threshold determination method in step S32, and thus will not be described again. In practice, the first similarity threshold and the second similarity threshold can be set as needed, as long as the highest similarity threshold, the lowest similarity threshold, and the first similarity threshold and the second similarity threshold are different. The present application does not limit this.

[0131] The clustering method of the present application utilizes the exclusive spatio-temporal domain information of each class cluster to further optimize the result of face clustering based on traditional image features. Specifically, a higher similarity threshold is set for the portrait pictures in each class cluster that are not in the exclusive spatio-temporal domain of the class cluster, and then the erroneous pictures in the class cluster are specifically removed, thereby improving the effect of face clustering. The spatio-temporal correlation coefficient is utilized to well measure the spatio-temporal correlation between each class cluster, realize dynamic threshold portrait clustering optimization, and then realize dynamic threshold portrait clustering optimization according to spatio-temporal information. Compared with fixed threshold portrait clustering, the robustness is stronger. The inverted index is adopted to speed up the search of the class cluster information in the similar spatio-temporal domain, thereby improving the system operation efficiency. The present application can simultaneously and fully consider the influence of time and space on face clustering, and improve the accuracy of face clustering.

[0132] In practice, when determining whether there is a spatio-temporal association between at least two first class clusters, a situation often occurs that when the first class cluster appears outside the spatio-temporal domain or has a single-dimensional overlap in time or space with the spatio-temporal domain, then at this time these images are likely to be abnormal images. Therefore, for this situation, the similarity threshold of image verification needs to be improved to correct the abnormal images.

[0133] Specifically, as shown in Figure 5 Figure 5 is a flowchart of a second embodiment of the clustering method of the present application. Before determining whether there is a spatio-temporal association between at least two first class clusters based on the spatio-temporal domain information corresponding to each first class cluster, it further includes:

[0134] Step S4: respectively determine whether there is the same time information and / or space information in the spatio-temporal domain information corresponding to each first class cluster.

[0135] Specifically, the time, space and position information of the images in each first class cluster are counted and compared to determine whether there is one or more images with the same time and / or space information.

[0136] Step S5: in response to the existence of the same time information and / or space information, determine the abnormal images based on the same time information and / or space information.

[0137] Specifically, if there is one or more images with the same time and / or space information, it can be determined that there is an abnormal image in the image, that is, there is an erroneous image that does not belong to the first class cluster, and then the image is determined as an abnormal image.

[0138] Step S6: remove the abnormal images from the first class cluster.

[0139] ​Specifically, after determining that a certain image is an abnormal image, the image needs to be removed from the first cluster to solve the abnormal situation and ensure that the images in the first cluster accurately reflect the real natural person's daily travel habits.

[0140] For example, assume that the time-space domain of a certain cluster is Time Area_1: [Time(8), Area(A); Time(9), Area(B)]. At this time, the cluster contains a total of 10 images, one of which corresponds to the time-space point [Time(3), Area(A)], that is, the image is captured at 3 a.m. at point A, which obviously does not match the time-space information of most images in the cluster, and therefore the image is extremely likely to be an image of a different person from the remaining 9 images in the file.

[0141] By setting a suitable similarity threshold for internal verification of the cluster, it can be determined whether the image and the remaining 9 images in the cluster are images of the same person. If the image and the remaining 9 images in the cluster are not images of the same person, the cluster is an error cluster, the image is an error image, and needs to be removed. By setting a higher similarity threshold for the image in each cluster that does not belong to the time-space domain of the cluster, the error images in the cluster can be removed.

[0142] It should be noted that the error image removed in step S6 needs to be separately set as a new cluster, and the system needs to assign a unique cluster ID to it. The image can be part of other clusters in the future, or it can remain the only image in the cluster.

[0143] Directly clustering images from massive image capture data often has poor results. The reason is that the probability of similar images is higher in large data than in small data. Grouping and re-clustering massive image data using time-space information is a new solution to improve the effect of image clustering.

[0144] After clustering images based on traditional image features, each cluster can be quickly assigned a dedicated time-space domain through probability statistics or association relationship mining, and then a truly fine-grained time-space area division from the perspective of each cluster can be achieved. The so-called time-space domain is composed of multiple time-space points and reflects the real natural person's daily travel habits corresponding to the cluster.

[0145] Using the dedicated time-space domain information of each cluster, the error correction and multi-file merging in image clustering can be completed, and the effect of image clustering can be optimized.

[0146] The clustering method of the present application utilizes the exclusive spatio-temporal domain information of each cluster to perform error correction optimization of portrait clustering. Specifically, a higher similarity threshold is set for the portrait pictures in each cluster that are not in the exclusive spatio-temporal domain of the cluster, thereby targetedly eliminating the error pictures in the cluster. The inverted index is used to speed up the search for the spatio-temporal cluster information in the similar spatio-temporal domain, thereby improving the system operation efficiency. By setting the spatio-temporal correlation coefficient, the dynamic threshold portrait clustering optimization is realized, which is more robust than the fixed threshold portrait clustering.

[0147] The clustering method of the present application first determines whether the plurality of clusters have a spatio-temporal correlation relationship based on the spatio-temporal domain information, and determines whether to cluster the plurality of clusters based on the similarity of the plurality of clusters having a time correlation relationship, thereby solving the problem of one person with multiple files and improving the clustering accuracy.

[0148] Please refer to Figure 6 The structure schematic diagram of an embodiment of the clustering device of the present application specifically includes an acquisition module 41, a determination module 42, and a clustering module 43.

[0149] The acquisition module 41 is configured to acquire at least two first clusters and the spatio-temporal domain information corresponding to each first cluster, wherein the spatio-temporal domain information represents the spatio-temporal position relationship of the object corresponding to the first cluster.

[0150] Further, after acquiring the first cluster, the spatio-temporal domain information corresponding to the first cluster is further acquired, wherein the spatio-temporal domain information represents the spatio-temporal position relationship of the object corresponding to the first cluster. For example, the first cluster is the cluster of object A, and the spatio-temporal domain information represents the time and place relationship of the travel of object A. One spatio-temporal domain information is composed of multiple fixed time and space places, and has the order of time and space. Specifically, each image shooting will exist a time, a space geographical position corresponding to the image, and the spatio-temporal domain information of the image can also be acquired at the same time when the face feature information of the image is acquired through the clustering algorithm.

[0151] The determination module 42 is configured to determine whether there is a spatio-temporal correlation between the at least two first clusters based on the spatio-temporal domain information corresponding to each first cluster.

[0152] In an embodiment, the determination module 42 determines the correlation between each first cluster and the time and space points by arranging the correlation in the form of inverted index based on the spatio-temporal domain information corresponding to each first cluster. Specifically, all the time and space point identifiers involved are used as indexes, and the information connected by each time and space point index is the cluster information of the corresponding spatio-temporal domain containing the time and space point. Different spatio-temporal domains are used as inverted indexes, and the cluster information of the first cluster corresponding to the spatio-temporal domain is connected after each index. The first cluster information includes cluster ID and the number of images in the cluster.

[0153] The correlation between the above various clusters and the time-space points is organized in a way of inverted index, which can accelerate the search for the clusters in the similar time-space domain, and further improve the system operation efficiency.

[0154] In an embodiment, the determining module 42 determines whether the time-space domain information corresponding to the at least two first clusters has the same time-space point; and in response to having the same time-space point, determines that there is a time-space correlation between the at least two first clusters.

[0155] The clustering module 43 is configured to, in response to the time-space correlation between the at least two first clusters, cluster the at least two first clusters based on the similarity between the at least two first clusters.

[0156] In an embodiment, the clustering module 43 is configured to determine a time-space correlation coefficient between the at least two first clusters; determine a similarity threshold value based on the time-space correlation coefficient; and in response to the similarity between the at least two first clusters being greater than the similarity threshold value, cluster the at least two first clusters.

[0157] In an embodiment, the clustering module 43 is configured to calculate an intersection of the time-space domain information corresponding to the at least two first clusters; calculate a union of the time-space domain information corresponding to the at least two first clusters; and determine the time-space correlation coefficient based on the intersection and the union.

[0158] In an embodiment, the clustering module 43 is configured to determine the similarity threshold value by using a preset highest similarity threshold value, a preset lowest similarity threshold value, and the time-space correlation coefficient.

[0159] In an embodiment, the clustering module 43 is configured to calculate the average centroid of the at least two first clusters respectively; and calculate the similarity between the at least two first clusters based on the average centroid of the at least two first clusters.

[0160] Specifically, the time-space domain information is generally composed of multiple time-space points. If the first cluster contains only one image, the time-space domain information of the first cluster can only be composed of one time-space point. Such a cluster is difficult to reflect the daily travel rule of the real person corresponding to the cluster. That is, in the above embodiment, the first cluster (ID: Cluster3) containing only one image cannot reflect the daily travel rule of the real person corresponding to the cluster, that is, the first cluster (ID: Cluster3) is temporarily not considered to be merged.

[0161] In the embodiment, the first type of clusters that need to be merged based on the similarity are the first type of cluster 1 (ID: Cluster1) and the first type of cluster 2 (ID: Cluster2), and the time-space intersection of the first type of cluster 1 (ID: Cluster1) and the first type of cluster 2 (ID: Cluster2) is (Time: T2, Area: B) (Time: T3, Area: C). Based on the similarity between the first type of cluster 1 and the first type of cluster 2, it is determined whether to merge the first type of cluster 1 and the first type of cluster 2.

[0162] In practice, when the number of images to be processed is less than the preset number of images, a higher similarity threshold needs to be set. For details, refer to the content of step S32 described above, which will not be repeated here.

[0163] The clustering device of the application determines whether there is a time-space association between at least two first type of clusters by obtaining the time-space domain information corresponding to each first type of cluster, and determining whether there is a time-space association between at least two first type of clusters based on the time-space domain information corresponding to each first type of cluster. If there is a time-space association, at least two first type of clusters are clustered based on the similarity between at least two first type of clusters. By using the time-space domain information of each cluster, it is determined whether there is a time-space association between different clusters, and then it is determined whether at least two clusters can be clustered, thereby solving the problem of one person with multiple profiles and improving the accuracy of clustering.

[0164] Please refer to Figure 13 The structure of an embodiment of the electronic device of the application is shown in the figure. The electronic device includes a memory 202 and a processor 201 connected to each other.

[0165] The memory 202 is used to store program instructions for implementing the method of any one of the above devices.

[0166] The processor 201 is used to execute the program instructions stored in the memory 202.

[0167] The processor 201 can also be called a CPU (Central Processing Unit). The processor 201 can be an integrated circuit chip with signal processing capability. The processor 201 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0168] The memory 202 can be a memory stick, a TF card, etc., and can store all information in the electronic device of the device, including input raw data, computer programs, intermediate running results and final running results. It stores and retrieves information according to the location specified by the controller. With the memory, the electronic device has a memory function and can work normally. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to the purpose, and there is also a classification method of dividing external memory and internal memory. The external memory is usually a magnetic medium or an optical disc, etc., which can store information for a long time. The internal memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data, and the data will be lost when the power is turned off or disconnected.

[0169] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0170] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0171] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0172] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions capable of causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods in the embodiments of the present application.

[0173] Please refer to Figure 14 , the structure schematic diagram of an embodiment of the computer readable storage medium of the present application. The storage medium of the present application stores a program file 203 capable of realizing all the methods described above, wherein the program file 203 can be stored in the storage medium in the form of a software product, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods in the embodiments of the present application. The storage device described above includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media capable of storing program codes, or a computer, a server, a mobile phone, a tablet, and other terminal devices.

[0174] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A clustering method characterized by, The method comprises: obtaining at least two first clusters, obtaining spatio-temporal correlation information corresponding to the first clusters based on the snapshot time and the snapshot location of the images in the first clusters, and obtaining spatio-temporal domain information corresponding to the first clusters based on the spatio-temporal correlation information, wherein the spatio-temporal domain information represents the spatio-temporal position relationship of the objects corresponding to the first clusters; wherein the first clusters are obtained by clustering the images in a database; determining whether there is a spatio-temporal correlation between at least two first clusters based on the spatio-temporal domain information corresponding to each first cluster; in response to the existence of a spatio-temporal correlation between at least two first clusters, clustering at least two first clusters based on the similarity between at least two first clusters; wherein the step of clustering at least two first clusters based on the similarity between at least two first clusters comprises: calculating the intersection of the spatio-temporal domain information corresponding to at least two first clusters; and calculating the union of the spatio-temporal domain information corresponding to at least two first clusters; determining a spatio-temporal correlation coefficient based on the intersection and the union; determining a similarity threshold based on the spatio-temporal correlation coefficient; in response to the similarity between at least two first clusters being greater than the similarity threshold, clustering at least two first clusters.

2. The clustering method of claim 1, wherein, The step of determining whether there is a spatio-temporal correlation between at least two first clusters based on the spatio-temporal domain information corresponding to each first cluster comprises: determining whether the spatio-temporal domain information corresponding to at least two first clusters has the same time-space point; in response to having the same time-space point, determining that there is a spatio-temporal correlation between at least two first clusters.

3. The clustering method of claim 2, wherein, The step of determining a similarity threshold based on the spatio-temporal correlation coefficient comprises: determining the similarity threshold using a preset highest similarity threshold, a preset lowest similarity threshold, and the spatio-temporal correlation coefficient.

4. The clustering method of claim 1, wherein, Before the step of clustering at least two first clusters in response to the similarity between at least two first clusters being greater than the similarity threshold, it comprises: respectively calculating the average centroid of at least two first clusters; calculating the similarity between at least two first clusters based on the average centroid of at least two first clusters.

5. The clustering method of claim 1, wherein, In response to the number of images in at least one of the at least two first clusters being less than a preset number, the step of determining a similarity threshold based on the spatio-temporal correlation coefficient comprises: determining a first similarity threshold based on the spatio-temporal correlation coefficient; The step of clustering at least two first clusters in response to the similarity between at least two first clusters being greater than the similarity threshold comprises: in response to the similarity between at least two first clusters being greater than the first similarity threshold, clustering at least two first clusters; In response to the number of images in at least two first clusters being greater than a preset number, the step of determining a similarity threshold based on the spatio-temporal correlation coefficient comprises: determining a second similarity threshold based on the spatio-temporal correlation coefficient; The clustering of the at least two first clusters in response to the similarity between the at least two first clusters being greater than the similarity threshold value comprises: The clustering of the at least two first clusters in response to the similarity between the at least two first clusters being greater than the second similarity threshold value; The first similarity threshold value is greater than the second similarity threshold value.

6. The clustering method of claim 1, wherein, The step of determining whether there is a spatio-temporal correlation between the at least two first clusters based on the spatio-temporal domain information corresponding to each of the first clusters comprises: Determining whether there is the same time information and / or space information in the spatio-temporal domain information corresponding to each of the first clusters respectively; In response to the same time information and / or space information, determining an abnormal image based on the same time information and / or space information; Excluding the abnormal image from the first cluster.

7. A clustering apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire at least two first clusters, obtain spatio-temporal correlation information corresponding to the first clusters based on the snapshot time and the snapshot location of the images in the first clusters, obtain spatio-temporal domain information corresponding to the first clusters based on the spatio-temporal correlation information, and the spatio-temporal domain information represents the spatio-temporal position relationship of the object corresponding to the first clusters; wherein the first clusters are obtained by clustering the images in the database; A determination module is configured to determine whether there is a spatio-temporal correlation between the at least two first clusters based on the spatio-temporal domain information corresponding to each of the first clusters; A clustering module is configured to cluster the at least two first clusters based on the similarity between the at least two first clusters in response to the existence of the spatio-temporal correlation between the at least two first clusters; wherein the clustering module is further configured to calculate the intersection of the spatio-temporal domain information corresponding to the at least two first clusters, and calculate the union of the spatio-temporal domain information corresponding to the at least two first clusters; Determine a spatio-temporal correlation coefficient based on the intersection and the union; Determine a similarity threshold value based on the spatio-temporal correlation coefficient; In response to the similarity between the at least two first clusters being greater than the similarity threshold value, the at least two first clusters are clustered.

8. An electronic device, comprising: Comprise: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the clustering method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A program file is stored, and the program file can be executed to implement the clustering method according to any one of claims 1-6.

Images