Method, device, electronic device, and storage medium for generating spatiotemporal information of clusters

Abstract

The present invention provides a method, apparatus, electronic device, and storage medium for generating spatiotemporal information for clusters. The method comprises: obtaining a first cluster; obtaining spatiotemporal association information corresponding to the first cluster based on the capture time and capture location of images in the first cluster; and obtaining spatiotemporal information corresponding to the first cluster based on the spatiotemporal association information. By assigning a dedicated spatiotemporal domain to each first cluster, the effect of portrait clustering is optimized and improved.

Description

Technical Field

[0001] The present invention relates to the field of video image processing technology, and in particular to a method, device, electronic device, and storage medium for generating spatiotemporal information of a cluster. Background Art

[0002] With the widespread adoption of intelligent video surveillance equipment, a massive amount of portrait images is accumulated daily. Using portrait clustering methods to organize portrait images into person-by-person units is a common approach. However, clustering directly from massive amounts of captured portrait data often yields poor results. For example, clustering can result in multiple files for the same person (i.e., multiple clusters corresponding to the same person), resulting in poor clustering results. Summary of the Invention

[0003] The present invention provides a method, device, electronic device, and storage medium for generating spatiotemporal information of a cluster, which can improve the accuracy of clustering.

[0004] To solve the above technical problems, the first technical solution provided by the present invention is: to provide a method for generating spatiotemporal domain information of a cluster, including: obtaining a first cluster; obtaining spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the image corresponding to the first cluster; and obtaining the spatiotemporal domain information corresponding to the first cluster based on the spatiotemporal correlation information.

[0005] Among them, the first cluster contains at least two images; the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the images corresponding to the first cluster includes: sorting the capture time and the capture location corresponding to the images in chronological order to generate sequence information; using an association mining algorithm to mine spatiotemporal correlation relationships based on the sequence information to obtain the spatiotemporal correlation information.

[0006] Among them, the step of sorting the capturing time and the capturing location corresponding to the image in chronological order to generate sequence information includes: encoding the spatiotemporal information of the capturing time and the capturing location corresponding to the image to obtain an identifier representing the spatiotemporal relationship of the image; and sorting the identifiers corresponding to the image in chronological order to generate the sequence information.

[0007] Among them, the step of using an association mining algorithm to perform spatiotemporal association mining based on the sequence information to obtain the spatiotemporal association information includes: using an association mining algorithm to perform spatiotemporal association mining based on the sequence information to obtain multiple strongly correlated identifiers; decoding the multiple strongly correlated identifiers based on the mapping relationship encoded by the spatiotemporal information to obtain the spatiotemporal association information.

[0008] Among them, before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capturing time and capturing location of the image corresponding to the first cluster, it includes: in response to the capturing time and capturing location of multiple images being the same, only retaining the capturing time and capturing location of one of the images.

[0009] Among them, before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the image corresponding to the first cluster, it includes: dividing the capture time and the capture location by week to obtain the capture time and the corresponding capture location in the first time range, and obtaining the capture time and the corresponding capture location in the second time range.

[0010] The first cluster includes an image; the capture time and the capture location corresponding to the image are used as the spatiotemporal information corresponding to the first cluster.

[0011] In order to solve the above technical problems, the second technical solution provided by the present invention is: to provide a device for generating spatiotemporal domain information of a cluster, including: a cluster acquisition module, used to obtain a first cluster; a mining module, used to obtain the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the image corresponding to the first cluster; and a spatiotemporal domain information acquisition module, used to obtain the spatiotemporal domain information corresponding to the first cluster based on the spatiotemporal correlation information.

[0012] To solve the above technical problems, the third technical solution provided by the present invention is: to provide an electronic device, comprising: a memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute any of the above methods.

[0013] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, which can be executed to implement any of the above methods.

[0014] The present invention, unlike existing techniques, obtains spatiotemporal correlation information corresponding to the first cluster by acquiring the capture time and location of the images corresponding to the first cluster. Based on this spatiotemporal correlation information, the present invention also obtains spatiotemporal domain information corresponding to the first cluster. By assigning a dedicated spatiotemporal domain to each first cluster, the present invention optimizes and improves the effectiveness of portrait clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0016] Figure 1 This is a flow chart of the first embodiment of the clustering method of the present application;

[0017] Figure 2 for Figure 1 A schematic flow chart of an embodiment of step S2;

[0018] Figure 3 A schematic diagram showing the relationship between whether the spatiotemporal domain information corresponding to at least two first-type clusters are arranged using an inverted index and whether they have the same spatiotemporal point;

[0019] Figure 4 for Figure 1 A schematic flow chart of an embodiment of step S3;

[0020] Figure 5 This is a flow chart of a second embodiment of the clustering method of the present application;

[0021] Figure 6 This is a schematic structural diagram of an embodiment of a clustering device of the present application;

[0022] Figure 7 for Figure 1 A flowchart of an embodiment of obtaining spatiotemporal information corresponding to the first cluster in step S1;

[0023] Figure 8 for Figure 7 A flow chart of an embodiment of step S12;

[0024] Figure 9 for Figure 8 A flow chart of an embodiment of step S121;

[0025] Figure 10 Schematic diagram of spatiotemporal information encoding in one embodiment of the present application;

[0026] Figure 11 for Figure 8 A flow chart of an embodiment of step S122;

[0027] Figure 12 A schematic structural diagram of an embodiment of a device for generating spatiotemporal information of a cluster of the present application;

[0028] Figure 13This is a schematic structural diagram of an embodiment of the electronic device of the present application;

[0029] Figure 14 This is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0030] The prior art offers a method and system for one-person, one-file face clustering based on a dynamic algorithm. Key features of this method include capturing facial photos in real time and recording the capture time and location. Long and short features are extracted from the captured facial photos, where long features are more common in key locations than short features, and short features are less common in key locations than long features. The captured facial photos are then compared and clustered to obtain one-person, one-file data for analysis, revealing the facial activity trajectory and frequency. This analysis of the one-person, one-file data reveals the facial activity trajectory and frequency at different time periods and locations. However, the face clustering proposed in this method essentially uses weighted construction of long and short features based on key image information points, and employs a dynamic algorithm to optimize the face clustering. Finally, the face clustering results are combined with capture time and location information to display the trajectory. This face clustering method fails to fully utilize both temporal and spatial information.

[0031] Existing technologies also include a face clustering method and device. While this design considers feature similarity matching and spatial and temporal distance constraints, and assigns different weights based on the order in which images were captured, attempting to optimize face clustering, the weighting is limited only by the temporal dimension and fails to fully consider the impact of both temporal and spatial information on face clustering.

[0032] The present application provides a clustering method that can fully consider the influence of time and space on clustering at the same time, thereby improving the accuracy of clustering. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0033] See Figure 1 , Figure 1 This is a flowchart of the first embodiment of the clustering method of the present application, which specifically includes:

[0034] Step S1: obtaining at least two first clusters and spatiotemporal information corresponding to each first cluster, where the spatiotemporal information represents the spatiotemporal position relationship of objects corresponding to the first clusters.

[0035] This embodiment uses human faces as an example to illustrate that a clustering algorithm is used to cluster images in a database based on facial features to obtain at least two first-class clusters. The clustering algorithm includes one or any combination of k-means and DBSCAN.

[0036] Furthermore, after obtaining the first cluster, the present application further obtains the spatiotemporal domain information corresponding to the first cluster, and the spatiotemporal domain information represents the spatiotemporal position relationship of the objects corresponding to the first cluster. For example, if the first cluster is the cluster of object A, then the spatiotemporal domain information represents the time and location relationship of the travel of object A. A spatiotemporal domain information is composed of multiple groups of fixed time and space locations, and has a time and space order. Specifically, when each image is taken, there will be a time and space geographical location corresponding to the image. When the facial feature information of the image is obtained through the clustering algorithm, the spatiotemporal domain information of the image can also be obtained at the same time.

[0037] For details, see Figure 7 , Figure 7 for Figure 1 The flowchart of an embodiment of obtaining the spatiotemporal information corresponding to the first cluster in step S1 specifically includes:

[0038] Step S11: Obtain the first cluster.

[0039] Specifically, based on the similarity of the images in the database, the feature sets corresponding to the images in the database are clustered to obtain a plurality of first clusters. It is understood that clustering algorithms include but are not limited to k-means and DBSCAN.

[0040] In one embodiment, all portrait cluster information within a specified time period is obtained. In practice, the selection of the time period primarily depends on the duration of stable operation of the portrait system. Generally, all portrait cluster information within the last three months can be selected. The portrait cluster information includes a first cluster ID and all portrait information contained within the first cluster. The portrait information primarily includes information such as the capture time, capture location, image type, and image ID. The image type can be a face image or a body image, meaning that the cluster can contain only face images, only body images, or both.

[0041] Specifically, a feature extraction algorithm, such as a convolutional neural network algorithm, is used to extract features from images in a database, thereby obtaining a feature set. The feature set is divided into K groups, and K objects are randomly selected as initial cluster centers. The distance, i.e., the similarity, between each object and each cluster center is then calculated. The similarity is then compared with a similarity threshold. Each object with a similarity greater than the similarity threshold is assigned to a corresponding cluster center, thereby obtaining multiple first-class clusters.

[0042] Step S12: Based on the capture time and capture location of the images in the first cluster, obtain the spatiotemporal correlation information corresponding to the first cluster.

[0043] In one embodiment, spatiotemporal correlation relationship mining is performed based on the capture time and capture location of the images in the first cluster to obtain spatiotemporal correlation information corresponding to the first cluster.

[0044] To fully leverage spatiotemporal correlation mining, it is necessary to quantify and calibrate the capture time and location for each cluster. In one embodiment, a typical method for quantifying and calibrating the time dimension, or capture time, can be to divide the continuous capture time into hourly periods based on a daily cycle, that is, to quantize the capture time into 24 different time periods on an hourly basis. The capture location can be independent longitude and latitude information, or it can be a unique spatial domain ID information composed of multiple front-end capture devices.

[0045] See Figure 8 , Figure 8 for Figure 7 The flowchart of an embodiment of step S12 specifically includes:

[0046] Step S121: sorting the capture time and capture location corresponding to the image in chronological order to generate sequence information.

[0047] Specifically, all images in the first cluster are sorted according to the time of capture. The sorted images have a certain continuity in time and can also form a certain order in terms of capture locations, thereby forming the time and space sequence information of a real natural person's travel.

[0048] In one embodiment, in order to efficiently utilize the existing association relationships, when sorting the capture time and capture location corresponding to the image in chronological order, the capture time and capture location corresponding to the image can be encoded with spatiotemporal information to obtain identifiers that represent the spatiotemporal relationship of the image, and the identifiers corresponding to the image can be sorted in chronological order to generate sequence information.

[0049] For details, see Figure 9-10 , Figure 9 for Figure 8 A flow chart of an embodiment of step S121 is shown in FIG. Figure 10 Schematic diagram of spatiotemporal information encoding in one embodiment of the present application.

[0050] like Figure 9 As shown, step S121 includes:

[0051] Step S1211: encoding the spatiotemporal information of the capture time and capture location corresponding to the image to obtain an identifier representing the spatiotemporal relationship of the image.

[0052] Specifically, the capture time and location of each image in the first cluster are combined and encoded. For example, the encoding scheme is: [Time(X), Area(Y)]=>U(Z). Here, Time(X) represents the capture time converted to hours, Area(Y) represents the capture location, and U(Z) is the unique identifier of this encoding scheme. In this way, a spatiotemporal information encoding dictionary is constructed, where the key value (keyword) is [Time(X), Area(Y)] and the value (return value) is U(Z). This facilitates spatiotemporal information conversion and can efficiently utilize existing association relationship methods.

[0053] In one embodiment, if Figure 10 As shown in the figure, it is assumed that there are three first-class clusters, and the first, second, and third first-class clusters are represented by circles, triangles, and squares respectively. Among them, the first first-class cluster has 5 pictures, which are represented by spatiotemporal information coding as [Time(7), Area(A)], [Time(8), Area(A)], [Time(9), Area(B)], [Time(16), Area(B)], and [Time(17), Area(A)]. The corresponding 5 different identifiers are U(1), U(2), U(3), U(4), and U(5). Similarly, the second first-class cluster has 5 pictures, which are represented by spatiotemporal information coding as [Time(8), Area(B)], [Time(9), Area(C)], [Time(10), Area(D)], [Time(18), Area(D)], and [Time(17), Area(A)]. The corresponding 5 different identifiers are U(6), U(7), U(8), U(9), and U(10). The third first-category cluster has 6 images, which are represented by spatiotemporal information encoding as [Time(8), Area(E)], [Time(8), Area(F)], [Time(9), Area(G)], [Time(17), Area(G)], [Time(17), Area(F)], and [Time(18), Area(E)]. The corresponding 6 different identifiers are U(11), U(12), U(13), U(14), U(15), and U(16).

[0054] Step S1212: sorting the identifiers corresponding to the images in chronological order to generate sequence information.

[0055] Specifically, sequence information is a unique identifier obtained by dividing the cluster information by day based on each first-class cluster and encoding it in chronological order. Therefore, after encoding the time and space information of a certain day of the three first-class clusters mentioned above, the corresponding sequence information can be obtained as follows:

[0056] The data in the first cluster of the first type consists of the following sequence information: U(1), U(2), U(3), U(4), U(5).

[0057] The data in the second first-category cluster consists of the following sequence information: U(6), U(7), U(8), U(9), U(10).

[0058] The data in the third first-category cluster consists of the following sequence information: U(11), U(12), U(13), U(14), U(15), U(16).

[0059] By aggregating information of multiple days for each first-class cluster, each first-class cluster will contain multiple pieces of useful sequence information.

[0060] Step S122: mining spatiotemporal association relationships based on sequence information using an association mining algorithm to obtain spatiotemporal association information.

[0061] Specifically, by setting a minimum support and a minimum confidence for each first-class cluster and then mining its associations, a unique identifier sequence combination encoding strongly correlated spatiotemporal information can be obtained, thereby improving the overall system utilization. The association mining algorithm used in step S122 can be an Apriori, FP-Growth, or other association mining method.

[0062] See Figure 11 , Figure 11 for Figure 8 Flowchart of an embodiment of step S122 in FIG. The step of mining spatiotemporal association relationships based on sequence information using an association mining algorithm to obtain spatiotemporal association information includes:

[0063] Step S1221: Use an association mining algorithm to mine spatiotemporal associations based on sequence information to obtain multiple strongly correlated identifiers.

[0064] Specifically, in one embodiment, Figure 10As shown, assuming that through association mining, we can obtain: the strongly correlated spatiotemporal information coding unique identifier sequence combinations in the first first-category cluster are U(2)—>U(3) and U(4)—>U(5). The strongly correlated spatiotemporal information coding unique identifier sequence combinations in the second first-category cluster are U(7)—>U(8) and U(9)—>U(10). The strongly correlated spatiotemporal information coding unique identifier sequence combinations in the third first-category cluster are U(11), U(12)—>U(13) and U(14), and U(15)—>U(16).

[0065] Step S1222: Decode multiple strongly correlated identifiers based on the mapping relationship of the spatiotemporal information encoding to obtain spatiotemporal correlation information.

[0066] Specifically, because the results directly obtained through association mining are not intuitive and not conducive to subsequent use, an inverse spatiotemporal information encoding dictionary can be obtained through the mapping relationships of the spatiotemporal information encoding dictionary, and the spatiotemporal relationship decoding of the obtained results can be performed. Furthermore, the mapping relationships of the spatiotemporal information encoding dictionary are used to swap the key and value values ​​of the dictionary to generate an inverse spatiotemporal information encoding dictionary.

[0067] In one embodiment, the strong spatiotemporal association rules obtained by decoding the association relationship mining through the inverse spatiotemporal information coding dictionary are as follows:

[0068] The first first-class cluster: [Time(8), Area(A)]—>[Time(9), Area(B)] and [Time(16), Area(B)]—>[Time(17), Area(A)]. That is, the real person represented by the first first-class cluster often appears at point A at 8 o'clock and moves to point B at 9 o'clock. In addition, the real person also often appears at point B at 16 o'clock and moves to point A at 17 o'clock.

[0069] Similarly, we can get the second first-class cluster: [Time(9), Area(C)]—>[Time(10), Area(D)] and [Time(18), Area(D)]—>[Time(19), Area(C)].

[0070] The third first-category cluster: [Time(8), Area(E)], [Time(8), Area(F)] -> [Time(9), Area(G)] and [Time(17), Area(G)], [Time(17), Area(F)] -> [Time(18), Area(E)]. That is, the real person represented by the third first-category cluster often appears at point E after 8 o'clock, then passes through point F, and moves to point G after 9 o'clock. In addition, they often appear at point G after 17 o'clock, then pass through point F, and move to point E at 18 o'clock.

[0071] Therefore, the inverse space-time information coding dictionary is obtained through the mapping relationship of the space-time information coding dictionary, and the space-time relationship decoding is performed on the obtained result to obtain the intuitive time and space travel information of real natural persons, which is convenient for improving the effect of the subsequent face clustering algorithm.

[0072] Step S13: obtaining the spatiotemporal domain information corresponding to the first cluster based on the spatiotemporal correlation information.

[0073] Specifically, the strong temporal and spatial correlation information obtained for each first cluster in step S1222 is used to generate exclusive temporal and spatial domain information for each first cluster. The temporal and spatial domain at this time can be defined as Time Area_N, for example: [Time(X1), Area(Y1); ...].

[0074] For example, in one embodiment, the first first-type cluster includes two exclusive spatiotemporal domain information, namely: TimeArea_1: [Time(8), Area(A); Time(9), Area(B)] and TimeArea_2: [Time(16), Area(B); Time(17), Area(A)].

[0075] The second first-category cluster contains two exclusive spatiotemporal domain information, namely: Time Area_1: [Time(9), Area(C); Time(10), Area(D)] and Time Area_2: [Time(18), Area(D); Time(19), Area(C)].

[0076] The third first-category cluster contains two exclusive spatiotemporal domain information, namely: Time Area_1: [Time(8), Area(E); Time(8), Area(F); Time(9), Area(G)] and Time Area_2: [Time(17), Area(G); Time(17), Area(F); Time(18), Area(E)].

[0077] Obviously, the Time Area_1 and Time Area_2 of the three first-class clusters are different. Therefore, it can be concluded that each first-class cluster has its own unique spatiotemporal information.

[0078] Before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the images corresponding to the first cluster, the method includes: in response to the capture time and capture location of multiple images being the same, retaining the capture time and capture location of only one image.

[0079] Specifically, if the images in a first-category cluster all involve the same capture time and the same capture location, then this first-category cluster only contains one spatiotemporal domain, that is, Time Area_1 is [Time(X1), Area(Y1)], where X1 is the only time point in the first-category cluster and Y1 is the only capture location in the first-category cluster.

[0080] In another embodiment, there may be a situation where the first cluster includes an image, and the capture time and capture location corresponding to the image are used as the spatiotemporal information corresponding to the first cluster.

[0081] Specifically, if a first-class cluster contains only one image, then the capture time and location information corresponding to this image is the only temporal and spatial information in this first-class cluster. In this case, the capture time and location information of this image are used as the spatiotemporal information of this first-class cluster. That is, the spatiotemporal information Time Area_1 of this first-class cluster is [Time(X1), Area(Y1)], where X1 is the only time point in this first-class cluster, and Y1 is the only capture location in this first-class cluster.

[0082] Specifically, in one embodiment, when determining the spatiotemporal information of the first cluster, the first cluster is screened to determine the spatiotemporal information for clusters containing at least two images whose capture times or locations differ. This can further improve system resource utilization. For clusters containing only one image, the capture time and location of that image can be directly determined as the spatiotemporal information.

[0083] The first cluster selected contains at least two images, which can facilitate the subsequent spatiotemporal correlation mining. If the first cluster contains only one image, then there is no need to mine its spatiotemporal correlation. Therefore, step S12 requires that the first cluster contain at least two images, which can also improve system resource utilization to a certain extent.

[0084] In addition, before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the image corresponding to the first cluster, the following steps are included:

[0085] The capturing time and the capturing location are divided into weekly cycles to obtain the capturing time and the corresponding capturing location in a first time range, and to obtain the capturing time and the corresponding capturing location in a second time range.

[0086] Specifically, dividing the capture time and capture location based on a weekly cycle means dividing a weekly cycle into two time ranges: a first time range is five days in a week, and a second time range is two days of a weekend.

[0087] In one embodiment, taking a week as an example, the travel trajectory of the same real natural person for five days in a week may be different from the travel trajectory for two days on the weekend. In step S12, after filtering out the required clustering information, data can be divided for each first-class cluster according to weekdays and weekends to obtain the weekday time-space domain and the weekend time-space domain. This can better reflect the daily travel habits of the real natural person corresponding to the first-class cluster.

[0088] See Figure 12 , Figure 12 This is a flow chart of an embodiment of the device for generating spatiotemporal information of clusters of the present application, which specifically includes: a cluster acquisition module 11 , a mining module 12 , and a spatiotemporal information acquisition module 13 .

[0089] The cluster acquisition module 11 is used to acquire a first cluster.

[0090] The mining module 12 is configured to obtain the spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the images in the first cluster.

[0091] Specifically, mining module 12 utilizes an association mining algorithm to mine spatiotemporal associations based on sequence information, obtaining multiple strongly correlated identifiers. Based on the mapping relationships of the spatiotemporal information encoding, these identifiers are decoded to obtain spatiotemporal association information. By using the mapping relationships of the spatiotemporal information encoding dictionary to obtain an inverse spatiotemporal information encoding dictionary, the resulting spatiotemporal relationship decoding is performed to obtain intuitive temporal and spatial travel information of real individuals, facilitating the improvement of the subsequent face clustering algorithm.

[0092] In one embodiment, the mining module 12 is used to sort the capture time and capture location corresponding to the image in chronological order to generate sequence information; and use the association mining algorithm to mine the spatiotemporal association relationship based on the sequence information to obtain the spatiotemporal association information.

[0093] In another embodiment, the mining module 12 is used to encode the spatiotemporal information of the capture time and capture location corresponding to the image to obtain an identifier representing the spatiotemporal relationship of the image; and sort the identifiers corresponding to the image in chronological order to generate sequence information.

[0094] In one embodiment, the mining module 12 mines spatiotemporal associations based on sequence information using an association mining algorithm to obtain multiple strongly correlated identifiers; and decodes the multiple strongly correlated identifiers based on a mapping relationship encoded by spatiotemporal information to obtain spatiotemporal association information.

[0095] In one embodiment, in response to the same capture time and capture location of multiple images, only the capture time and capture location of one of the images is retained.

[0096] In one embodiment, the capture time and capture location are divided into weekly periods to obtain the capture time and corresponding capture location in a first time range, and to obtain the capture time and corresponding capture location in a second time range.

[0097] The spatiotemporal information acquisition module 13 is configured to obtain the spatiotemporal information corresponding to the first cluster based on the spatiotemporal correlation information.

[0098] In one embodiment, when the first cluster includes an image, the spatiotemporal information acquisition module 13 uses the capture time and capture location corresponding to the image as the spatiotemporal information corresponding to the first cluster.

[0099] For example, the time-space information corresponding to each first-class cluster is represented as: Time Area_N: [Time(X1), Area(Y1); ...], where N is the sequence number of the time-space domain, X represents time, and Y represents location. Specifically, each time-space domain Time Area consists of multiple time-space points [Time(X1), Area(Y1)].

[0100] For example, suppose the spatiotemporal information of a first cluster is Time Area_1: [Time(8), Area(A); Time(9), Area(B)]. Its specific meaning is that the objects corresponding to the first cluster, that is, the real natural persons, are likely to depart from location A at 8 am and arrive at location B at 9 am. Therefore, it can be seen that the spatiotemporal information of the first cluster in step S1 actually reflects the daily travel habits of the real natural persons corresponding to the first cluster. The purpose of portrait clustering is to have one file for each person, that is, to associate and map all the snapshot images of each person with the ID card image of the person. The real natural person here means that after portrait clustering, each cluster is correctly associated and mapped with the corresponding ID card image, and the person corresponding to the ID card is the real natural person of the cluster. One file for each person means that a large number of portrait images are grouped into people, and all the snapshot images of each person are classified into a set. The images in the set can reflect the travel relationship of the corresponding real natural person. However, after image clustering, there is often a problem of multiple files for one person, which will make the corresponding travel relationship confusing and not be used for subsequent applications such as target tracking. In this regard, the present application merges multiple clusters according to the spatiotemporal information of the clustered clusters, thereby solving the problem of one person having multiple files and improving the accuracy of clustering.

[0101] Step S2: determining whether there is a spatiotemporal correlation between at least two first-type clusters based on the spatiotemporal domain information corresponding to each first-type cluster.

[0102] Specifically, after obtaining the spatiotemporal domain information corresponding to each first cluster, the correlation between each first cluster and the spatiotemporal point is arranged using an inverted index. The inverted index originates from the need to find records based on the value of an attribute in practical applications. Each item in this index table includes an attribute value and the address of each record with the attribute value. Since the attribute value is not determined by the record, but the location of the record is determined by the attribute value, it is called an inverted index. That is to say, all the spatiotemporal point identifiers involved are used as indexes, and the information connected by each spatiotemporal point index is the cluster information of the corresponding spatiotemporal domain containing the spatiotemporal point. Using different spatiotemporal domains as inverted indexes, each index is connected to the cluster information of the first cluster corresponding to the spatiotemporal domain. Among them, the first cluster information includes: cluster ID and the number of images in the cluster.

[0103] For example, suppose the first cluster 1 (ID is Cluster1) contains 5 images, and the corresponding spatiotemporal domain information is [Time(T1), Area(A); Time(T2), Area(B); Time(T3), Area(C)]. The first cluster 2 (ID is Cluster2) contains 6 images, and the corresponding spatiotemporal domain is [Time(T2), Area(A), Time(T2), Area(B); Time(T3), Area(C)]. The first cluster 3 (ID is Cluster3) contains 1 image, and the corresponding exclusive spatiotemporal domain is [Time(T1), Area(A)].

[0104] By organizing the correlations between the clusters and spatiotemporal points using an inverted index, we can expedite the search for clusters in similar spatiotemporal domains, thereby improving system efficiency. In one embodiment, step S2 only considers the spatiotemporal information corresponding to the first cluster, rather than the spatiotemporal information corresponding to each image within the cluster. This is primarily to focus on the primary spatiotemporal relationships within the cluster, avoid interference from noise signals, and improve system robustness.

[0105] For details, see Figure 2 , Figure 2 for Figure 1 The flowchart of an embodiment of step S2 specifically includes:

[0106] Step S21: Determine whether the spatiotemporal domain information corresponding to at least two first-type clusters has the same spatiotemporal point.

[0107] For details, please combine Figure 3 , Figure 3This is a schematic diagram of the relationship between whether the spatiotemporal information corresponding to at least two first clusters have the same spatiotemporal points using an inverted index. The relationship between the first cluster 1 (ID is Cluster1), the first cluster 2 (ID is Cluster2), and the first cluster 3 (ID is Cluster3) and the spatiotemporal points is arranged using an inverted index. Among them, the first cluster 1 (ID is Cluster1) and the first cluster 3 (ID is Cluster3) are related to the spatial point A at time T1; the first cluster 2 (ID is Cluster2) is related to the spatial point A at time T2; the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) are related to the spatial point B at time T2; the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) are related to the spatial point C at time T3. That is, the first cluster 1 (ID is Cluster1) and the first cluster 3 (ID is Cluster3) have the same time and space point (T1, A); the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have the same time and space point (T2, B); the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have the same time and space point (T3, C).

[0108] Step S22: In response to having the same time and space points, it is determined that there is a time and space association between at least two first-type clusters.

[0109] Specifically, the first cluster 1 (ID is Cluster1) and the first cluster 3 (ID is Cluster3) have the same time and space point (T1, A), and the first cluster 1 (ID is Cluster1) and the first cluster 3 (ID is Cluster3) have a time and space association; the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have the same time and space point (T2, B), and the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have a time and space association; the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have the same time and space point (T3, C), and the first cluster 1 (ID is Cluster1) and the first cluster 2 (ID is Cluster2) have a time and space association.

[0110] Step S3: In response to the spatiotemporal association between the at least two first-type clusters, clustering the at least two first-type clusters based on the similarity between the at least two first-type clusters.

[0111] Specifically, spatiotemporal information generally consists of multiple spatiotemporal points. If the first cluster contains only one image, its spatiotemporal information will only consist of one spatiotemporal point. Such a cluster will not be able to reflect the daily travel patterns of the real person corresponding to the cluster. In other words, in the above embodiment, the first cluster (with the ID Cluster3) containing only one image cannot reflect the daily travel patterns of the corresponding real person, so merging the first cluster (with the ID Cluster3) is not considered for the time being.

[0112] That is, in this embodiment, the first clusters that need to be merged based on similarity are first cluster 1 (ID: Cluster 1) and first cluster 2 (ID: Cluster 2). The time-space intersection of first cluster 1 (ID: Cluster 1) and first cluster 2 (ID: Cluster 2) is (Time: T2, Area: B) (Time: T3, Area: C). Based on the similarity between first cluster 1 and first cluster 2, it is determined whether to merge first cluster 1 with first cluster 2.

[0113] For details, please see Figure 4 , Figure 4 for Figure 1 The flowchart of an embodiment of step S3 specifically includes:

[0114] Step S31: Determine the spatiotemporal correlation coefficient between at least two first-type clusters.

[0115] Specifically, in this embodiment, the Jaccard coefficient is used to measure the spatiotemporal correlation coefficient between clusters.

[0116] Specifically, the intersection of the spatiotemporal domain information corresponding to at least two first-class clusters is calculated; and the union of the spatiotemporal domain information corresponding to at least two first-class clusters is calculated. Assume that the set of spatiotemporal points contained in the spatiotemporal domain information of the first-class cluster 1 is M, and the set of spatiotemporal points contained in the spatiotemporal domain information of the first-class cluster 2 is N. Then the intersection of the spatiotemporal domain information corresponding to at least two first-class clusters can be expressed as: |M∩N|. The union of the spatiotemporal domain information corresponding to at least two first-class clusters can be expressed as: |M∪N|. The spatiotemporal correlation coefficient is determined based on the intersection and the union. In one embodiment, the spatiotemporal correlation coefficient is expressed as α, α=|M∩N| / |M∪N|. By setting the spatiotemporal correlation coefficient, dynamic threshold portrait clustering optimization is achieved, which is more robust than the fixed threshold portrait clustering in the prior art.

[0117] Step S32: Determine a similarity threshold based on the spatiotemporal correlation coefficient.

[0118] Specifically, based on the determination of the above-mentioned spatiotemporal correlation coefficient α, the spatiotemporal correlation coefficient α can be used to dynamically adjust the similarity thresholds of at least two first-type clusters.

[0119] Specifically, the steps of determining the similarity threshold based on the spatiotemporal correlation coefficient include:

[0120] The similarity threshold is determined using a preset maximum similarity threshold, a preset minimum similarity threshold, and a spatiotemporal correlation coefficient.

[0121] For example, in one embodiment, the highest similarity threshold is defined as Sim_max and the lowest similarity threshold is defined as Sim_min, and the similarity threshold is:

[0122] Sim=Sim_max–(Sim_max-Sim_min)·α

[0123] Step S33: In response to the similarity between the at least two first-type clusters being greater than a similarity threshold, clustering the at least two first-type clusters.

[0124] Specifically, after obtaining the aforementioned similarity threshold Sim, the similarity between at least two first-class clusters needs to be compared with the similarity threshold Sim to determine whether the at least two first-class clusters can be clustered. Therefore, before determining whether the similarity between at least two first-class clusters is greater than the similarity threshold, the similarity between the two clusters needs to be calculated.

[0125] Specifically, the average centroids of at least two first clusters are calculated. For example, the average centroid of first cluster 1 is calculated using all centroids in first cluster 1, and the average centroid of first cluster 2 is calculated using all centroids in first cluster 2. The similarity between at least two first clusters is calculated based on the average centroids of the at least two first clusters. Specifically, after calculating the average centroids of first cluster 1 and first cluster 2, the similarity between first cluster 1 and first cluster 2 is calculated based on the average centroids of first cluster 1 and first cluster 2. The similarity between first cluster 1 and first cluster 2 is the distance between the average centroids of first cluster 1 and first cluster 2. Distance metrics include, but are not limited to, Euclidean distance and cosine distance. After determining the similarity between first cluster 1 and first cluster 2, the relationship between the similarity and a similarity threshold is determined. When the similarity is greater than the similarity threshold, the at least two first clusters are clustered, that is, first cluster 1 and first cluster 2 are clustered together. When the similarity is less than the similarity threshold, first cluster 1 and first cluster 2 are not clustered.

[0126] In one embodiment of the present application, in order to further improve the accuracy of clustering, it is necessary to determine the number of images in the first cluster. If the number of images in at least one of the at least two first clusters is less than a preset number, a first similarity threshold is determined based on the spatiotemporal correlation coefficient. In response to the similarity between the at least two first clusters being greater than the first similarity threshold, the at least two first clusters are clustered. In one embodiment, the preset number is 2. If the first cluster contains only a few 2 images, that is, only one image, then its spatiotemporal domain information can only be composed of one time and space point. Such a cluster is difficult to reflect the daily travel patterns of the real natural person corresponding to the cluster. That is, in the above embodiment, the first cluster (ID is Cluster3) with only one image cannot reflect the daily travel patterns of the corresponding real natural person. At this time, if the first cluster (ID is Cluster3) is to be merged, a larger first similarity threshold is set to improve accuracy.

[0127] In response to the number of images in at least two first clusters being greater than a preset number, a second similarity threshold is determined based on a spatiotemporal correlation coefficient; in response to the similarity between at least two first clusters being greater than the second similarity threshold, the at least two first clusters are clustered; and the first similarity threshold is greater than the second similarity threshold.

[0128] In the above embodiment, the first similarity threshold is greater than the second similarity threshold. That is, when the number of images to be processed is less than the number of images preset in this method, such as when there is only one image, a higher similarity standard needs to be set when determining the similarity between the image and the preset similarity threshold to ensure the accuracy of the clustered images.

[0129] In addition, the method for determining the first similarity threshold and the second similarity threshold here is the same as the method for determining the similarity threshold in step S32, and therefore will not be repeated here. In practice, the first similarity threshold and the second similarity threshold can be specifically set as needed, as long as the highest similarity threshold and the lowest similarity threshold are different from the first similarity threshold and the second similarity threshold. This application does not impose any restrictions on this.

[0130] The clustering method of the present application utilizes the exclusive spatiotemporal information of each cluster to further optimize the results of face clustering based on traditional image features. Specifically, a higher similarity threshold is set for portrait images in each cluster that are not in the exclusive spatiotemporal domain of the cluster, and then the erroneous images in the cluster are eliminated in a targeted manner, thereby improving the effect of face clustering. The spatiotemporal correlation coefficient is used to measure the spatiotemporal correlation between clusters, and the optimization of portrait clustering with dynamic thresholds is achieved, and then the optimization of portrait clustering with dynamic thresholds set according to spatiotemporal information is achieved. Compared with portrait clustering with fixed thresholds, it is more robust. The inverted index method is used to speed up the search for cluster information in similar spatiotemporal domains, thereby improving the efficiency of system operation. The present application can simultaneously and fully consider the influence of time and space on face clustering, and improve the accuracy of face clustering.

[0131] In practice, when determining whether there is a spatiotemporal correlation between at least two first-class clusters, it's common to find captured images that appear outside the spatiotemporal domain or overlap with that domain in either time or space. These images are likely anomalous. Therefore, it's necessary to increase the similarity threshold for image verification to correct for these anomalous images.

[0132] Specifically, such as Figure 5 As shown, Figure 5 This is a flow chart of the second embodiment of the clustering method of the present application. Before determining whether there is a spatiotemporal association between at least two first-class clusters based on the spatiotemporal domain information corresponding to each first-class cluster, the method further includes:

[0133] Step S4: determining whether the same time information and / or space information exists in the time-space domain information corresponding to each first cluster.

[0134] Specifically, statistics and comparison of time, space and position information are performed on the images in each first cluster to determine whether there is one or more images with the same time and / or space information.

[0135] Step S5: In response to the existence of the same time information and / or space information, an abnormal image is determined based on the same time information and / or space information.

[0136] Specifically, if there are one or more images with the same time and / or space information, then it can be determined that there must be an abnormal image in the image, that is, there is an erroneous image that does not belong to the first cluster, and the image is determined to be an abnormal image.

[0137] Step S6: remove abnormal images from the first cluster.

[0138] Specifically, after determining that a certain image is an abnormal image, it needs to be removed from the first cluster to resolve the abnormal situation and ensure that the images in the first cluster accurately reflect the daily travel habits of real natural persons.

[0139] For example, assume that the spatiotemporal domain of a cluster is Time Area_1: [Time(8), Area(A); Time(9), Area(B)]. At this time, the cluster contains a total of 10 images, and the spatiotemporal point corresponding to one of the images is [Time(3), Area(A)]. That is, the image was captured at point A at 3 a.m., which is obviously inconsistent with the spatiotemporal information of most images in the cluster. Therefore, this image is very likely not an image of the same person as the remaining 9 images in the archive.

[0140] By setting an appropriate intra-cluster similarity threshold, we determine whether the image and the remaining nine images within the cluster are images of the same person. If the image and the remaining nine images within the cluster are not images of the same person, the cluster is considered an incorrect cluster, and the image is an incorrect image and needs to be removed. By setting a higher similarity threshold for portrait images within each cluster that are not in the spatiotemporal domain of the cluster, we can specifically remove incorrect images from the cluster.

[0141] It should be noted that the erroneous image removed in step S6 needs to be individually set up in a new cluster, and the system needs to assign it a unique cluster ID. This image can be used as part of other clusters in the future, or it can always be the only image in the cluster.

[0142] Directly clustering portraits from massive amounts of snapshot data often yields poor results. This is because large amounts of portrait data are more likely to contain similar images than smaller amounts. Leveraging spatiotemporal information to group and cluster massive amounts of portrait data offers a new approach to improving clustering performance.

[0143] Each cluster derived from portrait clustering based on traditional image features can be quickly assigned a unique spatiotemporal domain through probabilistic statistics or association mining, thereby achieving true, fine-grained spatiotemporal domain segmentation from the perspective of each cluster. A spatiotemporal domain, composed of multiple spatial and temporal points, reflects the daily travel habits of the real people corresponding to that cluster.

[0144] By utilizing the unique spatiotemporal information of each cluster, we can complete the misalignment correction and multi-file merging in portrait clustering, thereby optimizing the effect of portrait clustering.

[0145] The clustering method of the present application utilizes the exclusive spatiotemporal information of each cluster to perform misalignment correction optimization for portrait clustering. Specifically, a higher similarity threshold is set for portrait images within each cluster that are not in the exclusive spatiotemporal domain of the cluster, thereby specifically eliminating erroneous images in the cluster. An inverted index is used to speed up the search for cluster information in similar spatiotemporal domains, thereby improving the operating efficiency of the system. By setting the spatiotemporal correlation coefficient, dynamic threshold portrait clustering optimization is achieved, which is more robust than fixed threshold portrait clustering.

[0146] The clustering method of the present application first determines whether multiple clusters have a spatiotemporal correlation based on spatiotemporal domain information, and determines whether to cluster the multiple clusters based on the similarity of the multiple clusters with a temporal correlation. This can solve the problem of one person having multiple files and improve the clustering accuracy.

[0147] See Figure 6 , is a structural diagram of an embodiment of the clustering device of the present application, which specifically includes: an acquisition module 41, a determination module 42 and a clustering module 43.

[0148] The acquisition module 41 is configured to acquire at least two first clusters and spatiotemporal information corresponding to each first cluster, where the spatiotemporal information represents a spatiotemporal position relationship of objects corresponding to the first clusters.

[0149] Furthermore, after obtaining the first cluster, the present application further obtains the spatiotemporal domain information corresponding to the first cluster, and the spatiotemporal domain information represents the spatiotemporal position relationship of the objects corresponding to the first cluster. For example, if the first cluster is the cluster of object A, then the spatiotemporal domain information represents the time and location relationship of the travel of object A. A spatiotemporal domain information is composed of multiple groups of fixed time and space locations, and has a time and space order. Specifically, when each image is taken, there will be a time and space geographical location corresponding to the image. When the facial feature information of the image is obtained through the clustering algorithm, the spatiotemporal domain information of the image can also be obtained at the same time.

[0150] The determining module 42 is configured to determine whether there is a spatiotemporal correlation between at least two first-type clusters based on the spatiotemporal domain information corresponding to each first-type cluster.

[0151] In one embodiment, determination module 42 determines the correlation between each first cluster and the spatiotemporal point using an inverted index based on the spatiotemporal information corresponding to each first cluster. Specifically, all involved spatiotemporal point identifiers are used as indices, and each spatiotemporal point index is connected to the cluster information corresponding to the spatiotemporal domain containing the spatiotemporal point. Using different spatiotemporal domains as inverted indices, each index is connected to the cluster information corresponding to the first cluster covering that spatiotemporal domain. The first cluster information includes the cluster ID and the number of images within the cluster.

[0152] By organizing the correlation between the above clusters and time-space points using the inverted index method, we can speed up the search for cluster information in similar time-space domains, thereby improving the system operation efficiency.

[0153] In one embodiment, the determination module 42 determines whether the spatiotemporal information corresponding to at least two first clusters has the same spatiotemporal point; in response to having the same spatiotemporal point, it determines that there is a spatiotemporal association between the at least two first clusters.

[0154] The clustering module 43 is configured to cluster the at least two first-type clusters based on the similarity between the at least two first-type clusters in response to the spatiotemporal association between the at least two first-type clusters.

[0155] In one embodiment, the clustering module 43 is configured to determine a spatiotemporal correlation coefficient between at least two first-class clusters; determine a similarity threshold based on the spatiotemporal correlation coefficient; and cluster the at least two first-class clusters in response to the similarity between the at least two first-class clusters being greater than the similarity threshold.

[0156] In one embodiment, the clustering module 43 calculates the intersection of the spatiotemporal information corresponding to at least two first clusters; calculates the union of the spatiotemporal information corresponding to at least two first clusters; and determines the spatiotemporal correlation coefficient based on the intersection and the union.

[0157] In one embodiment, the clustering module 43 determines the similarity threshold using a preset highest similarity threshold, a preset lowest similarity threshold, and a spatiotemporal correlation coefficient.

[0158] In one embodiment, the clustering module 43 calculates the average centroids of the at least two first clusters respectively; and calculates the similarity between the at least two first clusters based on the average centroids of the at least two first clusters.

[0159] Specifically, spatiotemporal information generally consists of multiple spatiotemporal points. If the first cluster contains only one image, its spatiotemporal information will only consist of one spatiotemporal point. Such a cluster will not be able to reflect the daily travel patterns of the real person corresponding to the cluster. In other words, in the above embodiment, the first cluster (with the ID Cluster3) containing only one image cannot reflect the daily travel patterns of the corresponding real person, so merging the first cluster (with the ID Cluster3) is not considered for the time being.

[0160] In this embodiment, the first clusters to be merged based on similarity are first cluster 1 (ID: Cluster 1) and first cluster 2 (ID: Cluster 2). The time-space intersection of first cluster 1 (ID: Cluster 1) and first cluster 2 (ID: Cluster 2) is (Time: T2, Area: B) (Time: T3, Area: C). Based on the similarity between first cluster 1 and first cluster 2, it is determined whether to merge first cluster 1 with first cluster 2.

[0161] In practice, when the number of images to be processed is less than the preset number of images, a higher similarity threshold standard needs to be set. For details, please refer to the content of step S32 above, which will not be repeated here.

[0162] The clustering device of the present application obtains at least two first-class clusters and the spatiotemporal information corresponding to each first-class cluster. Combining the spatiotemporal information corresponding to each first-class cluster, the device determines whether there is a spatiotemporal correlation between the at least two first-class clusters. If there is a spatiotemporal correlation, the at least two first-class clusters are clustered based on the similarity between the at least two first-class clusters. Using the spatiotemporal information of each cluster, the device determines whether there is a spatiotemporal correlation between different clusters, and further determines whether at least two clusters can be clustered. This solves the problem of one person having multiple files and improves clustering accuracy.

[0163] See Figure 13 , is a structural diagram of an embodiment of an electronic device of the present application, the electronic device includes a memory 202 and a processor 201 connected to each other.

[0164] The memory 202 is used to store program instructions for implementing any of the above-mentioned device methods.

[0165] The processor 201 is configured to execute program instructions stored in the memory 202 .

[0166] The processor 201 may also be referred to as a CPU (Central Processing Unit). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0167] The memory 202 can be a memory stick, a TF card, etc., which can store all the information in the electronic device of the device, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory. It stores and retrieves information according to the location specified by the controller. Only with memory can the electronic device have a memory function and ensure normal operation. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, system server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application.

[0172] See also Figure 14 , which is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The storage medium of the present application stores a program file 203 that can implement all the above methods, wherein the program file 203 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application. The aforementioned storage device includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0173] The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating spatiotemporal information of clusters, characterized in that: include: Get the first type of cluster; Based on the capture time and capture location of the images in the first cluster, obtaining spatiotemporal correlation information corresponding to the first cluster; Obtaining spatiotemporal domain information corresponding to the first cluster based on the spatiotemporal correlation information; Based on the spatiotemporal information corresponding to each first cluster, correcting an erroneous first cluster and / or merging multiple first clusters; The step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and the capture location of the images in the first cluster includes: Sorting the capture times and capture locations corresponding to the images in chronological order to generate sequence information; A correlation mining algorithm is used to mine spatiotemporal correlations based on the sequence information to obtain the spatiotemporal correlation information.

2. The method according to claim 1, characterized in that The step of sorting the capture times and capture locations corresponding to the images in chronological order to generate sequence information includes: Encoding the spatiotemporal information of the capture time and capture location corresponding to the image to obtain an identifier representing the spatiotemporal relationship of the image; The identifiers corresponding to the images are sorted in chronological order to generate the sequence information.

3. The method according to claim 2, characterized in that The step of mining the spatiotemporal association relationship based on the sequence information using the association mining algorithm to obtain the spatiotemporal association information includes: Using an association mining algorithm to mine spatiotemporal associations based on the sequence information, to obtain a plurality of strongly correlated identifiers; The multiple strongly correlated identifiers are decoded based on the mapping relationship of the spatiotemporal information encoding to obtain the spatiotemporal association information.

4. The method according to claim 1, wherein Before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and the capture location of the images in the first cluster, the method includes: In response to the capturing time and the capturing location of multiple images being the same, only the capturing time and the capturing location of one of the images is retained.

5. The method according to claim 4, characterized in that Before the step of obtaining the spatiotemporal correlation information corresponding to the first cluster based on the capture time and the capture location of the images in the first cluster, the method includes: The capturing time and the capturing location are divided into weekly periods to obtain the capturing time and the corresponding capturing location in a first time range, and to obtain the capturing time and the corresponding capturing location in a second time range.

6. The method according to claim 1, characterized in that The first cluster contains an image; The capture time and the capture location corresponding to the image are used as the spatiotemporal domain information corresponding to the first cluster.

7. A device for generating spatiotemporal information of clusters, characterized in that: include: A cluster acquisition module, used to acquire a first cluster; a mining module, configured to obtain spatiotemporal correlation information corresponding to the first cluster based on the capture time and capture location of the images in the first cluster; a spatiotemporal information acquisition module, configured to obtain spatiotemporal information corresponding to the first cluster based on the spatiotemporal correlation information, and to correct erroneous first clusters and / or merge multiple first clusters based on the spatiotemporal information corresponding to each first cluster; The mining module is further configured to sort the capture time and capture location corresponding to the image in chronological order to generate sequence information; A correlation mining algorithm is used to mine spatiotemporal correlations based on the sequence information to obtain the spatiotemporal correlation information.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A program file is stored, and the program file can be executed to implement the method according to any one of claims 1 to 6.

Images