A method and apparatus for clustering portraits

Abstract

Embodiments of the present application provide a portrait clustering method and device, comprising: determining different first portrait pictures and second portrait pictures from a portrait picture set, and determining picture similarity; the picture similarity is used to represent the similarity of portrait features and the similarity of space-time features; the portrait picture set comprises multiple portrait pictures collected by different lenses in a first period; based on the human flow prediction results of the corresponding lenses corresponding to the first portrait picture and the second portrait picture, the similarity threshold of portrait clustering is determined; different human flow prediction results correspond to different similarity thresholds; the human flow prediction result is obtained according to the real result of the human flow of the corresponding lens in a second period; the first period is after the second period, and the first period and the second period are adjacent in time and equal in length; if the picture similarity is higher than the similarity threshold, it is determined that the first portrait picture and the second portrait picture contain the same person. The above method is used to improve the accuracy of portrait clustering.

Description

Technical Field

[0001] The present application relates to the field of network technology, and in particular to a method and device for clustering human portraits. Background Art

[0002] By gradually building a comprehensive facial recognition system covering public areas, it is possible to capture facial snapshots of passersby from surveillance video captured by camera equipment. Using facial image clustering technology, the facial recognition system aggregates real-time facial snapshots to generate a portrait profile for each person (essentially, a collection of facial snapshots of the same person obtained through a clustering algorithm). This allows the construction of identity profiles based on these profiles, which can be used for user profiling and user flow monitoring. Identity profile compilation plays a crucial role in retail and surveillance scenarios.

[0003] Existing methods for clustering portrait images to obtain portrait profiles typically use the similarity between captured facial images for clustering. While this method can generate portrait profiles, in practice, occlusions such as masks and glasses, as well as occlusion between people's faces, can easily lead to clustering errors, resulting in low portrait profile accuracy.

[0004] Therefore, there is an urgent need for a portrait clustering method and device to improve the accuracy of portrait clustering. Summary of the Invention

[0005] Embodiments of the present invention provide a portrait clustering method and apparatus for improving the accuracy of portrait clustering.

[0006] In a first aspect, an embodiment of the present invention provides a portrait clustering method, the method comprising:

[0007] Determining a first portrait image and a second portrait image from the portrait image set;

[0008] Determining a picture similarity between the first portrait picture and the second portrait picture; the picture similarity is used to represent the similarity of portrait features and the similarity of spatiotemporal features; the portrait picture set includes multiple portrait pictures collected at different camera positions during a first time period, and the first portrait picture and the second portrait picture are two different portrait pictures in the portrait picture set;

[0009] Determining a similarity threshold for portrait clustering based on a first crowd flow prediction result corresponding to a first checkpoint at which the first portrait image was captured and a second crowd flow prediction result corresponding to a second checkpoint at which the second portrait image was captured; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained based on actual crowd flow results at the corresponding checkpoint during a second time period; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration;

[0010] In response to the picture similarity being higher than the similarity threshold, it is determined that the first portrait picture and the second portrait picture include the same person.

[0011] In the above method, a first portrait image and a second portrait image are determined from a portrait image set. The portrait image set can be captured images or video frames from a video. For example, the portrait image set can be portrait images captured during a first time period from surveillance videos from various checkpoint cameras. For the first and second portrait images in the portrait image set, a similarity is determined between the first and second portrait images. This similarity is used to characterize the similarity of portrait features and the similarity of spatiotemporal features. In this way, the similarity of the images includes both the similarity of portrait features and the similarity of spatiotemporal features. That is, the similarity of the images is a comprehensive similarity that takes into account both the spatiotemporal and portrait features of the two portrait images. A clustering result is determined based on the similarity between the image similarities and a similarity threshold corresponding to the crowd flow prediction result. This allows the clustering result to take into account spatiotemporal factors and the influence of different crowd flows on similarity, thereby improving the accuracy of the clustering result.

[0012] Optionally, the pedestrian flow prediction result is obtained based on the actual pedestrian flow result of the corresponding checkpoint in the second period, including:

[0013] Determine the actual result of the pedestrian flow at each checkpoint during the second period based on the portrait images during the second period;

[0014] The actual results of the pedestrian flow at each checkpoint in the second time period are input into the pedestrian flow prediction model to obtain the pedestrian flow prediction results of each checkpoint in the first time period; wherein, the pedestrian flow prediction model includes time-dependent parameters and space-dependent parameters; the time-dependent parameters are used to characterize the time granularity of pedestrian flow statistics; the space-dependent parameters are used to characterize the connectivity relationship between each checkpoint.

[0015] In the above method, the actual pedestrian flow results at each checkpoint during the second period are input into the pedestrian flow prediction model to obtain the pedestrian flow prediction results for each checkpoint during the first period. Because the pedestrian flow prediction model includes time-dependent parameters for characterizing the temporal granularity of pedestrian flow statistics and spatially dependent parameters for characterizing the connectivity between checkpoints, the pedestrian flow prediction results are obtained by comprehensively considering the optimal temporal granularity and the optimal spatial connectivity between checkpoints, improving the accuracy of the pedestrian flow prediction results. Furthermore, based on the spatiotemporal features derived from the pedestrian flow prediction results or the adjusted similarity threshold, more accurate clustering results can be obtained.

[0016] Optionally, the picture similarity is used to characterize the similarity of portrait features and the similarity of spatiotemporal features, including: determining the portrait feature similarity based on the first portrait picture and the second portrait picture;

[0017] Determining spatiotemporal features of the first portrait image based on the first pedestrian flow prediction result and the historical average pedestrian flow of the first checkpoint;

[0018] Determining the spatiotemporal characteristics of the second portrait image based on the second pedestrian flow prediction result and the historical average pedestrian flow of the second checkpoint;

[0019] determining a degree of temporal and spatial feature matching between the first portrait picture and the second portrait picture based on the temporal and spatial features of the first portrait picture and the temporal and spatial features of the second portrait picture;

[0020] The picture similarity is determined according to the portrait feature similarity and the spatiotemporal feature matching degree.

[0021] In the above method, the spatiotemporal characteristics of the first portrait image are determined based on the first crowd flow prediction result and the historical average crowd flow at the first checkpoint; the spatiotemporal characteristics of the second portrait image are determined based on the second crowd flow prediction result and the historical average crowd flow at the second checkpoint. In this way, the spatiotemporal characteristics of the first and second portrait images can be obtained. Furthermore, the spatiotemporal feature matching degree of the two portrait images is determined based on the spatiotemporal features of the two portrait images. This allows the spatiotemporal factors to be taken into account in the clustering result of the two portrait images determined based on the spatiotemporal matching degree, thereby improving the accuracy of the clustering result.

[0022] Optionally, determining the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree includes:

[0023] Obtaining, based on the portrait feature similarity, a probability that the first portrait in the first portrait picture and the second portrait in the second portrait picture are the same person and a probability that the second portrait is not the same person, wherein the same person probability is a probability that the first portrait picture and the second portrait picture contain the same person under the portrait feature similarity, and the sum of the same person probability and the not-the-same-person probability is 1;

[0024] The image similarity is determined according to the spatiotemporal feature matching degree, the same-person probability, and the non-same-person probability.

[0025] In the above method, based on the similarity of portrait features, the probability that the first and second portrait images contain the same person and the probability that they do not contain the same person are determined based on the similarity of the portrait features. In this way, accurate image similarity, including spatiotemporal features, can be obtained based on the spatiotemporal feature matching, the probability of not containing the same person, and the probability of containing the same person. The sum of the probability of not containing the same person and the probability of containing the same person is 1, and the probability of containing the same person can be equal to the portrait feature similarity.

[0026] Optionally, determining the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree includes:

[0027] The image similarity is determined by the following formula:

[0028]

[0029] Among them, prior represents the temporal and spatial feature matching degree; sim represents the similarity of portrait features, and prob represents the image similarity.

[0030] In the above method, the formula can accurately obtain the image similarity that takes into account the temporal and spatial factors.

[0031] Optionally, determining a similarity threshold for portrait clustering based on a first pedestrian flow prediction result of a first checkpoint where the first portrait image is located during the first time period and a second pedestrian flow prediction result of a second checkpoint where the second portrait image is located during the first time period includes:

[0032] Determining a pedestrian flow prediction mean based on the first pedestrian flow prediction result and the second pedestrian flow prediction result;

[0033] A similarity threshold for portrait clustering is determined based on the predicted mean of the crowd flow.

[0034] In the above method, the corresponding similarity threshold is obtained based on the average predicted value of the pedestrian flow. In this way, different similarity thresholds are corresponding to different average predicted values ​​of the pedestrian flow. This allows the similarity threshold to take into account the factors affecting the pedestrian flow, thereby improving the accuracy of the similarity threshold.

[0035] Optionally, each traffic prediction interval has a corresponding initial similarity threshold;

[0036] Determining a similarity threshold for portrait clustering based on the predicted mean of the crowd flow includes:

[0037] Obtaining an initial similarity threshold corresponding to the pedestrian flow prediction interval to which the pedestrian flow prediction mean belongs;

[0038] Determine a true mean of the number of people flow based on a first true result of the number of people flow at the first checkpoint during the first time period and a second true result of the number of people flow at the second checkpoint during the first time period;

[0039] According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain a similarity threshold for portrait clustering.

[0040] In the above method, the similarity threshold determined according to the predicted mean of pedestrian flow may have deviations. Therefore, the similarity threshold is readjusted according to the size and difference relationship between the actual pedestrian flow result and the predicted mean of pedestrian flow, so that the similarity threshold is closer to the actual similarity, thereby improving the accuracy of the clustering results.

[0041] Optionally, also include:

[0042] According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain the similarity threshold of the portrait clustering, including:

[0043] In response to the predicted mean of the human flow being higher than the actual mean of the human flow, increasing the initial similarity threshold;

[0044] In response to the predicted mean of the human flow being lower than the actual mean of the human flow, the initial similarity threshold is lowered.

[0045] In the above method, if the mean of the predicted flow rates for the two portrait images at the checkpoint is higher than the mean of the actual flow rates for the two portrait images at the checkpoint, it is considered that the occlusion probability of the two portrait images is increased, and the similarity between the two portrait images is likely to increase. Therefore, the similarity threshold should be increased accordingly to improve clustering accuracy. If the mean of the predicted flow rates for the two portrait images at the checkpoint is lower than the mean of the actual flow rates for the two portrait images at the checkpoint, it is considered that the occlusion probability of the two portrait images is decreased, and the similarity between the two portrait images is more accurate. Therefore, the similarity is likely to decrease. Therefore, the similarity threshold should be lowered accordingly to improve clustering accuracy.

[0046] In a second aspect, an embodiment of the present invention provides a portrait clustering device, the device comprising:

[0047] An acquisition module, configured to determine a first portrait image and a second portrait image from a portrait image collection;

[0048] a processing module, configured to determine a degree of similarity between the first portrait image and the second portrait image; the degree of similarity being used to characterize similarity in portrait features and similarity in spatiotemporal features; the portrait image collection comprising a plurality of portrait images collected at different camera positions during a first period of time, the first portrait image and the second portrait image being two different portrait images in the portrait image collection;

[0049] The processing module is further configured to determine a similarity threshold for portrait clustering based on a first crowd flow prediction result corresponding to a first checkpoint at which the first portrait image was captured and a second crowd flow prediction result corresponding to a second checkpoint at which the second portrait image was captured; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained based on actual crowd flow results at the corresponding checkpoint during a second time period; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration;

[0050] The processing module is further configured to, in response to the picture similarity being higher than the similarity threshold, determine that the first portrait picture and the second portrait picture include the same person.

[0051] In a third aspect, an embodiment of the present application further provides a computing device, comprising: a memory for storing programs; a processor for calling the programs stored in the memory, and executing the methods described in the various possible designs of the first aspect according to the obtained programs.

[0052] In a fourth aspect, an embodiment of the present application further provides a computer-readable non-volatile storage medium, comprising a computer-readable program. When a computer reads and executes the computer-readable program, the computer executes the method described in the various possible designs of the first aspect.

[0053] These implementations or other implementations of the present application will be more concise and understandable in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A schematic diagram of the architecture of portrait clustering provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the system architecture of a pedestrian flow prediction module provided by an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of a process flow of a portrait clustering method provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of a process flow of a portrait clustering method provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of a process flow of a portrait clustering method provided by an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of a portrait clustering device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0062] Figure 1 An embodiment of the present invention provides a system architecture for portrait clustering, wherein an image acquisition module 101 is configured to acquire portrait images from surveillance videos of each camera during a first time period and transmit the acquired portrait images to a similarity comparison module 102. Similarity comparison module 102 calculates the similarity of portrait features between any two portrait images. The similarity of portrait features between the two portrait images can be calculated by, for example, calculating the cosine similarity or Euclidean distance between the images. The specific method for calculating the similarity of portrait features between the two portrait images is not limited herein.

[0063] There are two ways for the similarity comparison module 102 to determine whether two portrait images can be clustered based on the similarity of the portrait features of the two portrait images:

[0064] One is: the similarity comparison module 102 determines the crowd flow prediction results corresponding to the two portrait pictures respectively from the crowd flow prediction module 104 according to the checkpoints and picture shooting time periods corresponding to the two portrait pictures (the crowd flow prediction results corresponding to the two portrait pictures are obtained from the predicted monitoring video feature matrix of the first time period based on the checkpoints and picture shooting time periods corresponding to the two portrait pictures, and the predicted monitoring video feature matrix of the first time period includes the crowd flow prediction results corresponding to each checkpoint at each time point in the first time period. The predicted monitoring video feature matrix of the first time period is obtained by inputting the real monitoring video feature matrix of the second time period (the real monitoring video feature matrix of the second time period includes the real crowd flow results corresponding to each checkpoint at each time point in the second time period) into the crowd flow prediction model. The first time period is after the second time period, and the first time period is adjacent to the second time period and the time periods are equal. In one example In the example, the prediction method may be to obtain the surveillance video feature matrix of the previous sliding window through a sliding window method to predict the surveillance video feature matrix of the next sliding window, wherein the surveillance video feature matrix of the next sliding window includes the predicted pedestrian flow value of the corresponding time period of the checkpoint. The historical average pedestrian flow of the two checkpoints corresponding to the two portrait images is obtained respectively (here, the historical average pedestrian flow of the two checkpoints corresponding to the two portrait images can be set in the similarity comparison module 102 or in the pedestrian flow prediction module 104, without limitation). The ratio of the pedestrian flow prediction results corresponding to the two portrait images to the historical average pedestrian flow of the corresponding checkpoints is calculated respectively. The ratios corresponding to the two portrait images are averaged to obtain the mean of the ratios. The mean is determined as the spatiotemporal feature matching degree. The spatiotemporal feature matching degree and the portrait feature similarity are input into the comprehensive feature similarity calculation formula to determine the comprehensive feature similarity of the two portrait images. The similarity comparison module 102 compares the comprehensive feature similarity with the similarity threshold corresponding to the pedestrian flow interval to which the pedestrian flow prediction results corresponding to the two portrait images belong. Here, each similarity threshold corresponds to a different range of pedestrian flow values. The pedestrian flow value ranges and corresponding similarity thresholds can be obtained based on a large amount of data statistics or determined based on professional experience, and there is no specific limitation here. In this way, the corresponding similarity thresholds can be set based on the pedestrian flow values, taking into account the impact of pedestrian flow on similarity in the comparison, thereby improving the accuracy of the clustering results.

[0065] If the comprehensive feature similarity is greater than the similarity threshold, it is determined that the portraits in the two portrait pictures belong to the same person, and the clustering module 103 clusters the portraits in the two portrait pictures. Otherwise, the portraits in the two portrait pictures do not belong to the same person, and accordingly, the clustering module 103 does not cluster the portraits in the two portrait pictures.

[0066] Another method: the similarity comparison module 102 determines the mean of the crowd flow prediction results corresponding to the two portrait images and the mean of the actual crowd flow corresponding to the two portrait images, and compares the mean of the crowd flow prediction results corresponding to the two portrait images with the mean of the actual crowd flow;

[0067] If the mean of the predicted flow rate is higher than the mean of the actual flow rate, the similarity threshold is increased;

[0068] If the similarity of the portrait features of the two portrait pictures is greater than the adjusted similarity threshold, it is determined that the portraits in the two portrait pictures belong to the same person, and the clustering module 103 clusters the portraits in the two portrait pictures. Otherwise, the portraits in the two portrait pictures do not belong to the same person, and accordingly, the clustering module 103 does not cluster the portraits in the two portrait pictures.

[0069] If the mean of the predicted result of the flow of people is lower than the mean of the actual flow of people minus the actual mean of the flow of people, the similarity threshold is lowered;

[0070] If the portrait feature similarity is greater than the lowered similarity threshold, it is determined that the portraits in the two portrait pictures belong to the same person. Similarly, the clustering module 103 clusters the portraits in the two portrait pictures. Otherwise, the portraits in the two portrait pictures do not belong to the same person. Accordingly, the clustering module 103 does not cluster the portraits in the two portrait pictures.

[0071] If the mean of the predicted flow rate is equal to the mean of the actual flow rate, the similarity threshold remains unchanged;

[0072] If the similarity of the portrait features of the two portrait pictures is greater than the similarity threshold, it is determined that the portraits in the two portrait pictures belong to the same person, and the clustering module 103 clusters the portraits in the two portrait pictures. Otherwise, the portraits in the two portrait pictures do not belong to the same person, and accordingly, the clustering module 103 does not cluster the portraits in the two portrait pictures.

[0073] Here, the extent to which the similarity threshold is lowered or raised can be determined according to the difference between the mean of the crowd flow prediction result and the mean of the actual crowd flow, and there is no specific restriction here.

[0074] The embodiment of the present invention provides a system architecture of a pedestrian flow prediction module, such as Figure 2As shown, the crowd flow prediction module 104 includes a crowd flow prediction model, which includes a space-dependent unit and a time-dependent unit. The space-dependent unit includes the distance relationship between each checkpoint, which is reflected by the number of steps a person walks (which can be reflected by the checkpoint adjacency matrix between each checkpoint). The time-dependent unit includes the time granularity taken for crowd flow prediction, such as predicting the crowd flow every hour with an hour as the time granularity. The checkpoint adjacency matrix in the space-dependent unit and the time granularity in the time-dependent unit are obtained based on a large amount of data training.

[0075] For example, first, an initial checkpoint adjacency matrix is ​​obtained. This can be accomplished by having a target person walk through each checkpoint within a set timeframe, and obtaining snapshots of the target person at each checkpoint. Since there are edge relationships between checkpoints that the target person passes through consecutively, that is, these edges can be considered the target person's path, the associations between checkpoints can be derived based on these edge relationships. The actual distance between checkpoints, calculated by mapping the target person's longitude and latitude to a map, can be used to determine the number of steps the target person took through the two checkpoints. Based on the number of steps between checkpoints and the associations between checkpoints, the initial checkpoint adjacency matrix can be derived.

[0076] Obtain ten days of surveillance data and determine the feature matrix of the surveillance video for the first 12 hours and the last 12 hours of each day. The feature matrix includes the checkpoint identifier and the hourly pedestrian flow at the checkpoint. In one example, the feature matrix is ​​as follows:

[0077]

[0078] Among them, the checkpoints in the first column are marked as 1, 2, and 3, which correspond to checkpoints 1, 2, and 3 respectively. The passenger flow of checkpoint 1 from 00:00 to 01:00 is 0 / h, the passenger flow from 01:00 to 02:00 is 2 / h, the passenger flow from 02:00 to 03:00 is 6 / h, the passenger flow from 03:00 to 04:00 is 1 / h, the passenger flow from 04:00 to 05:00 is 3 / h, and the passenger flow from 05:00 to 06:00 is 1 / h. The passenger flow from 00:00 to 06:00 is 2 / h, the passenger flow from 06:00 to 07:00 is 20 / h, the passenger flow from 07:00 to 08:00 is 23 / h, the passenger flow from 08:00 to 09:00 is 30 / h, the passenger flow from 09:00 to 10:00 is 10 / h, the passenger flow from 010:00 to 11:00 is 2 / h, and the passenger flow from 11:00 to 12:00 is 20 / h.

[0079] The passenger flow at checkpoint 2 from 00:00 to 01:00 is 0 / h, from 01:00 to 02:00 is 5 / h, from 02:00 to 03:00 is 7 / h... and from 11:00 to 12:00 is 14 / h.

[0080] The passenger flow at checkpoint 3 from 00:00 to 01:00 is 0 / h, from 01:00 to 02:00 is 3 / h, from 02:00 to 03:00 is 7 / h... and from 11:00 to 12:00 is 52 / h.

[0081] Then, the initial checkpoint adjacency matrix of each checkpoint is input into the pedestrian flow prediction model to be trained. As described above, the checkpoint adjacency matrix can be used to represent the number of steps people walk between checkpoints.

[0082] In the above example, the surveillance video feature matrices for the first 12 hours and the last 12 hours of each day are used as samples. The first 12 hours' surveillance video feature matrix is ​​input into the crowd flow prediction model to be trained, and the predicted surveillance video feature matrix for the last 12 hours is output by the trained crowd flow prediction model. Based on the difference between the predicted surveillance video feature matrix for the last 12 hours and the actual surveillance video feature matrix for the last 12 hours, the checkpoint adjacency matrix and crowd flow prediction-related parameters in the trained crowd flow prediction model are adjusted to obtain a trained crowd flow prediction model. The samples here can also be the surveillance video feature matrix for the last 12 hours of the previous day and the first 12 hours of the next day, or the surveillance video feature matrix for the first hour (the crowd flow in this feature matrix is ​​calculated at a time granularity of 5 minutes) and the surveillance video feature matrix for the last hour. In this way, by training the crowd flow prediction model according to different time granularities, the optimal time granularity that reflects the changing characteristics of crowd flow can be obtained, the accuracy of the similarity threshold determination can be improved, and the accuracy of the clustering results can be further improved.

[0083] Based on the above system architecture, the embodiment of the present application provides a portrait clustering method process, such as Figure 3 Shown, including:

[0084] Step 301: Determine a first portrait image and a second portrait image from a portrait image collection;

[0085] Step 302: Determine the image similarity between the first portrait image and the second portrait image; the image similarity is used to represent the similarity of portrait features and the similarity of spatiotemporal features; the portrait image collection includes multiple portrait images collected at different camera positions during a first period of time, and the first portrait image and the second portrait image are two different portrait images in the portrait image collection;

[0086] Step 303: Determine a similarity threshold for portrait clustering based on a first crowd flow prediction result corresponding to a first checkpoint at which the first portrait image was captured and a second crowd flow prediction result corresponding to a second checkpoint at which the second portrait image was captured; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained based on actual crowd flow results at the corresponding checkpoint during a second time period; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration;

[0087] Step 304: In response to the picture similarity being higher than the similarity threshold, determine that the first portrait picture and the second portrait picture contain the same person.

[0088] In the above method, a first portrait image and a second portrait image are determined from a portrait image set. The portrait image set can be captured images or video frames from a video. For example, the portrait image set can be portrait images captured during a first time period from surveillance videos captured by various checkpoint cameras. For the first and second portrait images in the portrait image set, a similarity is determined between the first and second portrait images. This similarity is used to characterize the similarity of portrait features and the similarity of spatiotemporal features. In this way, the similarity of the images includes both the similarity of portrait features and the similarity of spatiotemporal features. That is, the similarity of the images is a comprehensive similarity that takes into account both the spatiotemporal and portrait features of the two portrait images. A clustering result is determined based on the image similarity and a similarity threshold corresponding to the crowd flow prediction result. This allows the clustering result to take into account spatiotemporal factors and the influence of different crowd flows on similarity, thereby improving the accuracy of the clustering result.

[0089] An embodiment of the present invention provides a method for predicting pedestrian flow, wherein a pedestrian flow prediction result is obtained based on actual pedestrian flow results at a corresponding checkpoint in a second period, including:

[0090] Determine the actual result of the pedestrian flow at each checkpoint during the second period based on the portrait images during the second period;

[0091] The actual pedestrian flow results at each checkpoint during the second period are input into the pedestrian flow prediction model to obtain the predicted pedestrian flow results for each checkpoint during the first period. The pedestrian flow prediction model includes time-dependent parameters and spatial-dependent parameters. The time-dependent parameters are used to characterize the temporal granularity of pedestrian flow statistics, and the spatial-dependent parameters are used to characterize the connectivity between checkpoints. The actual pedestrian flow results at each checkpoint during the second period can be a surveillance video feature matrix of the surveillance video at each checkpoint during the second period at the corresponding temporal granularity parameter (the temporal granularity parameter in the pedestrian flow prediction model). This surveillance video feature matrix contains the pedestrian flow at each checkpoint at the temporal granularity during the second period. For example, if the temporal granularity parameter is 5 minutes and the second period is 1 hour long, the surveillance video feature matrix contains the pedestrian flow at each checkpoint during the periods xx:00-xx:05, xx:05-xx:10, xx:10-xx:15, and so on. The method for obtaining the passenger flow every 5 minutes can be based on a video flow algorithm. The video flow algorithm can be a video flow algorithm such as Yolov5+deepsort. There are no specific restrictions on the setting of the video flow algorithm. A custom video flow algorithm (written by yourself) can also be used. The spatial dependency parameter can be a checkpoint adjacency matrix. The checkpoint adjacency matrix is ​​used to represent the number of steps between checkpoints. It can be used to represent the connectivity between checkpoints and the time and speed of people walking between checkpoints.

[0092] An embodiment of the present invention provides a method for determining image similarity, wherein the image similarity is used to characterize the similarity of portrait features and the similarity of spatiotemporal features, including:

[0093] determining a similarity of portrait features based on the first portrait picture and the second portrait picture;

[0094] Determining spatiotemporal features of the first portrait image based on the first pedestrian flow prediction result and the historical average pedestrian flow of the first checkpoint;

[0095] Determining the spatiotemporal characteristics of the second portrait image based on the second pedestrian flow prediction result and the historical average pedestrian flow of the second checkpoint;

[0096] determining a degree of temporal and spatial feature matching between the first portrait picture and the second portrait picture based on the temporal and spatial features of the first portrait picture and the temporal and spatial features of the second portrait picture;

[0097] The picture similarity is determined according to the portrait feature similarity and the spatiotemporal feature matching degree.

[0098] Here, spatiotemporal features are obtained based on the first and second crowd flow prediction results for the first and second portrait images, as well as the historical average crowd flow at the checkpoints corresponding to the two portrait images. That is, the spatiotemporal features are incorporated into the crowd flow prediction results (the crowd flow prediction model includes a spatial dependency parameter—the checkpoint adjacency matrix—and a temporal dependency parameter—the time granularity parameter). This allows the similarity of the portrait features of the two portrait images to be corrected based on their spatiotemporal matching, resulting in a comprehensive similarity—the image similarity. In other words, by considering spatiotemporal factors in image similarity, the accuracy of the clustering results is improved.

[0099] An embodiment of the present invention provides a method for determining image similarity, which determines the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree, including:

[0100] Based on the portrait feature similarity, a probability of the first person in the first portrait image and a probability of the second person in the second portrait image being the same person are obtained, where the probability of the first person in the first portrait image and the second person in the second portrait image contain the same person at the portrait feature similarity, and the sum of the probability of the same person and the probability of the second person being the same person is 1. The image similarity is determined based on the spatiotemporal feature matching, the probability of the same person, and the probability of the second person being the same person. That is, based on the portrait feature similarity, the probability of the first person in the second portrait image and the second person in the second portrait image contain the same person at the portrait feature similarity is determined - the probability of the same person and the probability of the second person being the same person. In this way, image similarity is determined based on the spatiotemporal feature matching, the probability of the same person, and the probability of the second person being the same person, thereby improving the accuracy of image similarity and further improving the accuracy of the determination of whether the first and second portrait images contain the same person. In one example, the probability of the same person can be equal to the portrait feature similarity. Alternatively, because other factors generally affect the accuracy of calculating portrait feature similarity, the probability of the same person can be adjusted lower or higher accordingly. For example, if the image frame clarity of the surveillance videos of each checkpoint is low, resulting in a high similarity of portrait features, the corresponding ratio can be set to reduce the probability of the same person corresponding to the similarity of the portrait features.

[0101] An embodiment of the present invention provides a method for determining image similarity, which determines the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree, including:

[0102] The image similarity is determined by the following formula:

[0103]

[0104] Among them, prior represents the temporal and spatial feature matching degree; sim represents the similarity of portrait features, and prob represents the image similarity.

[0105] An embodiment of the present invention provides a method for determining a similarity threshold, which determines a similarity threshold for a portrait cluster based on a first pedestrian flow prediction result at a first checkpoint where a first portrait image is located during a first time period and a second pedestrian flow prediction result at a second checkpoint where a second portrait image is located during the first time period, including:

[0106] Determining a pedestrian flow prediction mean based on the first pedestrian flow prediction result and the second pedestrian flow prediction result;

[0107] Based on the predicted mean of the crowd flow, a similarity threshold for portrait clustering is determined. Here, the similarity threshold for portrait clustering is determined based on the mean of the crowd flow threshold. That is, different crowd flows have different similarity thresholds. This allows the clustering process to consider the impact of crowd flow on similarity results. This impact of crowd flow on similarity is incorporated into the clustering process through the similarity threshold, improving the accuracy of the clustering results.

[0108] The embodiment of the present invention provides a similarity threshold determination method, where each traffic flow prediction interval has a corresponding initial similarity threshold;

[0109] Determining a similarity threshold for portrait clustering based on the predicted mean of the crowd flow includes:

[0110] Obtaining an initial similarity threshold corresponding to the pedestrian flow prediction interval to which the pedestrian flow prediction mean belongs;

[0111] Determine a true mean of the number of people flow based on a first true result of the number of people flow at the first checkpoint during the first time period and a second true result of the number of people flow at the second checkpoint during the first time period;

[0112] Based on the relationship between the predicted mean and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain a similarity threshold for portrait clustering. In other words, the initial similarity threshold is not only determined based on the predicted mean of the crowd flow, but is also modified based on the actual mean of the crowd flow to obtain a similarity threshold for portrait clustering. This increases the weighting of the impact of actual crowd flow on similarity and improves the accuracy of clustering results.

[0113] An embodiment of the present invention provides a method for determining a similarity threshold, further comprising:

[0114] According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain the similarity threshold of the portrait clustering, including:

[0115] In response to the predicted mean of the human flow being higher than the actual mean of the human flow, increasing the initial similarity threshold;

[0116] In response to the predicted mean of human flow being lower than the actual mean of human flow, the initial similarity threshold is lowered. That is to say, because the greater the human flow, the greater the probability of occlusion, the easier it is to cause the similarity between the two portrait images to increase. Therefore, if the predicted mean of human flow is higher than the actual mean of human flow, it means that the initial similarity threshold is set low, and the initial similarity threshold needs to be raised, otherwise, it needs to be lowered. If the predicted mean of human flow is equal to the actual mean of human flow, it means that the initial similarity threshold corresponding to the predicted mean of human flow is accurate and does not need to be adjusted. An embodiment of the present invention provides a human flow prediction model, which is a graph convolutional layer model or a recursive neural network model + a convolutional neural network model.

[0117] Based on the above method flow, the embodiment of the present application provides a process of a portrait clustering method, such as Figure 4 Shown, including:

[0118] Step 401: Obtain a checkpoint adjacency matrix, and input the checkpoint adjacency matrix into a pedestrian flow prediction model to be trained.

[0119] Here, the distance between checkpoints is represented by the number of steps people walk between each checkpoint in the checkpoint adjacency matrix. Compared with directly using the actual distance between checkpoints to represent the distance between checkpoints, the relationship between people's walking time and distance can be integrated, thereby improving the accuracy of the pedestrian flow prediction results.

[0120] Step 402: Obtain historical surveillance videos and generate sample data based on the historical surveillance videos.

[0121] Here, the sample data is a surveillance video feature matrix for each time period. The length of each time period can be 5 minutes, 10 minutes, 15 minutes...1 hour, 2 hours...12 hours, etc. There is no specific restriction on the length of the time period.

[0122] The surveillance video feature matrix contains the checkpoint ID and the foot traffic at that checkpoint during the corresponding time period and in each sub-period. For example, checkpoint 1 corresponds to the time period 00:00-00:10, and each sub-period within 00:00-00:10 is 00:00-00:05, 00:05-00:10, 00:10-00:15, and 00:55-00:60, with a time period of 1 hour and a time granularity of 5 minutes.

[0123] Or the time period corresponding to checkpoint 1 is 00:00-12:00, and each sub-time period in 00:00-12:00 is 00:00-01:00, 01:00-02:00, 02:00-03:00...11:00-12:00 - the time period is 12 hours and the time granularity is 1 hour.

[0124] Here, the surveillance video feature matrix corresponding to each time period can be obtained for each checkpoint as sample data, such as the surveillance video feature matrix for the 1-hour time period and the surveillance video feature matrix for the 12-hour time period mentioned above (note that in order to ensure that the amount of sample data is equal, when the time period is 1 hour, the time granularity is 5 minutes, and when the time period is 12 hours, the time granularity is 1 hour, so as to ensure that the number of pedestrian flow data for each checkpoint in the sample data is 12). The surveillance video feature matrix of each time period is input into the pedestrian flow prediction model to be trained, and the optimal time granularity that best reflects the characteristics of pedestrian flow changes can be trained (in the above example, the time granularity is 5 minutes or 1 hour), so that when the trained pedestrian flow prediction model is applied, the surveillance video feature matrix can be determined according to the optimal time granularity (in the above example, the optimal time granularity can be 5 minutes or 1 hour).

[0125] Step 403: Input the sample data into the pedestrian flow prediction model to be trained to train the pedestrian flow prediction model to obtain a maturely trained pedestrian flow prediction model.

[0126] Here, the sample data includes the surveillance video feature matrix for the previous period and the (real) surveillance video feature matrix for the next period. The surveillance video feature matrix for the previous period is input into the crowd flow prediction model to be trained to obtain the predicted surveillance video feature matrix for the next period. Based on the difference between the real surveillance video feature matrix for the next period and the predicted surveillance video feature matrix for the next period, the checkpoint adjacency matrix and crowd flow-related parameters in the crowd flow prediction model to be trained are adjusted to obtain a mature crowd flow prediction model.

[0127] Step 404: Acquire the surveillance video of the second period, and acquire the predicted surveillance video feature matrix of the first period based on the surveillance video feature matrix of the second period.

[0128] Step 405: Acquire surveillance video of a first time period, where the first time period is after the second time period.

[0129] Step 406: Determine any two portrait images in the surveillance video of the first period, and calculate the similarity of portrait features of the two portrait images.

[0130] Step 407: Determine the crowd flow prediction results corresponding to the two portrait images respectively according to the predicted surveillance video feature matrix of the first time period and the camera positions and picture shooting time periods of the two portrait images.

[0131] Step 408: Obtain the historical average pedestrian flow of the two checkpoints corresponding to the two portrait images respectively.

[0132] Step 409: Determine the ratio of the crowd flow prediction results corresponding to the two portrait images to the historical average crowd flow of the corresponding checkpoints.

[0133] Step 410: Obtain the average of the ratios corresponding to the two portrait images.

[0134] Step 411: Determine the mean as the spatiotemporal feature matching degree, and determine the overall feature similarity - image similarity based on the spatiotemporal feature matching degree and the portrait feature similarity.

[0135] Step 412: Determine a similarity threshold based on the mean of the crowd flow prediction results of the two portrait images.

[0136] Here, the crowd flow prediction results in different data intervals correspond to different similarity thresholds.

[0137] For example, if the flow rate falls within the range [0, 50], the corresponding similarity threshold is 80%, and if the flow rate falls within the range [51, 100], the corresponding similarity threshold is 90%. Similarity thresholds corresponding to the flow rate value range can be obtained based on a large amount of data statistics or professional experience, and are not limited here.

[0138] Step 413: Whether the overall feature similarity is greater than the similarity threshold, if so, execute step 414; if not, execute step 415.

[0139] Step 414: cluster the two portrait images.

[0140] Step 415: Do not cluster the two portrait images.

[0141] It should be noted that the above method process is not unique. For example, steps 401 to 403 are the training process of the human flow prediction model, which does not need to be executed every time portrait clustering is performed. The above portrait clustering process is only an example and does not limit the specific implementation of the portrait clustering process.

[0142] Based on the above method flow, the embodiment of the present application further provides a process of a portrait clustering method, such as Figure 5 Shown, including:

[0143] Step 501: Obtain a checkpoint adjacency matrix, and input the checkpoint adjacency matrix into a pedestrian flow prediction model to be trained.

[0144] Step 502: Obtain historical surveillance videos and generate sample data based on the historical surveillance videos.

[0145] Step 503: Input the sample data into the pedestrian flow prediction model to be trained to train the pedestrian flow prediction model to obtain a maturely trained pedestrian flow prediction model.

[0146] Step 504: Obtain the surveillance video of the second time period, and obtain the predicted surveillance video feature matrix of the first time period based on the surveillance video of the second time period.

[0147] Step 505: Acquire surveillance video of a first time period, where the first time period is after the second time period.

[0148] Step 506: Determine any two portrait images in the surveillance video of the first period, and calculate the similarity of portrait features of the two portrait images.

[0149] Step 507: Determine the crowd flow prediction results corresponding to the two portrait images respectively according to the predicted surveillance video feature matrix of the first time period and the camera positions and picture shooting time periods of the two portrait images.

[0150] Step 508: Determine the mean of the crowd flow prediction results corresponding to the two portrait images and the actual mean of the crowd flow.

[0151] Step 509: Compare the mean of the crowd flow prediction results corresponding to the two portrait images with the actual mean of the crowd flow. If the mean of the crowd flow prediction results is higher than the actual mean of the crowd flow, proceed to step 510. If the mean of the crowd flow prediction results is lower than the actual mean of the crowd flow, proceed to step 511. If the mean of the crowd flow prediction results is equal to the actual mean of the crowd flow, proceed to step 512.

[0152] Step 510: Increase the similarity threshold.

[0153] Step 511: Lower the similarity threshold.

[0154] Step 512: The similarity threshold remains unchanged.

[0155] Step 513: Check whether the portrait feature similarity is greater than the adjusted similarity threshold. If yes, proceed to step 514; if no, proceed to step 515.

[0156] Step 514: cluster the two portrait images.

[0157] Step 515: Do not cluster the two portrait images.

[0158] It should be noted that the above method process is not unique. For example, steps 501 to 503 are the training process of the human flow prediction model, which does not need to be executed every time portrait clustering is performed. The above portrait clustering process is only an example and does not limit the specific implementation of the portrait clustering process.

[0159] Based on the same concept, an embodiment of the present invention provides a portrait clustering device. Figure 6 A schematic diagram of a portrait clustering device provided in an embodiment of the present application is shown in FIG. Figure 6 including:

[0160] An acquisition module 601 is configured to determine a first portrait image and a second portrait image from a portrait image collection;

[0161] Processing module 602 is configured to determine image similarity between the first portrait image and the second portrait image; the image similarity is used to represent similarity in portrait features and similarity in spatiotemporal features; the portrait image collection includes multiple portrait images collected at different camera positions during a first period of time, and the first portrait image and the second portrait image are two different portrait images in the portrait image collection;

[0162] The processing module 602 is further configured to determine a similarity threshold for portrait clustering based on a first crowd flow prediction result corresponding to a first checkpoint at which the first portrait image is captured and a second crowd flow prediction result corresponding to a second checkpoint at which the second portrait image is captured; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained based on actual crowd flow results at the corresponding checkpoint during a second time period; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration;

[0163] The processing module 602 is further configured to, in response to the picture similarity being higher than the similarity threshold, determine that the first portrait picture and the second portrait picture include the same person.

[0164] Optionally, the processing module 602 is specifically configured to obtain the pedestrian flow prediction result based on the actual pedestrian flow result of the corresponding checkpoint in the second time period, including:

[0165] Determine the actual result of the pedestrian flow at each checkpoint during the second period based on the portrait images during the second period;

[0166] The actual results of the pedestrian flow at each checkpoint in the second time period are input into the pedestrian flow prediction model to obtain the pedestrian flow prediction results of each checkpoint in the first time period; wherein, the pedestrian flow prediction model includes time-dependent parameters and space-dependent parameters; the time-dependent parameters are used to characterize the time granularity of pedestrian flow statistics; the space-dependent parameters are used to characterize the connectivity relationship between each checkpoint.

[0167] Optionally, the processing module 602 is specifically configured to use the image similarity to characterize the similarity of portrait features and the similarity of spatiotemporal features, including:

[0168] determining a similarity of portrait features based on the first portrait picture and the second portrait picture;

[0169] Determining spatiotemporal features of the first portrait image based on the first pedestrian flow prediction result and the historical average pedestrian flow of the first checkpoint;

[0170] Determining the spatiotemporal characteristics of the second portrait image based on the second pedestrian flow prediction result and the historical average pedestrian flow of the second checkpoint;

[0171] determining a degree of temporal and spatial feature matching between the first portrait picture and the second portrait picture based on the temporal and spatial features of the first portrait picture and the temporal and spatial features of the second portrait picture;

[0172] The picture similarity is determined according to the portrait feature similarity and the spatiotemporal feature matching degree.

[0173] Optionally, the processing module 602 is specifically configured to determine the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree, including:

[0174] Obtaining, based on the portrait feature similarity, a probability that the first portrait in the first portrait picture and the second portrait in the second portrait picture are the same person and a probability that the second portrait is not the same person, wherein the same person probability is a probability that the first portrait picture and the second portrait picture contain the same person under the portrait feature similarity, and the sum of the same person probability and the not-the-same-person probability is 1;

[0175] The image similarity is determined according to the spatiotemporal feature matching degree, the same-person probability, and the non-same-person probability.

[0176] Optionally, the processing module 602 is specifically configured to determine the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree, including:

[0177] The image similarity is determined by the following formula:

[0178]

[0179] Among them, prior represents the temporal and spatial feature matching degree; sim represents the similarity of portrait features, and prob represents the image similarity.

[0180] Optionally, the processing module 602 is specifically configured to determine a similarity threshold for portrait clustering based on a first pedestrian flow prediction result of a first checkpoint where the first portrait image is located during the first time period and a second pedestrian flow prediction result of a second checkpoint where the second portrait image is located during the first time period, including:

[0181] Determining a pedestrian flow prediction mean based on the first pedestrian flow prediction result and the second pedestrian flow prediction result;

[0182] A similarity threshold for portrait clustering is determined based on the predicted mean of the crowd flow.

[0183] Optionally, the processing module 602 is specifically configured to provide each traffic flow prediction interval with a corresponding initial similarity threshold;

[0184] Determining a similarity threshold for portrait clustering based on the predicted mean of the crowd flow includes:

[0185] Obtaining an initial similarity threshold corresponding to the pedestrian flow prediction interval to which the pedestrian flow prediction mean belongs;

[0186] Determine a true mean of the number of people flow based on a first true result of the number of people flow at the first checkpoint during the first time period and a second true result of the number of people flow at the second checkpoint during the first time period;

[0187] According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain a similarity threshold for portrait clustering.

[0188] Optionally, the processing module 602 is further configured to:

[0189] According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain the similarity threshold of the portrait clustering, including:

[0190] In response to the predicted mean of the human flow being higher than the actual mean of the human flow, increasing the initial similarity threshold;

[0191] In response to the predicted mean of the human flow being lower than the actual mean of the human flow, the initial similarity threshold is lowered.

[0192] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0193] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0196] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A portrait clustering method, characterized in that: The method includes: Determining a first portrait image and a second portrait image from the portrait image set; Determining a picture similarity between the first portrait picture and the second portrait picture; the picture similarity is used to represent the similarity of portrait features and the similarity of spatiotemporal features; the portrait picture set includes multiple portrait pictures collected at different camera positions during a first time period, and the first portrait picture and the second portrait picture are two different portrait pictures in the portrait picture set; Determining a crowd flow prediction mean based on a first crowd flow prediction result corresponding to a first checkpoint for collecting the first portrait image and a second crowd flow prediction result corresponding to a second checkpoint for collecting the second portrait image; determining a similarity threshold for portrait clustering based on the crowd flow prediction mean and a preset correspondence between the crowd flow prediction result and the similarity threshold; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained by inputting an actual crowd flow result of the corresponding checkpoint in the second time period into a crowd flow prediction model; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration; In response to the picture similarity being higher than the similarity threshold, determining that the first portrait picture and the second portrait picture include the same person; Among them, the temporal and spatial characteristics of the first portrait image are determined based on the first crowd flow prediction result and the historical average crowd flow of the first checkpoint, and the temporal and spatial characteristics of the second portrait image are determined based on the second crowd flow prediction result and the historical average crowd flow of the second checkpoint.

2. The method according to claim 1, wherein The crowd flow prediction result is obtained by inputting the actual crowd flow result of the corresponding checkpoint in the second period into the crowd flow prediction model, including: Determine the actual result of the pedestrian flow at each checkpoint during the second period based on the portrait images during the second period; The actual results of the pedestrian flow at each checkpoint in the second time period are input into the pedestrian flow prediction model to obtain the pedestrian flow prediction results of each checkpoint in the first time period; wherein, the pedestrian flow prediction model includes time-dependent parameters and space-dependent parameters; the time-dependent parameters are used to characterize the time granularity of pedestrian flow statistics; the space-dependent parameters are used to characterize the connectivity relationship between each checkpoint.

3. The method according to claim 1, wherein The image similarity is used to characterize the similarity of portrait features and the similarity of spatiotemporal features, including: determining a similarity of portrait features based on the first portrait picture and the second portrait picture; Determining spatiotemporal features of the first portrait image based on the first pedestrian flow prediction result and the historical average pedestrian flow of the first checkpoint; Determining the spatiotemporal characteristics of the second portrait image based on the second pedestrian flow prediction result and the historical average pedestrian flow of the second checkpoint; determining a degree of temporal and spatial feature matching between the first portrait picture and the second portrait picture based on the temporal and spatial features of the first portrait picture and the temporal and spatial features of the second portrait picture; The picture similarity is determined according to the portrait feature similarity and the spatiotemporal feature matching degree.

4. The method according to claim 3, wherein Determining the image similarity based on the portrait feature similarity and the spatiotemporal feature matching degree includes: Obtaining, based on the portrait feature similarity, a probability that the first portrait in the first portrait picture and the second portrait in the second portrait picture are the same person and a probability that the second portrait is not the same person, wherein the same person probability is a probability that the first portrait picture and the second portrait picture contain the same person under the portrait feature similarity, and the sum of the same person probability and the not-the-same-person probability is 1; The image similarity is determined according to the spatiotemporal feature matching degree, the same-person probability, and the non-same-person probability.

5. The method according to claim 1, wherein Determining a similarity threshold for portrait clustering based on a first pedestrian flow prediction result of a first checkpoint where the first portrait image is located during the first time period and a second pedestrian flow prediction result of a second checkpoint where the second portrait image is located during the first time period includes: Determining a pedestrian flow prediction mean based on the first pedestrian flow prediction result and the second pedestrian flow prediction result; A similarity threshold for portrait clustering is determined based on the predicted mean of the crowd flow.

6. The method according to claim 5, wherein Each traffic flow prediction interval has a corresponding initial similarity threshold; Determining a similarity threshold for portrait clustering based on the predicted mean of the crowd flow includes: Obtaining an initial similarity threshold corresponding to the pedestrian flow prediction interval to which the pedestrian flow prediction mean belongs; Determine a true mean of the number of people flow based on a first true result of the number of people flow at the first checkpoint during the first time period and a second true result of the number of people flow at the second checkpoint during the first time period; According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain a similarity threshold for portrait clustering.

7. The method according to claim 6, wherein Also includes: According to the relationship between the predicted mean of the crowd flow and the actual mean of the crowd flow, the initial similarity threshold is adjusted to obtain the similarity threshold of the portrait clustering, including: In response to the predicted mean of the human flow being higher than the actual mean of the human flow, increasing the initial similarity threshold; In response to the predicted mean of the human flow being lower than the actual mean of the human flow, the initial similarity threshold is lowered.

8. A portrait clustering device, characterized in that: The device comprises: An acquisition module, configured to determine a first portrait image and a second portrait image from a portrait image collection; a processing module, configured to determine a degree of similarity between the first portrait image and the second portrait image; the degree of similarity being used to characterize similarity in portrait features and similarity in spatiotemporal features; the portrait image collection comprising a plurality of portrait images collected at different camera positions during a first period of time, the first portrait image and the second portrait image being two different portrait images in the portrait image collection; The processing module is further configured to determine a predicted mean of crowd flow based on a first crowd flow prediction result corresponding to a first checkpoint for collecting the first portrait image and a second crowd flow prediction result corresponding to a second checkpoint for collecting the second portrait image; determine a similarity threshold for portrait clustering based on the predicted mean of crowd flow and a preset correspondence between the crowd flow prediction result and the similarity threshold; wherein different crowd flow prediction results correspond to different similarity thresholds; the crowd flow prediction result is obtained by inputting an actual result of crowd flow at the corresponding checkpoint in a second time period into a crowd flow prediction model; the first time period is after the second time period, and the first time period and the second time period are adjacent and have the same duration; The processing module is further configured to, in response to the picture similarity being higher than the similarity threshold, determine that the first portrait picture and the second portrait picture contain the same person; Among them, the temporal and spatial characteristics of the first portrait image are determined based on the first crowd flow prediction result and the historical average crowd flow of the first checkpoint, and the temporal and spatial characteristics of the second portrait image are determined based on the second crowd flow prediction result and the historical average crowd flow of the second checkpoint.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is run on a computer, the computer is enabled to implement the method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: memory for storing computer programs; A processor is configured to call the computer program stored in the memory and execute the method according to any one of claims 1 to 7 according to the obtained program.

Images