Target association method based on space-time constraint

Abstract

The invention discloses a time-space constraint-based target association method. The method comprises the following steps of: acquiring time-space information between each pair of cameras in a set area; constructing a dynamic camera topological structure according to the spatio-temporal information; and obtaining space-time constraint information according to the dynamic camera topological structure, and associating a target to be associated with a target in the dynamic camera topological structure according to the space-time constraint information. According to the method, the space-time constraint is acquired through the topological structure of the dynamic camera, and the to-be-associated target is associated with the target in the topological structure of the dynamic camera according to the space-time constraint. The interference of the negative sample is filtered according to the space-time constraint, so that the time in the target association process is greatly shortened, and the accuracy of target association is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to a target association method based on spatiotemporal constraints. Background Art

[0002] In a large-scale camera network, due to the different positions and angles of different cameras and the different time backgrounds in which the targets appear under different cameras, the images captured by the cameras usually have obvious background differences, large differences in lighting conditions, and different angles of pedestrian postures. This leads to large changes in pedestrian visual information, which increases the difficulty of large-scale target association tasks.

[0003] Currently, the application of spatiotemporal information in the field of target association mostly relies on the pedestrian transfer probability between different cameras to construct the camera topology structure. This requires large-scale labeled data, is very costly, and has poor portability, making it only suitable for applications in a small range. Some methods use the idea of ​​unsupervised learning and rely on visual information to organize camera networks. The effectiveness of spatiotemporal constraints depends on accurate visual models, which easily leads to a large number of false associations due to inconsistent appearance. In addition, the existing spatiotemporal information utilization model does not take into account the changes in road conditions at different times of the day. When faced with a large range of cross-spatiotemporal target association tasks, it is also impossible to dynamically adjust according to the retrieval situation. Summary of the Invention

[0004] The present invention provides a target association method based on spatiotemporal constraints to solve at least one of the problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a target association method based on time and space constraints, comprising:

[0007] Get the spatiotemporal information between each pair of cameras in the set area;

[0008] Constructing a dynamic camera topology structure according to the spatiotemporal information;

[0009] The spatiotemporal constraint information is acquired according to the dynamic camera topology structure, and the target to be associated is associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint information.

[0010] Optionally, obtaining the spatiotemporal information between each pair of cameras in the set area includes:

[0011] Get the path distance between each pair of cameras in the set area;

[0012] The path time of the sample target moving between each pair of cameras is collected according to the preset time interval to obtain the path time between each pair of cameras at each collection moment.

[0013] Optionally, constructing a dynamic camera topology structure according to the spatiotemporal information includes:

[0014] Statistical analysis is performed on the spatiotemporal information to obtain the spatiotemporal regularities between the cameras, and a dynamic camera topology structure is constructed based on the spatiotemporal regularities.

[0015] Optionally, performing statistical analysis on the spatiotemporal information to obtain spatiotemporal patterns between cameras includes:

[0016] A linear equation of the path time between each pair of cameras is determined by the path time between each pair of cameras at each acquisition moment, and an estimated path time between each pair of cameras at non-acquisition moments is obtained according to the linear equation.

[0017] Optionally, the linear equation of the path time between each pair of cameras is:

[0018]

[0019] Among them, Tij(dx,t) represents the path time between the i-th camera and the j-th camera at the non-collection time t on the dx-th day of the week, Tij(dx,t1) represents the path time between the i-th camera and the j-th camera at the collection time t1 on the dx-th day of the week obtained according to the sample target, and Tij(dx,t2) represents the path time between the i-th camera and the j-th camera at the collection time t2 on the dx-th day of the week obtained according to the sample target. t1 and t2 are the collection times before and after the non-collection time t, respectively.

[0020] Optionally, analyzing the spatiotemporal information to obtain the spatiotemporal regularity between the cameras further includes:

[0021] The nearest neighbor accumulation strategy is used to filter the noise of the path time estimation between each pair of cameras at non-collection moments.

[0022] Optionally, associating the target to be associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint information includes:

[0023] Acquire a spatial range to be searched according to the dynamic camera topology structure;

[0024] According to the dynamic camera topology, a preset starting point camera and cameras within the spatial range to be searched are obtained to perform path time estimation, and a time range to be searched for each camera is obtained;

[0025] The target to be associated is matched with all targets within the time range to be retrieved based on visual similarity to complete target association.

[0026] Optionally, before performing visual similarity matching between the target to be associated and the target within the time range to be retrieved, the method further includes:

[0027] Creating a search task based on the cameras in the space to be searched, and adding the search task to a global search task list;

[0028] The priority of the search task is determined according to the number of executions of the search task, the length of the associated trajectory, and the degree of matching of direction information, and the search tasks are executed in order of the priority.

[0029] Optionally, performing visual similarity matching between the target to be associated and the target within the time range to be retrieved includes:

[0030] Determine a first threshold based on the path time range, determine the visual similarity between the target to be associated and all targets within the time range to be retrieved, and if the similarity is higher than the first threshold, use the target to be associated as the starting camera for divergence, perform path time estimation on the starting camera and the cameras within the spatial range to be retrieved, and obtain the time range to be retrieved for each camera;

[0031] If the similarity is lower than the first threshold, it is determined that the current search task has failed, and a search task of the next priority is executed.

[0032] Optionally, determining the priority of the search task includes:

[0033] If the priority of the search task is higher than a preset threshold, executing the search task;

[0034] If the priority of the search task is lower than a preset threshold, the search task is abandoned.

[0035] The beneficial effects of the present invention are as follows:

[0036] This method uses dynamic camera topology to obtain spatiotemporal constraint information and then associates the target to be associated with the target in the dynamic camera topology based on this spatiotemporal constraint information. This method also filters out the interference of negative samples based on this spatiotemporal constraint information, significantly shortening the target association process and improving the accuracy of target association. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0038] Figure 1A flow chart showing a target association method based on spatiotemporal constraints according to the present invention;

[0039] FIG2( a ) shows a schematic diagram of the path time on a working day according to an embodiment of the present invention, and FIG2( b ) shows a schematic diagram of the path time on a non-working day according to an embodiment of the present invention;

[0040] FIG3( a ) shows a schematic diagram of the path time distribution before the nearest neighbor accumulation is used in an embodiment of the present invention; FIG3( b ) shows a schematic diagram of the path time distribution after the nearest neighbor accumulation is used in an embodiment of the present invention;

[0041] Figure 4 Schematic diagram showing images of the same pedestrian under different cameras according to an embodiment of the present invention;

[0042] Figure 5 A schematic diagram showing the track length and the number of tracks in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the following detailed description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0044] The present invention provides a target association method based on time and space constraints, which is characterized by comprising:

[0045] Get the spatiotemporal information between each pair of cameras in the set area;

[0046] Constructing a dynamic camera topology structure according to the spatiotemporal information;

[0047] The spatiotemporal constraint information is acquired according to the dynamic camera topology structure, and the target to be associated is associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint information.

[0048] The spatiotemporal information includes spatiotemporal information corresponding to working days and spatiotemporal information corresponding to non-working days. Dynamic camera topology structures corresponding to working days and dynamic camera topology structures corresponding to non-working days are respectively constructed based on the spatiotemporal information corresponding to working days and the spatiotemporal information corresponding to non-working days.

[0049] Figure 1This is a flow chart of the target association method based on spatiotemporal constraints of this embodiment. The target in this embodiment is the image captured by the camera. The spatiotemporal constraint information refers to the use of the time information and position information corresponding to the appearance of pedestrians or vehicles in different cameras, combined with the spatial distribution information of the camera network, to constrain the scope of the retrieval target during the target association process. By constructing a dynamic camera topology structure through the acquired spatiotemporal information, the problem of large-scale data labeling is solved. Target association based on spatiotemporal constraint information can not only correct the contradictions or misjudgments that occur when relying solely on visual information for association, but also narrow the retrieval scope and improve retrieval efficiency. This embodiment filters the interference of negative samples according to the spatiotemporal constraint information, greatly shortens the time of the target association process, and improves the accuracy of target association.

[0050] In a specific example, obtaining the spatiotemporal information between each pair of cameras in the set area includes obtaining the path distance between each pair of cameras in the set area;

[0051] The path time of the sample target moving between each pair of cameras is collected according to the preset time interval to obtain the path time between each pair of cameras at each collection moment.

[0052] In this example, spatiotemporal information includes path distance and path time. Using the path planning interface provided by AutoNavi's open platform, we obtain the path distance between each pair of cameras within a set area, and collect path time information for sample targets between each pair of cameras within the set area at preset time intervals. The path planning API is provided via HTTP, returning query data in JSON or XML format, and can return real-time prediction information based on the specific road conditions at the time the interface is called. The request parameters and return result parameters for this interface are shown in Tables 1 and 2, respectively.

[0053] Table 1 Route planning API request parameters

[0054] Parameter name meaning illustrate key Request service rights protection logo The service type key that users apply for on the Amap official website origin Starting point The latitude and longitude coordinates of the departure point, in the format of: precision, latitude destination End position The latitude and longitude coordinates of the destination point, in the format of: precision, latitude output Return data format type The return data type is json or xml

[0055] Table 2 Path planning API part of the return result parameters

[0056] name meaning illustrate road road name The name of the road from the starting point to the destination distance distance Path distance from the starting point to the destination orientation direction The destination is in the direction of the departure point duration Path time Estimated time from departure point to destination

[0057] This example registers on the AutoNavi Open Platform and applies for the key required for the "Web Service API." It then constructs a Uniform Resource Locator (URL) for the request, combining the key with the location of the desired camera. It then uses the Python requests library to perform a GET request to obtain the corresponding result. The result is then parsed, using the distance and duration fields to determine the path distance and time between the cameras.

[0058] This example uses the API provided by the navigation software open platform to quickly and easily obtain the path distance and path time data between any two camera positions, providing a large amount of accurate basic data for subsequent data statistical analysis.

[0059] In a possible implementation, the constructing of a dynamic camera topology structure according to the spatiotemporal information includes:

[0060] Performing statistical analysis on the spatiotemporal information to obtain spatiotemporal patterns between cameras, and constructing a dynamic camera topology structure based on the spatiotemporal patterns;

[0061] The dynamic camera topology structure includes a dynamic camera topology structure corresponding to working days and a dynamic camera topology structure corresponding to non-working days.

[0062] Continuing with the previous example, this example constructs a dynamic camera topology for weekdays. A camera topology describes the temporal and spatial relationships between different cameras. Traditional camera topologies are estimated by calculating the correlation between exit and entry events within the field of view of different cameras. However, to account for changing road conditions over time, this example obtains path time and distance data between each pair of cameras and then performs a statistical classification analysis on this data to derive temporal and spatial patterns between the cameras. This dynamic camera topology is then constructed based on these temporal and spatial patterns.

[0063] This example takes into account the differences in road conditions such as traffic flow and pedestrian behavior between weekdays and non-working days, and proposes to build dynamic camera topology structure models for weekdays and non-working days respectively to improve the accuracy of the model.

[0064] In one possible implementation, the statistical analysis of the spatiotemporal information to obtain the spatiotemporal regularity between the cameras includes:

[0065] A linear equation of the path time between each pair of cameras is determined by the path time between each pair of cameras at each acquisition moment, and an estimated path time between each pair of cameras at non-acquisition moments is obtained according to the linear equation.

[0066] Continuing with the previous example, in cities, people's travel patterns differ significantly between weekdays and non-weekdays. On weekdays, with the arrival of morning and evening rush hours, urban traffic exhibits two distinct shifts: for example, most people travel from their homes to their workplaces between 8:00 AM and 9:00 AM, returning from their workplaces around 5:00 PM and 6:00 PM. On non-weekdays, such as weekends and holidays, people's travel patterns are often irregular. This means that the spatiotemporal patterns corresponding to weekdays and non-weekdays differ significantly. This example categorizes the path time information for each pair of cameras in a given area into two categories, based on whether it falls on a weekday or not. This examines the distribution patterns of the data collected on weekdays and non-weekdays. Figure 2(a) shows the route time information for driving from home to work on weekdays, and Figure 2(b) shows the route time information for driving from home to work on weekends. The horizontal axes of Figures 2(a) and 2(b) represent the departure time, and the vertical axes represent the route time required for driving. According to Figure 2(a), the route time on weekdays shows a clear morning and evening peak distribution, and the route time at 8:00 a.m. and 6:00 p.m. is significantly higher than that at other times; according to Figure 2(b), the difference in departure and arrival time at different times on weekends is significantly reduced, and there is no obvious morning and evening peak distribution.

[0067] To eliminate noise interference and accumulate rich data information for subsequent work, and to obtain a more universal and accurate spatiotemporal range, this example uses the situation on weekdays as an example to obtain the spatiotemporal patterns corresponding to weekdays and construct a dynamic camera topology based on these spatiotemporal patterns. The preset time interval is set to 20 minutes, and the path distance and path time information between the 62 camera locations in the set area are called multiple times. For two consecutive months, an automated script is developed to call the path planning interface provided by the AutoNavi Map Open Platform three times at 0:00, 20:00, and 40:00 every hour between 0:00 and 23:59 every day. The path distance and path time between the two cameras closest to that time on that day are obtained, and the path distance and path time are stored in the database, combining the isolated camera points. This example collects path distance and path time data every 20 minutes. The collected data is relatively discrete, and there is a 20-minute time interval between adjacent data. In this example, it is considered that within a short time range, the change of road congestion is approximately regarded as continuous, that is, the road congestion at 10:10 am is very likely to be between 10:00 am and 10:20 am. When obtaining path time information at non-collection times, this example determines a linear equation based on the path time of the two adjacent moments, thereby calculating the estimated path time at the current collection moment. Specifically: by searching the database, obtain the collection moments t1 and t2 closest to the current non-collection moment t on each weekday in the database, and record the corresponding path times respectively. and Among them, abs(t-t1) means that the absolute value of t-t1 is the minimum, and abs(t-t2) means that the absolute value of t-t2 is the minimum. The linear equation of the path time between each pair of cameras at the non-collection time is determined based on the path time corresponding to the time points t1 and t2, and the path time estimation at the non-collection time is calculated based on the linear equation of the path time between each pair of cameras.

[0068] The linear equation for the path time between each pair of cameras is

[0069]

[0070] Among them, Tij(dx,t) represents the path time between the i-th camera and the j-th camera at the non-collection time t on the dx-th day of the week, Tij(dx,t1) represents the path time between the i-th camera and the j-th camera at the collection time t1 on the dx-th day of the week obtained according to the sample target, and Tij(dx,t2) represents the path time between the i-th camera and the j-th camera at the collection time t2 on the dx-th day of the week obtained according to the sample target. t1 and t2 are the collection times before and after the non-collection time t, respectively.

[0071] To obtain a path time estimate for a non-collection time t3 on a non-working day, a search can be performed based on the non-working day data within three months in the database to obtain the two times closest to the collection time t3 for each non-working day in the database, record the corresponding path times, and thus determine the linear equation corresponding to the non-working day. Based on the linear equation, the path time estimate for the non-collection time t3 can be obtained.

[0072] In a possible implementation, analyzing the spatiotemporal information to obtain the spatiotemporal regularity between the cameras further includes:

[0073] The nearest neighbor accumulation strategy is used to filter the noise of the path time estimation between each pair of cameras at non-collection moments to obtain the path time range.

[0074] Continuing with the previous example, for all non-collection moments t on all working days, the path time estimates Tij(dx,t) for all working days during the data collection period are obtained according to the above method. This example collects data from working days within three months and obtains path time estimates for approximately 60 discrete non-collection moments t. This example uses the nearest neighbor accumulation method to filter the noise of the path time estimates between each pair of cameras at non-collection moments, optimize the Gaussian distribution, and obtain accurate path time statistics, thereby obtaining a more accurate path time range. The nearest neighbor accumulation method is

[0075]

[0076] Among them, Count(η) represents the optimized frequency statistics value at position η, N(a) represents the frequency value at position a, [η-δ,η+δ] represents the cumulative interval, and η>δ.

[0077] The width of the accumulation interval is determined by the data density in the experiment. In this example, the width of the accumulation interval is set to 50. After the accumulation calculation, values ​​with a cumulative frequency of 1 are removed as noise and do not participate in the subsequent Gaussian distribution statistics. Figure 3 shows a schematic comparison of path times before and after the nearest neighbor accumulation strategy in this example. Figure 3(a) shows the path time distribution before the nearest neighbor accumulation strategy, and Figure 3(b) shows the path time distribution after the nearest neighbor accumulation strategy. As shown in Figures 3(a) and 3(b), the data distribution before the nearest neighbor accumulation strategy is relatively sparse, while the data distribution after the nearest neighbor accumulation strategy exhibits a more obvious Gaussian distribution. In this example, the nearest neighbor accumulation method is used to statistically fit the path time estimates at time t on all weekdays to obtain a Gaussian distribution. The interval [μ-3σ, μ+3σ] is selected as the path time range estimate at time t, and the first threshold is determined based on this path time range.

[0078] This example uses a combination of linear equations and Gaussian distribution to model the query time, obtain a suitable path time range, determine the preset lower threshold based on the path time range, and define a reasonable search range for target association.

[0079] In one possible implementation, associating the target to be associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint includes:

[0080] Acquire a spatial range to be searched according to the dynamic camera topology structure;

[0081] According to the dynamic camera topology, a preset starting point camera and cameras within the spatial range to be searched are obtained to perform path time estimation, and a time range to be searched for each camera is obtained;

[0082] The target to be associated is matched with the target within the time range to be retrieved based on visual similarity to complete the target association.

[0083] Continuing with the previous example, which collected path times and distances between different cameras, compiled statistics on the collected data, analyzed its spatiotemporal patterns, and constructed a dynamic camera topology corresponding to workdays. This method then calculated the path time ranges between each pair of cameras at any time during a workday. This example associates the target to be associated with the target in the dynamic camera topology based on spatiotemporal constraints.

[0084] The spatiotemporal constraints refer to the time and location information of the target appearing in different cameras. By analyzing and determining whether these information conforms to spatiotemporal patterns, we filter out irrelevant negative samples to improve the accuracy of target association. In this example, the spatiotemporal constraints provide a temporal and spatial search range for each target association task. Specifically, the spatiotemporal constraints determine the camera range required for the startle search and the time range for each camera's capture.

[0085] Through the dynamic camera topology structure, the 10 cameras with the shortest path from the predetermined starting camera are obtained as the spatial range to be searched; the path time of the starting camera and the cameras in the spatial range to be searched are estimated according to the dynamic camera topology structure to determine the time range to be searched for each camera; the target to be associated is matched with the target within the time range to be searched for visual similarity, and the existence of the target to be associated within the time range to be searched is judged based on the estimated path time at the current moment to achieve target association.

[0086] In a possible implementation, before performing visual similarity matching between the target to be associated and the target within the time range to be retrieved, the method further includes:

[0087] Creating a search task based on the cameras in the space to be searched, and adding the search task to a global search task list;

[0088] The priority of the search task is determined according to the number of executions of the search task, the length of the associated trajectory, and the degree of matching of direction information, and the search tasks are executed in order of the priority.

[0089] Continuing with the previous example, for each camera within the spatial range to be searched, a search task is created based on its search time, camera number, associated trajectory, and direction relative to the starting camera. Each search task is then prioritized based on its number of executions, associated trajectory length, and direction relative to the starting camera. The search tasks in the full set list are then prioritized and executed in order of priority.

[0090] This example uses the best-first retrieval algorithm for target association, calculates the priority of each search task, executes retrieval tasks in order based on the task priority, and dynamically adjusts retrieval tasks to minimize the negative impact of incorrect association at a certain hop in the retrieval process on the final retrieval results.

[0091] In a possible implementation, performing visual similarity matching between the target to be associated and the target within the time range to be retrieved includes:

[0092] Determine a first threshold based on the path time range, determine the visual similarity between the target to be associated and all targets within the time range to be retrieved, and if the similarity is higher than the first threshold, use the target to be associated as the starting camera for divergence, obtain the time range to be retrieved of the starting camera, perform path time estimation on the starting camera and the cameras within the spatial range to be retrieved, and again determine the visual similarity between the target to be associated and all targets within the time range to be retrieved;

[0093] If the similarity is lower than the first threshold, it is determined that the current search task has failed, and a search task of the next priority is executed.

[0094] Continuing with the above example, the first threshold is determined based on the path time range, and the visual similarity of the target to be associated is matched with all targets that appear in the search task with the highest priority within the time range to be retrieved.

[0095] If the visual similarity is higher than the first threshold, the camera corresponding to the target is used as the new starting camera for divergence, and the 10 cameras with the shortest path distance from the starting camera are re-acquired as the spatial range to be searched. The path time of the starting camera and the cameras in the spatial range to be searched is estimated, and the time range to be searched for each camera is determined. The target to be associated is again matched with the target with the highest priority in the new time range to be searched for visual similarity to obtain a matching result;

[0096] If the visual similarity is lower than the first threshold, the search is determined to have failed. At this time, the next search task within the time range to be retrieved is executed in order of priority, and the visual similarity of the target to be associated is matched with all targets appearing in the next search task. The similarity is again determined to be greater than the first threshold. According to the judgment result, the camera corresponding to the search task is used as the starting camera for divergence, or the next search task is executed according to the judgment result.

[0097] In a possible implementation, determining the priority of the search task includes:

[0098] If the priority of the search task is higher than a preset threshold, executing the search task;

[0099] If the priority of the search task is lower than a preset threshold, the search task is abandoned.

[0100] Continuing with the previous example, the priority of a search task is determined based on indicators such as the number of executions of the search task, the length of the associated trajectory, and the degree of direction information matching. The greater the number of executions, the lower the priority is; the longer the associated trajectory, the lower the priority is; and the greater the degree of deviation from the direction, the lower the priority is. The corresponding priority results are obtained based on the number of executions, the length of the associated trajectory, and the degree of direction information matching, and the average of the three priorities is calculated as the final search task priority.

[0101] The priority of the search task in the global search task list is determined according to a preset threshold. If the priority of the search task is higher than the preset threshold, the search task is executed; if the priority of the search task is lower than the preset threshold, the search task is abandoned.

[0102] In a specific example, the public dataset MSMT17 for person re-identification was used for data simulation. MSMT17 uses 15 cameras deployed on a campus, including 12 outdoor cameras and 3 indoor cameras. The data is collected in the morning, noon, and afternoon on four days in different weather conditions in one month. Figure 4 For images of the same ID captured by different cameras, due to the relatively small number of cameras in MSMT17 and the fact that most pedestrians appear in the field of view of only two or three cameras, this cannot fully reflect the large-scale target association scenarios targeted by the present invention, and lacks the temporal and spatial information of image acquisition. Therefore, this example simulates the camera number, path time, and geographic location information of pedestrian data in the MSMT17 dataset. This example selects an area three kilometers near a school and simulates a total of 62 camera locations, assigning them latitude and longitude information. Targets in the MSMT17 dataset that have the same camera label and are captured in the same time period are assigned to the same simulated camera target, while targets with the same camera label but different acquisition time labels are assigned to different simulated camera targets.

[0103] For the images in the 3060 IDs in the MSMT17 test set, they are labeled according to the camera label and acquisition time to obtain their virtual camera identifier k, and then randomly select an image in each ID as the first image that appears at the starting point, and randomly initialize the time and position information of the starting point in the preset 62 camera positions. For each image in the virtual camera k, first randomly select a camera from the 10 cameras closest to the starting point as the next hop position. Note that it is necessary to ensure that the position cannot be repeated with the determined position; determine the path time to the position based on the camera topology, and use the nearest neighbor accumulation method to obtain a normally distributed random number with a mean of 1 and a standard deviation of 0.1 to simulate the path time, and then determine the image acquisition time of the next hop position, and then record the time and corresponding position of the image in the virtual camera k in the database. Finally, use this position as the starting point and repeat the above process until all images are stored in the database. For example Figure 5 As shown, a total of 3060 trajectories are simulated in this example, among which the minimum trajectory length is 2 and the maximum is 21, and the number of trajectories with a trajectory length of 4 or 5 is close to 50%.

[0104] Two sets of experiments were conducted to test the performance of person re-identification (PRI) and the accuracy and efficiency of target association, respectively. One set of experiments used mAP, Rank-1, Rank-5, and Rank-10. Each retrieval step in each search task was considered a RII task when calculating these metrics. The second set of experiments used association accuracy and the average execution time of target association tasks. Association accuracy refers to the ratio of successfully associated complete trajectories to the total number of association tasks. The average execution time of an association task is an estimate of the execution time of each target association task, calculated by dividing the total association task time by the total number of association tasks. During testing, the first 100 trajectories from the 3060 trajectories were selected as test trajectories, and their starting points were used as query targets in turn, hoping to obtain complete associated trajectories. The remaining 2,000 or so trajectory images were distributed within the search range of these 100 query targets as interference to increase the association difficulty and improve retrieval accuracy. The two sets of experiments were conducted using the aforementioned settings. The CPU model used for this experiment was an i9-9900X with 64GB of RAM.

[0105] The test results of the two sets of experiments are shown in Tables 3 and 4, respectively. The "Target Association Algorithm with Integrating Spatiotemporal Constraints" in the table is the algorithm proposed in this example. The "Baseline Algorithm" is based on the pedestrian features extracted from BoT and is retrieved within a range later than the acquisition time of the query image. No more specific constraints are imposed on the time or spatial range of the retrieval.

[0106] Table 3. Pedestrian re-identification index test results of target association system

[0107] mAP (%) Rank -1(%) Rank -5 (%) Rank -10(%) Benchmark Algorithm 79.99 81.68 89.34 94.15 Target association integrating spatiotemporal constraints 95.22 95.30 98.90 99.45

[0108] Table 4 Target association system test results

[0109] Accuracy(%) Time(s) Benchmark Algorithm 29 207.50 Target association integrating spatiotemporal constraints 65 12.10

[0110] According to the experimental results in Tables 3 and 4, after adding spatiotemporal constraints, most of the interference from negative samples was filtered out, the number of gallery images in the re-identification test was reduced, and both the re-identification performance and the target association performance of the target association system were improved. Specifically, mAP improved by 15% after adding spatiotemporal constraints, Rank-1 improved by over 13%, and Rank-5 and Rank-10 improved by 8% and 5%, respectively. After adding spatiotemporal constraints, the target association system's association accuracy was also greatly improved, with the probability of successful association increasing from 29% to 65%. The average association task time was shortened by over 90%, significantly improving the system's operational efficiency.

[0111] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0112] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0113] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A target association method based on spatiotemporal constraints, characterized in that: include: Get the spatiotemporal information between each pair of cameras in the set area; Constructing a dynamic camera topology structure according to the spatiotemporal information; The spatiotemporal constraint information is acquired according to the dynamic camera topology structure, and the target to be associated is associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint information.

2. The method according to claim 1, characterized in that The acquisition of spatiotemporal information between each pair of cameras in the set area includes: Get the path distance between each pair of cameras in the set area; The path time of the sample target moving between each pair of cameras is collected according to the preset time interval to obtain the path time between each pair of cameras at each collection moment.

3. The method according to claim 2, characterized in that The step of constructing a dynamic camera topology structure according to the spatiotemporal information includes: Statistical analysis is performed on the spatiotemporal information to obtain the spatiotemporal regularities between the cameras, and a dynamic camera topology structure is constructed based on the spatiotemporal regularities.

4. The method according to claim 3, characterized in that The performing statistical analysis on the spatiotemporal information to obtain the spatiotemporal regularity between the cameras includes: A linear equation of the path time between each pair of cameras is determined by the path time between each pair of cameras at each acquisition moment, and an estimated path time between each pair of cameras at non-acquisition moments is obtained according to the linear equation.

5. The method according to claim 4, characterized in that The linear equation of the path time between each pair of cameras is: Among them, T ij (d x ,t) represents the distance between the i-th camera and the j-th camera on the d-th day of the week. x The path time at the non-collection time t of the day, T ij (d x ,t1) represents the distance between the i-th camera and the j-th camera obtained according to the sample target on the d-th day of the week. x The path time at the collection time t1 of the day, T ij (d x ,t2) represents the distance between the i-th camera and the j-th camera obtained according to the sample target on the d-th day of the week x The path time of the collection time t2 on the day, t1 and t2 are the collection times before and after the non-collection time t respectively.

6. The method according to claim 5, characterized in that Analyzing the spatiotemporal information to obtain the spatiotemporal patterns between the cameras further includes: The nearest neighbor accumulation strategy is used to filter the noise of the path time estimation between each pair of cameras at non-collection moments.

7. The method according to claim 1, characterized in that Associating the target to be associated with the target in the dynamic camera topology structure according to the spatiotemporal constraint information includes: Acquire a spatial range to be searched according to the dynamic camera topology structure; According to the dynamic camera topology, a preset starting point camera and cameras within the spatial range to be searched are obtained to perform path time estimation, and a time range to be searched for each camera is obtained; The target to be associated is matched with all targets within the time range to be retrieved based on visual similarity to complete target association.

8. The method according to claim 7, characterized in that Before performing visual similarity matching between the target to be associated and the target within the time range to be retrieved, the method further includes: Creating a search task based on the cameras in the space to be searched, and adding the search task to a global search task list; The priority of the search task is determined according to the number of executions of the search task, the length of the associated trajectory, and the degree of matching of direction information, and the search tasks are executed in order of the priority.

9. The method according to claim 8, characterized in that The visual similarity matching of the target to be associated with the target within the time range to be retrieved includes: Determine a first threshold based on the path time range, determine the visual similarity between the target to be associated and all targets within the time range to be retrieved, and if the similarity is higher than the first threshold, use the target to be associated as the starting camera for divergence, perform path time estimation on the starting camera and the cameras within the spatial range to be retrieved, and obtain the time range to be retrieved for each camera; If the similarity is lower than the first threshold, it is determined that the current search task has failed, and a search task of the next priority is executed.

10. The method according to claim 9, characterized in that Determining the priority of the search task includes: If the priority of the search task is higher than a preset threshold, executing the search task; If the priority of the search task is lower than a preset threshold, the search task is abandoned.

Images