A method and system for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features

Abstract

The present application relates to the field of trajectory reconstruction technology, and provides a surveillance video pedestrian trajectory reconstruction method and system integrating spatiotemporal features. The method establishes a regular hexagonal grid network in an urban area through grid division, and establishes a grid road network according to the deployment points and road network data of the video surveillance system in the urban area; based on the grid road network, a potential path set is constructed between the continuous sampling points of the target sampling trajectory; the spatiotemporal state features of the potential path are extracted and embedded to obtain the spatiotemporal state feature values ​​of the potential path in the potential path set to which it belongs; the speed of the potential path is corrected under the constraint of trajectory entropy, and based on the speed model of the target's regular movement, the spatiotemporal transfer feature values ​​of the potential paths between adjacent potential path sets are obtained; based on the spatiotemporal state feature values ​​and the spatiotemporal transfer feature values, the path that is closest to the target's moving spatiotemporal features is inferred from multiple potential paths to complete the trajectory reconstruction of the target.

Description

Technical Field

[0001] The present application relates to the technical field of trajectory reconstruction, and in particular to a method and system for reconstructing pedestrian trajectories from surveillance videos by integrating temporal and spatial features. Background Art

[0002] Trajectory reconstruction is to process the collected discrete trajectory data through specific methods to infer and restore the target's movement path. Pedestrian trajectory reconstruction is an important means to gain a deep understanding of pedestrian activity patterns in the city, which is crucial for public security management, especially crime prevention and investigation.

[0003] Traditional trajectory reconstruction methods mostly rely on GPS data, but in real applications, not all scenes are equipped with GPS devices or actively share location information. Especially when reconstructing pedestrian trajectories, due to factors such as privacy protection, it is often impossible to obtain the complete GPS trajectory data of pedestrians. With the continuous development of target re-identification technology, the monitoring of pedestrian images after target re-identification processing produces video trajectories, making it possible to associate pedestrians in massive video images; the trajectory data obtained based on pedestrian recognition technology provides a new data source for trajectory reconstruction, namely video trajectory.

[0004] According to different data sampling methods, trajectory data can be further divided into active sampling trajectory and passive sampling trajectory. GPS trajectory belongs to active sampling trajectory. This type of data is generated by the target actively sharing its location information through the GPS device. The sampling frequency and error are mainly determined by the performance of the GPS device. Video trajectory belongs to passive sampling trajectory. It forms trajectory sampling by capturing the target image through the surveillance camera. Its sampling frequency is limited by the monitoring deployment density and the target movement speed, and the sampling error mainly depends on the accuracy of the target re-identification algorithm.

[0005] Due to the fixed sampling position and the limited deployment range of surveillance cameras, the spatial features that video trajectories can present are more limited compared to GPS trajectories, and the uncertainty of pedestrian video trajectories is higher. At present, video trajectory reconstruction technology mainly restores low-sampling point trajectories to high-sampling trajectories and performs road network matching to reduce the uncertainty of pedestrian video trajectories; however, existing technologies and methods lack the ability to fully analyze the spatiotemporal characteristics of the movement path as a whole based on the study of pedestrians' regular movement patterns, and cannot effectively cope with the task of pedestrian trajectory video reconstruction.

[0006] Therefore, there is an urgent need to provide a technical solution to the above-mentioned deficiencies in the prior art. Summary of the invention

[0007] The purpose of this application is to provide a method and system for reconstructing pedestrian trajectories from surveillance videos that integrates temporal and spatial features, so as to solve or alleviate the problems existing in the above-mentioned prior art.

[0008] In order to achieve the above objectives, this application provides the following technical solutions:

[0009] The present application provides a method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features, comprising: step S1, establishing a grid-based pedestrian trajectory reconstruction system with a radius of A regular hexagonal grid network is prepared, and according to the obtained video surveillance system deployment point and road network data of the urban area, a road network adaptive rasterization processing is performed based on the grid coordinate system of the regular hexagonal grid network to obtain a grid road network; step S2, based on the grid road network, a potential path set of the target sampling trajectory is constructed between the continuous sampling points of the target sampling trajectory; wherein the potential path set contains multiple potential paths; the target sampling trajectory is the activity trajectory of the target collected by the video surveillance equipment deployed at the video surveillance system deployment point when the target is active in the urban area; step S3, The potential path is subjected to spatiotemporal state feature extraction and embedding to obtain the spatiotemporal state feature value of the potential path in the potential path set in which it is located; step S4, the potential path is subjected to velocity correction under trajectory entropy constraint, and based on the velocity model of the target's regular movement, the spatiotemporal transfer feature values ​​of the potential paths between adjacent potential path sets are obtained; step S5, based on a pre-constructed conditional random field model, according to the spatiotemporal state feature value and the spatiotemporal transfer feature value, the path that is closest to the spatiotemporal movement feature of the target is inferred from the multiple potential paths to complete trajectory reconstruction for the target.

[0010] Preferably, in step S1: step S11, according to the acquired video surveillance system deployment points and node data in the road network of the urban area, adaptively adjust the grid direction in the regular hexagonal grid network based on grid parameter optimization; step S12, map the video surveillance system deployment points to the road network, and add the video surveillance system deployment points as road network nodes, and add the monitorable paths of the video surveillance system deployment points as road network edges; step S13, map the road network nodes and the road network edges to the corresponding grid sequences in the regular hexagonal grid network to construct a grid initial road network; step S14, traverse the edges of the grid initial road network, delete and merge the edges that are completely located in the grid from the grid initial road network, and obtain the grid road network.

[0011] Preferably, in step S2, in response to the sampling time interval between any two consecutive sampling points being greater than or equal to a preset time length threshold , or, the straight-line distance between any two consecutive sampling points is greater than or equal to the preset distance length threshold , segment the corresponding target sampling trajectory, and use the segmented target sampling trajectory as the minimum unit of trajectory reconstruction to construct a potential path set of the target sampling trajectory.

[0012] Preferably, step S3 includes: step S31, respectively extracting features of each potential path in the potential path set corresponding to each target sampling trajectory to obtain the path state features of the corresponding potential path; step S32, coupling the path state features of the potential path with the time range and the space range to obtain the path state feature vector of the coupled spatiotemporal features; step S33, according to the embedding model:

[0013]

[0014] Embed the path state feature vector of the coupled spatiotemporal feature to obtain the spatiotemporal state feature value of the potential path in the potential path set in which it is located ; In the formula, Indicates that the target is and The target sampling trajectory moves between consecutive sampling points The corresponding potential path set a path state feature vector of each of the potential paths; are all positive integers; is the input layer of the embedding model, and the input vector dimension of the input layer is 14, and the output vector dimension is ; is the output layer of the embedding model, and the dimension of the output layer input vector is , the output vector dimension is 1; is the encoding layer of the embedding model, and the input vector dimension and output vector dimension of the encoding layer are ; are the network parameters of the embedding model.

[0015] Preferably, the path state feature includes: length feature, direction feature and monitoring coverage feature; correspondingly, in step S31, the feature extraction is performed on each potential path in the potential path set corresponding to each target sampling trajectory to obtain the path state feature of the corresponding potential path, including:

[0016] According to the formula:

[0017]

[0018] Determine the length characteristics of the potential path of movement of the target ;

[0019] In the formula, Indicates that the target is and The target sampling trajectory moving between consecutive sampling points; Represents the target sampling trajectory The corresponding potential path set potential pathways described; Represents the potential path The physical length of Represents the target sampling trajectory The corresponding and The straight-line distance between consecutive sampling points; is a positive integer;

[0020] According to the formula:

[0021]

[0022]

[0023] Determine the directional characteristics of the potential path when the target moves; wherein the directional characteristics include: heading deviation and steering bias ; The heading deviation Characterize the target and The degree of direction deviation when moving between consecutive sampling points; Characterize the target and The degree of oscillation when zigzagging between consecutive sampling points; is the total number of points contained in the edges corresponding to the potential path in the grid road network; is the starting point of the potential path and the edge corresponding to the grid road network The angle between the points; is the angle between the start and end points of the potential path; is the first edge corresponding to the potential path in the grid road network Point and The angle between the points;

[0024] According to the formula:

[0025]

[0026]

[0027]

[0028] Determine the coverage characteristics of the potential path when the target moves; wherein the coverage characteristics are characteristics of the deployment points of the video surveillance system where no sampling occurs when the target moves on the potential path, including: monitoring point quantity variables, monitoring coverage length variables, and monitoring coverage position variables;

[0029] In the formula, For the purpose of and When moving between consecutive sampling points, the number variable of the monitoring points on the potential path; For the purpose of and When moving between consecutive sampling points, the monitoring coverage length variable on the potential path; For the purpose of and When moving between consecutive sampling points, the monitoring coverage position variables on the potential path; For the purpose of and The deployment point of the video surveillance system where no sampling occurs on the potential path moving between consecutive sampling points the number of For the purpose of and The physical length of the monitoring coverage of the deployment points of the video monitoring system where no sampling occurs on the potential path moving between consecutive sampling points; For the purpose of and the maximum physical length in the monitoring coverage of the deployment points of the video surveillance system where no sampling occurs on the potential path moving between consecutive sampling points; For the purpose of and The minimum physical length of the monitoring coverage of the deployment points of the video monitoring system where no sampling occurs on the potential path moving between consecutive sampling points; For the purpose of and The monitoring coverage radius of the deployment points of the video monitoring system where no sampling occurs on the potential path moving between consecutive sampling points; For the purpose of and The shortest distance from the path in the monitoring coverage of the video surveillance system deployment point where no sampling occurs to the corresponding video surveillance system deployment point on the potential path moving between consecutive sampling points.

[0030] Preferably, in step S32, obtaining the path state feature vector of the coupled spatiotemporal feature includes: according to the formula:

[0031]

[0032] Determining the path state characteristic vector of the coupled spatiotemporal characteristics;

[0033] In the formula, Indicates that the target is and When moving between consecutive sampling points, the a path state feature vector of each of the potential paths; Respectively represent the target in and When moving between consecutive sampling points, the length characteristics, heading deviation, turning deviation, monitoring point quantity variable, monitoring coverage length variable, and monitoring coverage position variable of the potential path described in the article; Respectively represent the target in and When moving between consecutive sampling points, the The spatial and temporal scopes corresponding to the potential pathways; is a positive integer;

[0034] Among them, according to the formula:

[0035]

[0036]

[0037] Determine the target in and When moving between consecutive sampling points, the The spatial range of the potential paths and time range ;

[0038] In the formula, is the grid coordinate interval of the grid road network corresponding to the urban area; For the purpose of and When moving between consecutive sampling points, the The grid coordinate interval of the grid road network corresponding to each of the potential paths; is the number of time slots into which the unit time (1 day) is divided; For the and The sampling time range of the target is the number of consecutive sampling points.

[0039] Preferably, step S4 includes: step S41, according to the formula:

[0040]

[0041] Determining the mean and variance of the moving speed of the target when it moves;

[0042] In the formula, For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding moving speed mean; are all positive integers;

[0043] Indicates that the target is in the target sampling trajectory The corresponding potential path set Potential Paths The average moving speed on Indicates Potential Paths The spatiotemporal state characteristic value in the potential path set in which it is located;

[0044] For the purpose of and The target sampling trajectory moves between consecutive sampling points The variance of the corresponding moving speed; Sampling trajectory for the target The number of the potential paths in the corresponding potential path set; is a positive integer;

[0045] Step S42: traverse the target on the target sampling trajectory The potential path in the corresponding potential path set, in response to or , then according to the formula:

[0046]

[0047] The average movement speed of the target Correction is performed to obtain the target sampling trajectory of the target The corresponding potential path set Potential Paths Corrected movement speed on ;

[0048] In the formula, is the current step size of the speed; The sliding window step size for speed correction; Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding moving speed mean; For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding moving speed mean; is the current step length The weight of are all positive integers;

[0049] Step S43, traverse the potential paths between two adjacent potential path sets, according to the formula:

[0050]

[0051] Determine the spatiotemporal transfer probability of the target on the potential path between the continuous sampling points corresponding to two adjacent potential path sets ;

[0052] In the formula, For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding potential path set potential pathways described; For the purpose of and The target sampling trajectory moving between consecutive sampling points The corresponding potential path set potential pathways described; are all positive integers;

[0053] For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding moving speed mean; For the purpose of and The target sampling trajectory moves between consecutive sampling points The corresponding moving speed mean; For the purpose of and The target sampling trajectory moves between consecutive sampling points The standard deviation of the corresponding moving speed; For the purpose of and The target sampling trajectory moves between consecutive sampling points The standard deviation of the corresponding moving speed; Indicates that the target is in the target sampling trajectory The corresponding potential path set Potential Paths The speed of movement on Indicates that the target is in the target sampling trajectory The corresponding potential path set Potential Paths The speed of movement on Sample trajectories for the target The corresponding potential path set Potential Paths Corrected movement speed on ; Sample trajectories for the target The corresponding potential path set Potential Paths Corrected movement speed on ; Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of is the valid interval range of the normal distribution obeyed by the moving speed of the target;

[0054] Step S44, according to the formula:

[0055]

[0056] Determine the spatiotemporal transfer characteristic values ​​of the potential paths of the target between consecutive sampling points ;

[0057] In the formula, For the purpose of and The target sampling trajectory moves between consecutive sampling points The spatiotemporal transition probability of each potential path between the corresponding potential path set and its adjacent potential path set.

[0058] Preferably, according to the formula:

[0059]

[0060] Determine the target in and The target sampling trajectory moves between consecutive sampling points Trajectory entropy ;

[0061] In the formula, Sampling trajectory for the target The corresponding and The length of the sampling interval between consecutive sampling points; is the preset time length threshold; is the time length weight coefficient; Sampling trajectory for the target The corresponding and The length of the sampling interval between consecutive sampling points; is the preset distance length threshold; is the distance length weight coefficient.

[0062] The embodiment of the present application also provides a surveillance video pedestrian trajectory reconstruction system integrating spatiotemporal features, including: a rasterization unit configured to establish a radius of 100 in the urban area where the target activity is located by rasterization. A regular hexagonal grid network is provided, and according to the obtained video surveillance system deployment point and road network data of the urban area, a road network adaptive rasterization processing is performed based on the grid coordinate system of the regular hexagonal grid network to obtain a grid road network; a path set generation unit is configured to construct a potential path set of the target sampling trajectory between the continuous sampling points of the target sampling trajectory based on the grid road network through the shortest path search technology; wherein the potential path set contains multiple potential paths; the target sampling trajectory is the activity trajectory of the target collected by the video surveillance equipment deployed at the video surveillance system deployment point when the target is active in the urban area; a spatiotemporal state feature unit , configured to extract and embed the spatiotemporal state features of the potential path, and obtain the spatiotemporal state feature value of the potential path in the potential path set; a spatiotemporal transfer feature unit, configured to perform speed correction on the potential path under the trajectory entropy constraint, and obtain the spatiotemporal transfer feature values ​​of the potential paths between adjacent potential path sets based on the speed model of the target's regular movement; a trajectory reconstruction unit, based on a pre-constructed conditional random field model, infers and determines from multiple potential paths the path that is closest to the target's moving spatiotemporal features according to the spatiotemporal state feature values ​​and the spatiotemporal transfer feature values, so as to complete trajectory reconstruction for the target.

[0063] Beneficial effects:

[0064] In the method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features provided in the embodiment of the present application, according to the accuracy condition of trajectory reconstruction, a grid is divided to establish a radius of 1 in the urban area where the target is active. A regular hexagonal grid network is constructed, and according to the obtained video surveillance system deployment points and road network data of the urban area, the road network is adaptively rasterized based on the grid coordinate system of the regular hexagonal grid network to obtain a grid road network; then, based on the grid road network, a potential path set of the target sampling trajectory is constructed between the continuous sampling points of the target sampling trajectory through the shortest distance search technology; for multiple potential paths contained in the potential path set, spatiotemporal state features are extracted and embedded to obtain the spatiotemporal state feature values ​​of the potential paths in the potential path set to which they belong; then, the potential paths are speed-corrected under the constraint of trajectory entropy, and based on the speed model of the conventional movement of pedestrians, the spatiotemporal transfer feature values ​​of the potential paths between adjacent potential path sets are obtained; finally, based on the pre-constructed conditional random field model, according to the spatiotemporal state feature values ​​and the spatiotemporal transfer feature values, the path that is closest to the target's movement spatiotemporal features is inferred from multiple potential paths to complete the trajectory reconstruction of the target.

[0065] Therefore, through the trajectory reconstruction strategy based on the grid road network, the trajectory data and the road network data are unified to the same scale, the road network topology and the spatial characteristics of the path are retained, the road network complexity is reduced, and the efficiency and robustness of the trajectory reconstruction are significantly improved; through the spatiotemporal state characteristics and spatiotemporal transfer characteristics of the potential path, and integrated with the spatiotemporal information of the trajectory, according to the stability characteristics of the target movement speed, the trajectory entropy is used to effectively suppress the diffusion of uncertainty in the trajectory reconstruction process, and the path that is most similar to the target movement spatiotemporal characteristics is accurately screened out, which significantly improves the robustness and accuracy of the trajectory reconstruction, and realizes the effective reconstruction of the target video trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. Among them:

[0067] Figure 1 A flowchart of a method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to some embodiments of the present application;

[0068] Figure 2 A schematic diagram of a grid road network provided according to some embodiments of the present application;

[0069] Figure 3 A schematic diagram of spatiotemporal state characteristics of potential paths provided according to some embodiments of the present application;

[0070] Figure 4 A schematic diagram of potential paths of coupled spatiotemporal features provided according to some embodiments of the present application;

[0071] Figure 5 A schematic diagram of speed correction based on a sliding window according to some embodiments of the present application;

[0072] Figure 6 A schematic diagram of a normal distribution of the moving speed of a target provided according to some embodiments of the present application;

[0073] Figure 7 A schematic diagram of a conditional random field model provided according to some embodiments of the present application;

[0074] Figure 8 A schematic diagram of the structure of a surveillance video pedestrian trajectory reconstruction system integrating spatiotemporal features provided according to some embodiments of the present application. DETAILED DESCRIPTION

[0075] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. Each example is provided by way of explanation of the present application and does not limit the present application. In fact, it will be clear to those skilled in the art that modifications and variations may be made in the present application without departing from the scope or spirit of the present application. For example, a feature shown or described as a part of an embodiment may be used in another embodiment to produce yet another embodiment. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the embodiments of the present invention should belong to the scope of protection of the embodiments of the present invention.

[0076] In the process of pedestrian video trajectory reconstruction, trajectory data is collected at a certain sampling frequency, which inevitably leads to the trajectory data not fully reflecting the target's true movement path, so the trajectory data is also called uncertain trajectory. Trajectory reconstruction can reduce the uncertainty of trajectory data and provide data support for downstream tasks of trajectory mining such as mobile feature research. The uncertainty of trajectory data is mainly reflected in two aspects: on the one hand, the sampling data cannot accurately record the real-time position of the target; on the other hand, the trajectory data cannot show the real movement of the target during the resampling interval.

[0077] At present, the road network matching technology is mainly used to match the sampling points to the corresponding road network, thereby effectively reducing the uncertainty of trajectory data. However, as the sampling frequency decreases, there will still be path discontinuities after road network matching, and at this time, road network matching will not be able to further reduce the uncertainty of trajectory data.

[0078] The uncertainty of pedestrian video trajectories is mainly reflected in two aspects: First, pedestrian movement is quite random. On the one hand, pedestrian movement follows the road network, but is not strictly restricted by the road network; whether walking or riding a bicycle, pedestrians may switch freely between pedestrian paths, non-motorized vehicle paths and motor vehicle paths, and may even go against the flow, cross the road or change transportation methods; on the other hand, the monitoring points for obtaining pedestrian video trajectories are usually set on the side of the road or at intersections; the coverage of these monitoring points often covers an area, which may include multiple roads in different directions. Therefore, from the perspective of monitoring, the target may be considered to be moving freely in this area.

[0079] Second, pedestrian video trajectory data is more sparse. On the one hand, due to the limitation of monitoring image quality, it is impossible to accurately re-identify all targets; at the same time, since the deployment of monitoring points cannot fully cover every road, the video trajectory data of the same target often has varying degrees of missing; on the other hand, pedestrians move relatively slowly, resulting in a longer movement time between monitoring points, which significantly reduces the sampling frequency of pedestrian video trajectories (ultra-low sampling trajectory).

[0080] For the reconstruction of low-sampling trajectory data, two main methods are currently used. One method uses a heuristic search algorithm or a simple probability model to build a transition probability model between adjacent positions, such as combining a Markov chain to design a search algorithm. This type of method relies on a large amount of high-quality historical trajectory data, and uses data mining methods such as cluster analysis to discover frequent patterns of trajectory fragments and directly reconstruct sampling points into road network movement paths. However, this type of method lacks an effective mechanism to constrain the diffusion of uncertainty during transfer reasoning, and simple probability models are difficult to fully capture the spatiotemporal characteristics of the target trajectory.

[0081] Another type of method uses a sequence neural network model, which takes advantage of the neural network model in feature description to enhance the modeling ability of complex sequence conversion patterns and realize the recovery of low-sampling trajectories to high-sampling trajectories. However, this type of method does not fully consider that in low-sampling trajectory sequences, due to the large distance between the temporal and spatial ranges, the trajectory characteristics of the target in different temporal and spatial ranges are different, resulting in weak temporal and spatial correlation of the sampling sequence, making it difficult for the sequence neural network to play an effective role.

[0082] Moreover, the two methods currently used to reconstruct low-sampling trajectory data cannot effectively solve the problem of selecting parallel road sections when reconstructing low-sampling trajectories. Therefore, for the reconstruction of pedestrian video trajectories, firstly, the existing technology does not consider the unification of the data scale between pedestrian video trajectories and complex road networks, resulting in insufficient diversity of candidate paths and inaccurate reconstruction results; secondly, the existing methods do not consider the comprehensive spatiotemporal characteristics of distance, direction and monitoring coverage on the overall moving path, resulting in the inferred path being unable to reflect the moving characteristics of the target; thirdly, the existing methods do not constrain the uncertainty angle of the trajectory data, resulting in the diffusion of uncertainty during the reconstruction process.

[0083] Based on this, this application proposes a trajectory reconstruction method based on conditional random fields (CRF) with spatio-temporal feature embedding under trajectory information entropy constraints (CRF-STEEC for short) to reconstruct the movement path of pedestrians from low-sampled trajectory data. Figures 1 to 7 As shown, the surveillance video pedestrian trajectory reconstruction method integrating spatiotemporal features includes:

[0084] Step S1: Create a grid with a radius of A regular hexagonal grid network is constructed, and according to the acquired video surveillance system deployment point and road network data of the urban area, the road network is adaptively rasterized based on the grid coordinate system of the regular hexagonal grid network to obtain a grid road network.

[0085] In order to unify the data scale between the pedestrian video trajectory and the complex road network, in this application, the road network nodes and the points inside the road are mapped to the grid through adaptive gridding of the road network, so as to preserve the geometric characteristics of the road; and in the process of trajectory reconstruction, the grid coverage is used as the basic unit to effectively deal with the large randomness of pedestrian movement. In this way, through the trajectory reconstruction strategy based on the grid road network, the trajectory data and the road network data are unified to the same scale, the spatial characteristics of the road network topology and the path are preserved, the complexity of the road network is reduced, and the efficiency and stability of trajectory reconstruction are significantly improved.

[0086] In this application, according to the accuracy condition of trajectory reconstruction, the deployment points of the video surveillance system in the urban area are divided into In the regular hexagonal grid, the monitored paths corresponding to the deployment points of the video surveillance system are represented by the grids they pass through. When the road network is rasterized, the regular hexagonal grid can, on the one hand, make the nodes in the grid have more similar spatial features, and on the other hand, it is easy to rotate according to the regional road distribution in the city, adapting to the large number of non-horizontal or vertical roads in the city to achieve the best coverage.

[0087] Then, (step S11) according to the obtained video surveillance system deployment points in the urban area and the node data in the road network, the grid direction in the regular hexagonal grid network is adaptively adjusted based on the grid parameter optimization technology. (step S12) The video surveillance deployment situation in the urban area is mapped to the road network, the video surveillance system deployment points are added as nodes on the road network (i.e., road network nodes), and the monitorable paths corresponding to the video surveillance system deployment points are added as edges on the road network (road network edges).

[0088] (Step S13) Map the network nodes and network edges to the corresponding grid sequence in the regular hexagonal grid network to construct the grid initial network. That is, the network nodes and network edges are mapped to the corresponding grids in the regular hexagonal grid network, where the network nodes are represented by the corresponding grid coordinates as grid network nodes; the network edges are represented by the corresponding grid coordinate sequence as grid network edges. Here, in order to ensure the consistency of the geometric features of the grid initial network and the original network, the projection point of the center position of the regular hexagonal grid through which the edge of the original network passes on the edge is used as the inner point of the corresponding grid network edge.

[0089] Finally, (step S14) the edges of the grid initial road network are traversed, and the edges completely located in the grid are deleted and merged from the grid initial road network to obtain the grid road network. In other words, if an edge is completely in one grid, that is, the grid road network edges corresponding to the regular hexagonal grids passed by the edge of the original road network and the starting and ending points (path starting point, path end point) of the edge of the original road network are all in the same grid, then the edge corresponding to the path is deleted and merged in the grid initial road network, thereby achieving road network simplification, forming a grid road network, and completely retaining the effective topological structure and geometric characteristics of the path, providing effective road network basic data for subsequent trajectory reconstruction.

[0090] Step S2: construct a potential path set of the target sampling trajectory between consecutive sampling points of the target sampling trajectory based on the grid road network.

[0091] In this application, video surveillance equipment is deployed at the deployment points of the video surveillance system in the urban area to sample the targets in the area and obtain the target's activity trajectory, that is, the target sampling trajectory. Here, the target sampling trajectory is expressed as:

[0092]

[0093] in, is a positive integer, The number of deployment points for the video surveillance system where sampling occurs; For the goal A sampling of video surveillance system deployment points.

[0094] When the sampling time interval between any two consecutive sampling points (i.e., video surveillance system deployment points) is greater than or equal to the preset time length threshold , or, the straight-line distance between any two consecutive sampling points is greater than or equal to the preset distance length threshold , the corresponding target sampling trajectory is segmented; and the segmented target sampling trajectory is used as the minimum unit of trajectory reconstruction, and based on the grid road network, a potential path set of the target sampling trajectory is constructed between the continuous sampling points of the target sampling trajectory through the shortest path search technology.

[0095] That is, the target is located at any two consecutive video surveillance system deployment points (the and When moving at the first and The target sampling trajectory between consecutive sampling points is Said that when sampling points and The sampling time interval of the sampling points to the target is greater than the preset time length threshold , or, sampling points and The straight-line distance between the sampling points is greater than or equal to the preset distance length threshold When and The target sampling trajectory between consecutive sampling points Split to meet a preset time length threshold , preset distance length threshold requirements.

[0096] use Indicates that the target is and The target sampling trajectory when moving between consecutive sampling points A set of multiple potential paths obtained by segmentation; Represents the set of potential paths The potential paths; among them, is a positive integer, Sample trajectory for the target The set of potential paths The total number of potential paths in .

[0097] Step S3: extract and embed the spatiotemporal state features of the potential path to obtain the spatiotemporal state feature value of the potential path in the potential path set to which it belongs.

[0098] Based on the grid road network, a set of potential paths between consecutive sampling points is constructed, where each potential path includes the path length, the target's moving time on the path, and the average moving speed.

[0099]

[0100] Potential paths to the target The above information is described, where Represents potential path The physical length (i.e., path length) of Indicates that the target is in a potential path The moving time on Indicates that the target is in a potential path The average moving speed.

[0101] In this application, when reconstructing the trajectory, it is necessary to determine the similarity between the spatiotemporal state characteristics of the potential path and the target movement, and the spatiotemporal state characteristics of the target movement on the potential path are reflected by the static characteristics of the potential path - path state characteristics: length characteristics, direction characteristics and monitoring coverage characteristics. Among them, the length feature reflects the length characteristics of the potential path when the target moves in the corresponding sampling trajectory segment (target sampling trajectory); the direction feature reflects the direction characteristics of the potential path when the target moves in the corresponding sampling trajectory segment (target sampling trajectory); the monitoring coverage feature reflects the monitoring coverage characteristics of the potential path when the target moves in the corresponding sampling trajectory segment (target sampling trajectory).

[0102] Therefore, when determining the spatiotemporal state feature value of a potential path in its potential path set, first, (step S31) feature extraction is performed on each potential path in the potential path set corresponding to each target sampling trajectory to obtain the path state feature of the corresponding potential path. Specifically, according to the formula:

[0103]

[0104] Determine the length characteristics of potential paths for target movement In the formula, Indicates that the target is and The target sampling trajectory moving between consecutive sampling points; Represents the target sampling trajectory The corresponding potential path set potential paths; Represents potential path The physical length of Represents the target sampling trajectory The corresponding and The straight-line distance between consecutive sampling points; Is a positive integer.

[0105] According to the formula:

[0106]

[0107]

[0108] Determine the directional characteristics of the potential path of the target when it moves. The directional characteristics include: heading deviation and steering bias ; Heading deviation Characterize the target and The degree of direction deviation when moving between consecutive sampling points; steering deviation Characterize the target and The degree of oscillation when zigzagging between consecutive sampling points; is the total number of points contained in the edges corresponding to the potential paths in the grid road network; is the starting point of the potential path and the corresponding edge of the grid road network The angle between the points; is the angle between the start and end points of the potential path; is the edge of the potential path in the grid road network. Point and The angle between the points.

[0109] According to the formula:

[0110]

[0111]

[0112]

[0113] Determine the coverage characteristics of the potential path when the target moves. The coverage characteristics are the characteristics of the deployment points of the video surveillance system where no sampling occurs when the target moves on the potential path, including: monitoring point quantity variables, monitoring coverage length variables, and monitoring coverage position variables.

[0114] In the formula, For the goal and When moving between consecutive sampling points, the number of monitoring points on the potential path is variable; For the goal and When moving between consecutive sampling points, the monitoring coverage length variable on the potential path; For the goal and When moving between consecutive sampling points, the monitoring coverage position variables on the potential path; For the goal and The video surveillance system deployment point where no sampling occurs on the potential path between consecutive sampling points the number of For the goal and The physical length of the monitoring coverage of the deployment point of the video surveillance system where no sampling occurs on the potential path between consecutive sampling points; For the goal and The maximum physical length of the monitoring coverage of the deployment point of the video surveillance system where no sampling occurs on the potential path between consecutive sampling points; For the goal and The minimum physical length of the monitoring coverage of the deployment point of the video surveillance system where no sampling occurs on the potential path between consecutive sampling points; For the goal and The monitoring coverage radius of the deployment points of the video surveillance system where no sampling occurs on the potential path between consecutive sampling points; For the goal and The shortest distance from the path in the monitoring coverage of the video surveillance system deployment point where no sampling occurs to the corresponding video surveillance system deployment point on the potential path moving between consecutive sampling points.

[0115] Then, (step S32) after obtaining the path state characteristics of the potential path, the path state characteristics of the potential path are coupled in time range and space range to obtain a path state characteristic vector of coupled time and space characteristics. Specifically, according to the formula:

[0116]

[0117] Determine the path state characteristic vector of the coupled spatiotemporal characteristics. Indicates that the target is and When moving between consecutive sampling points, the The path state feature vector of potential paths; Respectively represent the target in and When moving between consecutive sampling points, the The length characteristics of the potential paths, heading deviation, turning deviation, monitoring point quantity variables, monitoring coverage length variables, and monitoring coverage position variables; Respectively represent the target in and When moving between consecutive sampling points, the The spatial and temporal scopes corresponding to the potential paths; Is a positive integer.

[0118] Among them, according to the formula:

[0119]

[0120]

[0121] Determine the target and When moving between consecutive sampling points, the The spatial range of potential paths and time range .

[0122] In the formula, is the grid coordinate interval of the grid road network corresponding to the urban area; For the goal and When moving between consecutive sampling points, the The grid coordinate interval of the grid road network corresponding to the potential path; is the number of time slots into which the unit time (1 day) is divided; For the and The sampling time range of the target is the number of consecutive sampling points.

[0123] Finally, (step S33), embed the path state feature vector of the coupled spatiotemporal features to obtain the spatiotemporal state feature value of the potential path in the potential path set. Specifically, according to the embedding model:

[0124]

[0125] Embed the path state feature vector of the coupled spatiotemporal features to obtain the spatiotemporal state feature value of the potential path in the potential path set where it is located .

[0126] In the formula, Indicates that the target is and The target sampling trajectory moving between consecutive sampling points The corresponding potential path set The path state feature vector of potential paths; All are positive integers. is the input layer of the embedding model, and the input vector of the input layer is 14, and the output vector dimension is ; is the output layer of the embedding model, and the input vector dimension of the output layer is , the output vector dimension is 1; is the encoding layer of the embedding model, and the input vector dimension and output vector dimension of the encoding layer are ; are the network parameters of the embedding model. Here, the dimension It is a custom dimension and can be customized to 64, 128, 256 or 512, etc.

[0127] Step S4: perform speed correction on the potential path under the constraint of trajectory entropy, and obtain the spatiotemporal state transfer characteristic values ​​of each potential path between adjacent potential path sets based on the speed model of the target's regular movement.

[0128] Since pedestrian video trajectories have ultra-low sampling characteristics, it is difficult to determine whether pedestrians have stopped between video monitoring points (between sampling points). However, the pedestrian's stop state and duration will affect the uncertainty of the trajectory data. In this application, speed correction is used to effectively reduce the uncertainty of the data itself in trajectory data reconstruction and prevent the spread of uncertainty during the reconstruction process.

[0129] Since the pedestrian's moving speed is in a stable state in the whole area within the trajectory segment, the moving speed between each group of adjacent sampling points in each trajectory segment obeys the normal distribution. Under the space-time constraints of , the moving speed of the target obeys the normal distribution, that is, the speed model of the pedestrian's regular movement is:

[0130]

[0131] in, For the target sampling trajectory The moving speed of the target under the space-time constraints; For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding movement speed variance.

[0132] Specifically, (step S41) according to the formula:

[0133]

[0134] Determine the mean and variance of the target's moving speed. The goal is and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; All are positive integers.

[0135] Indicates that the target is on the target sampling trajectory The corresponding potential path set Potential Paths The average moving speed on Indicates Potential Paths The spatiotemporal state feature value in the potential path set in which it is located; For the goal and The target sampling trajectory moving between consecutive sampling points The variance of the corresponding moving speed; Sample trajectory for the target The number of potential paths in the corresponding potential path set; is a positive integer;

[0136] In this application, the speed correction adopts the sliding window method to correct the target on the target sampling trajectory. The corresponding potential path set Potential Paths The abnormal speed of the front and rear Specifically, (step S42) traverses the target in the target sampling trajectory The potential path in the corresponding potential path set, when or , then according to the formula:

[0137]

[0138] Average moving speed to the target Correction is performed to obtain the target sampling trajectory The corresponding potential path set Potential Paths Corrected movement speed on . is the current step size of the speed; The sliding window step size for speed correction; Sample trajectory for the target The trajectory entropy of Sample trajectory for the target The trajectory entropy of Sample trajectory for the target The trajectory entropy of For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; is the current step length The weight of .

[0139] In order to accurately reconstruct the target's moving path, the characteristics of the target when it transfers from one potential path to another potential path between two consecutive adjacent sampling points are characterized by the spatiotemporal transfer feature, which is specifically manifested as the probability relationship between the target's moving speed on the potential path and the moving speed distribution on the potential path between adjacent sampling points. Specifically, (step S43) traverse the potential paths between two adjacent potential path sets, according to the formula:

[0140]

[0141] Determine the spatiotemporal transfer probability of the target on the potential path between the continuous sampling points corresponding to two adjacent potential path sets .

[0142] In the formula, For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding potential path set potential paths; For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding potential path set potential paths; All are positive integers.

[0143] For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; For the goal and The target sampling trajectory moving between consecutive sampling points The corresponding moving speed mean; For the goal and The target sampling trajectory moving between consecutive sampling points The standard deviation of the corresponding moving speed; For the goal and The target sampling trajectory moving between consecutive sampling points The standard deviation of the corresponding moving speed; Indicates that the target is on the target sampling trajectory The corresponding potential path set Potential Paths The speed of movement on Indicates that the target is on the target sampling trajectory The corresponding potential path set Potential Paths The speed of movement on Sampling trajectory for the target on the target The corresponding potential path set Potential Paths Corrected movement speed on ; Sampling trajectory for the target on the target The corresponding potential path set Potential Paths Corrected movement speed on ; Sample trajectory for the target The trajectory entropy of Sample trajectory for the target The trajectory entropy of is the valid interval range of the normal distribution obeyed by the target's moving speed.

[0144] Finally, (step S44) according to the formula:

[0145]

[0146] Determine the spatiotemporal transfer characteristic values ​​of the potential paths of the target between consecutive sampling points In the formula, For the goal and The target sampling trajectory moving between consecutive sampling points The spatiotemporal transition probability of each pair of potential paths between the corresponding potential path set and its adjacent potential path set.

[0147] Step S5: Based on the pre-constructed conditional random field model, according to the spatiotemporal state characteristic values ​​and the spatiotemporal transfer characteristic values, the path closest to the target's moving spatiotemporal characteristics is inferred from multiple potential paths to complete the trajectory reconstruction of the target.

[0148] In this application, if Figure 7 The conditional random fields (CRF) model constructed as shown in the figure takes the spatiotemporal range between consecutive sampling points as observation values, defines the potential paths between sampling points as different states of the trajectory to be reconstructed, and then infers the optimal state between consecutive sampling points. Considering the uniqueness of pedestrian video trajectory sampling, CRF can effectively handle the non-dependency between the conditional probability of the path state and the observation sequence.

[0149] The conditional random field model is trained based on the neural network gradient descent method to obtain the parameters of the embedded model Specifically, the parameters are trained by the maximum log-likelihood, where the loss function is:

[0150]

[0151] Among them, according to the formula:

[0152]

[0153] Determine the normalized potential function for the conditional random field model .

[0154] According to the formula:

[0155]

[0156] Determine the potential function of the conditional random field model .

[0157] According to the formula:

[0158]

[0159] Determine the partition function for the conditional random field model , is the global normalization factor in trajectory reconstruction.

[0160] Based on the conditional random field model, according to the spatiotemporal state eigenvalues ​​and spatiotemporal transfer eigenvalues, the Viterbi algorithm and dynamic programming technology are used to recursively construct the optimal path by optimizing the possibility to ensure that the sum of the spatiotemporal state characteristics and spatiotemporal transfer characteristics of the selected path in the potential path set is maximized, and the selected path is used as the reconstruction result to achieve trajectory reconstruction of the target.

[0161] Therefore, based on the effectively extracted spatiotemporal state characteristics and spatiotemporal transfer characteristics of the target when it moves on the target sampling trajectory, the conditional random field is used to utilize arbitrary non-dependent characteristics of the observation sequence, and the smaller part of uncertainty is used to constrain the larger part, thereby achieving effective control of uncertainty, reconstructing the target's moving path from low-sampling trajectory data, significantly improving the robustness and accuracy of trajectory reconstruction, and realizing effective reconstruction of pedestrian video trajectory.

[0162] The present application also provides a surveillance video pedestrian trajectory reconstruction system integrating spatiotemporal features, such as Figure 8 As shown, the system includes:

[0163] The gridding unit 801 is configured to establish a grid with a radius of 1 in the urban area where the target activity is located by grid division. A regular hexagonal grid network is obtained, and according to the video surveillance system deployment point and road network data of the urban area, the road network is adaptively rasterized based on the grid coordinate system of the regular hexagonal grid network to obtain a grid road network;

[0164] The path set generation unit 802 is configured to construct a potential path set of the target sampling trajectory between consecutive sampling points of the target sampling trajectory based on a grid road network by using a shortest path search technology; wherein the potential path set includes multiple potential paths; the target sampling trajectory is the activity trajectory of the target collected by the video surveillance equipment deployed at the deployment point of the video surveillance system when the target is active in the urban area;

[0165] The spatiotemporal state feature unit 803 is configured to extract and embed the spatiotemporal state features of the potential path to obtain the spatiotemporal state feature value of the potential path in its potential path set;

[0166] The spatiotemporal transfer feature unit 804 is configured to perform speed correction on the potential path under the trajectory entropy constraint, and obtain the spatiotemporal transfer feature value of each potential path between adjacent potential path sets based on the speed model of the target's regular movement;

[0167] The trajectory reconstruction unit 805, based on the pre-constructed conditional random field model, infers and determines the path that is closest to the target's moving spatiotemporal characteristics from multiple potential paths according to the spatiotemporal state feature values ​​and the spatiotemporal transfer feature values, so as to complete the trajectory reconstruction of the target.

[0168] The surveillance video pedestrian trajectory reconstruction system integrating spatiotemporal features provided in the embodiments of the present application can implement the steps and processes of the surveillance video pedestrian trajectory reconstruction method integrating spatiotemporal features of any of the above embodiments and achieve the same technical effects, which will not be described one by one here.

[0169] In the description of the present invention, the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0170] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features, characterized in that: include: Step S1, establishing a regular hexagonal grid network with a radius of R in the urban area where the target activity is located by grid division, and performing adaptive gridding of the road network based on the grid coordinate system of the regular hexagonal grid network according to the acquired video surveillance system deployment point and road network data of the urban area to obtain a grid road network; Step S2: constructing a potential path set of the target sampling trajectory between consecutive sampling points of the target sampling trajectory based on the grid road network; wherein the potential path set includes multiple potential paths; the target sampling trajectory is the activity trajectory of the target collected by the video surveillance equipment at the deployment point of the video surveillance system when the target is active in the urban area; Step S3, extracting and embedding the spatiotemporal state features of the potential path to obtain the spatiotemporal state feature value of the potential path in the potential path set in which the potential path is located; Step S4, performing speed correction on the potential path under trajectory entropy constraint, and obtaining the spatiotemporal transfer characteristic values ​​of the potential paths between adjacent potential path sets based on the speed model of the target's regular movement; Step S5: Based on the pre-constructed conditional random field model, according to the spatiotemporal state characteristic values ​​and the spatiotemporal transfer characteristic values, the path that is closest to the spatiotemporal characteristics of the movement of the target is inferred from all potential paths to complete the trajectory reconstruction of the target.

2. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 1, characterized in that: In step S1: Step S11, according to the acquired video surveillance system deployment points and node data in the road network in the urban area, the grid direction in the regular hexagonal grid network is adaptively adjusted using a grid parameter optimization technology; Step S12: Mapping the video surveillance system deployment point to the road network, and adding the video surveillance system deployment point as a road network node, and adding the monitorable path of the video surveillance system deployment point as a road network edge; Step S13: Mapping the road network nodes and the road network edges to a grid sequence corresponding to the regular hexagonal grid network to construct an initial grid road network; Step S14, traverse the edges of the grid initial road network, delete and merge the edges that are completely located in the grid from the grid initial road network, and obtain the grid road network.

3. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 1, characterized in that: In step S2, In response to the sampling time interval between any two consecutive sampling points being greater than or equal to a preset time length threshold ΔT, or the straight-line distance between any two consecutive sampling points being greater than or equal to a preset distance length threshold ΔL, the corresponding target sampling trajectory is segmented, and the segmented target sampling trajectory is used as the minimum unit of trajectory reconstruction to construct a potential path set of the target sampling trajectory.

4. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 1, characterized in that: Step S3 includes: Step S31, performing feature extraction on each potential path in the potential path set corresponding to each target sampling trajectory to obtain path state features of the corresponding potential path; Step S32: coupling the path state characteristics of the potential path in time range and space range to obtain a path state characteristic vector of coupled time and space characteristics; Step S33, according to the embedding model: The path state feature vector of the coupled spatiotemporal feature is embedded to obtain the spatiotemporal state feature value E of the potential path in the potential path set where it is located. e ; In the formula, represents the target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The path state feature vector of the kth potential path in the corresponding potential path set; i and k are both positive integers; Linear pre (14, dim) is the input layer of the embedding model, and the input vector dimension of the input layer is 14, and the output vector dimension is dim; Linear pos (dim, 1) is the output layer of the embedding model, and the input vector dimension of the output layer is dim, and the output vector dimension is 1; TransformerEncoder(dim) is the encoding layer of the embedding model, and the input vector dimension and output vector dimension of the encoding layer are both dim; W is the parameter of the embedding model.

5. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 4, characterized in that: The path status features include: length features, direction features and monitoring coverage features; Correspondingly, in step S31, the feature extraction is performed on each potential path in the potential path set corresponding to each target sampling trajectory to obtain the path state feature of the corresponding potential path, including: According to the formula: Determine the length feature var of the potential path of movement of the target len ; In the formula, represents the target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points; Represents the target sampling trajectory The kth potential path in the corresponding potential path set; Represents the potential path The physical length of Represents the target sampling trajectory The straight-line distance between the corresponding i-th and i+1-th consecutive sampling points; i and k are positive integers; According to the formula: Determine the directional characteristics of the potential path when the target moves; wherein the directional characteristics include: heading deviation and steering bias The heading deviation Characterizes the degree of deviation of the direction of the target when it moves between the i-th and i+1-th consecutive sampling points; the steering deviation Characterizes the swing degree of the target when it detours between the i-th and i+1-th consecutive sampling points; X is the total number of points contained in the edge corresponding to the potential path in the grid road network; h x h is the angle between the starting point of the potential path and the xth point on the edge corresponding to the grid road network; X is the angle between the starting point and the end point of the potential path; t x is the angle between the xth point and the x+1th point in the edge corresponding to the potential path in the grid road network; According to the formula: Determine the monitoring coverage characteristics of the potential path when the target moves; wherein the monitoring coverage characteristics are characteristics of the deployment points of the video monitoring system where no sampling occurs when the target moves on the potential path, including: monitoring point quantity variables, monitoring coverage length variables, and monitoring coverage position variables; In the formula, is the variable of the number of monitoring points on the potential path when the target moves between the i-th and i+1-th consecutive sampling points; is the monitoring coverage length variable on the potential path when the target moves between the i-th and i+1-th consecutive sampling points; The monitoring coverage position variable on the potential path when the target moves between the i-th and i+1-th consecutive sampling points; The video surveillance system deployment points where no sampling occurs on the potential path of the target moving between the i-th and i+1-th consecutive sampling points the number of The physical length of the monitoring coverage of the deployment point of the video monitoring system where no sampling occurs on the potential path of the target moving between the i-th and i+1-th consecutive sampling points; The maximum physical length in the monitoring coverage of the deployment points of the video monitoring system where no sampling occurs on the potential path of the target moving between the i-th and i+1-th consecutive sampling points; The minimum physical length in the monitoring coverage of the deployment points of the video monitoring system where no sampling occurs on the potential path of the target moving between the i-th and i+1-th consecutive sampling points; The monitoring coverage radius of the deployment points of the video monitoring system where no sampling occurs on the potential path of the target moving between the i-th and i+1-th consecutive sampling points; The shortest distance from the path in the monitoring coverage of the video surveillance system deployment point where no sampling occurs to the corresponding video surveillance system deployment point on the potential path of the target moving between the i-th and i+1-th consecutive sampling points.

6. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 4, characterized in that: In step S32, obtaining the path state feature vector of the coupled spatiotemporal feature includes: According to the formula: Determining the path state characteristic vector of the coupled spatiotemporal characteristics; In the formula, represents the path state feature vector of the kth potential path when the target moves between the i-th and i+1-th consecutive sampling points; var len , They respectively represent the length characteristics, heading deviation, turning deviation, monitoring point quantity variable, monitoring coverage length variable, and monitoring coverage position variable of the kth potential path when the target moves between the i-th and i+1-th continuous sampling points; Sv lp 、Tv lp Respectively represent the spatial range and time range corresponding to the kth potential path when the target moves between the i-th and i+1-th continuous sampling points; i and k are positive integers; Among them, according to the formula: Determine the spatial range Sv corresponding to the kth potential path when the target moves between the i-th and i+1-th continuous sampling points lp and time range Tv lp ; In the formula, (g x-min , g x-max , g y-min , g y-max , g z-min , g z-max ) is the grid coordinate interval of the grid road network corresponding to the urban area; The grid coordinate interval of the grid road network corresponding to the kth potential path when the target moves between the i-th and i+1-th consecutive sampling points; T slot.count is the number of time slots into which the unit time is divided; (T slot-min , T slot-max ) is the sampling time range of the i-th and i+1-th consecutive sampling points for the target.

7. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 1, characterized in that: Step S4 includes: Step S41, according to the formula: Determining the mean and variance of the moving speed of the target when it moves; In the formula, The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The corresponding moving speed mean; i and k are both positive integers; Indicates that the target is in the target sampling trajectory The kth potential path in the corresponding potential path set The average moving speed on E e (k) represents the potential path described in the kth item The spatiotemporal state characteristic value in the potential path set in which it is located; The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The variance of the corresponding moving speed; K is the target sampling trajectory The number of potential paths in the corresponding potential path set; i, k, K are positive integers; Step S42: traverse the target on the target sampling trajectory The potential path in the corresponding potential path set, in response to or According to the formula: The average movement speed of the target Correction is performed to obtain the target sampling trajectory of the target The kth potential path in the corresponding potential path set Corrected movement speed on Where v is the current step size for speed correction; a is the sliding window step size for speed correction; Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of The target sampling trajectory of the target moving between the ivth and i-v+1th consecutive sampling points The corresponding moving speed mean; The target sampling trajectory of the target moving between the i+vth and i+v+1th continuous sampling points The corresponding moving speed mean; η v is the weight of the current step size v for speed correction; i and v are both positive integers; Step S43, traverse the potential paths between two adjacent potential path sets, according to the formula: Determine the spatiotemporal transition probability P of the target on the potential path between the continuous sampling points corresponding to two adjacent potential path sets t ; In the formula, The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The jth potential path in the corresponding potential path set; The target sampling trajectory of the target moving between the i+1th and i+2th consecutive sampling points The corresponding potential path in the potential path set is the lth potential path; j and l are both positive integers; The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The corresponding moving speed mean; The target sampling trajectory of the target moving between the i+1th and i+2th consecutive sampling points The corresponding moving speed mean; The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The standard deviation of the corresponding moving speed; The target sampling trajectory of the target moving between the i+1th and i+2th consecutive sampling points The standard deviation of the corresponding moving speed; Indicates that the target is in the target sampling trajectory The corresponding potential path set is the jth potential path The speed of movement on Indicates that the target is in the target sampling trajectory The corresponding potential path set is the lth potential path The speed of movement on Sample trajectories for the target The corresponding potential path set is the lth potential path Corrected movement speed on ; Sample trajectories for the target The corresponding potential path set is the jth potential path Corrected movement speed on ; Sampling trajectory for the target The trajectory entropy of Sampling trajectory for the target The trajectory entropy of the target; α is the effective interval range of the normal distribution obeyed by the moving speed of the target; Step S44, according to the formula: Determine the spatiotemporal transfer characteristic value E of the potential path between two consecutive sampling points of the target t ; In the formula, The target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points The spatiotemporal transition probability of each potential path between the corresponding potential path set and its adjacent potential path set.

8. The method for reconstructing pedestrian trajectories from surveillance videos by integrating spatiotemporal features according to claim 7, characterized in that: According to the formula: Determine the target sampling trajectory of the target moving between the i-th and i+1-th consecutive sampling points Trajectory entropy In the formula, Sampling trajectory for the target The corresponding sampling interval time length of the i-th and i+1-th continuous sampling points; ΔT is the preset time length threshold; ω1 is the time length weight coefficient; Sampling trajectory for the target The sampling interval distance length of the corresponding i-th and i+1-th continuous sampling points; ΔL is the preset distance length threshold; ω2 is the distance length weight coefficient.

9. A surveillance video pedestrian trajectory reconstruction system integrating spatiotemporal features, characterized in that: include: A rasterization unit is configured to establish a regular hexagonal grid network with a radius of R in the urban area where the target activity is located through raster division, and to perform adaptive rasterization processing of the road network based on the grid coordinate system of the regular hexagonal grid network according to the acquired video surveillance system deployment point and road network data of the urban area to obtain a grid road network; A path set generating unit is configured to construct a potential path set of the target sampling trajectory between consecutive sampling points of the target sampling trajectory based on the grid road network by using the shortest path search technology; wherein the potential path set includes multiple potential paths; the target sampling trajectory is the activity trajectory of the target collected by the video surveillance equipment deployed at the deployment point of the video surveillance system when the target is active in the urban area; a spatiotemporal state feature unit configured to extract and embed the spatiotemporal state features of the potential path to obtain the spatiotemporal state feature value of the potential path in the potential path set in which the potential path is located; a space-time transfer feature unit configured to perform speed correction on the potential path under the constraint of trajectory entropy, and obtain space-time transfer feature values ​​of the potential paths between adjacent potential path sets based on the speed model of the target's regular movement; The trajectory reconstruction unit, based on a pre-constructed conditional random field model, infers and determines the path that is closest to the moving spatiotemporal characteristics of the target from the multiple potential paths according to the spatiotemporal state characteristic values ​​and the spatiotemporal transfer characteristic values, so as to complete the trajectory reconstruction of the target.

Images