Multi-monitoring device active tracking method and device based on road network behavior rule prediction

Abstract

The present application relates to the field of computer vision and geographic information system, and particularly relates to a multi-monitoring device active tracking method and device based on road network behavior law prediction. Heterogeneous correlation and space-time dependence between monitoring road network devices are coded by using multi-source data composed of road network data, monitoring device external parameters, target clustering data and external time characteristic data, a space-time graph convolution model composed of a graph convolution network and a time series neural network is used to construct a target space-time behavior law model under the monitoring road network, and target detection technology and image coordinate to road network latitude and longitude coordinate conversion technology are used to combine the moving direction and speed of the target to assist in realizing fast and accurate sparse multi-monitoring device active tracking and trajectory description.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and geographic information systems, and in particular to a multi-monitoring device active tracking method and device based on road network behavior pattern prediction. Background Art

[0002] In an urban public security monitoring system, by real-time monitoring and tracking moving targets (such as small-scale objects like people, non-motor vehicles, motor vehicles, etc.) on the road surface, it helps urban managers better understand and analyze the activity patterns of urban people and the occurrence of events, and has important functions and significance in the fields of behavior analysis, criminal investigation analysis, traffic management, etc. To achieve this goal, cross-camera linkage technology between multiple monitoring devices is usually adopted to ensure the smooth tracking of targets between different cameras, so as to obtain all-round and multi-angle target information for in-depth analysis and processing.

[0003] At present, in the fields of computer vision and geographic information systems, there are mainly two methods for cross-camera tracking of multiple monitoring targets: high-density cross-camera tracking and low-density cross-camera tracking. Among them, the former is the current mainstream method, which is usually applicable to areas with high-density coverage of monitoring devices, and realizes cross-camera linkage without blind spots through technologies such as boundary coincidence relay. However, in real-world scenarios, the monitoring areas are usually of low density, and the application of such methods is very limited and the practicability is poor. For low-density cross-camera tracking technology, due to the independence of low-density monitoring devices from each other and the lack of linkage between them, the target often loses tracking after switching cameras, resulting in the missing and fragmentation of the tracking target's path. Generally, external signals such as satellite positioning are required to assist tracking, or it depends on algorithms to perform global full-volume search on all monitoring devices in the area, which requires a large amount of equipment or computing power. In most scenarios, the cross-camera recognition of targets by low-density multiple monitoring devices still needs to rely on manual intervention to summarize, induce and predict the moving routes of targets, which is time-consuming and laborious, bringing huge challenges to the current target tracking technology that relies on modern urban monitoring systems. Summary of the Invention

[0004] To solve the problems existing in the prior art, embodiments of the present invention provide a multi-monitoring device active tracking method and device based on road network behavior pattern prediction, which use multi-source data composed of road network data, external parameters of monitoring devices, target aggregation data, and external time feature data to encode the heterogeneous correlation and spatio-temporal dependence between monitoring road network devices, use a spatio-temporal graph convolutional model composed of a graph convolutional network and a temporal neural network to construct a target spatio-temporal behavior pattern model under the monitoring road network, and use target detection technology and image coordinate to road network longitude and latitude coordinate conversion technology to combine the moving direction and speed of the target to assist in realizing fast and accurate sparse multi-monitoring device active tracking and trajectory characterization.

[0005] To solve any of the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0006] This invention provides a method for active tracking of multiple monitoring devices based on road network behavior pattern prediction, comprising:

[0007] Receive target image features and initial monitoring points of the tracked target;

[0008] From the video images of the initial monitoring point, determine the consecutive video image frames containing the target image features as target video image frames, and determine the position information of the tracked target in each target video image frame;

[0009] Based on the position information of the tracked target in each target video image frame, calculate the movement information of the tracked target in the monitoring road network where the initial monitoring point is located;

[0010] The current multi-directional traffic data of the monitored road network is obtained, and the current multi-directional traffic data and the movement information are calculated using a pre-built spatiotemporal convolution model to obtain the predicted monitoring point of the tracked target in the monitored road network at the next moment.

[0011] The predicted arrival time of the tracked target to each predicted monitoring point is calculated based on the distance between each predicted monitoring point and the initial monitoring point, as well as the movement information.

[0012] For each predicted monitoring point, the target image feature is searched in the video image corresponding to the predicted arrival time, and the predicted monitoring point where the target image feature is found is taken as the actual monitoring point.

[0013] Furthermore, calculating the movement information of the tracked target within the monitoring network where the initial monitoring point is located, based on the target's position information in each target video image frame, further includes:

[0014] For each target video image frame, the location information is transformed from the image coordinate system of the target video image frame to the latitude and longitude coordinate system of the monitoring road network where the initial monitoring point is located, so as to obtain the latitude and longitude coordinates of the tracked target;

[0015] The latitude and longitude coordinates of the tracked target corresponding to all target video image frames are fitted in the order of the target video image frames to obtain the motion information.

[0016] Furthermore, the steps for constructing the spatiotemporal convolutional model include:

[0017] Obtain information parameters of each monitoring point in the monitoring road network, and calculate the spatial dependency characteristics of the monitoring road network based on the information parameters of each monitoring point;

[0018] Historical traffic data of each monitoring point is obtained, and the time dependence characteristics of the monitored road network are calculated based on the historical traffic data of each monitoring point.

[0019] The spatial dependency features and temporal dependency features are fused to obtain fused features;

[0020] The fused features are input into a two-layer graph convolutional network for graph convolution calculation to obtain the spatial dynamic correlation features of each monitoring point.

[0021] Obtain the external time characteristic data of the monitored road network;

[0022] The external time feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation to obtain the spatiotemporal convolution model.

[0023] Furthermore, the external temporal feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation, resulting in the spatiotemporal convolution model, which further includes:

[0024] The spatial dynamic correlation features are connected at each time step through a flattening layer and a fully connected layer, and then input into the encoding layer.

[0025] The external time feature data is normalized and fully connected, and then input into each time step of the decoding layer.

[0026] The recurrent neural network is trained to obtain the spatiotemporal convolution model.

[0027] Furthermore, the method also includes:

[0028] Obtain the current external time characteristic data of the monitored road network;

[0029] The calculation of the current multi-directional traffic data and the movement information using a pre-built spatiotemporal convolution model to obtain the predicted monitoring point location of the tracked target in the monitoring road network at the next moment further includes:

[0030] The current multi-directional traffic data and the current external time feature data are input into the spatiotemporal convolution model for calculation to obtain the predicted monitoring point of the tracked target in the monitoring road network at the next moment.

[0031] Furthermore, the method also includes:

[0032] Calculate the probability that the tracked target will appear at each predicted monitoring point in the next time step;

[0033] For each predicted monitoring point, searching for the target image features in the video image corresponding to the predicted arrival time further includes:

[0034] The target image features are searched sequentially in the video images corresponding to the predicted arrival time for the predicted monitoring points, in descending order of probability value.

[0035] Furthermore, after using the predicted monitoring points based on the identified target image features as the actual monitoring points, the method further includes:

[0036] Using the actual monitoring point as the initial monitoring point, the step of repeatedly determining the continuous video image frames containing the target image feature from the video images of the initial monitoring point as the target video image frames is repeated until the target image feature does not appear in the video images corresponding to all predicted monitoring points.

[0037] The movement information of the tracked target at all initial monitoring points is fitted to obtain the complete movement information of the tracked target.

[0038] Furthermore, searching for the target image features in the video image corresponding to the predicted arrival time further includes:

[0039] Based on the predicted arrival time, determine the corresponding time window;

[0040] The target image features are searched in the video images corresponding to the time window.

[0041] On the other hand, embodiments of the present invention also provide a multi-monitoring device for active tracking based on road network behavior pattern prediction, comprising:

[0042] The target tracking information receiving unit is used to receive the target image features and initial monitoring points of the tracked target;

[0043] The target tracking unit is used to determine, from the video images of the initial monitoring point, consecutive video image frames containing the target image features as target video image frames, and to determine the position information of the tracked target in each target video image frame;

[0044] A target movement calculation unit is used to calculate the movement information of the target in the monitoring road network where the initial monitoring point is located, based on the position information of the target in each target video image frame.

[0045] The predictive monitoring point calculation unit is used to acquire the current multi-directional traffic data of the monitored road network, and use a pre-built spatiotemporal convolution model to calculate the current multi-directional traffic data and the movement information to obtain the predicted monitoring point of the tracked target in the monitored road network at the next moment.

[0046] The predicted arrival time calculation unit is used to calculate the predicted arrival time of the tracked target to each predicted monitoring point based on the distance between each predicted monitoring point and the initial monitoring point and the movement information.

[0047] The actual monitoring point determination unit is used to search for the target image features in the video image corresponding to the predicted arrival time for each predicted monitoring point, and to take the predicted monitoring point where the target image features are found as the actual monitoring point.

[0048] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described method.

[0049] The beneficial effects of the embodiments of the present invention are as follows:

[0050] This specification's embodiments pre-construct a spatiotemporal convolutional model for predicting target movement in a monitored road network. When tracking a target, the target's location information is first determined, and then its movement information is calculated based on this location information. Current multi-directional traffic data of the monitored road network is acquired, and the pre-constructed spatiotemporal convolutional model is used to calculate the predicted monitoring point in the monitored road network where the target will appear at the next moment. Then, based on the distance between each predicted monitoring point and the initial monitoring point, and the movement information, the predicted arrival time of the target at each predicted monitoring point is calculated. For each predicted monitoring point, the target image features are searched in the video image corresponding to the predicted arrival time, and the predicted monitoring point where the target image features are found is taken as the actual monitoring point. This specification's embodiments achieve accurate tracking of targets in a monitored road network. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1The diagram shown is a flowchart of a multi-monitoring device active tracking method based on road network behavior pattern prediction in an embodiment of the present invention.

[0053] Figure 2 The diagram shown is a schematic representation of a multi-monitoring device for active tracking based on road network behavior pattern prediction in an embodiment of the present invention.

[0054] Figure 3 The diagram shown is a structural schematic of a computer device in an embodiment of the present invention.

[0055] [Explanation of Figure Markers]:

[0056] 201. Target tracking information receiving unit;

[0057] 202. Target tracking and search unit;

[0058] 203. Target movement calculation unit;

[0059] 204. Predictive monitoring point calculation unit;

[0060] 205. Predicted arrival time calculation unit;

[0061] 206. Unit for determining actual monitoring points;

[0062] 302. Computer equipment;

[0063] 304. Processing equipment;

[0064] 306. Storage resources;

[0065] 308. Drive mechanism;

[0066] 310. Input / output module;

[0067] 312. Input devices;

[0068] 314. Output devices;

[0069] 316. Presentation equipment;

[0070] 318. Graphical User Interface;

[0071] 320. Network interface;

[0072] 322. Communication link;

[0073] 324. Communication bus. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0076] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0077] To address the problems existing in the prior art, this invention provides a multi-monitoring active tracking method based on road network behavior pattern prediction. It utilizes multi-source data composed of road network data, monitoring device extrinsic parameters, target clustering data, and external time feature data to encode the heterogeneous correlations and spatiotemporal dependencies among monitoring road network devices. A spatiotemporal graph convolution model composed of graph convolutional networks and temporal neural networks is used to construct a target spatiotemporal behavior pattern model under the monitoring road network. Target detection technology and image coordinate to road network latitude and longitude coordinate conversion technology are used, combined with the target's movement direction and speed, to assist in achieving fast and accurate active tracking and trajectory characterization of sparse multi-monitoring devices.

[0078] Figure 1 The diagram illustrates a flowchart of a multi-monitoring device active tracking method based on road network behavior pattern prediction, according to an embodiment of the present invention. This diagram depicts the process of tracking targets in a monitored road network, but based on conventional or non-creative labor, it may include more or fewer operational steps. The order of steps listed in the embodiment is merely one possible execution order among many, and does not represent the only possible execution order. In actual system or device products, the methods shown in the embodiment or the accompanying drawings can be executed sequentially or in parallel. Specifically, as shown... Figure 1As shown, the method may include:

[0079] Step 101: Receive the target image features and initial monitoring points of the tracked target;

[0080] Step 102: Determine consecutive video image frames containing the target image features from the video images of the initial monitoring point as target video image frames, and determine the position information of the tracked target in each target video image frame;

[0081] In this step, for the target being tracked, firstly, based on the input target image features and the initial monitoring point, target detection and other technologies are used to perform feature comparison in the video image of the initial monitoring point, continuously locate and track the target, and obtain the specific position of the target in the video image (anchor frame coordinates B).

[0082] The formula for expressing the anchor frame coordinates B is: B = [x] B ,y B ,w B ,h B ].

[0083] Among them, (x B ,y B ) represents the bottom coordinates of the target anchor frame, w B h represents the width of the target anchor frame. B Indicates the height of the target anchor frame.

[0084] Step 103: Calculate the movement information of the tracked target in the monitoring road network where the initial monitoring point is located based on the position information of the tracked target in each target video image frame;

[0085] In this embodiment of the specification, calculating the movement information of the tracked target in the monitoring road network where the initial monitoring point is located, based on the position information of the tracked target in each target video image frame, further includes:

[0086] For each target video image frame, the location information is transformed from the image coordinate system of the target video image frame to the latitude and longitude coordinate system of the monitoring road network where the initial monitoring point is located, so as to obtain the latitude and longitude coordinates of the tracked target;

[0087] The latitude and longitude coordinates of the tracked target corresponding to all target video image frames are fitted in the order of the target video image frames to obtain the motion information.

[0088] Specifically, for video images containing detected targets, the latitude and longitude coordinates of the targets on the road network map are calculated using an image coordinate to road network coordinate conversion method. Here, the field-of-view coordinate mapping method is used. That is, the nearest monitoring distance V of the monitoring equipment in the vertical field of view is first calculated using the height, tilt angle, horizontal field of view h, and vertical field of view v of the monitoring equipment. min and the furthest monitoring distance V max Then, through the bottom coordinates (x) of the target anchor frame B ,y B Calculate the tilt angle θ and distance d of the target relative to the orientation of the monitoring equipment in the road network coordinate system. s Finally, based on the longitude (lat), latitude (lon), and orientation of the monitoring equipment... And the actual latitude and longitude of the target in the road network coordinate system are calculated from the Earth's radius R;

[0089] Formula for calculating the tilt angle θ: θ = x B ·hh / 2;

[0090] relative distance d s Calculation formula:

[0091] Formula for calculating longitude lat′:

[0092] Latitude Lon′ calculation formula:

[0093] Step 104: Obtain the current multi-directional traffic data of the monitored road network, and use a pre-built spatiotemporal convolution model to calculate the current multi-directional traffic data and the movement information to obtain the predicted monitoring point of the tracked target appearing in the monitored road network at the next moment;

[0094] In the embodiments of this specification, the steps for constructing the spatiotemporal convolution model include:

[0095] Obtain information parameters of each monitoring point in the monitoring road network, and calculate the spatial dependency characteristics of the monitoring road network based on the information parameters of each monitoring point;

[0096] Historical traffic data of each monitoring point is obtained, and the time dependence characteristics of the monitored road network are calculated based on the historical traffic data of each monitoring point.

[0097] The spatial dependency features and temporal dependency features are fused to obtain fused features;

[0098] The fused features are input into a two-layer graph convolutional network for graph convolution calculation to obtain the spatial dynamic correlation features of each monitoring point.

[0099] Obtain the external time characteristic data of the monitored road network;

[0100] The external time feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation to obtain the spatiotemporal convolution model.

[0101] Specifically, the monitoring equipment is first calibrated by recording its installation height, pitch angle, orientation, and latitude and longitude coordinates. Based on the road network data and the latitude and longitude coordinates of the monitoring equipment, the non-Euclidean road network distance and connectivity between the monitoring equipment are obtained.

[0102] Then, the spatial topology of the monitored road network is defined as an undirected graph G = (V, E, M), where E represents the set of monitoring points in the monitored road network, N represents the number of monitoring points, and E∈e ij This represents a set of edges from monitoring point i to point j. This is an adjacency matrix, representing the correlation weights between monitoring points. The larger the weight, the higher the correlation between the points.

[0103] Then, based on the road network distance, road network connectivity, and orientation between monitoring points, the spatial dependency of the monitoring road network is established, which is represented as a spatial distance map G. 1 = (V, E, M) 1 Spatial structure diagram G 2 = (V, E, M) 2 ) and spatial orientation diagram G 3 = (V, E, M) 3 Among them, the spatial distance map G 1 The actual distance d between monitoring points on the road network ij The reciprocal of the value is used as the spatial distance-related adjacency matrix M. 1 Weight K 1 Spatial structure correlation diagram G 2 Using the neighborhood connectivity between monitoring points as the spatial structure-related adjacency matrix M 2 Weight K 2 Spatial Orientation Diagram G 3 The horizontal orientation angle θ between monitoring points ij (Maximum phase difference is 180 degrees) serves as the spatial orientation-related adjacency matrix M. 3 Weight K 3 ;

[0104] Weight K 1 Calculation formula: K 1 =1 / d ij ;

[0105] Weight K 2 Calculation formula:

[0106] Weight K 3 Calculation formula: K 3 =1-sinθ ij ;

[0107] Then, based on the target cluster data of each monitoring point, the time dependency of the monitoring road network is established. A one-hour sliding window is used to move and aggregate the target traffic data at each monitoring point, with each cycle lasting ten minutes (based on an average target movement speed of 1.2 meters per second and an average monitoring point distance of 800 meters), to obtain the historical traffic time series. Furthermore, considering the directional movement of targets between monitoring points, for the target traffic data of each cycle, the existing traffic data is further grouped according to the monitoring points where these targets will appear in the next cycle (i.e., outflow directionality), resulting in a multi-directional traffic time series matrix.

[0108] Then, based on external time characteristic data composed of weekdays, holidays, time of day, weather, and temperature, the monitoring network's time autocorrelation, periodicity, and trend modeling are further enhanced. Among these, the weekday item d... t Indicates the current day of the week; the holiday item h t Indicates whether it is a holiday or public holiday; time item p t This indicates the current time of day (each hour is a time interval), weather item w t The categories are categorized into eight types of weather: sunny, cloudy, overcast, light rain, showers, fog, and thunderstorms. The temperature category is c. t This indicates the current temperature in degrees Celsius.

[0109] Work item d t Formula: d t = r, r = 1, 2, ..., 7;

[0110] Holiday items h t formula:

[0111] Time period item o t Formula: o t =s,s=1,2,...,24;

[0112] Weather item w t Formula:w t =k, k = 1, 2, ..., 8;

[0113] Then, the adjacency matrix is ​​standardized by first adding the identity matrix I to the existing adjacency matrix M. N To prevent information about the graph nodes themselves from being omitted, and in the identity matrix I NA trainable hyperparameter λ is added before the node; the larger λ is, the more important the node's features are, resulting in a self-transitive adjacency matrix. Then use the degree matrix. (Adjacency matrix) (degree) of adjacency matrix A weighted average of rows and columns is applied to ensure that nodes with low degrees have a greater influence on their neighbors, while nodes with high degrees have their influence distributed among more neighbors, resulting in a less significant impact on their neighbors. This process yields the standardized adjacency matrix.

[0114] Self-transitive adjacency matrix Calculation formula:

[0115] Normalized adjacency matrix Calculation formula:

[0116] Then the normalized adjacency matrix Repeat the expansion L times to make it consistent with the multi-directional flow time series matrix. Having the same dimension, and after performing feature fusion operations (matrix multiplication) on multiple graphs separately, they are input into a two-layer graph convolutional network for graph convolution calculation to extract the spatial dynamic correlation between monitored road site locations. After the graph convolution calculation is completed, the spatial distance graph G is processed. 1 Spatial structure diagram G 2 And spatial orientation diagram G 3 The graph convolutional network outputs O i Weighted fusion is performed, where the hyperparameter β of the weighted fusion is... i It can also be learned;

[0117] Formula for calculating graph convolutional networks:

[0118] Where W0 and W1 are convolution kernels; X represents the input features, which is the multi-directional flow time series matrix.

[0119] Output graph weighted fusion calculation formula: O = β 1 O 1 +β 2 O 2 +β 3 O 3 ;

[0120] Where O represents the output after fusion.

[0121] Then, a recurrent neural network (LSTM Long Short-Term Memory network) with an encoding-decoding structure is used to establish the temporal dynamic correlation of the monitored road network. First, each time step of the weighted fusion output graph O is connected through a flattening layer (·) and a fully connected layer FCN (·), and then input into the LSTM encoding layer with a feature size of [L, N×N]. Then, the external temporal feature data E = [d, h, o, t] is normalized by a normalization operation (·) and a fully connected operation, and then input into each time step of the LSTM decoding layer. The LSTM neural network and the graph convolutional network together form a spatiotemporal graph convolutional model to model the spatiotemporal behavior of targets under the monitored road network.

[0122] LSTM neural network calculation formula:

[0123] f(O, E) = LSTM decoder (LSTM encoder (FCN(flatten(O))))+norm(FCN(E))).

[0124] When calculating the predicted monitoring points, the current external time characteristic data of the monitored road network is obtained;

[0125] The calculation of the current multi-directional traffic data and the movement information using a pre-built spatiotemporal convolution model to obtain the predicted monitoring point location of the tracked target in the monitoring road network at the next moment further includes:

[0126] The current multi-directional traffic data and the current external time feature data are input into the spatiotemporal convolution model for calculation to obtain the predicted monitoring point of the tracked target in the monitoring road network at the next moment.

[0127] Step 105: Calculate the predicted arrival time of the tracking target to each predicted monitoring point based on the distance between each predicted monitoring point and the initial monitoring point and the movement information.

[0128] In the embodiments of this specification, the movement information includes the direction of movement and the speed of movement. Therefore, the predicted arrival time of the tracked target to each predicted monitoring point can be calculated based on the distance and the speed of movement.

[0129] Step 106: For each predicted monitoring point, search for the target image feature in the video image corresponding to the predicted arrival time, and take the predicted monitoring point where the target image feature is found as the actual monitoring point.

[0130] Specifically, searching for the target image features in the video image corresponding to the predicted arrival time further includes:

[0131] Based on the predicted arrival time, determine the corresponding time window;

[0132] The target image features are searched in the video images corresponding to the time window.

[0133] In the embodiments of this specification, the time window can be an empirical or experimental value, such as 3 minutes, and this specification does not impose any limitations. For example, the target image features are searched in the video images corresponding to the time windows 3 minutes before and after the predicted arrival time (a total of 6 minutes).

[0134] In this embodiment of the specification, to speed up the search, the method further includes:

[0135] Calculate the probability that the tracked target will appear at each predicted monitoring point in the next time step;

[0136] For each predicted monitoring point, searching for the target image features in the video image corresponding to the predicted arrival time further includes:

[0137] The target image features are searched sequentially in the video images corresponding to the predicted arrival time for the predicted monitoring points, in descending order of probability value.

[0138] In the embodiments of this specification, the spatiotemporal convolutional model can predict the probability of target flow direction at the current monitoring point, and weight and fuse this probability based on the actual movement direction of the target (monitoring points in the same direction as the target have a higher probability), to obtain the final retrieval priority of monitoring points where the target may appear in the next moment. The retrieval priority represents the predicted probability of the tracked target appearing at the monitoring point in the next moment. This improves search efficiency.

[0139] Furthermore, after using the predicted monitoring points based on the identified target image features as the actual monitoring points, the method further includes:

[0140] Using the actual monitoring point as the initial monitoring point, the step of repeatedly determining the continuous video image frames containing the target image feature from the video images of the initial monitoring point as the target video image frames is repeated until the target image feature does not appear in the video images corresponding to all predicted monitoring points.

[0141] The movement information of the tracked target at all initial monitoring points is fitted to obtain the complete movement information of the tracked target.

[0142] This specification's embodiments pre-construct a spatiotemporal convolutional model for predicting target movement in a monitored road network. When tracking a target, the target's location information is first determined, and then its movement information is calculated based on that location information. Current multi-directional traffic data of the monitored road network is obtained from target clustering data. The pre-constructed spatiotemporal convolutional model is used to calculate the predicted monitoring point where the tracked target will appear in the monitored road network at the next moment. Then, based on the distance between each predicted monitoring point and the initial monitoring point, and the movement information, the predicted arrival time of the tracked target at each predicted monitoring point is calculated. For each predicted monitoring point, the target image features are searched in the video image corresponding to the predicted arrival time. The predicted monitoring point where the target image features are found is taken as the actual monitoring point. This specification's embodiments achieve accurate tracking of targets in a monitored road network.

[0143] Based on the same inventive concept, embodiments of the present invention also provide an active tracking device for multiple monitoring devices based on road network behavior pattern prediction, such as... Figure 2 As shown, it includes:

[0144] The target tracking information receiving unit 201 is used to receive the target image features and initial monitoring points of the tracked target.

[0145] The target tracking unit 202 is used to determine, from the video images of the initial monitoring point, consecutive video image frames containing the target image features as target video image frames, and to determine the position information of the tracked target in each target video image frame;

[0146] The target movement calculation unit 203 is used to calculate the movement information of the target in the monitoring road network where the initial monitoring point is located, based on the position information of the target in each target video image frame.

[0147] The predictive monitoring point calculation unit 204 is used to acquire the current multi-directional traffic data of the monitored road network, and use a pre-built spatiotemporal convolution model to calculate the current multi-directional traffic data and the movement information to obtain the predicted monitoring point of the tracked target in the monitored road network at the next moment.

[0148] The predicted arrival time calculation unit 205 is used to calculate the predicted arrival time of the tracking target to each predicted monitoring point based on the distance between each predicted monitoring point and the initial monitoring point and the movement information.

[0149] The actual monitoring point determination unit 206 is used to search for the target image feature in the video image corresponding to the predicted arrival time for each predicted monitoring point, and to take the predicted monitoring point where the target image feature is found as the actual monitoring point.

[0150] The beneficial effects obtained by the above-described device are the same as those obtained by the above-described method, and will not be described in detail in the embodiments of the present invention.

[0151] like Figure 3 The diagram illustrates the structure of a computer device according to an embodiment of the present invention. The apparatus in this invention can be the computer device described in this embodiment, executing the method of the present invention as described above. The computer device 302 may include one or more processing devices 304, such as one or more central processing units (CPUs), each processing unit implementing one or more hardware threads. The computer device 302 may also include any storage resource 306 for storing information of any kind, such as code, settings, data, etc. Non-limitingly, for example, the storage resource 306 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any storage resource can use any technology to store information. Further, any storage resource can provide volatile or non-volatile retention of information. Further, any storage resource may represent a fixed or removable component of the computer device 302. In one case, when the processing device 304 executes associated instructions stored in any storage resource or combination of storage resources, the computer device 302 can perform any operation of the associated instructions. The computer device 302 also includes one or more drive mechanisms 308 for interacting with any storage resource, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0152] Computer device 302 may also include an input / output module 310 (I / O) for receiving various inputs (via input device 312) and providing various outputs (via output device 314). A specific output mechanism may include a presentation device 316 and an associated graphical user interface (GUI) 318. In other embodiments, the input / output module 310 (I / O), input device 312, and output device 314 may be omitted, and the device may function solely as a computer device within a network. Computer device 302 may also include one or more network interfaces 320 for exchanging data with other devices via one or more communication links 322. One or more communication buses 324 couple the components described above together.

[0153] Communication link 322 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 322 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0154] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0155] This invention also provides a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to perform the above-described method.

[0156] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0157] It should also be understood that, in the embodiments of the present invention, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the present invention, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0158] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0160] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, or may be electrical, mechanical, or other forms of connection.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0162] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this invention should not be construed as a limitation of this invention.

Claims

1. A multi-monitoring device active tracking method based on road network behavior pattern prediction, characterized in that, include: Receive target image features and initial monitoring points of the tracked target; From the video images of the initial monitoring point, determine the consecutive video image frames containing the target image features as target video image frames, and determine the position information of the tracked target in each target video image frame; Based on the position information of the tracked target in each target video image frame, calculate the movement information of the tracked target in the monitoring road network where the initial monitoring point is located; The current multi-directional traffic data of the monitored road network is obtained, and the current multi-directional traffic data and the movement information are calculated using a pre-built spatiotemporal convolution model to obtain the predicted monitoring point of the tracked target in the monitored road network at the next moment. The predicted arrival time of the tracked target to each predicted monitoring point is calculated based on the distance between each predicted monitoring point and the initial monitoring point, as well as the movement information. For each predicted monitoring point, the target image feature is searched in the video image corresponding to the predicted arrival time, and the predicted monitoring point where the target image feature is found is taken as the actual monitoring point. The steps for constructing the spatiotemporal convolution model include: Information parameters of each monitoring point in the monitoring road network are obtained, and the spatial dependency characteristics of the monitoring road network are calculated based on the information parameters of each monitoring point. Specifically, the monitoring equipment is calibrated, and its installation height, pitch angle, orientation, and latitude / longitude coordinates are recorded. Based on the road network data and the latitude / longitude coordinates of the monitoring equipment, the non-Euclidean road network distances and connectivity between the monitoring equipment are obtained. The spatial topology of the monitoring road network is defined as an undirected graph. ,in This represents the set of monitoring points in the monitored road network. N This represents the number of monitoring points. Representative monitoring points i Arrive at the point j A set of edges, The adjacency matrix represents the correlation weights between monitoring points. The spatial dependencies of the monitoring road network are established based on the non-Euclidean road network distances, connectivity, and orientations between monitoring points, and are represented as spatial distance graphs. Spatial structure diagram and spatial orientation diagram Among them, spatial distance map Based on the actual distance of the road network between monitoring points The reciprocal of the value is used as the spatial distance-related adjacency matrix. weight Spatial structure related diagrams Using the neighborhood connectivity between monitoring points as the spatial structure-related adjacency matrix weight Spatial orientation diagram The horizontal angle between monitoring points As a spatial orientation-related adjacency matrix weight ; Weight Calculation formula: ; Weight Calculation formula: ; Weight Calculation formula: ; Historical traffic data of each monitoring point is acquired, and the time dependence characteristics of the monitoring road network are calculated based on the historical traffic data of each monitoring point. Specifically, the time dependence of the monitoring road network is established based on the target cluster data of each monitoring point. The target traffic data of each monitoring point is moved and summarized in a one-hour sliding window, with each ten-minute period as a cycle, to obtain the historical traffic time series. For the target traffic data of each cycle, the target traffic data is grouped according to the monitoring points where these targets appear in the next cycle to obtain a multi-directional traffic time series matrix. The spatial dependency features and temporal dependency features are fused to obtain fused features; The fused features are input into a two-layer graph convolutional network for graph convolution calculation to obtain the spatial dynamic correlation features of each monitoring point. Obtain the external time characteristic data of the monitored road network; The external time feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation to obtain the spatiotemporal convolution model.

2. The method according to claim 1, characterized in that, Calculating the movement information of the tracked target in the monitoring road network where the initial monitoring point is located, based on the position information of the tracked target in each target video image frame, further includes: For each target video image frame, the location information is transformed from the image coordinate system of the target video image frame to the latitude and longitude coordinate system of the monitoring road network where the initial monitoring point is located, so as to obtain the latitude and longitude coordinates of the tracked target; The latitude and longitude coordinates of the tracked target corresponding to all target video image frames are fitted in the order of the target video image frames to obtain the motion information.

3. The method according to claim 1, characterized in that, The external time feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation, and the resulting spatiotemporal convolution model further includes: The spatial dynamic correlation features are connected at each time step through a flattening layer and a fully connected layer, and then input into the encoding layer. The external time feature data is normalized and fully connected, and then input into each time step of the decoding layer. The recurrent neural network is trained to obtain the spatiotemporal convolution model.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the current external time characteristic data of the monitored road network; The calculation of the current multi-directional traffic data and the movement information using a pre-built spatiotemporal convolution model to obtain the predicted monitoring point location of the tracked target in the monitoring road network at the next moment further includes: The current multi-directional traffic data and the current external time feature data are input into the spatiotemporal convolution model for calculation to obtain the predicted monitoring point of the tracked target in the monitoring road network at the next moment.

5. The method according to claim 4, characterized in that, The method further includes: Calculate the probability that the tracked target will appear at each predicted monitoring point in the next time step; For each predicted monitoring point, searching for the target image features in the video image corresponding to the predicted arrival time further includes: The target image features are searched sequentially in the video images corresponding to the predicted arrival time for the predicted monitoring points, in descending order of probability value.

6. The method according to claim 1, characterized in that, After using the predicted monitoring points based on the found target image features as the actual monitoring points, the method further includes: Using the actual monitoring point as the initial monitoring point, the step of repeatedly determining the continuous video image frames containing the target image feature from the video images of the initial monitoring point as the target video image frames is repeated until the target image feature does not appear in the video images corresponding to all predicted monitoring points. The movement information of the tracked target at all initial monitoring points is fitted to obtain the complete movement information of the tracked target.

7. The method according to claim 1, characterized in that, Searching for the target image features in the video image corresponding to the predicted arrival time further includes: Based on the predicted arrival time, determine the corresponding time window; The target image features are searched in the video images corresponding to the time window.

8. A multi-monitoring active tracking device based on road network behavior pattern prediction, characterized in that, include: The target tracking information receiving unit is used to receive the target image features and initial monitoring points of the tracked target; The target tracking unit is used to determine, from the video images of the initial monitoring point, consecutive video image frames containing the target image features as target video image frames, and to determine the position information of the tracked target in each target video image frame; A target movement calculation unit is used to calculate the movement information of the target in the monitoring road network where the initial monitoring point is located, based on the position information of the target in each target video image frame. The predictive monitoring point calculation unit is used to acquire the current multi-directional traffic data of the monitored road network, and use a pre-built spatiotemporal convolution model to calculate the current multi-directional traffic data and the movement information to obtain the predicted monitoring point of the tracked target in the monitored road network at the next moment. The predicted arrival time calculation unit is used to calculate the predicted arrival time of the tracked target to each predicted monitoring point based on the distance between each predicted monitoring point and the initial monitoring point and the movement information. The actual monitoring point determination unit is used to search for the target image feature in the video image corresponding to the predicted arrival time for each predicted monitoring point, and to take the predicted monitoring point where the target image feature is found as the actual monitoring point. The steps for constructing the spatiotemporal convolution model include: Information parameters of each monitoring point in the monitoring road network are obtained, and the spatial dependency characteristics of the monitoring road network are calculated based on the information parameters of each monitoring point. Specifically, the monitoring equipment is calibrated, and its installation height, pitch angle, orientation, and latitude / longitude coordinates are recorded. Based on the road network data and the latitude / longitude coordinates of the monitoring equipment, the non-Euclidean road network distances and connectivity between the monitoring equipment are obtained. The spatial topology of the monitoring road network is defined as an undirected graph. ,in This represents the set of monitoring points in the monitored road network. N This represents the number of monitoring points. Representative monitoring points i Arrive at the point j A set of edges, The adjacency matrix represents the correlation weights between monitoring points. The spatial dependencies of the monitoring road network are established based on the non-Euclidean road network distances, connectivity, and orientations between monitoring points, and are represented as spatial distance graphs. Spatial structure diagram and spatial orientation diagram Among them, spatial distance map Based on the actual distance of the road network between monitoring points The reciprocal of the value is used as the spatial distance-related adjacency matrix. weight Spatial structure related diagrams Using the neighborhood connectivity between monitoring points as the spatial structure-related adjacency matrix weight Spatial orientation diagram The horizontal angle between monitoring points As a spatial orientation-related adjacency matrix weight ; Weight Calculation formula: ; Weight Calculation formula: ; Weight Calculation formula: ; Historical traffic data of each monitoring point is acquired, and the time dependence characteristics of the monitoring road network are calculated based on the historical traffic data of each monitoring point. Specifically, the time dependence of the monitoring road network is established based on the target cluster data of each monitoring point. The target traffic data of each monitoring point is moved and summarized in a one-hour sliding window, with each ten-minute period as a cycle, to obtain the historical traffic time series. For the target traffic data of each cycle, the target traffic data is grouped according to the monitoring points where these targets appear in the next cycle to obtain a multi-directional traffic time series matrix. The spatial dependency features and temporal dependency features are fused to obtain fused features; The fused features are input into a two-layer graph convolutional network for graph convolution calculation to obtain the spatial dynamic correlation features of each monitoring point. Obtain the external time characteristic data of the monitored road network; The external time feature data and the spatial dynamic correlation features of each monitoring point are input into a recurrent neural network for calculation to obtain the spatiotemporal convolution model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 7.

Images