Trajectory prediction methods, devices and storage media

Abstract

This application provides a trajectory prediction method, apparatus, and storage medium, relating to the field of trajectory prediction. The method includes: acquiring a historical trajectory dataset and first trajectory data of a target object, the first trajectory data including N trajectory sequences traversed by the target object, each trajectory sequence representing the target object's movement from one camera node to another in a monitoring system; calculating the prior probability and conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data; calculating the predicted probability of the at least one candidate camera node based on the prior probability and conditional probability; and determining the candidate camera node with the highest predicted probability from the at least one candidate camera node as the next camera node the target object will traverse.

Description

Technical Field

[0001] This application relates to the field of trajectory prediction, and more particularly to a trajectory prediction method, apparatus and storage medium. Background Technology

[0002] Intelligent surveillance, as part of social security systems, is ubiquitous in public places. Current technologies typically utilize real-time monitoring systems to detect and track targets in real time.

[0003] Specifically, the prediction of target trajectories across cameras usually involves constructing a topology between cameras and inferring the next possible camera view based on the topological association between cameras and the position where the pedestrian disappears from the surveillance screen.

[0004] However, in complex scenarios, such as those involving multiple spaces including indoors and outdoors, and underground, the spaces are discontinuous, making it difficult to establish topological relationships between cameras and thus hindering accurate trajectory prediction. Therefore, existing intelligent surveillance systems have relatively low accuracy in predicting pedestrian trajectories. Summary of the Invention

[0005] This application provides a trajectory prediction method, apparatus, and storage medium, which can solve the problem of low accuracy in predicting pedestrian trajectories in existing intelligent monitoring systems.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] In a first aspect, this application provides a trajectory prediction method, which includes: acquiring a historical trajectory dataset and first trajectory data of a target object, the first trajectory data including N trajectory sequences traversed by the target object, each trajectory sequence being the trajectory of the target object moving from one camera node to another in a monitoring system; calculating the prior probability and conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data, the prior probability being the probability that at least one candidate camera node is the next camera node after the current camera node, and the conditional probability being the probability that the historical trajectory dataset contains the first trajectory data; calculating the predicted probability of at least one candidate camera node based on the prior probability and the conditional probability; determining the candidate camera node with the highest predicted probability from the at least one candidate camera node as the next camera node to be traversed by the target object; wherein, at least one candidate camera node is a camera node on a target trajectory in at least one target combined trajectory, the target trajectory being a trajectory matching at least two adjacent camera nodes corresponding to the first trajectory data, and at least one target combined trajectory being a trajectory obtained by combining trajectory sequences between every two adjacent camera nodes in the historical dataset.

[0008] Based on the above technical solution, the trajectory prediction method provided in this application can calculate the prior probability of at least one candidate camera node in the monitoring system based on the acquired historical trajectory dataset and the trajectory data of the object to be queried, thereby obtaining the probability that the at least one candidate camera node is the next camera node of the current camera node; and calculate the conditional probability of at least one candidate camera node in the monitoring system, thereby obtaining the probability that the historical trajectory dataset contains the first trajectory data. Then, based on the prior probability and conditional probability, the predicted probability of the at least one candidate camera node is calculated, and the candidate camera node with the highest predicted probability is taken as the next camera node passed by the object to be queried. Therefore, by using the historical trajectory dataset and the trajectory data of the object to be queried, the trajectory prediction of the object to be queried can be transformed into the prediction of the probability of the next camera node most likely to be passed by the object to be queried. Thus, even when it is impossible to construct the indoor and outdoor environmental topology, the next camera node most likely to be passed by the object to be queried can be predicted, thereby realizing the trajectory prediction of the object to be queried and improving the accuracy of the trajectory prediction of the object to be queried.

[0009] In a first possible implementation of the first aspect, the state transition probability between every two adjacent camera nodes in the monitoring system is calculated based on the historical trajectory dataset, and a state transition probability matrix is ​​generated; based on the state transition probability matrix and the first trajectory data, the path probability of the first trajectory data is calculated, where the path probability is the probability of the first trajectory data appearing in the historical trajectory dataset; based on the path probability and the state transition probability matrix, the conditional probability of at least one candidate camera node is calculated.

[0010] In the second possible implementation of the first aspect, the above-mentioned calculation of the state transition probability between every two adjacent camera nodes in the monitoring system based on the historical trajectory dataset, and the generation of the state transition probability matrix, includes: establishing a Markov model based on the historical trajectory dataset, wherein the Markov model is used to characterize that there are two directed transition states between every two adjacent camera nodes; using the Markov model, calculating the state transition probability between every two adjacent camera nodes in the monitoring system according to the target parameters of each camera node in the monitoring system, and generating the state transition probability matrix, wherein the target parameters include the attribute information of each camera node at the current time.

[0011] In a third possible implementation of the first aspect, the above-mentioned calculation of the path probability of the first trajectory data based on the state transition probability matrix and the first trajectory data includes: determining at least one target state transition probability; each target state transition probability is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix; and calculating the product of all target state transition probabilities in the at least one target state transition probability to obtain the path probability of the first trajectory data.

[0012] In the fourth possible implementation of the first aspect, the above-mentioned calculation of the conditional probability of at least one candidate camera node based on the path probability and the state transition probability matrix includes: calculating the target ratio of the trajectory data of the target trajectory passing through at least one candidate camera node to the target ratio of the trajectory data of the historical trajectory dataset passing through at least one candidate camera node based on the state transition probability matrix; and calculating the conditional probability of at least one candidate camera node based on the target ratio.

[0013] In the fifth possible implementation of the first aspect, after obtaining the historical trajectory dataset and the first trajectory data of the object to be queried, the method further includes: decomposing the historical trajectory dataset into at least one second trajectory data, the second trajectory data containing trajectory sequences between every two adjacent camera nodes in the monitoring system; combining the at least one second trajectory sequence to obtain at least one target combined trajectory, each target combined trajectory including multiple second trajectory sequences from the at least one second trajectory sequence; matching the at least one target combined trajectory with the first trajectory data; for each target combined trajectory, if a second trajectory sequence in a target combined trajectory matches the first trajectory data, then all camera nodes in a target combined trajectory are used as candidate camera nodes.

[0014] Secondly, this application provides a trajectory prediction device, comprising: an acquisition module, a calculation module, and a determination module. The acquisition module is used to acquire a historical trajectory dataset and first trajectory data of a target object. The first trajectory data includes N trajectory sequences traversed by the target object, each trajectory sequence representing the trajectory of the target object moving from one camera node to another in a monitoring system. The calculation module is used to calculate, based on the historical trajectory dataset and the first trajectory data acquired by the acquisition module, a prior probability and a conditional probability of at least one candidate camera node in the monitoring system. The prior probability is the probability that at least one candidate camera node is the next camera node after the current camera node, and the conditional probability is the probability that the historical trajectory dataset contains the first trajectory data. The calculation module is further used to calculate a predicted probability of at least one candidate camera node based on the prior probability and the conditional probability. The aforementioned determining module is used to determine the candidate camera node with the highest predicted probability calculated by the calculation module from at least one candidate camera node, as the next camera node that the query object will pass through; wherein, at least one candidate camera node is a camera node on the target trajectory in at least one target combined trajectory, the target trajectory is the trajectory that matches at least two adjacent camera nodes corresponding to the first trajectory data, and at least one target combined trajectory is the trajectory obtained by combining the trajectory sequences between every two adjacent camera nodes in the historical dataset.

[0015] In the first possible implementation of the second aspect, the aforementioned calculation module is specifically used to: calculate the state transition probability between every two adjacent camera nodes in the monitoring system based on the historical trajectory dataset, and generate a state transition probability matrix; calculate the path probability of the first trajectory data based on the state transition probability matrix and the first trajectory data, wherein the path probability is the probability of the first trajectory data appearing in the historical trajectory dataset; and calculate the conditional probability of at least one candidate camera node based on the path probability and the state transition probability matrix.

[0016] In the second possible implementation of the second aspect, the above-mentioned calculation module is specifically used to: establish a Markov model based on the historical trajectory dataset, the Markov model being used to characterize that there are two directed transition states between every two adjacent camera nodes; using the Markov model, according to the target parameters of each camera node in the monitoring system, calculate the state transition probability between every two adjacent camera nodes in the monitoring system, and generate a state transition probability matrix, the target parameters including the attribute information of each camera node at the current time.

[0017] In a third possible implementation of the second aspect, the aforementioned calculation module is specifically used to: determine at least one target state transition probability; each target state transition probability is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix; calculate the product of all target state transition probabilities in the at least one target state transition probability to obtain the path probability of the first trajectory data.

[0018] Thirdly, this application provides a trajectory prediction apparatus, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the trajectory prediction method as described in the first aspect and any possible implementation of the first aspect.

[0019] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the trajectory prediction method as described in the first aspect and any possible implementation thereof.

[0020] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a trajectory prediction device, cause the trajectory prediction device to perform the trajectory prediction method as described in the first aspect and any possible implementation thereof.

[0021] In a sixth aspect, embodiments of this application provide a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the trajectory prediction method as described in the first aspect and any possible implementation thereof.

[0022] Specifically, the chip provided in this application embodiment also includes a memory for storing computer programs or instructions. Attached Figure Description

[0023] Figure 1 One of the flowcharts for a trajectory prediction method provided in the embodiments of this application;

[0024] Figure 2 A second flowchart illustrating a trajectory prediction method provided in this application embodiment;

[0025] Figure 3 One of the schematic diagrams illustrating a trajectory prediction method provided in this application embodiment;

[0026] Figure 4 This is a second schematic diagram illustrating an example of a trajectory prediction method provided in this application.

[0027] Figure 5This is a schematic diagram of the structure of a trajectory prediction device provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of another trajectory prediction device provided in an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0030] The trajectory prediction method, apparatus, and storage medium provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings, which can solve the problem of low accuracy in predicting pedestrian trajectories in existing intelligent monitoring systems.

[0031] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0032] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0033] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0034] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0036] With the rapid integration and development of communication technologies and infrastructure such as the Internet, various mobile terminals such as smart electronic devices and wearable devices have greatly facilitated people's daily lives, making it possible to obtain real and massive amounts of mobile trajectory data, thus providing data support for the mining of mobile target trajectory patterns. In today's big data era, trajectory data is complex and diverse, not only containing the behavioral trajectory patterns of mobile targets, but also recording the real-time dynamics of the targets.

[0037] Intelligent surveillance, as part of social security systems, is ubiquitous in public places, such as crowded train stations, supermarkets, and schools, where numerous cameras are installed. Real-time monitoring systems can detect and track targets in real time, and promptly alert authorities in case of anomalies. However, for a long time, people have simply saved the surveillance videos and only reviewed them when an anomaly occurred, manually capturing the footage—a time-consuming and labor-intensive process that lacks real-time response measures.

[0038] The problem of target tracking across cameras addresses the target tracking issue in real-time monitoring systems. Cross-camera tracking builds upon single-camera target tracking, delving deeper into data fusion and matching for targets spanning multiple cameras to achieve the tracking objective. Research on cross-camera personnel tracking has significant practical implications because real-world targets often move between locations and span considerable time periods, inevitably leading to targets appearing on multiple different cameras for tracking.

[0039] Pedestrian trajectory prediction mainly refers to predicting the specific movement path of a target in the future given a portion of the target's current movement trajectory. By analyzing the current trajectory data of the target, the trajectory pattern of the target can be obtained, and the unknown trajectory of the moving target can be predicted, providing support for decision-making and implementation.

[0040] However, in cross-camera personnel tracking within intelligent surveillance systems, which combine numerous cameras in a complex area, cross-camera target tracking presents challenges. These challenges arise because the complex area includes mixed indoor and outdoor, underground and above-ground zones, making camera topology generation difficult; insufficient camera density leads to monitoring interruptions; and discontinuous target trajectories make target trajectory prediction difficult. Furthermore, for continuous target tracking, current target detection algorithms cannot simultaneously perform real-time target detection on multiple surveillance videos, resulting in low accuracy for real-time surveillance video target detection tasks.

[0041] To address the issue of low accuracy in pedestrian trajectory prediction in existing intelligent monitoring systems, this application provides a trajectory prediction method. Based on acquired historical trajectory datasets and the trajectory data of the target object, the method calculates the prior probability of at least one candidate camera node in the monitoring system, obtaining the probability that this candidate camera node is the next camera node after the current camera node. It also calculates the conditional probability of the at least one candidate camera node in the monitoring system, obtaining the probability that the historical trajectory dataset contains the first trajectory data. Then, based on the prior and conditional probabilities, the predicted probability of the at least one candidate camera node is calculated, and the candidate camera node with the highest predicted probability is selected as the next camera node the target object will pass through. Therefore, by using historical trajectory datasets and the trajectory data of the target object, trajectory prediction can be transformed into a prediction of the probability of the target object passing through the next camera node most likely to be reached. This allows for trajectory prediction of the target object even when indoor / outdoor environmental topology cannot be constructed, improving the accuracy of trajectory prediction.

[0042] The trajectory prediction method in this application is applied to scenarios where targets are continuously tracked through video surveillance.

[0043] like Figure 1 The diagram shows a flowchart of a trajectory prediction method provided in an embodiment of this application. The method includes the following steps S101 to S104:

[0044] S101. Obtain the historical trajectory dataset and the first trajectory data of the object to be queried.

[0045] The first trajectory data includes N trajectory sequences of the object to be queried, each trajectory sequence being the trajectory of the object to be queried moving from one camera node to another in the monitoring system.

[0046] In this embodiment of the application, the monitoring system described above can be a monitoring network composed of multiple camera nodes, where each camera node represents a camera and the camera's capture range.

[0047] Optionally, the above trajectory sequence can be the trajectory of one camera node moving directly to another camera node in the monitoring system, or it can be the trajectory of one camera node moving to another camera node through other camera nodes in the monitoring system.

[0048] Optionally, the aforementioned first trajectory data may be a known, but incomplete, trajectory of the object to be queried.

[0049] In this embodiment of the application, the aforementioned historical trajectory dataset can be a collection of regularized historical trajectory data.

[0050] Optionally, the original historical trajectory data can be obtained first. This trajectory data may consist of a series of discrete points with unstructured attributes and cannot be used directly.

[0051] Optionally, the original historical trajectory data may include attribute information such as the camera node's device number, location information, and time information.

[0052] For example, a single piece of original historical trajectory data may include: 1, 30.624806, 104.136604, 1, 2014 / 8 / 3 21:18:46. Here, 1 is the device number of the camera node corresponding to the original historical trajectory data, 30.624806 and 104.136604 are the latitude and longitude of the camera node, and 2014 / 8 / 3 21:18:46 is the time information corresponding to the camera node and the original historical trajectory data.

[0053] Alternatively, the original historical trajectory data can be regularized using formula (1).

[0054] T = {<l1,t1> ,<l2,t2> , ... <l N , t N >} Formula (1)

[0055] Where T can represent regularized historical trajectory data, l N It can represent the Nth camera node, t N The time corresponding to this historical trajectory data can be represented as t. N time.

[0056] It can be understood that, through formula (1), the original historical trajectory can be transformed into a sequence of binary pairs with spatiotemporal characteristics, and the discrete points in the original historical trajectory data can be transformed into ordered sequence elements. Furthermore, each sequence element consists of the camera node number and the shooting time.

[0057] It should be noted that when regularizing the original historical trajectory data using formula (1), the original historical trajectory data may contain a large number of dwell points of the queried object, resulting in excessively long individual trajectory data, and the trajectory probability will also be different under different spatiotemporal backgrounds. Therefore, the original historical trajectory data can be segmented according to the time threshold, and the dwell points of the same queried object at the same location can be merged into one element of the binary sequence.

[0058] Optionally, after step S101 above, the trajectory prediction method provided in this application embodiment may further include steps S105 to S108 as described below.

[0059] S105. Decompose the historical trajectory dataset into at least one second trajectory data.

[0060] The second trajectory data mentioned above includes the trajectory sequence between every two adjacent camera nodes in the monitoring system.

[0061] Optionally, "decomposing the historical trajectory dataset into at least one second trajectory data" can be understood as: decomposing the historical trajectory dataset into at least one trajectory sequence.

[0062] It should be noted that the adjacency relationship between two adjacent camera nodes can be based on the adjacency of the binary elements representing the camera nodes in the historical trajectory data, rather than geographical or topological adjacency.

[0063] S106. Combine at least one second trajectory data to obtain at least one target combined trajectory.

[0064] Each target combined trajectory includes multiple second trajectory data from at least one second trajectory data.

[0065] In this embodiment of the application, each target combined trajectory may consist of at least two second trajectory data.

[0066] Optionally, different target combination trajectories may contain some of the same second trajectory data.

[0067] For example, a historical trajectory dataset can be decomposed into trajectory data 1, trajectory data 2, and trajectory data 3. Then, by combining the decomposed trajectory data, target combined trajectory 1, target combined trajectory 2, target combined trajectory 3, and target combined trajectory 4 can be obtained. Specifically, target combined trajectory 1 includes trajectory data 1 and trajectory data 2, target combined trajectory 2 includes trajectory data 1 and trajectory data 3, target combined trajectory 3 includes trajectory data 2 and trajectory data 3, and target combined trajectory 4 includes trajectory data 1, trajectory data 2, and trajectory data 3.

[0068] S107. Match at least one target combination trajectory with the first trajectory data.

[0069] S108. For each target combination trajectory, if the second trajectory sequence in a target combination trajectory matches the first trajectory data, then all camera nodes in a target combination trajectory are used as candidate camera nodes.

[0070] Optionally, at least one second trajectory sequence can be combined to obtain at least one target combined trajectory, and then the at least one target combined trajectory can be matched with the second trajectory data. For each target combined trajectory, if at least one trajectory sequence matches the second trajectory data, all camera nodes in the target combined trajectory can be used as candidate camera nodes.

[0071] Therefore, if a target trajectory partially matches the first trajectory data, the camera nodes on the target trajectory can be used as candidate camera nodes for the queried object to pass through at the next moment. This significantly increases the number of possible trajectories, thereby improving the accuracy of trajectory prediction.

[0072] S102. Based on the historical trajectory dataset and the first trajectory data, calculate the prior probability and conditional probability of at least one candidate camera node in the monitoring system.

[0073] The prior probability is the probability that at least one candidate camera node is the next camera node after the current camera node, and the conditional probability is the probability that the historical trajectory dataset includes the first trajectory data.

[0074] In this embodiment of the application, the above-mentioned at least one candidate camera node is a camera node on the target trajectory in the target combined trajectory. The target trajectory is a trajectory that matches at least two camera nodes corresponding to the first trajectory data. The target combined trajectory is a trajectory obtained by combining the trajectory sequences between every two adjacent camera nodes in the historical trajectory dataset.

[0075] It should be noted that compared to the multiple paths available through complex road monitoring networks, existing historical trajectories exhibit sparsity, which limits trajectory prediction based on historical trajectories. Specifically, because partial matching of the query trajectory can only find one historical trajectory, a particular camera may directly become the predicted trajectory, while other cameras have a zero probability. However, in reality, other cameras still have a certain probability of being the next camera the query target will pass through. This phenomenon, where the next possible candidate camera nodes for the query trajectory are partially missing due to limitations in historical trajectory data, is called the trajectory sparsity problem.

[0076] In this embodiment of the application, the probability that each candidate camera node is the next camera node of the current camera node can be used as the prior probability of each candidate camera node.

[0077] Optionally, combined Figure 1 ,like Figure 2As shown, the step S102 above, "calculating the conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data", may include the following steps S102a to S102c.

[0078] S102a. Based on the historical trajectory dataset, calculate the state transition probability between every two adjacent camera nodes in the monitoring system, and generate a state transition probability matrix.

[0079] Optionally, step S102a may include steps a and b below.

[0080] a. Establish a Markov model based on historical trajectory datasets.

[0081] Among them, the Markov model can be used to characterize that there are two directed transition states between every two adjacent camera nodes in the monitoring system.

[0082] In this embodiment, there are two directed transition states between adjacent camera nodes, which can represent the movement of the object to be queried from one camera node to another.

[0083] b. Using a Markov model, based on the target parameters of each camera node in the monitoring system, calculate the state transition probability between every two adjacent camera nodes in the monitoring system, and generate a state transition probability matrix.

[0084] The target parameters may include the attribute information of each camera node at the current moment.

[0085] Optionally, the attribute information of a camera node may include the camera node's device number, location information, time information, status information, and information of its neighboring camera nodes.

[0086] In this embodiment of the application, the state transition probability can be the conditional probability of the current value given a certain value at the previous time.

[0087] For example, in the Markov model P st =P(x i =t|x i-1 In the expression =s), we can represent the probability that the current state is t given that the previous state was s.

[0088] Optionally, a state transition probability matrix can be generated based on the state transition probability between every two adjacent camera nodes in the monitoring system to quantify the relationship between adjacent camera nodes.

[0089] For example, such as Figure 3As shown, the monitoring system contains nine camera nodes: l1, l2, l3, l4, l5, l6, l7, l8, and l9. There are two directed transition states p between every two adjacent camera nodes; for example, there is a transition state p between camera node l1 and camera node l2. 12 and p 21 Calculate the state transition probability p between all adjacent camera nodes, such as... Figure 4 As shown, the state transition probability matrix M can be generated, thereby quantifying the relationship between adjacent camera nodes.

[0090] It should be noted that when performing the above steps S102a to S102c, step S102 specifically includes steps S102a to S102d, wherein step S102d is: calculating the prior probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data.

[0091] In this embodiment of the application, the process of establishing the state transition probability matrix based on the Markov model is actually the process of decomposing all trajectories in the historical trajectory dataset. Each element in the matrix represents the probability of one-step reachability between adjacent camera nodes after decomposition.

[0092] Optionally, after decomposing the historical trajectory, the number of decomposed trajectories increases dramatically. Therefore, the probability of a camera node reaching another camera node in s steps can be calculated using formula (2), that is, by raising each element in the matrix to the power of s. Finally, the total transition probability between one camera node and another camera node is the sum of the transition probabilities corresponding to all possible steps s.

[0093]

[0094] Where i can represent the starting camera node of the trajectory, j can represent the ending camera node of the trajectory, i→j can represent the trajectory, L can represent the state of the camera node on the trajectory, and 1.2 is empirical data.

[0095] Thus, by calculating the state transition probabilities between all adjacent camera nodes and generating a state transition probability matrix, the relationship between adjacent camera nodes can be quantified. Therefore, the prediction of the trajectory of the object to be queried can be transformed into the prediction of the next camera node of the current camera node, avoiding the construction of camera node topology relationships in complex environments and improving the accuracy of trajectory prediction.

[0096] S102b: Calculate the path probability of the first trajectory data based on the state transition probability matrix and the first trajectory data.

[0097] The path probability can be the probability that the first trajectory data appears in the historical trajectory dataset.

[0098] Optionally, step S102b may include steps c to d below.

[0099] c. Determine at least one target state transition probability.

[0100] Here, the state transition probability of each target is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix.

[0101] d. Calculate the product of all target state transition probabilities in at least one target state transition probability to obtain the path probability of the first trajectory data.

[0102] In this way, the path probability of the first trajectory data can be calculated by using the state transition probabilities between all adjacent camera nodes in the first trajectory data, so as to calculate the conditional probability of the candidate camera nodes in the subsequent calculation.

[0103] S102c: Based on the path probability and state transition probability matrix, calculate the conditional probability of at least one candidate camera node.

[0104] Optionally, step S102c may include steps e and f below.

[0105] e. Based on the state transition probability matrix, calculate the target trajectory data that passes through at least one candidate camera node, and the target ratio of the trajectory data that passes through at least one candidate camera node in the historical trajectory dataset.

[0106] In this embodiment of the application, the ratio of the number of trajectories that pass through at least one candidate camera node to the number of trajectories that pass through at least one candidate camera node in the historical trajectory dataset can be calculated based on the state transition probability matrix, i.e., the target ratio.

[0107] Optionally, using formula (3), for each candidate camera, the number of target trajectories passing through the candidate camera node can be counted first, and then the number of all trajectories passing through the candidate camera node in the historical trajectory dataset can be counted. Finally, the target ratio can be calculated.

[0108]

[0109] Wherein, the numerator represents the number of trajectories that pass through at least one of the candidate camera nodes, and the denominator represents the total number of trajectories that pass through that single camera node, T q It can represent the first trajectory data, l j It can represent the j-th camera node.

[0110] f. Based on the target ratio, calculate the conditional probability of at least one candidate camera node.

[0111] Optionally, the conditional probability of at least one candidate camera node can be calculated using formula (4).

[0112]

[0113] Where s can represent the starting camera node, j can represent the ending camera node, and p can represent any camera node between the starting and ending camera nodes (i.e., a candidate camera node). c→j p can represent the state transition probability from camera node c to camera node j in the state transition probability matrix. s→j P(T) can represent the state transition probability from camera node s to camera node j in the state transition probability matrix. q ) can represent the path probability of the first trajectory data.

[0114] Thus, by calculating the conditional probability based on the path probability and the ratio of trajectory data of the target trajectory passing through at least one candidate camera node to trajectory data of the historical trajectory dataset passing through at least one candidate camera node, when there is no trajectory in the historical trajectory dataset that matches the first trajectory data, the camera node on the target trajectory can be used as a candidate camera node to predict the trajectory of the object to be queried, thereby improving the accuracy of trajectory prediction.

[0115] S103. Based on prior probability and conditional probability, calculate the predicted probability of at least one candidate camera node.

[0116] In this embodiment of the application, the predicted probability of at least one candidate camera node can be calculated by using the Bayesian inference formula to obtain formula (5) based on the prior probability and conditional probability.

[0117]

[0118] Among them, P(T) q |l j P(l) can represent the conditional probability of camera node j. j ) can represent the prior probability of camera node j, and k can represent the k-th camera node among the n camera nodes in the monitoring system.

[0119] In this embodiment of the application, a trajectory prediction model can be established based on the Bayesian inference idea, which transforms the prediction of the trajectory of the object to be queried into calculating the probability that each camera node may be the next camera node that the object to be queried will pass under the condition that a known part of the trajectory of the object to be queried (i.e., the first trajectory data) exists.

[0120] S104. Determine the candidate camera node with the highest predicted probability from at least one candidate camera node, and use it as the next camera node that the queried object will pass through.

[0121] It is understandable that the candidate camera node with the highest predicted probability is the next camera node that the queried object is most likely to move to.

[0122] The trajectory prediction method provided in this application can calculate the prior probability of at least one candidate camera node in the monitoring system based on the acquired historical trajectory dataset and the trajectory data of the object to be queried, thereby obtaining the probability that the at least one candidate camera node is the next camera node of the current camera node; and calculate the conditional probability of the at least one candidate camera node in the monitoring system, thereby obtaining the probability that the historical trajectory dataset contains the first trajectory data. Then, based on the prior probability and conditional probability, the predicted probability of the at least one candidate camera node is calculated, and the candidate camera node with the highest predicted probability is taken as the next camera node passed by the object to be queried. Therefore, based on the historical trajectory dataset and the trajectory data of the object to be queried, the trajectory prediction of the object to be queried can be transformed into the prediction of the probability of the next camera node most likely to be passed by the object to be queried. Thus, even when it is impossible to construct the indoor and outdoor environmental topology, the next camera node most likely to be passed by the object to be queried can be predicted, thereby realizing the trajectory prediction of the object to be queried and improving the accuracy of the trajectory prediction of the object to be queried.

[0123] This application embodiment can divide the trajectory prediction device into functional modules or functional units according to the above method examples. For example, each function can be divided into its own functional modules or functional units, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module or functional unit. The module or unit division in this application embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation.

[0124] like Figure 5 The diagram shown is a schematic diagram of a trajectory prediction device provided in an embodiment of this application. The device includes: an acquisition module 51, a calculation module 52, and a determination module 53.

[0125] The acquisition module 51 is used to acquire historical trajectory dataset and first trajectory data of the object to be queried. The first trajectory data includes N trajectory sequences traversed by the object to be queried, each trajectory sequence being the trajectory of the object to be queried moving from one camera node to another in the monitoring system. The calculation module 52 is used to calculate the prior probability and conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data acquired by the acquisition module 51. The prior probability is the probability that at least one candidate camera node is the next camera node after the current camera node, and the conditional probability is the probability that the historical trajectory dataset contains the first trajectory data. Based on the prior probability and conditional probability, the predicted probability of at least one candidate camera node is calculated. The determination module 53 is used to determine the candidate camera node with the highest predicted probability calculated by the calculation module 52 from at least one candidate camera node, as the next camera node to be traversed by the object to be queried. At least one candidate camera node is a camera node on the target trajectory in at least one target combination trajectory. The target trajectory is the trajectory matching at least two adjacent camera nodes corresponding to the first trajectory data. At least one target combination trajectory is the trajectory obtained by combining the trajectory sequences between every two adjacent camera nodes in the historical dataset.

[0126] In one possible implementation, the aforementioned calculation module 52 is specifically used for:

[0127] Based on the historical trajectory dataset, calculate the state transition probability between every two adjacent camera nodes in the monitoring system and generate a state transition probability matrix; based on the state transition probability matrix and the first trajectory data, calculate the path probability of the first trajectory data, which is the probability of the first trajectory data appearing in the historical trajectory dataset; based on the path probability and the state transition probability matrix, calculate the conditional probability of at least one candidate camera node.

[0128] In one possible implementation, the aforementioned calculation module 52 is specifically used for:

[0129] Based on historical trajectory datasets, a Markov model is established. The Markov model is used to represent that there are two directed transition states between every two adjacent camera nodes. Using the Markov model, the state transition probability between every two adjacent camera nodes in the monitoring system is calculated according to the target parameters of each camera node in the monitoring system, and a state transition probability matrix is ​​generated. The target parameters include the attribute information of each camera node at the current time.

[0130] In one possible implementation, the aforementioned calculation module 52 is specifically used for:

[0131] Determine at least one target state transition probability; each target state transition probability is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix; calculate the product of all target state transition probabilities in the at least one target state transition probability to obtain the path probability of the first trajectory data.

[0132] In one possible implementation, the aforementioned calculation module 52 is specifically used for:

[0133] Based on the state transition probability matrix, calculate the target trajectory data that passes through at least one candidate camera node, and the target ratio of the trajectory data that passes through at least one candidate camera node in the historical trajectory dataset.

[0134] Based on the target ratio, calculate the conditional probability of at least one candidate camera node.

[0135] In one possible implementation, the aforementioned calculation module 52 is specifically used for:

[0136] Based on the state transition probability matrix, calculate the target trajectory data that passes through at least one candidate camera node, and the target ratio of the trajectory data that passes through at least one candidate camera node in the historical trajectory dataset.

[0137] Based on the target ratio, calculate the conditional probability of at least one candidate camera node.

[0138] In one possible implementation, the above-mentioned device further includes: a decomposition module, a combination module, and a matching module.

[0139] The aforementioned decomposition module is used to decompose the historical trajectory dataset into at least one second trajectory data after the acquisition module acquires the historical trajectory dataset and the first trajectory data of the object to be queried. The second trajectory data contains the trajectory sequence between every two adjacent camera nodes in the monitoring system. The aforementioned combination module is used to combine the at least one second trajectory data obtained by the decomposition module to obtain at least one target combined trajectory. Each target combined trajectory includes multiple second trajectory data from the at least one second trajectory data. The aforementioned matching module is used to match the at least one target combined trajectory obtained by the combination module with the first trajectory data. The aforementioned determination module 53 is also used to, for each target combined trajectory, if the second trajectory data in a target combined trajectory matches the first trajectory data, then all camera nodes in a target combined trajectory are selected as candidate camera nodes.

[0140] The trajectory prediction device provided in this application can calculate the prior probability of at least one candidate camera node in the monitoring system based on the acquired historical trajectory dataset and the trajectory data of the object to be queried, thereby obtaining the probability that the at least one candidate camera node is the next camera node of the current camera node; and calculate the conditional probability of the at least one candidate camera node in the monitoring system, thereby obtaining the probability that the historical trajectory dataset contains the first trajectory data. Then, based on the prior probability and conditional probability, the predicted probability of the at least one candidate camera node is calculated, and the candidate camera node with the highest predicted probability is taken as the next camera node passed by the object to be queried. Therefore, based on the historical trajectory dataset and the trajectory data of the object to be queried, the trajectory prediction of the object to be queried can be transformed into the prediction of the probability of the next camera node most likely to be passed by the object to be queried. Thus, even when it is impossible to construct the indoor and outdoor environmental topology, the next camera node most likely to be passed by the object to be queried can be predicted, thereby realizing the trajectory prediction of the object to be queried and improving the accuracy of the trajectory prediction of the object to be queried.

[0141] When implemented in hardware, the acquisition module 51, calculation module 52, determination module 53, decomposition module, combination module, and matching module in this embodiment can be integrated onto the processor. Specific implementation methods are as follows: Figure 6 As shown.

[0142] Figure 6 A schematic diagram of another possible structure of the trajectory prediction device involved in the above embodiments is shown. The trajectory prediction device includes a processor 302 and a communication interface 303. The processor 302 is used to control and manage the operation of the trajectory prediction device, for example, executing the steps performed by the acquisition module 51, calculation module 52, determination module 53, decomposition module, combination module, and matching module, and / or other processes for performing the techniques described herein. The communication interface 303 is used to support communication between the trajectory prediction device and other network entities. The trajectory prediction device may also include a memory 301 and a bus 304, the memory 301 being used to store the program code and data of the trajectory prediction device.

[0143] The memory 301 may be a memory in a trajectory prediction device, and the memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state drive; the memory may also include a combination of the above types of memory.

[0144] The processor 302 described above can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0145] Bus 304 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 304 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0146] Figure 7 This is a schematic diagram of the structure of chip 170 provided in an embodiment of this application. Chip 170 includes one or more (including two) processors 1710 and communication interfaces 1730.

[0147] Optionally, the chip 170 also includes a memory 1740, which may include read-only memory and random access memory, and provides operation instructions and data to the processor 1710. A portion of the memory 1740 may also include non-volatile random access memory (NVRAM).

[0148] In some implementations, memory 1740 stores elements such as execution modules or data structures, or subsets thereof, or extended sets thereof.

[0149] In this embodiment of the application, the corresponding operation is executed by calling the operation instructions stored in the memory 1740 (the operation instructions can be stored in the operating system).

[0150] The processor 1710 described above can implement or execute various exemplary logic blocks, units, and circuits described in conjunction with the disclosure of this application. The processor can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, units, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0151] The memory 1740 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state drive; the memory may also include combinations of the above types of memory.

[0152] The Bus 1720 can be an Extended Industry Standard Architecture (EISA) bus, etc. The Bus 1720 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0153] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the trajectory prediction method in the above method embodiments.

[0155] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the trajectory prediction method in the method flow shown in the above method embodiments.

[0156] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0157] Embodiments of the present invention provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the trajectory prediction method described above.

[0158] Since the trajectory prediction device, computer-readable storage medium, and computer program product in the embodiments of the present invention can be applied to the above method, the technical effects they can achieve can also be referred to the above method embodiments. The embodiments of the present invention will not be described again here.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0160] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0162] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A trajectory prediction method, characterized in that, The method includes: Acquire historical trajectory dataset and first trajectory data of the object to be queried. The first trajectory data includes N trajectory sequences traversed by the object to be queried, and each trajectory sequence is the trajectory of the object to be queried moving from one camera node to another camera node in the monitoring system. Based on the historical trajectory dataset and the first trajectory data, calculate the prior probability and conditional probability of at least one candidate camera node in the monitoring system. The prior probability is the probability that the at least one candidate camera node is the next camera node of the current camera node, and the conditional probability is the probability that the historical trajectory dataset contains the first trajectory data. Based on the prior probability and the conditional probability, calculate the predicted probability of the at least one candidate camera node; The candidate camera node with the highest predicted probability is determined from the at least one candidate camera node, and is used as the next camera node that the queried object needs to pass through. Wherein, the at least one candidate camera node is a camera node on the target trajectory in at least one target combined trajectory, the target trajectory is a trajectory that matches at least two adjacent camera nodes corresponding to the first trajectory data, and the at least one target combined trajectory is a trajectory obtained by combining the trajectory sequences between every two adjacent camera nodes in the historical trajectory dataset; The step of calculating the conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data includes: Based on the historical trajectory dataset, calculate the state transition probability between every two adjacent camera nodes in the monitoring system, and generate a state transition probability matrix. Based on the state transition probability matrix and the first trajectory data, the path probability of the first trajectory data is calculated, where the path probability is the probability of the first trajectory data appearing in the historical trajectory dataset. Based on the path probability and the state transition probability matrix, the conditional probability of the at least one candidate camera node is calculated.

2. The method according to claim 1, characterized in that, Based on the historical trajectory dataset, the process of calculating the state transition probability between every two adjacent camera nodes in the monitoring system, and generating a state transition probability matrix, includes: Based on the historical trajectory dataset, a Markov model is established, which is used to represent that there are two directed transition states between every two adjacent camera nodes. Using the Markov model, based on the target parameters of each camera node in the monitoring system, the state transition probability between every two adjacent camera nodes in the monitoring system is calculated, and a state transition probability matrix is ​​generated. The target parameters include the attribute information of each camera node at the current time.

3. The method according to claim 1, characterized in that, The step of calculating the path probability of the first trajectory data based on the state transition probability matrix and the first trajectory data includes: Determine at least one target state transition probability; each target state transition probability is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix. The path probability of the first trajectory data is obtained by multiplying the product of all target state transition probabilities in at least one target state transition probability.

4. The method according to claim 1, characterized in that, The calculation of the conditional probability of the at least one candidate camera node based on the path probability and the state transition probability matrix includes: Based on the state transition probability matrix, calculate the target trajectory data passing through the at least one candidate camera node, and the target ratio of the trajectory data passing through the at least one candidate camera node in the historical trajectory dataset; Based on the target ratio, the conditional probability of the at least one candidate camera node is calculated.

5. The method according to any one of claims 1 to 3, characterized in that, After obtaining the historical trajectory dataset and the first trajectory data of the object to be queried, the method further includes: The historical trajectory dataset is decomposed into at least one second trajectory data, which contains the trajectory sequence between every two adjacent camera nodes in the monitoring system. The at least one second trajectory data is combined to obtain the at least one target combined trajectory, and each target combined trajectory includes multiple second trajectory data from the at least one second trajectory data; Match the at least one target combined trajectory with the first trajectory data; For each target combination trajectory, if the second trajectory data in a target combination trajectory matches the first trajectory data, then all camera nodes in the target combination trajectory are used as candidate camera nodes.

6. A trajectory prediction device, characterized in that, The device includes: an acquisition module, a calculation module, and a determination module; The acquisition module is used to acquire historical trajectory dataset and first trajectory data of the object to be queried. The first trajectory data includes N trajectory sequences traversed by the object to be queried, and each trajectory sequence is the trajectory of the object to be queried moving from one camera node to another camera node in the monitoring system. The calculation module is used to calculate the prior probability and conditional probability of at least one candidate camera node in the monitoring system based on the historical trajectory dataset and the first trajectory data obtained by the acquisition module. The prior probability is the probability that the at least one candidate camera node is the next camera node of the current camera node, and the conditional probability is the probability that the historical trajectory dataset contains the first trajectory data. The calculation module is also used to calculate the predicted probability of the at least one candidate camera node based on the prior probability and the conditional probability. The determining module is used to determine the candidate camera node with the highest predicted probability calculated by the calculation module from the at least one candidate camera node, and use it as the next camera node that the query object needs to pass through. Wherein, the at least one candidate camera node is a camera node on the target trajectory in at least one target combined trajectory, the target trajectory is a trajectory that matches at least two adjacent camera nodes corresponding to the first trajectory data, and the at least one target combined trajectory is a trajectory obtained by combining the trajectory sequences between every two adjacent camera nodes in the historical dataset; The computing module is specifically used for: Based on the historical trajectory dataset, calculate the state transition probability between every two adjacent camera nodes in the monitoring system, and generate a state transition probability matrix. Based on the state transition probability matrix and the first trajectory data, the path probability of the first trajectory data is calculated, where the path probability is the probability of the first trajectory data appearing in the historical trajectory dataset. Based on the path probability and the state transition probability matrix, the conditional probability of the at least one candidate camera node is calculated.

7. The apparatus according to claim 6, characterized in that, The computing module is specifically used for: Based on the historical trajectory dataset, a Markov model is established, which is used to represent that there are two directed transition states between every two adjacent camera nodes. Using the Markov model, based on the target parameters of each camera node in the monitoring system, the state transition probability between every two adjacent camera nodes in the monitoring system is calculated, and a state transition probability matrix is ​​generated. The target parameters include the attribute information of each camera node at the current time.

8. The apparatus according to claim 6, characterized in that, The computing module is specifically used for: Determine at least one target state transition probability; each target state transition probability is the state transition probability between every two adjacent camera nodes in the first trajectory data in the state transition probability matrix. The path probability of the first trajectory data is obtained by multiplying the product of all target state transition probabilities in at least one target state transition probability.

9. A trajectory prediction device, characterized in that, include: A processor and a communication interface; the communication interface is coupled to the processor, the processor being used to run computer programs or instructions to implement the trajectory prediction method as described in any one of claims 1-5.

10. A computer-readable storage medium storing instructions, characterized in that, When the computer executes the instruction, the computer performs the trajectory prediction method as described in any one of claims 1-5.

Images