A method and device for simulating pedestrian flow trajectory

Abstract

The present invention provides a method and device for simulating pedestrian trajectory. The method comprises: obtaining video data to be analyzed, extracting initial trajectory motion state data and object feature data of pedestrians in a target time period based on the video data; obtaining the trajectory simulation model to generate simulated trajectory motion state data corresponding to the next time period based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; and outputting corresponding simulated pedestrian trajectory data based on the simulated trajectory motion state data. The pedestrian trajectory simulation method provided by the present invention can accurately and effectively simulate and obtain pedestrian trajectory data, thereby effectively improving the efficiency and robustness of pedestrian trajectory simulation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly to a method and device for simulating a human flow trajectory. In addition, the present invention also relates to an electronic device and a processor-readable storage medium. Background Art

[0002] In recent years, with the rapid development of the economy and society, crowd flow simulation monitoring and simulation technology have been used more and more widely. Through crowd flow simulation, we can know in advance whether pedestrians in the target area will flow normally, so as to ensure pedestrian safety, which is also crucial to ensure public safety. In addition, for newly built building scenes, such as newly built stations, shopping malls, pedestrian streets, etc., crowd flow simulation can help designers know the rationality of their design: designers often hope that in actual operation, the designed roads can ensure the normal flow of pedestrians and a relatively uniform passage, which can be verified by using crowd flow simulation in the design. At present, the existing methods of pedestrian trajectory prediction usually consider pedestrian intentions, the interaction of the surrounding environment, and the social relations of pedestrians, and are mainly divided into microscopic methods and macroscopic methods. Among them, microscopic methods such as social force models, borrowing from the research paradigm of particle physics, model the mutual interaction between people and people, people and the environment into interaction forces, and then simulate the trajectory behavior of pedestrians; some macroscopic methods borrow from the research paradigm of fluid mechanics, modeling space as a scalar field, and pedestrians move in the direction of gradient descent. Both of these methods are knowledge-driven and cannot use data to correct the assumptions reflected in the formulas, which means they are only applicable to specific ideal scenarios. Existing data-driven methods are usually based on long-short term memory networks, which have high limitations and cannot accurately and effectively simulate and obtain pedestrian trajectory data. Therefore, how to design an efficient pedestrian trajectory simulation solution has become a problem that needs to be solved urgently. Summary of the invention

[0003] To this end, the present invention provides a method and device for simulating a crowd flow trajectory, so as to solve the defect that the crowd flow trajectory simulation scheme in the prior art has high limitations, thereby resulting in poor accuracy of crowd flow trajectory data simulation.

[0004] In a first aspect, the present invention provides a method for simulating a flow of people, comprising:

[0005] Acquire video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data;

[0006] Based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model, the trajectory simulation model is obtained to generate simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data;

[0007] Based on the simulated trajectory motion state data, corresponding simulated crowd flow trajectory data are output.

[0008] Further, the obtaining of the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model specifically includes:

[0009] The initial trajectory motion state data and object feature data are input into a preset encoder module to obtain a node embedding vector and an edge embedding vector in the heterogeneous gated information transfer network, the node embedding vector and the edge embedding vector are input into the trajectory simulation model, and after decoding processing by a preset decoder module, the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model is obtained.

[0010] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: characterizing the interaction relationship of pedestrians based on the information transmission mechanism of the heterogeneous gated information transmission network to complete the pedestrian interaction relationship modeling and obtain the corresponding first trajectory simulation model; using the first trajectory simulation model as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector.

[0011] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: on the basis of pedestrian interaction relationship modeling, environmental objects are introduced as additional interactors, and the interaction relationship between people and environmental objects is characterized based on the information transmission mechanism of the heterogeneous gated information transmission network to complete the interaction relationship modeling between people and environmental objects and obtain a corresponding second trajectory simulation model; the second trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein the environmental objects include the pedestrian's destination and obstacles around the pedestrian.

[0012] Furthermore, before inputting the node embedding vector and the edge embedding vector into the trajectory simulation model, the method further includes:

[0013] On the basis of modeling the interaction relationship between pedestrians and the interaction relationship between people and environmental objects, based on the information transmission mechanism of the heterogeneous gated information transmission network, a preset gating structure is used to control the proportion of transmitted messages, and the influence of target objects in the pedestrian's field of view on the pedestrian's behavior during walking is modeled, so as to simulate the influence of target objects in the field of view on the transmitted information.

[0014] Furthermore, the initial trajectory motion state data includes the pedestrian's two-dimensional coordinate trajectory, motion direction, acceleration, moving speed and displacement data; and the object feature data includes the pedestrian's feature data and the feature data of the environmental object.

[0015] Furthermore, outputting corresponding simulated crowd flow trajectory data based on the simulated trajectory motion state data specifically includes: performing trajectory iterative simulation based on the simulated trajectory motion state data to obtain complete simulated crowd flow trajectory data.

[0016] In a second aspect, the present invention further provides a pedestrian flow trajectory simulation device, comprising:

[0017] A data acquisition unit, used to acquire video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data;

[0018] A trajectory simulation unit, for obtaining the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data;

[0019] The trajectory data output unit is used to output corresponding simulated human flow trajectory data based on the simulated trajectory motion state data.

[0020] Furthermore, the trajectory simulation unit is specifically used for:

[0021] The initial trajectory motion state data and object feature data are input into a preset encoder module to obtain a node embedding vector and an edge embedding vector in the heterogeneous gated information transfer network, the node embedding vector and the edge embedding vector are input into the trajectory simulation model, and after decoding processing by a preset decoder module, the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model is obtained.

[0022] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a first modeling unit, used to characterize the interaction relationship of pedestrians based on the information transmission mechanism of the heterogeneous gated information transmission network, so as to complete the pedestrian interaction relationship modeling and obtain the corresponding first trajectory simulation model; the first trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector.

[0023] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a second modeling unit, which is used to introduce environmental objects as additional interactors on the basis of pedestrian interaction relationship modeling, and characterize the interaction relationship between people and environmental objects based on the information transmission mechanism of the heterogeneous gated information transmission network to complete the interaction relationship modeling between people and environmental objects and obtain a corresponding second trajectory simulation model; the second trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein the environmental objects include the destination of the pedestrian and the obstacles around the pedestrian.

[0024] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a third modeling unit, which is used to control the proportion of transmitted messages based on the information transmission mechanism of the heterogeneous gated information transmission network on the basis of modeling the interaction relationship between pedestrians and the interaction relationship between people and environmental objects, and to model the influence of target objects in the pedestrian's field of view on the pedestrian's behavior during walking, so as to simulate the influence of target objects in the field of view on the transmitted information.

[0025] Furthermore, the initial trajectory motion state data includes the pedestrian's two-dimensional coordinate trajectory, motion direction, acceleration, moving speed and displacement data; and the object feature data includes the pedestrian's feature data and the feature data of the environmental object.

[0026] Furthermore, the trajectory data output unit is specifically used to: perform trajectory iterative simulation based on the simulated trajectory motion state data to obtain complete simulated crowd flow trajectory data.

[0027] In a third aspect, the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for simulating pedestrian flow trajectories as described in any one of the above are implemented.

[0028] In a fourth aspect, the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the pedestrian trajectory simulation method as described in any one of the above items are implemented.

[0029] The pedestrian trajectory simulation method provided by the present invention extracts the initial trajectory motion state data and object feature data of pedestrians in a target time period from the video data to be analyzed, performs trajectory prediction based on the initial trajectory motion state data, the object feature data and a preset trajectory simulation model, obtains the simulated trajectory motion state data corresponding to the next time period, and obtains simulated pedestrian trajectory data based on the simulated trajectory motion state data. The method can accurately and effectively simulate and obtain pedestrian trajectory data, thereby effectively improving the efficiency and robustness of pedestrian trajectory simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flow chart of a method for simulating a flow of human flow trajectories provided by an embodiment of the present invention;

[0032] Figure 2 is a specific schematic diagram of the method for simulating the trajectory of human flow provided by an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of a heterogeneous graph representing multiple interactive relationships in a method for simulating a pedestrian flow trajectory provided by an embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of a visual field effect in a method for simulating a flow of people provided in an embodiment of the present invention;

[0035] Figure 5 It is a schematic diagram of a heterogeneous message transmission network of multiple interactive relationships in a method for simulating a flow of people trajectory provided by an embodiment of the present invention;

[0036] Figure 6 is a structural schematic diagram of a pedestrian flow trajectory simulation device provided by an embodiment of the present invention;

[0037] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The following is a detailed description of an embodiment of the method for simulating the flow of people according to the present invention. Figure 1 As shown, it is a flow chart of the method for simulating the flow of people provided by an embodiment of the present invention, and the specific implementation process includes the following steps:

[0040] Step 101: Obtain video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data.

[0041] Specifically, the initial trajectory motion state data includes the pedestrian's two-dimensional coordinate trajectory, motion direction, acceleration, moving speed, and displacement data; the object feature data includes the pedestrian's feature data and the environmental object's feature data.

[0042] Step 102: Based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model, obtain the trajectory simulation model to generate simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data.

[0043] Specifically, based on the initial trajectory motion state data, object feature data and the preset trajectory simulation model, the trajectory simulation model generates the simulated trajectory motion state data corresponding to the next time period, and the corresponding implementation process includes: inputting the initial trajectory motion state data and object feature data into the preset encoder module, obtaining the node embedding vector and edge embedding vector in the heterogeneous gated message passing network, inputting the node embedding vector and the edge embedding vector into the trajectory simulation model, and after decoding by the preset decoder module, obtaining the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model. Among them, the heterogeneous gated message passing network (Multi-Graph Gated Message-Passing Network) is one of the graph neural networks.

[0044] In the embodiment of the present invention, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, the following modeling needs to be performed in advance.

[0045] Firstly, based on the information transmission mechanism of the heterogeneous gated information transmission network, the interaction relationship of pedestrians is characterized to complete the pedestrian interaction relationship modeling and obtain the corresponding first trajectory simulation model; the first trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector.

[0046] Specifically, pedestrian interaction modeling based on heterogeneous gated information transfer networks aims to characterize various interactive relationships of pedestrians, such as avoidance mechanisms, walking with friends, etc., through the information transfer mechanism of heterogeneous gated information transfer networks. Because it is not restricted by the assumptions in sociological theory, it learns the inherent laws of human interactive behaviors in a data-driven way, has good noise adaptability and scene generalization, and can support sufficiently complex interactive behavior modeling. The output algorithm can be applied to pedestrian flow simulation in open areas, such as squares, pedestrian streets, etc. In the actual implementation process, when constructing a heterogeneous graph of relationships in the simulated trajectory motion state data, all pedestrians, obstacles, and destinations are determined as corresponding nodes, and a heterogeneous graph with two types of edges is constructed. Figure 3 As shown, 301 is a pedestrian, 302 is an obstacle, 303 is a destination, 304 is a closed edge, and 305 is a pedestrian-destination edge. There are two types of edges in this scheme: closed edges and pedestrian-destination edges. The adjacent edges are constructed by a fast k-nearest neighbor (KNN) algorithm, which usually corresponds to the repulsive force generated by avoiding collisions. A closed edge can connect a pedestrian to another pedestrian, or a pedestrian to an obstacle. In an embodiment of the present invention, the wall is represented by discrete dense obstacles (similar to a fence). When the distance between two environmental objects is less than a predefined radius R, the two environmental objects are connected. The pedestrian-destination edge can simulate the relationship between a pedestrian and its corresponding destination. At each time step (i.e., time period), a pedestrian has only one destination, but the destination can be dynamic. For example, a pedestrian has a series of destinations, also called waypoints. Once the pedestrian arrives at the destination, he needs to switch to the next destination within a specific time period after staying for a period of time.

[0047] Furthermore, on the basis of pedestrian interaction relationship modeling, environmental objects are introduced as additional interactors, and the interaction relationship between people and environmental objects is characterized based on the information transmission mechanism of the heterogeneous gated information transmission network to complete the interaction relationship modeling between people and environmental objects and obtain the corresponding second trajectory simulation model; the second trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein the environmental objects include the pedestrian's destination and obstacles around the pedestrian.

[0048] Specifically, the pedestrian-environmental object interaction framework based on heterogeneous gated information transfer networks is based on pedestrian modeling interaction, and introduces environmental objects as an additional interactor, and uses the information transfer mechanism of heterogeneous gated information transfer networks to model the interaction relationship between people and destinations, people and obstacles and other environmental objects. The corresponding output algorithm can be applied to crowd simulation scenes such as shopping malls. In the actual implementation process, when modeling the interaction of multiple environmental objects based on heterogeneous graph convolutional networks (heterogeneous gated information transfer networks), it is considered that pedestrians want to go to the corresponding destination, and there are other pedestrians and obstacles and other environmental objects on the way. It is necessary to consider the reaction of pedestrians and find the direction to the destination. In this process, pedestrians are actually collecting information about the destination, such as relative position. Avoid collisions with surrounding pedestrians and obstacles and other environmental objects. In this process, pedestrians collect information about objects (environmental objects) around pedestrians and respond after comprehensive consideration. According to previous studies on pedestrian behavior modeling, there are several different types of interactions. In order to model this, this solution designs two different types of relationships to ensure that new information of environmental objects can be fully transmitted, and the present invention designs corresponding heterogeneous gated information transfer networks. The information of the sending node and the receiving node is propagated to the edge together with the information of the edge to generate a new edge embedding vector. The calculation can be expressed as follows:

[0049]

[0050] Among them, the variable Represent the characteristics of user i and user j in the l-1th layer respectively, and represents the characteristics of the kth type of edge between user i and user j at layer l and layer l-1 respectively. For k = 1, 2, ..., K, for each type of edge (different relationship), select f e，k As a multi-layer perceptron with a gating mechanism, different relations are not shared because the interactions are completely different. Figure 5 As shown, the node update function can be expressed as follows:

[0051]

[0052] Among them, the variable represents the characteristics of user i at layer l-1, and N represents the characteristics of the kth type of edge between user i and user j at the lth layer and the l-1th layer respectively. K (i) represents the neighbor subset of the i-th node on the K-th type of edge. The present invention first connects the embedding vectors N1(i), ..., N obtained from the edges of different relations. K (i) and use f a(·) Summarize the information, the present invention combines it with Use together to update The present invention selects f a (·) and f n (·)Implemented as a multi-layer perceptron.

[0053] Furthermore, based on the modeling of pedestrian interaction relationships and the interaction relationships between people and environmental K objects, based on the information transmission mechanism of the heterogeneous gated information transmission network, a preset gating structure can be used to control the proportion of transmitted messages, and model the influence of target objects in the pedestrian's field of view on pedestrian behavior during walking, so as to simulate the influence of target objects in the field of view on the transmitted information.

[0054] Specifically, the pedestrian vision effect is modeled to improve the crowd flow simulation effect. Based on pedestrian interaction and human-environment interaction, the effect of pedestrian vision on pedestrian behavior during walking is modeled. That is, pedestrians receive and process information from different directions differently. Here, the present invention uses a gated structure to learn the propagation method. In the actual implementation process, when modeling the pedestrian vision effect based on a heterogeneous gated information transmission network, the pedestrian's reaction is greatly affected by the vision. Figure 4 As shown, 401 is the field of vision of the pedestrian. In the present invention, the influence of the pedestrian field of vision effect on the simulation of pedestrian trajectories is explicitly modeled. Considering that pedestrians collect information about objects within their field of vision, they will be affected by the field of vision effect, that is, pedestrians tend to ignore objects behind them because they cannot see behind them. On the other hand, the gating structure in heterogeneous gated information transfer networks has been proven to be an effective tool for controlling how much information should be retained and is widely used in long short-term memory networks. In order to simulate the influence of field of vision on information reception, the present invention designs a gating function to control how much information should be propagated to the receiving node, as shown below:

[0055] f g，k (·)=f Sig，noid (f MLP (·)),

[0056] where f g，k (·) represents the gating function for the kth type of edge, f Sigmoid (·) represents the sigmoid activation function that can output continuous values ​​in the range (0,1), f MLP (·) represents the multi-layer perceptron function. The present invention uses a multi-layer perceptron to calculate the value from the sender (sending node), the receiver (receiving node) and their relative positions. This means using information from the sender, the receiver and the edge to calculate how much information should be propagated to the sender. The physical meaning is the proportion of propagated messages. It should be noted that the gating function can have different variants depending on the data set and task. Therefore, the final message propagation function can be expressed as:

[0057]

[0058] Among them, the variable denote the features of user i and user j at the l-1th layer, respectively. For k = 1, 2, ..., K, and represents the characteristics of the kth type of edge between user i and user j at the lth layer and the l-1th layer, respectively, g，k (·) represents the gating function for the kth type of edge, f MLP，k (·) represents the multilayer perceptron function for the k-th type of edge.

[0059] Step 103: Based on the simulated trajectory motion state data, output corresponding simulated crowd flow trajectory data.

[0060] After the modeling is completed, the present invention also includes a model training process. In the trajectory simulation model training process, for all input sample data, due to different dimensions and value ranges, it is necessary to normalize them in advance to facilitate better training of the trajectory simulation model. Specifically, the present invention can train the trajectory simulation model based on the mean square error (Microsoft Service Engine, MSE) loss of the predicted acceleration and the actual value in the sample data. During training, the present invention shields non-pedestrian nodes when calculating the loss function, that is, only the prediction error of the pedestrian node is accumulated as the loss function. During training, the mean square error between the predicted acceleration and the actual value in the sample data is calculated, and a mature trajectory simulation model is obtained through iterative training for testing. During testing, the initial trajectory motion state data of several starting time steps (i.e., the target time period) can be given, and the simulated trajectory motion state data of the next time step (next time period) can be simulated, and then the simulated trajectory motion state data is used to iteratively simulate the new trajectory to train the trajectory simulation model again.

[0061] like Figure 2As shown, a specific implementation process of the present invention is described as follows: Based on the information propagation mechanism of the heterogeneous gated information transmission network, it is possible to characterize the prediction of the next trajectory motion state information of the pedestrian (i.e., simulated trajectory motion state information) by propagating information between pedestrians. For example, a graph with pedestrians as nodes is first constructed. On the one hand, there is a friendship relationship between pedestrians. On the other hand, when the distance between pedestrians is less than a given threshold, an edge is connected between the corresponding nodes; then, the coordinates, moving speed, etc. of the pedestrians are used as the initial trajectory motion state data, and the initial trajectory motion state data of the pedestrian is encoded into the embedding vector of a node, and the relative motion state of the two pedestrians is encoded into the embedding vector of the edge; then, the above information is propagated through the heterogeneous gated information transmission network, and the final node embedding vector is obtained by comprehensive analysis; the final node embedding vector is decoded by the decoder module to obtain the simulated trajectory motion state data corresponding to the next time period of the pedestrian. In this way, the state of the previous time period can be iteratively simulated to generate the state of the next time period, and finally the crowd simulation result (i.e., simulated crowd trajectory data) for a long period of time is obtained.

[0062] Specifically, the encoder module of step 201 is executed to take the pedestrian's initial trajectory motion state data and object feature data as input to generate embedding vectors of nodes and edges. The initial trajectory motion state data may include the speed of each object in the last T time steps (target time period), the speed is obtained by differentiating the position, and the unique hot encoding of each type of object. The features of the edge include relative position and distance, which can be expressed as the difference between the position vectors of two objects and the modulus of the difference. The representation set of the graph of the heterogeneous gated information transfer network is defined as:

[0063] G (0) ={x (0) , e (0)}

[0064] where x (0) , e (0) They represent the representation of the nodes and edges of the 0th layer respectively. The set of these embedded vectors is denoted as the graph representation G (0) , is fed into the next module processor to further propagate the information on the graph.

[0065] The processor module is a key part of the model, which propagates information between nodes and edges. The present invention designs a novel structure: a heterogeneous gated message passing network, which is used to model various types of interactions. By executing step 202, the processor module performs message passing based on the heterogeneous gated message passing network. The transmission process can be expressed as follows:

[0066] G (1) =f MessagePassing (G (0) )

[0067] G (L) =f MessagePassing (G (L-1) ),

[0068] Among them, f MessagePassing (·) denotes a message passing network (i.e., a heterogeneous gated message passing network), L denotes letting the information pass through the message passing network L times, and G (0) , ..., G (L) Represents the graph representation from layer 0 to layer L, including node and edge representations.

[0069] The final node embedding vector looks like this:

[0070] a t+1 =f MLP (x (L) ),

[0071] where a t+1 represents the acceleration at time t+1, x (L) represents the node representation of the Lth layer, that is, the last layer. This formula is used for any node. The acceleration of the next time step (next time period) is predicted based on the final node embedding vector, where f MLP (·) represents a multilayer perceptron. The acceleration is calculated as the difference between two adjacent velocities as follows,

[0072] a t =v t -v t -1, v t =s t -s t-1 ·

[0073] Among them, a t represents the acceleration at time t, v t and v t-1 They represent the speed at time t and time t-1 respectively, s t and t-1 denote the positions at time t and time t-1 respectively. Accordingly, we get a t+1 After that, it can be calculated as:

[0074] v t+1 =v t +a t+1 ,s t+1 =s t +v t+1 .

[0075] Among them, a t+1 represents the acceleration at time t+1, v t and v t+1 They represent the speed at time t and time t+1, st and t+1 Respectively represent the positions at time t and time t+1. Therefore, the present invention calculates the simulated trajectory motion state data of the next time period through the decoding module, and finally recursively generates a series of trajectories, that is, simulated human flow trajectory data.

[0076] It should be noted that the biggest difference between the trajectory simulation model of the present invention and the crowd flow simulation based on sociological theory is that it does not presuppose any assumptions and is completely data-driven to learn the interaction rules of pedestrians. Precisely because the algorithm framework is not subject to very idealized sociological theory assumptions, it can simulate a more realistic crowd flow distribution. Compared with the existing pedestrian trajectory prediction, this trajectory simulation model pays more attention to long-term crowd flow simulation, and while the model pays attention to the prediction error of individual trajectories in long-term crowd flow, the trajectory simulation model also pays attention to the simulation error of group crowd flow distribution, the latter of which often plays a more important role in wind direction prevention and control.

[0077] In the actual implementation process, if you want to use the initial state of the crowd flow distribution in the environment (such as the initial trajectory motion state data and object feature data) and the destinations of different pedestrians to simulate the crowd flow trajectory data, you can use the trajectory simulation model to recursively generate the simulated trajectory motion state data to finally construct the complete simulated crowd flow trajectory data. When modeling, the user needs to abstract the destination into a point and model the obstacles (such as walls and fences) as discrete points to construct the training data. You can use the preset pedestrian monitoring and recognition algorithm to extract (1) the two-dimensional coordinate trajectory of the crowd flow; (2) the destination of the pedestrian; (3) the coordinates of the obstacle as sample data from the sample video data. Through a series of such sample data training models, the interactive relationship between pedestrians, pedestrians and destinations, and pedestrians and obstacles can be learned.

[0078] In the process of training the trajectory simulation model, a similar windowing method can be used to take the initial trajectory motion state data of pedestrians in the first n time periods as input to predict the acceleration of each pedestrian in the next step, so that the speed and displacement can be calculated. After obtaining the simulated trajectory motion state data, the present invention can dynamically analyze the trajectory curve of individuals and the trajectory distribution of groups, that is, simulate the trajectory data of pedestrian flow.

[0079] In addition, in the actual implementation process, if you want to use the initial state of the crowd distribution in the environment (such as the initial trajectory motion state data and object feature data) and the relay point (way-point) sequence of different pedestrians to simulate the crowd trajectory data, you can use the trajectory simulation model to recursively generate the simulated trajectory motion state data to finally construct a complete simulated crowd trajectory data. Compared with the first implementation method, this implementation method is suitable for more complex crowd simulation scenarios, such as in an airport environment. The training mode of this implementation method is the same as that of the previous implementation method in the training phase. In the use phase, the difference is that the destination of the pedestrian needs to be dynamically specified: each user has a relay point sequence. At the beginning, the destination of the pedestrian is the first relay point in the sequence. When the pedestrian arrives at the first relay point (the distance from the first relay point is less than a given threshold) and stays for a full residence time, the destination is switched to the next location in the relay point sequence, and then the pedestrian is attracted by the new destination, and goes back and forth until the user reaches the final destination to obtain complete simulated crowd trajectory data. In the manner described above, the trajectory of human flow can be simulated, and in this process, the interactions between pedestrians, pedestrians and destinations (current relay points), and pedestrians and obstacles are taken into account. After obtaining the simulated trajectory motion state data, the present invention can dynamically analyze the trajectory curve of individuals and the trajectory distribution of groups, that is, simulate the trajectory data of human flow. Different from the previous implementation scheme, this implementation scheme can simulate scenes with larger scales and complex environments, such as airports and stations, which is of reference significance for human flow bottleneck analysis, heat map analysis, and scene activity planning guidance.

[0080] Furthermore, after obtaining the simulated crowd flow trajectory data, it is possible to know in advance whether there will be abnormal flow of pedestrians in the area by analyzing the pedestrian movement characteristics corresponding to the simulated crowd flow trajectory data, thereby issuing early warning of abnormal events; or, based on the simulated crowd flow trajectory data, evaluating how the epidemic spreads among the crowd; or, based on the simulated crowd flow trajectory data, evaluating the rationality of newly built building scenes, such as newly built stations, shopping malls, pedestrian streets, etc., crowd flow simulation can help designers understand the rationality of their designs.

[0081] The crowd trajectory simulation method described in the embodiment of the present invention extracts the initial trajectory motion state data and object feature data of pedestrians in the target time period from the video data to be analyzed, performs trajectory prediction based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model, obtains the simulated trajectory motion state data corresponding to the next time period, and obtains simulated crowd trajectory data based on the simulated trajectory motion state data. It can accurately and effectively simulate and obtain crowd trajectory data, effectively improving the efficiency and robustness of crowd trajectory simulation.

[0082] Corresponding to the above-mentioned method for simulating a flow of people, the present invention also provides a device for simulating a flow of people. Since the embodiment of the device is similar to the above-mentioned method embodiment, the description is relatively simple. For relevant details, please refer to the description of the above-mentioned method embodiment. The embodiment of the device for simulating a flow of people described below is only illustrative. Please refer to Figure 6 As shown, it is a structural schematic diagram of a pedestrian trajectory simulation device provided by an embodiment of the present invention.

[0083] The human flow trajectory simulation device described in the present invention specifically includes the following parts:

[0084] A data acquisition unit 601 is used to acquire video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data;

[0085] The trajectory simulation unit 602 is used to obtain the trajectory simulation model to generate simulated trajectory motion state data corresponding to the next time period based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data;

[0086] The trajectory data output unit 603 is used to output corresponding simulated human flow trajectory data based on the simulated trajectory motion state data.

[0087] Furthermore, the trajectory simulation unit is specifically used for:

[0088] The initial trajectory motion state data and object feature data are input into a preset encoder module to obtain a node embedding vector and an edge embedding vector in the heterogeneous gated information transfer network, the node embedding vector and the edge embedding vector are input into the trajectory simulation model, and after decoding processing by a preset decoder module, the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model is obtained.

[0089] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a first modeling unit, used to characterize the interaction relationship of pedestrians based on the information transmission mechanism of the heterogeneous gated information transmission network, so as to complete the pedestrian interaction relationship modeling and obtain the corresponding first trajectory simulation model; the first trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector.

[0090] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a second modeling unit, which is used to introduce environmental objects as additional interactors on the basis of pedestrian interaction relationship modeling, and characterize the interaction relationship between people and environmental objects based on the information transmission mechanism of the heterogeneous gated information transmission network to complete the interaction relationship modeling between people and environmental objects and obtain a corresponding second trajectory simulation model; the second trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein the environmental objects include the destination of the pedestrian and the obstacles around the pedestrian.

[0091] Furthermore, before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, it also includes: a third modeling unit, which is used to control the proportion of transmitted messages based on the information transmission mechanism of the heterogeneous gated information transmission network on the basis of modeling the interaction relationship between pedestrians and the interaction relationship between people and environmental objects, and to model the influence of target objects in the pedestrian's field of view on the pedestrian's behavior during walking, so as to simulate the influence of target objects in the field of view on the transmitted information.

[0092] Furthermore, the initial trajectory motion state data includes the pedestrian's two-dimensional coordinate trajectory, motion direction, acceleration, moving speed and displacement data; and the object feature data includes the pedestrian's feature data and the feature data of the environmental object.

[0093] Furthermore, the trajectory data output unit is specifically used to: perform trajectory iterative simulation based on the simulated trajectory motion state data to obtain complete simulated crowd flow trajectory data.

[0094] The crowd trajectory simulation device described in the embodiment of the present invention extracts the initial trajectory motion state data and object feature data of pedestrians in the target time period from the video data to be analyzed, performs trajectory prediction based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model, obtains the simulated trajectory motion state data corresponding to the next time period, and obtains simulated crowd trajectory data based on the simulated trajectory motion state data. It can accurately and effectively simulate and obtain crowd trajectory data, effectively improving the efficiency and robustness of crowd trajectory simulation.

[0095] Corresponding to the above-mentioned human flow trajectory simulation method, the present invention also provides an electronic device. Since the embodiment of the electronic device is similar to the above-mentioned method embodiment, the description is relatively simple. For relevant parts, please refer to the description of the above-mentioned method embodiment. The electronic device described below is only exemplary. Figure 7As shown, it is a schematic diagram of the physical structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include: a processor 701, a memory 702, a communication bus 703 (i.e., the above-mentioned device bus) and a search engine 705, wherein the processor 701 and the memory 702 complete mutual communication through the communication bus 703, and communicate with the outside through the communication interface 704. The processor 701 can call the logic instructions in the memory 702 to execute the pedestrian trajectory simulation method, which includes: obtaining the video data to be analyzed, extracting the initial trajectory motion state data and object feature data of the pedestrian in the target time period based on the video data; based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model, obtaining the trajectory simulation model to generate the simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; based on the simulated trajectory motion state data, outputting the corresponding simulated pedestrian trajectory data.

[0096] In addition, the logic instructions in the above-mentioned memory 702 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a storage chip, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0097] On the other hand, an embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program stored on a processor-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer can execute the pedestrian trajectory simulation method provided by the above-mentioned method embodiments. The method comprises: obtaining video data to be analyzed, extracting initial trajectory motion state data and object feature data of pedestrians in a target time period based on the video data; based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model, obtaining the trajectory simulation model to generate simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; based on the simulated trajectory motion state data, outputting corresponding simulated pedestrian trajectory data.

[0098] On the other hand, an embodiment of the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, it is implemented to execute the pedestrian trajectory simulation method provided by the above embodiments. The method includes: obtaining video data to be analyzed, extracting the initial trajectory motion state data and object feature data of pedestrians in the target time period based on the video data; based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model, obtaining the trajectory simulation model to generate simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; based on the simulated trajectory motion state data, outputting corresponding simulated pedestrian trajectory data.

[0099] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for simulating pedestrian flow trajectories, characterized in that: include: Acquire video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data; Based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model, the trajectory simulation model is obtained to generate simulated trajectory motion state data corresponding to the next time period; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; The step of obtaining the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model based on the initial trajectory motion state data, the object feature data and the preset trajectory simulation model specifically includes: Inputting the initial trajectory motion state data and object feature data into a preset encoder module, obtaining a node embedding vector and an edge embedding vector in the heterogeneous gated information delivery network, inputting the node embedding vector and the edge embedding vector into the trajectory simulation model, and decoding and processing by a preset decoder module to obtain the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model; Before the node embedding vector and the edge embedding vector are input into the trajectory simulation model, the method further includes: characterizing the interaction relationship of pedestrians based on the information transfer mechanism of the heterogeneous gated information transfer network to complete the pedestrian interaction relationship modeling and obtain the corresponding first trajectory simulation model; using the first trajectory simulation model as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; on the basis of the pedestrian interaction relationship modeling, introducing environmental objects as additional interactors, characterizing the interaction relationship between people and environmental objects based on the information transfer mechanism of the heterogeneous gated information transfer network to complete the pedestrian interaction relationship modeling The interaction relationship between pedestrians and environmental objects is modeled to obtain a corresponding second trajectory simulation model; the second trajectory simulation model is used as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein the environmental objects include the destination of the pedestrian and the obstacles around the pedestrian; based on the modeling of the interaction relationship between pedestrians and the interaction relationship between people and environmental objects, based on the information transmission mechanism of the heterogeneous gated information transmission network, a preset gating structure is used to control the proportion of transmitted messages, and the influence of the target objects in the pedestrian's field of view on the pedestrian's behavior during walking is modeled to simulate the influence of the target objects in the field of view on the transmitted information; Based on the simulated trajectory motion state data, corresponding simulated crowd flow trajectory data are output.

2. The method for simulating the flow of people according to claim 1, characterized in that: The initial trajectory motion state data includes the pedestrian's two-dimensional coordinate trajectory, motion direction, acceleration, moving speed and displacement data; the object feature data includes the pedestrian's feature data and the environmental object's feature data.

3. The method for simulating the flow of people according to claim 1, characterized in that: The outputting corresponding simulated crowd flow trajectory data based on the simulated trajectory motion state data specifically includes: performing trajectory iterative simulation based on the simulated trajectory motion state data to obtain complete simulated crowd flow trajectory data.

4. A pedestrian flow trajectory simulation device, characterized in that: include: A data acquisition unit, used to acquire video data to be analyzed, and extract initial trajectory motion state data and object feature data of pedestrians within a target time period based on the video data; A trajectory simulation unit is used to obtain the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model based on the initial trajectory motion state data, object feature data and a preset trajectory simulation model; wherein the trajectory simulation model is a neural network model obtained by modeling and training based on a preset heterogeneous gated information transfer network and sample data; the trajectory simulation unit is specifically used to: input the initial trajectory motion state data and object feature data into a preset encoder module, obtain the node embedding vector and edge embedding vector in the heterogeneous gated information transfer network, input the node embedding vector and the edge embedding vector into the trajectory simulation model, and obtain the simulated trajectory motion state data corresponding to the next time period generated by the trajectory simulation model after decoding and processing by a preset decoder module; before inputting the node embedding vector and the edge embedding vector into the trajectory simulation model, it also includes: characterizing the interaction relationship of pedestrians based on the information transfer mechanism of the heterogeneous gated information transfer network to complete the modeling of pedestrian interaction relationship , obtain the corresponding first trajectory simulation model; use the first trajectory simulation model as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; on the basis of pedestrian interaction relationship modeling, introduce environmental objects as additional interactors, characterize the interaction relationship between people and environmental objects based on the information transmission mechanism of the heterogeneous gated information transmission network, so as to complete the interaction relationship modeling between people and environmental objects, and obtain the corresponding second trajectory simulation model; use the second trajectory simulation model as the trajectory simulation model to obtain the input node embedding vector and the edge embedding vector; wherein, the environmental objects include the destination of pedestrians and obstacles around pedestrians; on the basis of pedestrian interaction relationship modeling and the interaction relationship between people and environmental objects, based on the information transmission mechanism of the heterogeneous gated information transmission network, use the preset gating structure to control the proportion of transmitted messages, and model the influence of target objects in the pedestrian's field of view on pedestrian behavior during walking, so as to simulate the influence of target objects in the field of view on transmitted information; The trajectory data output unit is used to output corresponding simulated human flow trajectory data based on the simulated trajectory motion state data.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for simulating the trajectory of human flow as described in any one of claims 1 to 3 are implemented.

6. A processor-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for simulating the trajectory of human flow as described in any one of claims 1 to 3 are implemented.

Images