System and method for estimating future traffic density in environment

Abstract

The present disclosure provides a system and method for estimating future traffic density in an environment. The method includes receiving at least one partial trajectory and a sequence of observation vectors for at least one object in an environment. The at least one object is represented by a plurality of particles. The method comprises the following steps: processing the at least one partial trajectory by using a trajectory prediction model to predict the position of each particle in a plurality of particles at a future moment; and processing the observation vector sequence using an entering particle prediction model to predict the probability that entering particles are observed at each entry point at future moments in time. A future traffic density is estimated based on the predicted location of each particle and the predicted probability that an incoming particle is observed at each entry point at a future time.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to traffic density estimation, and more particularly to systems and methods for estimating future traffic density in an environment. BACKGROUND

[0002] Prediction of traffic density in an environment is critical for congestion prediction and route planning. For example, transportation researchers, transportation engineers, road engineers, city planners, policy makers, economists, vehicle manufacturers, and commuters who rely on transportation every day have placed significant interest on understanding traffic density in a city area comprising multiple roads. Likewise, understanding of traffic density in an indoor space is important, for example, to determine a schedule for deploying service robots in the indoor space.

[0003] Model-driven approaches can be used to predict traffic density. Model-driven approaches rely on network topology that describes the system of interest. For example, simulation models utilize predefined network topology and system logic to simulate traffic conditions. However, model-driven approaches require laborious tuning and calibration processes, and significant human involvement to produce optimal network topology and system logic. Thus, there remains a need for a system and method for predicting traffic density in an environment. SUMMARY

[0004] It is an object of some embodiments to estimate future traffic density in an environment. The environment can be an indoor space (e.g., an office space) or an outdoor space (e.g., a city area comprising roads). The environment comprises a plurality of entry points and exit points. The present disclosure provides a traffic density estimation model for estimating future traffic density in the environment. The traffic density estimation model comprises a trajectory prediction model, an entering particle prediction model, and an iterative sampling model. Some embodiments are based on the recognition that traffic density in an environment depends on the movement of objects (e.g., people) present in the environment at a given time. The trajectory prediction model accounts for the movement of existing objects by predicting trajectories of the existing objects.

[0005] A partial trajectory of an object is applied as input to the trajectory prediction model. The object moves in the environment from an entry point of the plurality of entry points to an exit point of the plurality of exit points. The partial trajectory is represented by a sequence of positions, e.g., [1, 3, 2, 0, 0, 0, 0, 0, 0, 0] of length 9. For example, there can be six different positions (1-6) in total in the environment. The partial trajectory can depict that at time instance tl the object is at position 1 in the environment, at time instance t2 the object is at position 3 in the environment, and at time instance t3 the object is at position 2 in the environment. This partial trajectory is input to the trajectory prediction model to predict the position of the object at the next time instance t4 (also referred to as a future time instance).

[0006] At the first prediction step, the partial trajectories are input to the trajectory prediction model, and the trajectory prediction model outputs a probability distribution of the position of the object at multiple locations, e.g., six locations (1-6). The probability distribution indicates the probability of the object being at each of the six locations at time instance t4. For example, the probability distribution indicates that the probability of the object being at location 5 at time instance t4 is 0.6.

[0007] Some implementations are based on the recognition that the object can be represented by multiple particles, e.g., five particles, and each particle is processed independently. The number of particles can be specified by a user. Each particle can follow a different trajectory. To this end, the position of each particle at time instance t4 is to be predicted. To predict the position of each particle at time instance t4, a location is sampled from the probability distribution, and the sampled location is appended to the trajectory of the particle. For example, for the trajectory of particle-1 of the five particles, the position of particle-1 at time instance t4 is predicted to be 5 by sampling from the probability distribution. Likewise, for the trajectory of each particle, the position at time instance t4 is predicted to determine the updated trajectories corresponding to the five particles.

[0008] Some implementations are based on the recognition that for an environment with constant entering and exiting objects, the traffic density estimation model should take into account the objects that can initiate new trajectories at the entry points at the next time instance to accurately estimate the future traffic density. In other words, entering particles are to be predicted to accurately estimate the future traffic density. The traffic density estimation model includes an entering particle prediction model for predicting the entering particles.

[0009] Observation vector sequence is input to the entering particle prediction model. Each element of the observation vector is a binary variable, where 1 indicates the presence of an entering particle, and 0 indicates no observation. For example, the observation vector sequence can be obtained from a camera installed in the environment. In an example, if at time i = 2, an object (e.g., a person) starts moving from location 3 and is captured by the camera, then z2[3] = 1 and all other entries of z2(vector) are zero. The entering particle prediction model outputs a multinomial probability vector. The multinomial probability vector indicates the probability of observing an entering particle at each entry point at the next time instance. Further, based on the multinomial probability vector, the next observation vector is predicted and appended to the observation vector sequence to determine an updated observation vector sequence.

[0010] Further, the updated trajectories corresponding to the five particles and the updated observation vector sequence are input to the iterative sampling model. Based on the updated trajectories and the updated observation vector sequence, the iterative sampling model estimates the future traffic density at the next time instance.

[0011] In some implementations, multiple partial trajectories of different objects are input to the trajectory prediction model. For example, at a first prediction step, a partial trajectory of object-1, a partial trajectory of object-2, and a partial trajectory of object-3 are input to the trajectory prediction model. The trajectory prediction model outputs a probability distribution of the position of object-1 at the next time instant over six locations (1-6), a probability distribution of the position of object-2 at the next time instant over six locations, and a probability distribution of the position of object-3 at the next time instant over six locations. Further, the probability distribution of the position of object-1, the probability distribution of the position of object-2 129b, and the probability distribution of the position of object-3 131b are averaged to determine an aggregated multinomial probability vector. The aggregated multinomial probability vector indicates the final prediction probability of the position of the objects over the six locations.

[0012] The aggregated multinomial probability vector and the multinomial probability vector are input to an iterative sampling model. Based on the multinomial probability vector and the aggregated multinomial probability vector, the iterative sampling model estimates a future traffic density at the next time instant. In particular, the iterative sampling model outputs a sample of future objects from the average vector of the multinomial probability vector and the aggregated multinomial probability vector, which indicates the future traffic density.

[0013] Further, in some implementations, a service robot can be deployed in an environment based on the estimated future traffic density. For example, the environment can be an indoor space that includes multiple bays. The service robot can be deployed to one or more of the multiple bays based on the estimated future traffic density in the one or more bays. Additionally or alternatively, in some implementations, the environment is an outdoor space that includes multiple roads and traffic lights at intersections of the multiple roads. The traffic lights can be controlled based on the estimated future traffic density in the environment.

[0014] Accordingly, one embodiment discloses a system for estimating future traffic density in an environment, where the environment includes a plurality of entry points and a plurality of exit points. The system includes: at least one processor; and a memory having instructions stored thereon, which cause the at least one processor of the system to perform the following operations: receive, for at least one object, at least one partial trajectory and a sequence of observation vectors, where the at least one object is moving from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and where the at least one object is represented by a plurality of particles; perform the following processing: (1) process the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a location of the at least one object at a future time at different locations in the environment; and predict a location of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) process the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimate a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

[0015] Accordingly, another embodiment discloses a method for estimating future traffic density in an environment, where the environment includes a plurality of entry points and a plurality of exit points. The method includes: receiving, for at least one object, at least one partial trajectory and a sequence of observation vectors, where the at least one object is moving from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and where the at least one object is represented by a plurality of particles; performing the following processing: (1) processing the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a location of the at least one object at a future time at different locations in the environment; and predicting a location of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) processing the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimating a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

[0016] Accordingly, yet another embodiment discloses a non-transitory computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for estimating future traffic density in an environment, wherein the environment includes a plurality of entry points and a plurality of exit points. The method includes receiving, for at least one object, at least one partial trajectory and a sequence of observation vectors, wherein the at least one object moves from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and wherein the at least one object is represented by a plurality of particles; performing a process of: (1) processing the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a location of the at least one object at a future time at different locations in the environment; and predicting a location of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) processing the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimating a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1A

[0018] Figure 1A A block diagram of a traffic density estimation model for estimating future traffic density according to an embodiment of the disclosure is shown.

[0019] Figure 1B

[0020] Figure 1B A schematic diagram of a trajectory prediction model that considers movement of existing objects by predicting trajectories of the existing objects according to an embodiment of the disclosure is shown.

[0021] Figure 1C

[0022] Figure 1C A prediction of a location of each particle at a next time according to an embodiment of the disclosure is shown.

[0023] Figure 1D

[0024] Figure 1D A schematic diagram for predicting entering particles using an entering particle prediction model according to an embodiment of the disclosure is shown.

[0025] Figure 1E

[0026] Figure 1E ​​​​​​​​​​Estimation of future traffic density according to an embodiment of the disclosure is shown.

[0027] [ Figure 1F ]

[0028] Figure 1F A schematic diagram for determining a polynomial probability vector according to an embodiment of the disclosure is shown.

[0029] [ Figure 1G ]

[0030] Figure 1G Estimation of future traffic density based on a polynomial probability vector and a polynomial probability vector according to an embodiment of the disclosure is shown.

[0031] [ Figure 2 ]

[0032] Figure 2 A block diagram of a system for estimating future traffic density according to an embodiment of the disclosure is shown.

[0033] [ Figure 3 ]

[0034] Figure 3 A block diagram of an architecture of a transformer decoder of a trajectory prediction model according to an embodiment of the disclosure is shown.

[0035] [ Figure 4 ]

[0036] Figure 4 A block diagram of an architecture of a transformer decoder of an entry particle prediction model according to an embodiment of the disclosure is shown.

[0037] [ Figure 5 ]

[0038] Figure 5 Deployment of a service robot in an indoor space based on an estimated future traffic density according to an embodiment of the disclosure is shown.

[0039] [ Figure 6 ]

[0040] Figure 6 Control of a traffic light in an outdoor space according to an embodiment of the disclosure is shown.

[0041] [ Figure 7A ]

[0042] Figure 7A A first part of a block diagram of a method for estimating a future traffic density in an environment according to an embodiment of the disclosure is shown.

[0043] [ Figure 7B ]

[0044] Figure 7B A second portion of a block diagram showing a method for estimating future traffic density in an environment according to embodiments of the present disclosure is shown.

[0045] [ Figure 8 ]

[0046] Figure 8 A schematic diagram of a computing device that can be used to implement the systems and methods of the present disclosure is shown. DETAILED DESCRIPTION

[0047] The presently disclosed embodiments will be further explained with reference to the drawings. The depicted drawings are not necessarily to scale, and generally emphasis has been placed upon illustrating the principles of the presently disclosed embodiments.

[0048] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure can be practiced without these specific details. In other instances, devices and methods are shown in block diagram form in order to avoid obscuring the present disclosure.

[0049] As used in this specification and claims, the terms “for example,” “for instance,” and “such as,” and the verbs “comprising,” “having,” “including,” and their other verb forms, when used to describe the disclosure, are each meant to encompass the other embodiments. The term “based on” means at least partially based on. Furthermore, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. Any headings used herein are for convenience only and do not interpret the scope or meaning of the embodiments.

[0050] An object of some embodiments is to estimate future traffic density in an environment. The environment can be an indoor space (e.g., an office space) or an outdoor space (e.g., an urban area including roads). The environment includes a plurality of entry points and exit points. The present disclosure provides a traffic density estimation model for estimating future traffic density in an environment.

[0051] Figure 1A A block diagram of a traffic density estimation model 101 for estimating future traffic density 109 according to embodiments of the present disclosure is shown. The traffic density estimation model 101 includes a trajectory prediction model 103, an entering particle prediction model 105, and an iterative sampling model 107. Some embodiments are based on the recognition that the traffic density in an environment depends on the movement of objects (e.g., people) present in the environment at a given time. The trajectory prediction model 103 takes into account the movement of existing objects by predicting the trajectories of the existing objects, as described below with reference to Figure 1BThe.

[0052] Figure 1B A schematic diagram illustrating the operation of the trajectory prediction model 103 that considers the movement of an existing object by predicting the trajectory of the existing object, according to an embodiment of the present disclosure, is shown. A partial trajectory 111 of an object is applied as input to the trajectory prediction model 103. The object moves in the environment from an entry point among a plurality of entry points to an exit point among a plurality of exit points. The partial trajectory 111 is represented by a sequence of positions of length 9 [1, 3, 2, 0, 0, 0, 0, 0, 0, 0] at different time instants, such as time instants t1, t2, t3, t4, t5, t6, t7, t8, t9, respectively. For the purpose of explanation, it is assumed that there are a total of six different positions (1-6) in the environment. The partial trajectory 111 depicts that the position of the object in the environment at time instant t1 is 1, the position of the object in the environment at time instant t2 is 3, and the position of the object in the environment at time instant t3 is 2. Such a partial trajectory is input to the trajectory prediction model 103 to predict the position of the object at the next time instant t4 (also referred to as a future time instant).

[0053] At a first prediction step, the partial trajectory 111 is input to the trajectory prediction model 103 and the trajectory prediction model 103 outputs a probability distribution 113 of the position of the object over the six positions (1-6). The probability distribution 113 indicates the probability of the object being at each of the six positions (1-6) at time instant t4. For example, the probability distribution 113 indicates that the probability of the object being at position 5 at time instant t4 is 0.6.

[0054] Some embodiments are based on the recognition that the object can be represented by a plurality of particles (e.g., five particles) and each particle is processed independently. The number of particles can be specified by a user. Each particle can follow a different trajectory. To this end, the position of each particle at time instant t4 is to be predicted. To predict the position of each particle at time instant t4, positions are sampled from the probability distribution 113 and the sampled positions are appended to the trajectory of the particle. For example, for the trajectory 115 of particle-1, the position of particle-1 at time instant t4 is predicted to be 5 by sampling from the probability distribution 113. Likewise, for the trajectory of each particle, the position at time instant t4 is predicted to determine the updated trajectories 117 corresponding to the five particles.

[0055] Further, at a second prediction step, the updated trajectories 117 are used to predict the position of each particle at time instant t5, as described below with reference to Figure 1C The.

[0056] Figure 1CA prediction of the location of each particle at time t5 is shown in accordance with an embodiment of the present disclosure. The updated trajectories 117 corresponding to the five particles are input to the trajectory prediction model 103, and the trajectory prediction model 103 outputs a probability distribution for each particle (as probability distribution 113). A location for each particle is sampled from the corresponding probability distribution, and the sample location is appended to the corresponding trajectory. For example, the updated trajectory 117a for particle-1 is input to the trajectory prediction model 103, and the trajectory prediction model 103 outputs a probability distribution of the location of particle-1 over six locations (1-6). A location is sampled from the probability distribution of the location of particle-1, and the sample location (e.g., location 3) is appended to the updated trajectory 117a of particle-1 to determine a new updated trajectory 119a for particle-1. Likewise, the locations of the other particles at time t5 are predicted to determine new updated trajectories 119.

[0057] Similarly, in a third prediction step, the new updated trajectories 119 are further used to predict the location of each particle at time t6. In this way, the location of each particle is iteratively predicted until the location is predicted for time t9 to complete the trajectories.

[0058] Some embodiments are based on the recognition that for an environment with constant entering and exiting objects, the traffic density estimation model 101 should take into account objects that can initiate new trajectories at the entry point at the next time instant to accurately estimate future traffic density. In other words, entering particles are to be predicted to accurately estimate future traffic density. The traffic density estimation model 101 includes an entering particle prediction model 105 for predicting entering particles. The entering particle prediction model 105 is described below with reference to FIG. 4. Figure 1D The entering particle prediction model 105 is explained.

[0059] Figure 1D A schematic diagram is shown for predicting entering particles using the entering particle prediction model 105 in accordance with an embodiment of the present disclosure. The observation vector sequence 121 is input to the entering particle prediction model 105. Each element of the observation vector sequence 121 is a binary variable, where 1 indicates the presence of an entering particle, and 0 indicates no observation. For example, the observation vector sequence 121 can be obtained from a camera installed in the environment. At time i = 2, an object (e.g., a person) starts moving from location 3 and is captured by the camera, then z2[3] = 1 and all other entries of z2(vector) are zero. The entering particle prediction model 105 outputs a multinomial probability vector 123. The multinomial probability vector 123 indicates the probability of observing an entering particle at each entry point at the next time instant. For the purpose of explanation, the number of entry points is considered to be six (1-6). Each shade in the multinomial probability vector 123 depicts a different probability. Further, based on the multinomial probability vector 123, the next observation vector z7 is predicted and appended to the observation vector sequence 121 to determine an updated observation vector sequence 125.

[0060] Based on the updated trajectories 117 and the updated observation vector sequence 125, the future traffic density 109 at the next time instant can be estimated, as explained below with reference to Figure 1E

[0061] Figure 1E An estimation of the future traffic density 109 according to an embodiment of the present disclosure is shown. The updated trajectories 117 and the updated observation vector sequence 125 are input to the iterative sampling model 107. Based on the updated trajectories 117 and the updated observation vector sequence 125, the iterative sampling model 107 estimates the future traffic density 109 at the next time instant.

[0062] In some embodiments, multiple partial trajectories of different objects, such as the partial trajectories 111, are input to the trajectory prediction model 103. For example, as Figure 1F shown at a first prediction step, multiple trajectories, such as the partial trajectory of object-1 127a, the partial trajectory of object-2 129a, and the partial trajectory of object-3 131a, are input to the trajectory prediction model 103. The trajectory prediction model 103 outputs a probability distribution of the location of object-1 127b at the next time instant over the six locations (1-6), a probability distribution of the location of object-2 129b at the next time instant over the six locations, and a probability distribution of the location of object-3 131b at the next time instant over the six locations. Further, the probability distribution 127b, the probability distribution 129b, and the probability distribution 131b are averaged to determine an aggregated multinomial probability vector 133. The aggregated multinomial probability vector 133 indicates the final prediction probability of the location of the objects over the six locations.

[0063] ​Further, based on the aggregated polynomial probability vector 133 and the polynomial probability vector 123, the iterative sampling model 107 estimates the future traffic density 109 at the next time instant, as explained below with reference to Figure 1G

[0064] Figure 1G An estimation of the future traffic density 109 based on the aggregated polynomial probability vector 133 and the polynomial probability vector 123 according to an embodiment of the present disclosure is shown. The polynomial probability vector 123 and the aggregated polynomial probability vector 133 are input to the iterative sampling model 107. Based on the polynomial probability vector 123 and the aggregated polynomial probability vector 133, the iterative sampling model 107 estimates the future traffic density 109 at the next time instant. In particular, the iterative sampling model 107 outputs a sample of future objects from a mean vector of the polynomial probability vector 123 and the aggregated polynomial probability vector 133, the mean vector being indicative of the future traffic density. The mean vector can be given mathematically as

[0065]

[0066] wherein, denote the probability distribution 127b, the probability distribution 129b and the probability distribution 131b, and is the polynomial probability vector 123.

[0067] Some embodiments of the present disclosure provide a system for estimating a future traffic density 109 based on a trajectory prediction model 103, an entering particle prediction model 105 and an iterative sampling model 107. Such a system is described below in Figure 2 .

[0068] Figure 2 A block diagram of a system 200 for estimating a future traffic density 109 in an environment according to an embodiment of the present disclosure is shown. The system 200 comprises a network interface controller (NIC) 209 adapted to connect the system 200 to a network 211 (also referred to as a communication channel) by a bus 207. Through the network 211, which can be wireless or wired, the system 200 receives, for at least one object, at least one partial trajectory and a sequence of observation vectors 213. The at least one object moves in the environment from an entry point of a plurality of entry points to an exit point of a plurality of exit points. The at least one object is represented by a plurality of particles.

[0069] ​Further, in some implementations, a human-machine interface (HMI) 205 within the system 200 connects the system 200 to the keyboard 201 and the pointing device 203. The pointing device 203 can include a mouse, trackball, touchpad, joystick, trackpad, stylus, or touch screen, among others. Further, the system 200 includes an application interface 219 for connecting the system 200 to an application device 221 to perform various operations. Additionally, the system 200 can be linked through the bus 207 to a display interface 223 adapted to connect the system 200 to a display device 225, such as a computer monitor, television, projector, or mobile device, among others.

[0070] The system 200 also includes a processor 215 and a memory 217 that stores instructions executable by the processor 215. The processor 215 can be a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 217 can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems. The memory 217 is configured to store the trajectory prediction model 103, the entering particle prediction model 105, and the iterative sampling model 107.

[0071] The processor 215 is configured to process the at least one partial trajectory with the trajectory prediction model 103 trained to predict a probability distribution (e.g., the probability distribution 113) of a location of the at least one object at a future time at different locations in the environment. The processor 215 is further configured to predict a location of each particle in the plurality of particles at the future time based on the predicted probability distribution. The processor 215 is further configured to process the sequence of observation vectors with the entering particle prediction model 105 trained to predict a probability of observing an entering particle at each entry point at the future time. The processor 215 is further configured to estimate the future traffic density 109 at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time using the iterative sampling model 107.

[0072] In some implementations, the trajectory prediction model 103 includes a stack of transformer decoders. Each transformer decoder includes a plurality of layers as described below with reference to Figure 3

[0073] Figure 3 A block diagram illustrating an architecture of a transformer decoder 300 of the trajectory prediction model 103 according to an implementation of the present disclosure is shown. The transformer decoder 300 includes an input embedding and position encoding layer 301, a masked multi-head attention layer 303, a first addition and normalization layer 305, a multi-head attention layer 307, a second addition and normalization layer 309, a feed forward layer 311, and a third addition and normalization layer 313. The partial trajectory 111 is input to the embedding and position encoding layer 301.​

[0074] The input embedding and position encoding layer 301 is configured to produce a high-dimensional representation of the partial trajectory 111. For example, the input embedding and position encoding layer 301 is configured to process the input sequence (i.e., the partial trajectory 111). The input embedding layer takes the input sequence and maps each position to a high-dimensional vector representation. This embedding allows the trajectory prediction model 103 to model the meaning of the input sequence in a continuous vector space, which can capture complex relationships between positions in the trajectory. Unlike recurrent neural networks, which can process the input sequence sequentially and naturally capture position information, the transformer processes the entire input sequence in parallel, which means it needs a way to encode the relative position of each position in the input sequence. Position encoding addresses such a problem by adding a position encoding vector to each embedding, which encodes the position of each position in the input sequence. The position encoding vectors are learned during training and are designed to capture the relative position of each position in the input sequence in a way that is compatible with the attention mechanism used in the transformer.

[0075] The masked multi-head attention layer 303 is configured to ensure that the trajectory prediction model 103 predicts the position of the object based on the positions observed in the input sequence by selectively focusing on different parts of the input sequence when processing the input sequence. For example, first, the input sequence is divided into multiple “heads”, each of which can focus on a different part of the input sequence. Next, a “query” vector is created for each position in the input sequence. The query vector represents the information that the trajectory prediction model 103 tries to extract from that position. Then, the masked multi-head attention layer 303 computes a “score” for each position in the input sequence based on how well its query vector matches other positions in the input sequence. The computed scores reflect how important each position is for the current task of the trajectory prediction model. Finally, the trajectory prediction model 103 uses the computed scores to weight the input sequence, effectively amplifying important parts of the input sequence while ignoring irrelevant parts. The “masking” part of the masked multi-head attention layer 303 comes into play when processing sequences of variable length. Specifically, the masked multi-head attention layer 303 applies a mask to the scores to ensure that the trajectory prediction model 103 only focuses on positions in the input sequence that have already been processed and not on any future positions.

[0076] The first add and normalize layer 305 is configured to add the outputs of the input embedding and position encoding layer 301 and the masked multi-head attention layer 303 and normalize the resulting sum. The add and normalize layer consists of two parts: an element-wise summation operation (“add” part) and a layer normalization operation (“normalize” part). For example, the output from the masked multi-head attention layer 303 is added to the input of the first add and normalize layer 305. Such an operation is performed element-wise, which means that each element in the two tensors is added together. The resulting tensor is then normalized using layer normalization. Layer normalization is a technique for normalizing values in a tensor across a feature dimension (the last dimension in the tensor). Specifically, for each feature dimension, the mean and standard deviation of the values are computed, and then the values are shifted and scaled to have zero mean and unit variance. This normalization helps the trajectory prediction model 103 better handle vanishing and exploding gradients during training and can improve performance.

[0077] The multi-head attention layer 307 is configured to compute multiple attention functions with different learned projections. For example, the multi-head attention layer 307 is configured to allow the trajectory prediction model 103 to attend to different parts of the input sequence in parallel, and then combine the information from these different “heads” to produce the final output. For example, first, the input sequence (i.e., the output sequence from the first add and normalize layer 305) is transformed into three vectors: a query vector Q, a key vector K, and a value vector V. The three vectors are computed using learned linear transformations. Next, the three vectors are split into multiple “heads,” each of which can attend to a different part of the input sequence. Specifically, each head applies its own linear transformation to the query, key, and value vectors, producing a set of “projected” vectors for that head. An attention mechanism is then applied to each head individually. For each head, the multi-head attention layer 307 computes a “score” vector, which represents how well each position in the input sequence matches the query vector. The score vector is computed using the dot product between the query vector and the key vector for that position. The score vector is then normalized using a softmax function to produce a set of attention weights, which indicate how much that head should attend to each position in the input sequence. Finally, the value vector for each head is weighted and combined by the corresponding attention weights to produce a single output vector for that head. The output vectors are concatenated together to form the final output of the multi-head attention layer 307.

[0078] The second add and normalize layer 309 is configured to add and normalize the normalization and output by the first add and normalize layer 305 with the multiple attention functions output by the multi-head attention layer 307.

[0079] The feedforward layer 311 is a fully connected neural network configured to transform the normalized output of the second addition and normalization layer 309 using linear transformations. The fully connected neural network includes two linear transformations separated by a non-linear activation function. The first linear transformation projects the input tensor onto a higher dimensional space. The activation function applies a non-linear transformation to the output of the first linear transformation. Such a transformation allows the trajectory prediction model 103 to capture complex patterns and relationships in the input tensor. The second linear transformation maps the output of the activation function back down to the original dimensions of the input tensor.

[0080] The third addition and normalization layer 313 is configured to add and normalize the normalized output from the second addition and normalization layer 309 with the transformed normalized output of the feedforward layer 311.

[0081] In embodiments, the trajectory prediction model 103 is trained using a cross-entropy loss function. For example, the trajectory prediction model 103 is trained on a training dataset that includes a plurality of samples of pre-processed partial trajectories of people. Each partial trajectory is fed into the trajectory prediction model 103 and the trajectory prediction model 103 predicts the next location of the corresponding person. The cross-entropy loss function computes the negative log of the prediction of the location of the at least one object predicted for the true next destination and then averages the loss function over all samples in the training dataset, giving a single value that the trajectory prediction model attempts to minimize during training.

[0082] Further, in some implementations, the entering particle prediction model 105 includes a stack of transformer decoders. Each transformer decoder includes a plurality of layers as described below with reference to Figure 4

[0083] Figure 4 A block diagram illustrating an architecture of a transformer decoder 400 of the entering particle prediction model 105 according to embodiments of the present disclosure is shown. The transformer decoder 400 includes a masked multi-head attention layer 401, a first addition and normalization layer 403, a multi-head attention layer 405, a second addition and normalization layer 407, a feedforward layer 409, and a third addition and normalization layer 411.

[0084] ​The masked multi-head attention layer 401 is configured to ensure that the entering particle prediction model 105 predicts entering particles based on the observation position in the observation vector sequence 121. For example, the masked multi-head attention layer 401 is configured to selectively focus on different parts of the input sequence (e.g., the observation vector sequence 121) when processing the input sequence. For example, first, the input sequence is divided into multiple “heads”, each of which can focus on a different part of the input sequence. Next, a “query” vector is created for each position in the input sequence. The query vector represents the information that the entering particle prediction model 105 is trying to extract from that position. Then, the masked multi-head attention layer 401 computes a “score” for each position in the input sequence based on how well its query vector matches other positions in the input sequence. The computed scores reflect how important each position is for the current task of the entering particle prediction model. Finally, the entering particle prediction model 105 uses the computed scores to weight the input sequence, effectively amplifying important parts of the input sequence while ignoring irrelevant parts. The “masking” part of the masked multi-head attention layer 401 comes into play when processing sequences of variable length. Specifically, the masked multi-head attention layer 401 applies a mask to the scores to ensure that the entering particle prediction model 105 only focuses on positions in the input sequence that have already been processed and not on any future positions.

[0085] The first addition and normalization layer 403 is configured to add the outputs of the masked multi-head attention layer 401 and normalize the resulting sum. The addition and normalization layer consists of two parts: an element-wise summation operation (“addition” part) and a layer normalization operation (“normalization” part). For example, the outputs from the masked multi-head attention layer 401 are added to the inputs of the first addition and normalization layer 403. Such an operation is performed element-wise, which means that each element in the two tensors is added together. The resulting tensor is then normalized using layer normalization. Layer normalization is a technique for normalizing values in a tensor across the feature dimension (the last dimension in the tensor). Specifically, for each feature dimension, the mean and standard deviation of the values are computed, and then the values are shifted and scaled to have zero mean and unit variance. This normalization helps the entering particle prediction model 105 better handle vanishing and exploding gradients during training and can improve performance.

[0086] The multi-head attention layer 405 is configured to compute multiple attention functions using different learned projections. For example, the multi-head attention layer 405 is configured to allow the particle prediction model 105 to attend to different parts of the input sequence in parallel, and then combine the information from these different “heads” to produce the final output. For example, first, the input sequence (i.e., the output sequence from the first addition and normalization layer 403) is transformed into three vectors: a query vector Q, a key vector K, and a value vector V. The three vectors are computed using learned linear transformations. Next, the three vectors are split into multiple “heads,” each of which can attend to a different part of the input sequence. Specifically, each head applies its own linear transformation to the query vector, the key vector, and the value vector, producing a set of “projected” vectors for that head. An attention mechanism is then applied to each head individually. For each head, the multi-head attention layer 307 computes a “score” vector that represents how well each position in the input sequence matches the query vector. The score vector is computed using the dot product between the query vector and the key vector for that position. The score vector is then normalized using a softmax function to produce a set of attention weights that indicate how much that head should attend to each position in the input sequence. Finally, the value vector for each head is weighted and combined by the corresponding attention weights to produce a single output vector for that head. The output vectors are concatenated together to form the final output of the multi-head attention layer 405.

[0087] The second addition and normalization layer 407 is configured to add and normalize the normalized output from the first addition and normalization layer 403 with the multiple attention functions output by the multi-head attention layer 405.

[0088] The feedforward layer 409 is a fully connected neural network that is configured to transform the normalized output of the second addition and normalization layer 407 using linear transformations. The fully connected neural network includes two linear transformations separated by a non-linear activation function. The first linear transformation projects the input tensor onto a higher dimensional space. The activation function applies a non-linear transformation to the output of the first linear transformation. This transformation allows the particle prediction model 105 to capture complex patterns and relationships in the input tensor. The second linear transformation maps the output of the activation function back down to the original dimensions of the input tensor.

[0089] The third addition and normalization layer 411 is configured to add and normalize the normalized output of the second addition and normalization layer 407 with the transformed normalized output of the feedforward layer 409.

[0090] In implementations, the entering particle prediction model 105 is trained with a mean squared error loss function. For example, the entering particle prediction model 105 is trained on a training dataset comprising samples of observation vectors obtained from the camera. Each sample is applied to the entering particle prediction model 105, and the entering particle prediction model 105 predicts a corresponding entering particle. The mean squared error loss function computes the average of the squared differences between the predicted entering particle and the corresponding ground truth.

[0091] Some implementations are based on the recognition that the system 200 can be used to deploy service robots in an environment based on the estimated future traffic density 109. Service robots generally assist humans by performing dirty, mundane, remote, or repetitive jobs. For example, the environment can be an indoor space comprising a plurality of bays. The processor 215 is configured to deploy a service robot to at least one of the plurality of bays based on an estimate of the future traffic density in the at least one of the plurality of bays. Such implementations are described below in Figure 5 .

[0092] Figure 5 Deployment of service robots in an indoor space 500 according to implementations of the present disclosure is shown. The indoor space 500 can be an office space or a floor of a building comprising a plurality of bays, such as bay 501, bay 503, bay 505, bay 507, bay 509, bay 511, bay 513, bay 515, and bay 517. The number of service robots to be deployed to a bay is based on the future traffic density in the bay. For example, for bay 505, the future traffic density at time t = 3 is estimated to be 16 people / bay. Using a scaling factor (e.g., 0.25), the number of service robots to be deployed to bay 505 is determined to be 0.25 x 16 = 4. To this end, the processor 215 deploys four service robots, such as service robot 519a, service robot 519b, service robot 519c, and service robot 519d, to bay 505.

[0093] In some implementations, if the number of service robots already deployed to bay 505 exceeds the number of service robots corresponding to the estimated future traffic density, a redundant service robot is invoked, e.g., the redundant service robot is moved back to the service dispatch area 521. For example, if the number of service robots to be deployed in bay 505 is four, but there are already five service robots in bay 505, the processor 215 moves one of the five service robots back to the service dispatch area 521.

[0094] Deploying service robots based on future traffic density in this way minimizes contact between service robots and people in the environment, thereby improving efficiency and reducing the operational cost of service robots.

[0095] Additionally or alternatively, in some implementations, the environment is the outdoor space comprising multiple roads and traffic lights at their intersections. The traffic lights can be controlled based on estimated future traffic density in the environment, as follows: Figure 6 As described in [the text].

[0096] Figure 6 Control of traffic lights in an outdoor space 600 according to an embodiment of the present disclosure is illustrated. The outdoor space 600 may be an area of ​​a city containing multiple roads (e.g., road 601, road 603, road 605, and road 607). Furthermore, the outdoor space 600 includes traffic lights 609 at the intersections of the multiple roads. Traffic lights 609 are communicatively connected to system 200. Vehicles (such as vehicles 611a, 611b, and 611c) move on road 601. Vehicles may be autonomous or manually driven. System 200 estimates the future traffic density in area 613. Based on the estimated future traffic density, processor 215 controls traffic lights 609. For example, based on the estimated future traffic density, processor 215 may control the duration of red traffic lights and / or the duration of green traffic lights, thereby minimizing the waiting time for vehicles at traffic lights 609.

[0097] In addition, see the following reference Figure 7A and Figure 7B Describe a general method for estimating future traffic density in an environment (indoor or outdoor space).

[0098] Figure 7A and Figure 7B A block diagram of a method 700 for estimating future traffic density in an environment according to an embodiment of the present disclosure is shown. At block 701, method 700 includes receiving at least one partial trajectory (such as partial trajectory 111) and an observation vector sequence (such as observation vector sequence 121) for at least one object. The at least one object moves in the environment from an entry point among a plurality of entry points to an exit point among a plurality of exit points. The at least one object is represented by a plurality of particles.

[0099] At box 703, method 700 includes processing at least one partial trajectory using a trajectory prediction model (i.e., trajectory prediction model 103), which is trained to predict the probability distribution (e.g., probability distribution 113) of the position of at least one object at different locations in the environment at future time points. At box 705, method 700 includes predicting the position of each of a plurality of particles at future time points based on the predicted probability distribution. To predict the position of each of the plurality of particles at future time points based on the predicted probability distribution, the position of each particle is sampled from the predicted probability distribution.

[0100] At block 707, the method 700 includes processing the sequence of observation vectors (e.g., processing the sequence of observation vectors 121) with an entering particle prediction model (e.g., the entering particle prediction model 105) trained to predict a probability of observing an entering particle at each entry point at a future time. In particular, the entering particle prediction model outputs a multinomial probability vector (e.g., the multinomial probability vector 123) that indicates probabilities of observing an entering particle at each entry point at a future time.

[0101] At block 709, the method 700 includes estimating a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

[0102] Figure 8 A schematic diagram of a computing device that can be used to implement the system 200 and method 700 of the present disclosure is shown. The computing device 800 includes a power supply 801, a processor 803, a memory 805, a storage 807, all of which are connected to a bus 809. In addition, a high-speed interface 811, a low-speed interface 813, a high-speed expansion port 815, and a low-speed connection port 817 can be connected to the bus 809. Furthermore, a low-speed expansion port 819 is connected to the bus 809. In addition, an input interface 821 can be connected to an external receiver 823 and an output interface 825 via the bus 809. A receiver 827 can be connected to an external transmitter 829 and a transmitter 831 via the bus 809. An external memory 833, an external sensor 835, a machine 837, and an environment 839 can also be connected to the bus 809. In addition, one or more external input / output devices 841 can be connected to the bus 809. A network interface controller (NIC) 843 can be adapted to connect to a network 845 through the bus 809, where data or other data can be presented on third-party display devices, third-party imaging devices, and / or third-party printing devices external to the computing device 800.

[0103] The memory 805 can store instructions that are executable by the computing device 800, as well as any data that can be utilized by the methods and systems of the present disclosure. The memory 805 can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems. The memory 805 can be one or more volatile memory units and / or one or more non-volatile memory units. The memory 805 can also be another form of computer-readable media, such as a magnetic or optical disk.

[0104] The storage device 807 can be adapted to store supplementary data and / or software modules used by the computing device 800. The storage device 807 can include a hard drive, an optical drive, a thumb-drive, an array of drives, or any combinations thereof. Further, the storage device 807 can include a computer-readable medium, such as a floppy disk device, a hard drive device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. Instructions can be stored in the information carrier. The instructions can be executed by one or more processing devices (for example, the processor 803) to implement one or more methods, such as those described above.

[0105] The computing device 800 can be linked through the bus 809 to a display interface or user interface (HMI) 847 adapted to connect the computing device 800 to a display device 849 and a keyboard 851, where the display device 849 can include a computer monitor, a camera, a television, a projector, or a mobile device, among others. In some implementations, the computing device 800 can include a printer interface to connect to a printing device, where the printing device can include a liquid inkjet printer, a solid ink printer, a large-format commercial printer, a thermal printer, a UV printer, or a dye-sublimation printer, among others.

[0106] The high-speed interface 811 manages bandwidth-intensive operations for the computing device 800, while the low-speed interface 813 manages lower bandwidth-intensive operations. Such allocation of functions is exemplary only. In some implementations, the high-speed interface 811 can be coupled to the memory 805, the user interface (HMI) 848, the keyboard 851, and the display device 849 (for example, through a graphics processor or accelerator), and the high-speed expansion ports 815, which can accept various expansion cards including, for example, graphics cards, sound cards, or network cards that can be used to connect the computing device 800 to an external communications network. In implementations, the low-speed interface 813 is coupled to the storage device 807 and the low-speed expansion port 817 via the bus 809. The low-speed expansion port 817, which can include various communication ports (for example, USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices 841. The computing device 800 can be connected to a server 853 and a rack server 855. The computing device 800 can be implemented in several different forms. For example, the computing device 800 can be implemented as part of the rack server 855.

[0107] This description provides only the exemplary implementations, and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the following description of the exemplary implementations will provide those skilled in the art with enabling descriptions of how one or more exemplary implementations can be performed. Various changes can be made to the function and arrangement of elements without departing from the concept and scope of the disclosed subject matter as set forth in the appended claims.

[0108] In the following description, specific details are given to provide a thorough understanding of the embodiments. However, a person of ordinary skill in the art will understand that the embodiments can be practiced without these specific details. For example, the systems, processes, and other elements in the disclosed subject matter can be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques can be shown without detailed description in order to avoid obscuring the embodiments. Additionally, like numbers in different drawings can represent the same element or different aspects of the same element.

[0109] Furthermore, various embodiments can be described as a process, which can be depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart can describe operations as a sequential process, many of the operations can be performed in parallel, or concurrently, or in any order. In addition, the order of the operations can be re-arranged. A process can be terminated when its operations are completed, but could also occur indefinitely. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function.

[0110] Furthermore, embodiments of the subject matter described can be implemented, at least in part, manually or automatically. Manual or automatic implementation can occur over a time period, such as at one time, once the process is initiated, repeatedly (for example, on a circuit or platform), continually or a single time. In addition, embodiments of the subject matter described can be implemented using software, hardware, firmware, middleware, microcode, hardware description languages, or any combination thereof. If implemented in software, the program code or code segments can be stored in a machine-readable medium. Processors can execute the necessary tasks.

[0111] The various methods or processes outlined herein can be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software can be written using any of a number of suitable programming languages and / or programming or scripting tools, and also can be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Typically, the functionality of the program modules can be combined or distributed as desired in various embodiments.

[0112] Embodiments of the present disclosure can be embodied as a method, of which an example has been provided. The acts performed as part of the method can be ordered in any suitable way. Accordingly, embodiments can be constructed in which acts are performed in an order different than illustrated, which can include performing some acts simultaneously, even though shown as serial process acts in illustrative embodiments.

[0113] Moreover, the functions described in this disclosure can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, some embodiments of this disclosure can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Furthermore, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0114] According to embodiments of the present disclosure, the term "data processing apparatus" can encompass all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0115] A computer program (which can also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.

[0116] A computer program can be deployed for execution by one computer or by multiple computers located in one site or distributed across multiple sites and interconnected by a communication network. Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0117] Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0118] To provide for interaction with a user, implementations of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s client device in response to requests received from the web browser.

[0119] Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN") and a wide area network ("WAN"), e.g., the Internet.

[0120] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0121] While this disclosure has described certain preferred embodiments, it is understood that various other alterations and modifications will become apparent to the skilled artisan after reviewing this disclosure. Accordingly, it is the intention of the appended claims to cover all such variations and modifications as fall within the true spirit and scope of the present disclosure.

Claims

1. A system for estimating future traffic density in an environment, wherein, The environment includes a plurality of entry points and a plurality of exit points, the system comprising: at least one processor; and a memory having instructions stored thereon that cause the at least one processor of the system to perform the following operations: receiving, for at least one object, at least one partial trajectory and a sequence of observation vectors, wherein the at least one object is moving from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and wherein the at least one object is represented by a plurality of particles; performing the following processes: (1) processing the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a location of the at least one object at a future time in different locations in the environment; and predicting a location of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) processing the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimating a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

2. The system of claim 1, wherein, To predict the location of each particle of the plurality of particles at the future time based on the predicted probability distribution, the at least one processor is further configured to sample the location of each particle from the predicted probability distribution.

3. The system of claim 1, wherein, The trajectory prediction model comprises a stack of transformer decoders, and wherein each transformer decoder comprises: an input embedding and position encoding layer configured to produce a high-dimensional representation of the at least one partial trajectory; a masked multi-head attention layer configured to ensure that the trajectory prediction model predicts the location of the object based on observed locations in the at least one partial trajectory; a first addition and normalization layer configured to add and normalize an output of the input embedding and position encoding layer and the masked multi-head attention layer; a multi-head attention layer configured to compute a plurality of attention functions with different learned projections; a second addition and normalization layer configured to add and normalize a normalized sum output by the first addition and normalization layer and the plurality of attention functions output by the multi-head attention layer; a feed-forward layer configured to transform a normalized output of the second addition and normalization layer using a linear transformation; and a third addition and normalization layer configured to add and normalize the normalized output of the second addition and normalization layer and a transformed normalized output of the feed-forward layer.

4. The system of claim 1, wherein, The entering particle prediction model comprises a stack of transformer decoders, and wherein each transformer decoder comprises: an input embedding and position encoding layer configured to produce a high-dimensional representation of the sequence of observation vectors; a masked multi-head attention layer configured to ensure that the entering particle prediction model predicts the probability of observing an entering particle at each entry point at a future time based on observed locations in the sequence of observation vectors; a first addition and normalization layer configured to add and normalize an output of the input embedding and position encoding layer and the masked multi-head attention layer; a multi-head attention layer configured to compute a plurality of attention functions with different learned projections; a second addition and normalization layer configured to add and normalize a normalized sum output by the first addition and normalization layer and the plurality of attention functions output by the multi-head attention layer; a feed-forward layer configured to transform a normalized output of the second addition and normalization layer using a linear transformation; and a third addition and normalization layer configured to add and normalize the normalized output of the second addition and normalization layer and a transformed normalized output of the feed-forward layer. a masked multi-head attention layer configured to ensure that the entering particle prediction model predicts entering particles based on observation positions of the sequence of observation vectors; a first additive and normalization layer configured to add outputs of the masked multi-head attention layer and normalize the resulting sum; a multi-head attention layer configured to compute a plurality of attention functions with different learned projections; a second additive and normalization layer configured to add and normalize the normalized sum output by the first additive and normalization layer with the plurality of attention functions output by the multi-head attention layer; a feed-forward layer configured to transform the normalized output of the second additive and normalization layer using a linear transformation; and a third additive and normalization layer configured to add and normalize the normalized output of the second additive and normalization layer with the transformed normalized output of the feed-forward layer.

5. The system of claim 1, wherein, The trajectory prediction model is trained with a cross-entropy loss function, and wherein the cross-entropy loss function computes a negative log likelihood of the predicted position of the at least one object for a true next destination.

6. The system of claim 1, wherein, The entering particle prediction model is trained with a mean squared error loss function, and wherein the mean squared error loss function computes an average of squared differences between predicted entering particles and corresponding ground truths.

7. The system of claim 1, wherein, The environment is an indoor space comprising a plurality of compartments, and wherein the at least one processor is further configured to deploy at least one service robot to at least one compartment of the plurality of compartments based on the estimated future traffic density in the environment.

8. The system of claim 1, wherein, The environment is an outdoor space comprising a plurality of roads and traffic lights at intersections of the plurality of roads, and wherein the at least one processor is further configured to control the traffic lights based on the estimated future traffic density in the environment.

9. A method for estimating future traffic density in an environment, wherein, The environment comprises a plurality of entry points and a plurality of exit points, the method comprising: receiving, for at least one object, at least one partial trajectory and a sequence of observation vectors, wherein the at least one object moves from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and wherein the at least one object is represented by a plurality of particles; performing the following processes: (1) processing the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a position of the at least one object at a future time at different locations in the environment; and predicting a position of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) processing the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimating a future traffic density at the future time based on the predicted position of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

10. The method of claim 9, wherein, To predict a position of each particle of the plurality of particles at the future time instant based on the predicted probability distribution, the method further comprises sampling the position of each particle from the predicted probability distribution.

11. The method of claim 9, wherein, The trajectory prediction model comprises a stack of transformer decoders, and wherein each transformer decoder comprises: an input embedding and position encoding layer configured to produce a high-dimensional representation of the at least one partial trajectory; a masked multi-head attention layer configured to ensure that the trajectory prediction model predicts the position of the object based on observed positions in the at least one partial trajectory; a first addition and normalization layer configured to add outputs of the input embedding and position encoding layer and the masked multi-head attention layer and to normalize the resulting sum; a multi-head attention layer configured to compute a plurality of attention functions with different learned projections; a second addition and normalization layer configured to add and normalize the normalized sum output by the first addition and normalization layer and the plurality of attention functions output by the multi-head attention layer; a feed-forward layer configured to transform the normalized output of the second addition and normalization layer using a linear transformation; and a third addition and normalization layer configured to add and normalize the normalized output of the second addition and normalization layer and the transformed normalized output of the feed-forward layer.

12. The method of claim 9, wherein, The entering particle prediction model comprises a stack of transformer decoders, and wherein each transformer decoder comprises: a masked multi-head attention layer configured to ensure that the entering particle prediction model predicts an entering particle based on observed positions of the sequence of observation vectors; a first addition and normalization layer configured to add outputs of the masked multi-head attention layer and to normalize the resulting sum; a multi-head attention layer configured to compute a plurality of attention functions with different learned projections; a second addition and normalization layer configured to add and normalize the normalized sum output by the first addition and normalization layer and the plurality of attention functions output by the multi-head attention layer; a feed-forward layer configured to transform the normalized output of the second addition and normalization layer using a linear transformation; and a third addition and normalization layer configured to add and normalize the normalized output of the second addition and normalization layer and the transformed normalized output of the feed-forward layer.

13. The method of claim 9, wherein, The trajectory prediction model is trained with a cross-entropy loss function, and wherein the cross-entropy loss function computes a negative logarithm of a predicted position of the at least one object for a true next destination.

14. The method of claim 9, wherein, The entering particle prediction model is trained with a mean squared error loss function, and wherein the mean squared error loss function computes an average of squared differences between predicted entering particles and corresponding ground truths.

15. The method of claim 9, wherein, The environment is an indoor space comprising a plurality of compartments, and wherein the method further comprises deploying at least one service robot to at least one compartment of the plurality of compartments based on the estimated future traffic density in the environment.

16. The method of claim 9, wherein, The environment is an outdoor space comprising a plurality of roads and traffic lights at intersections of the plurality of roads, and wherein the method further comprises controlling the traffic lights based on the estimated future traffic density in the environment.

17. A non-transitory computer-readable storage medium having embodied thereon a program executable by a processor to perform a method for estimating future traffic density in an environment, wherein, The environment comprises a plurality of entry points and a plurality of exit points, the method comprising: receiving, for at least one object, at least one partial trajectory and a sequence of observation vectors, wherein the at least one object moves from an entry point of the plurality of entry points to an exit point of the plurality of exit points, and wherein the at least one object is represented by a plurality of particles; performing the following processing: (1) processing the at least one partial trajectory with a trajectory prediction model trained to predict a probability distribution of a location of the at least one object at a future time at different locations in the environment; and predicting a location of each particle of the plurality of particles at the future time based on the predicted probability distribution; and (2) processing the sequence of observation vectors with an entering particle prediction model trained to predict a probability of observing an entering particle at each entry point at a future time; and estimating a future traffic density at the future time based on the predicted location of each particle and the predicted probability of observing an entering particle at each entry point at the future time.

18. The non-transitory computer-readable storage medium of claim 17, wherein, To predict the location of each particle of the plurality of particles at the future time based on the predicted probability distribution, the method further comprises sampling a location of each particle from the predicted probability distribution.

19. The non-transitory computer-readable storage medium of claim 17, wherein, The environment is an indoor space comprising a plurality of compartments, and wherein the method further comprises deploying at least one service robot to at least one compartment of the plurality of compartments based on the estimated future traffic density in the environment.

20. The non-transitory computer-readable storage medium of claim 17, wherein, The environment is an outdoor space comprising a plurality of roads and traffic lights at intersections of the plurality of roads, and wherein the method further comprises controlling the traffic lights based on the estimated future traffic density in the environment.

Images