Travel route generation method and system

Abstract

The invention discloses a travel route generation method and system, relates to the technical field of navigation, and solves the problems that in the prior art, large estimation errors exist in complex environments such as traffic flow in peak hours, and the quality of a planned travel route is low. According to the embodiment of the invention, multi-dimensional information of bus stops, online car-hailing hotspots and traffic grid units is fused through the space-time traffic map, fine-grained perception of traffic states is realized, and the map can comprehensively capture complex association between different traffic factors and road congestion degrees through combination of node attributes and passing time; then probability prediction of future passing time is carried out through the space-time diagram neural network, the prediction uncertainty is effectively quantified through the obtained prediction probability distribution, prediction errors caused by traffic flow abrupt change are reduced, and the quality of the finally generated travel route is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of navigation technology, in particular to a travel route generation method and system. BACKGROUND

[0002] With the acceleration of urbanization and the diversification of residents' travel needs, the urban transportation system is facing unprecedented pressure and challenges. Policies at all levels place more emphasis on the development of digital transportation, proposing to strengthen the use of digital technology to improve transportation services and urban management, accelerate the transformation of traditional facilities, deepen urban renewal, accelerate the intelligent transformation of transportation facilities, promote the application of new technologies such as 5G, Internet of Things, artificial intelligence in transportation facility construction and operation, and improve the ability of intelligent monitoring of transportation facilities, real-time sensing of road network operation, risk investigation and safety warning, and maintenance decision-making.

[0003] Under this background, travel planning methods based on intelligent algorithms are widely used, and the core goal is to provide users with the best travel route recommendation to achieve efficient use of transportation resources and improve travel experience.

[0004] However, current travel planning methods are mostly based on the travel environment of each road, and the prediction model predicts the congestion condition change of the path through the historical travel environment and the current travel environment of the road, which is difficult to fully explore the nonlinear dependence relationship between various traffic factors, resulting in a large prediction error in the face of complex environments such as peak traffic flow, and the quality of the planned path is low.

[0005] Therefore, there is a need for a travel route generation method and system. SUMMARY

[0006] In view of the large prediction error of the prior art in the face of complex environments such as peak traffic flow, and the low quality of the planned path, the present application provides a travel route generation method and system, which can reduce the prediction error in the face of complex environments such as peak traffic flow, and improve the quality of the planned path. The specific technical solutions are as follows: In a first aspect, the present application provides a travel route generation method, comprising: Obtain geographical information and traffic information of a target urban area, the geographical information including bus stops and a road network, and the traffic information including pickup and drop-off hotspots of online car-hailing services and travel times of roads in the road network; construct a spatio-temporal traffic graph based on the geographical information and the traffic information; input the spatio-temporal traffic graph into a spatio-temporal graph neural network to obtain a travel time prediction probability distribution of each road in the road network in a future preset time period output by the spatio-temporal graph neural network; receive a travel request of a user; input the travel request and the prediction probability distribution into a reinforcement learning model to obtain a planned route output by the reinforcement learning model; wherein the reinforcement learning model is obtained based on modeling of a route planning problem.

[0007] In the spatio-temporal traffic graph, nodes include first nodes corresponding to the bus stops, second nodes corresponding to the pickup and drop-off hotspots of online car-hailing services, and third nodes corresponding to traffic grid cells obtained by dividing the target urban area. A physical edge between two nodes corresponds to a road between the real positions of the two nodes. The attributes of each physical edge include the travel time of the corresponding road. The attributes of the first nodes include the number of waiting passengers at the corresponding bus stops. The attributes of the second nodes include the number of available vehicles at the corresponding pickup and drop-off hotspots of online car-hailing services. The attributes of the third nodes include the overall traffic state of the corresponding local area.

[0008] Preferably, the spatio-temporal traffic graph further includes logical edges. The first node is connected to the third node where the first node is located through a logical edge. The second node is connected to the third node where the second node is located and the fourth node adjacent to the third node through logical edges, respectively. The logical edges are used to establish the association of traffic states between nodes of different granularities. The spatial convolution layer of the spatio-temporal graph neural network includes a first branch, a second branch, a fusion unit, and a convolution unit. The first branch is used to process the connection relationship of the physical edges. The second branch is used to process the connection relationship of the logical edges. The fusion unit is used to fuse the processing results of the two branches. The convolution unit is used for forward propagation based on the fusion result.

[0009] Preferably, the reinforcement learning model is a multi-agent model. The reinforcement learning model includes a central policy network. The agents of the reinforcement learning model share the central policy network. The input of the travel request and the prediction probability distribution into the reinforcement learning model to obtain the planned route output by the reinforcement learning model includes: the reinforcement learning model establishes an agent based on the travel request; the agent takes the origin and destination based on the travel request, the travel preference weight of the user, and the vector obtained from the prediction probability distribution as a state, inputs the state into the central policy network, and obtains the planned route output by the central policy network.

[0010] Preferably, the reward function of the agent comprises a personal reward term and a system penalty term, the personal reward term is obtained by weighted summation based on the travel preference weight, and the cost, time and quantitative discomfort of the planned route; the system penalty term is obtained by weighted summation based on the local congestion change amount and the vehicle distribution imbalance change amount caused after the planned route is executed.

[0011] Preferably, the training process of the central strategy network comprises: training the central strategy network in a simulated traffic environment in stages through a curriculum learning strategy, and the difficulty of the training sequence is higher in the later training stage; the difficulty of the training stage increases with the increase of the number of agents, the traffic flow density and the probability of sudden event injection.

[0012] Preferably, the prediction probability distribution comprises a prediction standard deviation, and the planned route comprises K candidate routes; after the travel request and the prediction probability distribution are input into the reinforcement learning model to obtain the planned route output by the reinforcement learning model, the method further comprises: calculating the total expected utility and the total uncertainty of the candidate routes; wherein the personal reward term is obtained by weighted summation based on the travel preference weight, and the cost, time and quantitative discomfort of the planned route; the system penalty term is obtained by weighted summation based on the local congestion change amount and the vehicle distribution imbalance change amount caused after the planned route is executed; the reliability score of the candidate route is calculated based on the total expected utility and the total uncertainty; and the K candidate routes and the corresponding reliability scores are returned to the client of the user.

[0013] Preferably, after the spatio-temporal traffic graph is input into the spatio-temporal graph neural network to obtain the prediction probability distribution of the physical edge passing time in the future preset time period output by the spatio-temporal graph neural network, the method further comprises: periodically acquiring incremental data of the passing time of each road in the road network; inputting the incremental data into the spatio-temporal graph neural network to update the prediction probability distribution.

[0014] In a second aspect, the embodiments of the present application provide a travel route generation system, which is applied to the method of the first aspect, and the system comprises: The acquisition module is configured to acquire geographical information and traffic information of a target urban area, wherein the geographical information comprises bus stops and a road network, and the traffic information comprises pickup and drop-off hotspots of online car-hailing services and passing times of roads in the road network. The construction module is configured to construct a space-time traffic graph based on the geographic information and the traffic information; wherein, nodes in the space-time traffic graph include a first node corresponding to the bus stop, a second node corresponding to the online car-hailing pickup and drop-off hotspot, and a third node corresponding to a traffic grid unit, the traffic grid unit being obtained by dividing the target urban area; a physical edge between two nodes corresponds to a road between the real positions of the two nodes, and the attribute of each physical edge includes the passing time of the corresponding road; the attribute of the first node includes the number of waiting passengers at the corresponding bus stop, the attribute of the second node includes the number of available vehicles at the corresponding online car-hailing pickup and drop-off hotspot, and the attribute of the third node includes the overall traffic state of the corresponding local area; The prediction module is configured to input the space-time traffic graph into a space-time graph neural network to obtain a passing time prediction probability distribution of each road in the road network within a future preset time period output by the space-time graph neural network. The receiving module is configured to receive a travel request of a user. The planning module is configured to input the travel request and the prediction probability distribution into a reinforcement learning model to obtain a planning route output by the reinforcement learning model; wherein, the reinforcement learning model is obtained based on modeling of a route planning problem.

[0015] In a third aspect, an embodiment of the present application provides a computing device, comprising: a memory configured to store a program; and a processor configured to load the program to execute the method of the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising a stored program, wherein the program controls the device where the computer-readable storage medium is located to execute the method of the first aspect when the program is running.

[0017] Compared with the prior art, the present application has the following beneficial effects: the space-time traffic graph integrates the multi-dimensional information of the bus stop, the online car-hailing hotspot and the traffic grid unit, realizes fine-grained perception of the traffic state, and the combination of the node attribute and the passing time enables the graph to comprehensively capture the complex correlation between different traffic factors and the road congestion degree; then, the space-time graph neural network is used for probability prediction, the prediction probability distribution obtained effectively quantifies the prediction uncertainty, reduces the prediction error caused by traffic flow mutation, and improves the quality of the finally generated travel route. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual proportions.

[0019] Figure 1 A flowchart of a travel route generation method provided by an embodiment of the present application is shown in FIG. 1. Figure 2 A structural diagram of a travel route generation system provided by an embodiment of the present application is shown in FIG. 2. Figure 3 A structural diagram of a computing device provided by an embodiment of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0021] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] It should also be understood that the terms used in the present application specification are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should be further understood that the term “and / or” used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0024] To solve the problem that the traditional method has a large prediction error in the face of complex environments such as peak traffic flow, and the quality of the planned path is low, the present application provides a travel route generation method and system, which can reduce the prediction error in the face of complex environments such as peak traffic flow, and improve the quality of the planned path.

[0025] Please refer to Figure 1 , Figure 1 A flowchart of a travel route generation method provided by an embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the method comprises the following steps. Step 101, the computing device acquires geographical information and traffic information of a target city area.

[0026] The computing device can be a terminal or a server, specifically, the server can be a blade server, a high-density server, a rack server, a cabinet server, a general-purpose server, a graphics processing unit (GPU) server, a data processing unit (DPU) server, or an artificial intelligence (AI) server, etc.; the terminal can be a personal computer, a notebook computer, a smartphone, a tablet computer, an Internet of Things device, and a portable wearable device, etc.

[0027] Preferably, the computing device can be deployed with a travel integrated service platform, which includes a bus subsystem, a ride-hailing subsystem, and a bus station management system, and can manage different types of vehicles through these subsystems to obtain corresponding information; the travel integrated service platform can also be provided with various external interfaces to obtain information from partners or public channels.

[0028] Among them, the embodiments of the present application mainly focus on the travel planning of ground transportation, and the application scenarios include the daily commuting, business travel and leisure tourism of residents in areas where subways are not laid or difficult to lay. It can be understood that the embodiments of the present application can also include subways in the travel planning range, for example, the computing device can first determine the target subway path based on the user's travel request, and then plan the route from the starting point of travel to the starting point of the target subway path and the route from the ending point of the target subway path to the ending point of travel respectively using the subsequent steps in the embodiments; then integrate into a complete route, and then compare with the planned route of other ground transportation, select the optimal one or more routes and recommend to the user.

[0029] Among them, the target city area is the city area covered by the data that the computing device can obtain; by obtaining more complete data for travel route planning, better services can be provided for users.

[0030] Among them, the computing device can obtain geographic information and traffic information required for travel planning route through the travel integrated service platform from the database managed by itself or the public database with access permission.

[0031] Among them, the geographic information includes city boundaries, bus stops and road networks, and the traffic information includes bus routes and timetables, real-time GPS positions of buses, pick-up and drop-off hotspots of ride-hailing, real-time orders and trajectories of ride-hailing, travel times, average speeds and congestion indexes of roads in the road network. These data can be obtained from the interfaces of the respective subsystems of the travel integrated service platform, and / or open source maps, commercial maps, government open platforms, bus companies, and ride-hailing platforms.

[0032] Specifically, due to the involvement of business secrets and user privacy, the computing device can obtain aggregated and desensitized regional-level data from the online car-hailing subsystem or the cooperative online car-hailing platform.

[0033] Then, for the data obtained from different sources, the computing device can perform preprocessing work such as coordinate system unification, time alignment, data cleaning and standardization, and then perform step 102 to construct a unified spatio-temporal traffic graph.

[0034] Step 102, the computing device constructs a spatio-temporal traffic graph based on the geographic information and the traffic information.

[0035] Among them, the spatio-temporal traffic graph is a graph data structure that can uniformly represent the dynamic state of public transportation, online car-hailing and road network, capture spatial connection relationship, and integrate dynamic changes in time dimension, so as to realize the fusion of multi-modal traffic data.

[0036] Firstly, the computing device can divide the target city area into discrete grid cells, specifically, based on the city boundary, the target city area can be divided into a two-dimensional grid network with a fixed step length, where the size of each grid cell is a fixed value, and the index of the grid is uniquely determined by the row number and column number; Then map the longitude and latitude of the corresponding area to the grid system through the GIS system.

[0037] Then, the node set of the spatio-temporal traffic graph is constructed based on the grid cell.

[0038] Among them, the nodes in the spatio-temporal traffic graph include the first nodes corresponding to the bus stops, the second nodes corresponding to the online car-hailing pickup and drop-off hotspots, and the third nodes corresponding to the traffic grid cells.

[0039] Specifically, the online car-hailing pickup and drop-off hotspot represents the position where online car-hailing frequently picks up and drops off passengers, which can be obtained by clustering historical online car-hailing order data by the computing device, and each hotspot is assigned a unique identifier and mapped to the corresponding grid cell.

[0040] It can be understood that the granularity of the third node is different from that of the first node and the second node, and there can be multiple first nodes and second nodes in one third node. By associating the first nodes representing the public transportation mode and the second nodes representing the online car-hailing mode with the third nodes of a larger granularity area, they are uniformly included in the same graph structure, thereby supporting collaborative analysis of multi-modal data, realizing cross-modal integration, and avoiding data island situation.

[0041] The nodes can be attached with factors having influence on the traffic state, or directly represent the traffic state itself. Preferably, the attribute of the first node includes the number of waiting passengers at the corresponding bus stop, the attribute of the second node includes the number of available vehicles at the corresponding pickup and drop-off hotspot of the online car-hailing, and the attribute of the third node includes the overall traffic state of the corresponding local area.

[0042] Then, the computing device can define edges between the nodes in the graph to represent the connection relationship.

[0043] The physical edges between two nodes correspond to the roads between the real positions of the two nodes, and the attribute of each physical edge includes the travel time of the corresponding road. It can be understood that, in order to make the text indirect, the roads between the nodes appearing in the present application document refer to the roads between the real positions of the nodes.

[0044] The edges between the fine-grained nodes and the coarse-grained nodes can be the roads between the fine-grained nodes and the centers of the traffic grid cells, or the shortest roads between the fine-grained nodes and the edges of the traffic grid cells.

[0045] Preferably, the spatio-temporal traffic graph further includes logical edges, the first node and the third node where the first node is located are connected through the logical edges, and the second node and the third node where the second node is located and the fourth node adjacent to the third node are connected through the logical edges, respectively.

[0046] The physical edges are used to represent the connection of the actual geographical space; the logical edges are used to establish the traffic state association between the nodes of different granularities, and are specifically used to establish the influence relationship between the traffic states, so that the subsequent spatio-temporal graph neural network can conduct traffic influence based on the influence relationship.

[0047] Specifically, the edge weight of the physical edge can be the reciprocal of the corresponding road length; the logical edge can not be set with weight, or be set with equal initial weight.

[0048] Then, the computing device can attach dynamic attributes to each physical edge, which are updated based on real-time traffic data stream to capture the spatio-temporal changes of the traffic state.

[0049] The dynamic attributes include bus travel time, online car-hailing travel time, and road average speed. Specifically, the bus travel time is calculated based on bus GPS data in real time, and represents the expected travel time of the bus on the corresponding physical edge; the online car-hailing travel time is calculated based on online car-hailing trajectory data in real time, and represents the average travel time of the online car-hailing on the corresponding physical edge; and the road average speed can be calculated based on traffic sensors in real time, and represents the average vehicle speed on the corresponding physical edge.

[0050] Preferably, the computing device can obtain the latest data through a preset length of time window, and update the calculation of the dynamic attribute based on the latest data.

[0051] By setting logical edges between nodes of different granularities, the influence of a single bus station or a ride-hailing hotspot on a local area can be captured, and the traffic conditions of the corresponding local area can be aggregated to reflect the overall congestion degree. Conversely, the overall congestion degree of the local area can also affect the nodes and physical edges within the grid area. In this way, the subsequent spatio-temporal graph neural network can learn the complex correlation between the point data of the bus station or the ride-hailing hotspot and the overall surface data of the grid cell, forming a complete system that can perceive paths and the global.

[0052] Step 103, the computing device inputs the spatio-temporal traffic graph into the spatio-temporal graph neural network to obtain a predicted probability distribution of the travel time of each road in the road network within a future preset time period output by the spatio-temporal graph neural network.

[0053] Among them, after obtaining the spatio-temporal traffic graph and the dynamic and static attributes of the nodes and edges in the graph, the computing device can respectively extract and splice the dynamic and static attributes of the nodes and the dynamic and static attributes of the edges to obtain node features and edge features.

[0054] Exemplarily, The feature vector of the first node is [node type code, station capacity, number of lines, number of real-time vehicles, and estimated arrival time of the next bus]; The feature vector of the second node is [node type code, hotspot level, current number of available vehicles, average waiting time, and order density]; The feature vector of the third node is [node type code, main road type, average speed in the grid, total number of vehicles, and key intersection congestion index]; The feature vector of the physical edge is [edge type code, road level, length, basic speed limit, real-time bus travel time, ride-hailing travel time, and road average speed]. The logical edge does not have additional complex dynamic attributes.

[0055] It can be understood that all numerical features are normalized to eliminate the influence of dimensions.

[0056] Then, the computing device can input the spatio-temporal traffic graph and the above feature vectors into the spatio-temporal graph neural network. Among them, the computing device can generate a time series of graphs The graph snapshot at each time step contains three core components, namely the adjacency matrix A, the node feature matrix and the edge feature matrix .

[0057] where for a multi-relational graph containing physical edges and logical edges, the adjacency matrix and represent the physical connections and logical connections between nodes. Each row in the node feature matrix corresponds to a node's feature vector, and each row in the edge feature matrix corresponds to a physical edge's feature vector.

[0058] Preferably, the spatial convolution layer of the spatio-temporal graph neural network comprises a first branch, a second branch, a fusion unit and a convolution unit, the first branch is used to process the connection relationship of the physical edge, the second branch is used to process the connection relationship of the logical edge, the fusion unit is used to fuse the processing results of the two branches, and the convolution unit is used to perform forward propagation based on the fusion result.

[0059] where for each relation r∈{ , }), the first branch and the second branch use independent, trainable weight matrices to linearly transform the node features: , is the node feature of relation r.

[0060] Then the adjacency matrix corresponding to the relation is used to aggregate neighbor information: , is the node feature aggregation result of relation r; for a physical edge, it aggregates the information of road-connected neighbors; for a logical edge, it aggregates the information of state-associated neighbors.

[0061] All relation-specific aggregation results are then combined by the fusion unit to obtain the node feature representation of the current layer. The fusion method can be feature addition, weighted average or concatenation followed by neural network fusion.

[0062] Taking addition fusion as an example, the fusion calculation formula can be: where b is a bias term, () is an activation function, and H is the node feature representation of the current layer.

[0063] The expression of the forward propagation of the convolution unit can be: .

[0064] where the relation set R={ , }, = +I is the adjacency matrix of relation r plus the self-connection matrix I; is the degree matrix of , used for normalization; is a trainable weight matrix of the first layer for the relation r, is the node feature of the first layer.

[0065] By treating physical edges and logical edges differently in the convolution process and using multi-relation graph convolution technology, the neural network can learn independent propagation patterns and parameters for each type of connection, thereby capturing both the physical propagation of traffic flow on the road network and the logical influence of the state between facilities and regions, forming a more accurate and comprehensive dynamic representation of complex urban traffic systems.

[0066] In the spatial convolution process, the computing device can calculate the attention distribution with edge features in the edge feature matrix, so that the neural network can more intelligently select neighbors as learning objects.

[0067] wherein the attention weight is calculated as follows: ; wherein, is the attention weight, is a normalization function, is a multi-layer perceptron, is the feature vector of node i, is the feature vector of the neighbor node j of node i, is the physical edge feature between nodes i and j, and the convolution calculation of the logical edge does not use the attention weight here.

[0068] Then, the computing device can learn information from the neighbor nodes based on the attention weight and generate new features in combination with its own features.

[0069] Then, the computing device can input the feature sequence output by the spatial convolution into the temporal module. The temporal module applies a gated recurrent unit to each node's feature sequence to capture the periodic and trend changes of traffic flow, learning the temporal evolution patterns in the context of spatial correlation.

[0070] wherein the update gate and reset gate mechanisms of the gated recurrent unit can adaptively select to remember or forget historical information.

[0071] The spatio-temporal graph neural network includes one or more spatio-temporal convolution blocks, and the output layer thereof is used to output the prediction probability distribution of the travel time of each edge e in a future preset period.

[0072] wherein the future preset period can be 5 minutes, 10 minutes or 30 minutes from the current time.

[0073] The output layer can first output the mean and standard deviation of the predicted travel time, and then generate a corresponding prediction probability distribution assuming that the predicted travel time follows a Gaussian distribution.

[0074] Preferably, after obtaining the prediction probability distribution, the computing device can periodically obtain incremental data of travel times of roads in the road network; input the incremental data into the spatio-temporal graph neural network to update the prediction probability distribution.

[0075] Step 104, the computing device receives a travel request of a user.

[0076] After the spatio-temporal graph neural network is deployed, the computing device can obtain the current and predicted traffic conditions of roads in the target city area, and then plan a travel route for the user based thereon.

[0077] Specifically, the user can send a travel request to the travel comprehensive service platform through a client, the travel request including the user's current starting point, the user's destination, the departure time, and the user's unique identifier. The computing device can obtain the user's travel preferences based on the unique identifier.

[0078] Step 105, the computing device inputs the travel request and the prediction probability distribution into a reinforcement learning model to obtain a planned route output by the reinforcement learning model.

[0079] The reinforcement learning model is based on a route planning problem. The computing device can model the route planning problem as a reinforcement learning problem, thereby generating a travel route that meets individual needs and takes into account system efficiency, i.e., a planned route.

[0080] Preferably, the reinforcement learning model is a multi-agent model, and the reinforcement learning model includes a central policy network, and agents of the reinforcement learning model share the central policy network. The computing device can establish an agent based on the travel request using the reinforcement learning model; the agent takes as a state the starting point and the destination based on the travel request, the travel preference weight of the user, and the vector of the prediction probability distribution, and inputs the state into the central policy network to obtain a planned route output by the central policy network.

[0081] The computing device can model each travel request entering the system as an independent agent, and use the spatio-temporal traffic graph and the spatio-temporal graph neural network as an environment to form a multi-agent learning framework. The environment will dynamically change according to the joint action of all agents, thereby giving corresponding rewards and updating states.

[0082] By modeling each travel request as an independent agent and sharing a central policy network, the model can explicitly consider the interactions between agents, meaning that agents take into account the influence of other agents when planning routes.

[0083] Each agent includes a state space, an action space, and a reward function.

[0084] Specifically, the states in the state space represent all the personal and traffic state information known to the agent at the current moment. The expression for the state vector can be: ,in, This is a user-specific parameter vector, which specifically includes the route's starting and ending locations and the user's travel preference weights. G is the travel execution state vector, which specifically includes the time consumed, the cost incurred, and the current traffic mode, which is the traffic mode or current state that the user is currently using; G is the global traffic state embedding, which is the predicted probability distribution of the predicted travel time for each road output by the spatiotemporal graph neural network.

[0085] Specifically, actions in the action space include decisions that can be made in the current state, including macro-level transportation modes or combinations of modes, and parameterized actions corresponding to the transportation modes, such as pick-up points in the ride-hailing mode, and bus routes and stops in the bus mode. For example, the combination of modes may include a hybrid mode of bus and ride-hailing.

[0086] The reward function is used to calculate the reward given to the agent based on the state change after the agent performs an action.

[0087] Preferably, the reward function of the intelligent agent includes a personal reward and a system penalty. The personal reward is obtained by weighting and summing the travel preference weights, the cost, time, and quantification inappropriateness of the planned route. The system penalty is obtained by weighting and summing the changes in local congestion and the changes in vehicle distribution imbalance caused by executing the planned route.

[0088] The expression for the reward function includes: ,in, As a reward value, For individual rewards, This is a system penalty item. These are the weighting coefficients.

[0089] The calculation formula for individual rewards includes: ; in, Costs incurred after the action is performed. The time spent performing the action an indicator of discomfort caused by quantized actions, 、 、 is the travel preference weight from user i. can be calculated based on the number of transfers, walking distance, and car congestion of the action. Car congestion can be inferred based on the number of waiting passengers at bus stops and predicted travel time.

[0090] wherein the system penalty term is used to penalize the negative impact of the agent's behavior on the traffic system, and its calculation formula includes: ; wherein, is the local congestion change caused by the agent's action on the relevant road grid cells, which can be calculated by the change in predicted travel time; is the change in the imbalance of the spatial distribution of online car-hailing services caused by the agent's action; and are the set weight coefficients.

[0091] wherein the shared policy network allows knowledge to be quickly transferred and generalized among all agents. The learning experience of one user can immediately benefit all other users, greatly improving learning efficiency and policy consistency.

[0092] Preferably, the training process of the central policy network includes: through a curriculum learning strategy, the central policy network is trained in a simulated traffic environment in stages, and the later the training stage in the training sequence, the higher the difficulty of the training stage; The difficulty of the training stage increases with the increase of the number of agents, the density of traffic flow and the probability of sudden events.

[0093] wherein the computing device can prepare a simulated environment of a target urban area, and use a simulator to train the central policy network. The core of the simulator is a discrete event simulator, which advances in time steps, manages all agents, vehicles and traffic states.

[0094] Specifically, the simulator is designed to be highly configurable and dynamically responsive. When an agent performs an action, the simulator calls a spatio-temporal graph neural network to predict the impact of these actions on future traffic states, thereby dynamically updating the environment state. This enables the simulator to truly reflect the feedback loop between agent decision-making and environmental changes.

[0095] wherein the computing device can prepare a series of training scene sequences with increasing difficulty as a curriculum, and each scene is defined by a set of parameters, including the number of concurrent agents, traffic flow density and mode, and the probability of injecting sudden events. The sudden event can include traffic accidents, road construction, and severe weather.

[0096] By training the central policy network of the multi-agent reinforcement learning model, the central policy network can learn how to consider the route selection influence and global influence between agents and output a globally optimal planning route under adverse traffic conditions of a large number of concurrent agents.

[0097] The massive state-action-reward data obtained through distributed exploration and the gradual progression of curriculum learning ensure that the decision of the central policy network is robust, efficient, and can be generalized to most travel scenarios. The value of the decision of a single agent lies not in the route itself, but in the central policy network's deeper understanding of the approach to finding the corresponding strategy under a specific global state.

[0098] After the central policy network is trained and converges, the central policy network can perform forward reasoning based on the input state and finally output a planning route. The central policy network outputs a complete probability distribution on the action space, and then selects the K different action sequences with the highest probability values from the distribution.

[0099] Specifically, the central policy network can start from the root state, simulate several steps into the future, and generate K complete trajectories from the starting point to the ending point through Monte Carlo Tree Search (MCTS) or Beam Search. These K routes cover pure public transportation, pure online car hailing, mixed mode, and other possibilities.

[0100] Preferably, the predicted probability distribution includes a predicted standard deviation, and the planning route includes K candidate routes. After the planning route output by the reinforcement learning model, the computing device can calculate the total expected utility and total uncertainty of the candidate routes. The total expected utility is calculated based on the personal reward items at each step during the execution of the candidate route, and the total uncertainty is calculated based on the predicted standard deviation of all physical edges corresponding to the candidate route. The reliability score of the candidate route is calculated based on the total expected utility and the total uncertainty. The K candidate routes and the corresponding reliability scores are returned to the client of the user.

[0101] The total expected utility is calculated as follows: The calculation formula of the total expected utility includes: ; wherein, is a discount factor, represents the candidate route, and is used to balance the immediate reward and future reward; t is the step number of the candidate route.

[0102] The total uncertainty is calculated as follows: The calculation formula of the total uncertainty includes: ; wherein, a proportion of a total predicted travel time of the route to a predicted travel time of the edge e, a predicted standard deviation of the edge e.

[0103] wherein the total expected utility and the reliability score are in a positive proportional relationship, and the total uncertainty and the reliability score are in an inverse proportional relationship, and the computing device can calculate the relative reliability score of each candidate route based on this principle.

[0104] In the embodiments of the present application, the multi-dimensional information of the bus stops, the online car-hailing hotspots and the traffic grid cells is fused through the space-time traffic graph, realizing fine-grained perception of the traffic state. The combination of the node attributes and the travel time enables the graph to comprehensively capture the complex correlation between different traffic factors and the road congestion degree. Then, the prediction probability distribution obtained through the space-time graph neural network effectively quantifies the prediction uncertainty, reduces the prediction error caused by the sudden change of the traffic flow, and improves the quality of the finally generated path.

[0105] The method provided in the embodiments of the present application is described above, and the system provided in the embodiments of the present application will be described below.

[0106] Please refer to Figure 2 , Figure 2 The structure of a travel route generation system provided in the embodiments of the present application is shown in FIG. 2, which includes: Figure 2 An acquisition module 201 is configured to acquire geographical information and traffic information of a target city area, wherein the geographical information includes bus stops and a road network, and the traffic information includes online car-hailing pickup and drop-off hotspots and travel times of roads in the road network. A construction module 202 is configured to construct a space-time traffic graph based on the geographical information and the traffic information. The nodes in the space-time traffic graph include first nodes corresponding to the bus stops, second nodes corresponding to the online car-hailing pickup and drop-off hotspots, and third nodes corresponding to traffic grid cells, wherein the traffic grid cells are obtained by dividing the target city area. The physical edges between two nodes correspond to roads between the real positions of the two nodes, and the attributes of each physical edge include the travel time of the corresponding road. The attributes of the first nodes include the number of waiting passengers of the corresponding bus stops, the attributes of the second nodes include the number of available vehicles of the corresponding online car-hailing pickup and drop-off hotspots, and the attributes of the third nodes include the overall traffic state of the corresponding local area. A prediction module 203 is configured to input the space-time traffic graph into a space-time graph neural network to obtain a travel time prediction probability distribution of each road in the road network in a future preset time period output by the space-time graph neural network. A receiving module 204 is configured to receive a travel request of a user. ​The planning module 205 is used to input the travel request and the predicted probability distribution into the reinforcement learning model to obtain the planned route output by the reinforcement learning model; wherein, the reinforcement learning model is obtained by modeling based on the route planning problem.

[0107] Preferably, the predicted probability distribution includes the prediction standard deviation, and the planned route includes K candidate routes; the system 20 also includes an output module 206, used to calculate the total expected utility and total uncertainty of the candidate routes; wherein, the total expected utility is the negative value of the sum of the individual rewards at each step in the execution of the candidate routes, and the total uncertainty is calculated based on the prediction standard deviation of all physical edges corresponding to the candidate routes; the reliability score of the candidate routes is calculated based on the total expected utility and the total uncertainty; and the K candidate routes and their corresponding reliability scores are returned to the user's client.

[0108] Preferably, the system 20 further includes an update module 207, which periodically acquires incremental data of the travel time of each road in the road network; inputs the incremental data into the spatiotemporal graph neural network to update the predicted probability distribution.

[0109] The travel route generation system provided in this application embodiment can be understood by referring to the relevant content in the foregoing method embodiment section, and will not be repeated here.

[0110] like Figure 3 As shown, Figure 3 This is a schematic diagram of a possible logical structure of a computing device provided in an embodiment of this application. The computing device 30 includes a processor 301, a communication interface 302, a memory 303, and a bus 304. The processor 301, the communication interface 302, and the memory 303 are interconnected via the bus 304. In an embodiment of this application, the processor 301 is used to control and manage the operation of the computing device 30. For example, the processor 301 is used to execute... Figure 1 The steps in the embodiments and / or other processes used in the techniques described herein. Communication interface 302 is used to support communication by computing device 30. Memory 303 is used to store program code and data of computing device 30.

[0111] The processor 301 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, transistor logic, hardware components, or any combination thereof. It can implement or execute various example logical blocks, modules, and circuits described in connection with the disclosure. The processor can also be a combination of computing functionality, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, or the like. The bus 304 can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, or the like. For ease of representation, Figure 3 Only one bus is shown in the figure, but there can be more than one bus, or types of buses.

[0112] In another embodiment of the present application, a computer readable storage medium is also provided, which includes instructions, when the instructions are run on a computer, cause the computer to execute the method described in the above embodiments. Figure 1 In another embodiment of the present application, a computer readable storage medium is also provided, which includes instructions, when the instructions are run on a computer, cause the computer to execute the method described in the above embodiments.

[0113] Those skilled in the art can appreciate that the units of each example described in connection with the embodiments disclosed herein can be realized in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components of each example have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0115] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0116] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0117] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a mobile hard disk, a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the description of the present application.

Claims

1. A method for generating travel routes, characterized in that, The method includes: Obtain geographic and traffic information for the target city area. The geographic information includes bus stops and road networks, and the traffic information includes ride-hailing pick-up and drop-off hotspots and travel times on each road in the road network. A spatiotemporal traffic map is constructed based on the geographic information and the traffic information. The nodes in the spatiotemporal traffic map include a first node corresponding to the bus stop, a second node corresponding to the ride-hailing pick-up / drop-off hotspot, and a third node corresponding to a traffic grid unit, which is obtained by dividing the target city area. The physical edges between two nodes correspond to roads between the actual locations of the two nodes, and the attributes of each physical edge include the travel time of the corresponding road. The attributes of the first node include the number of people waiting at the corresponding bus stop, the attributes of the second node include the number of available vehicles at the corresponding ride-hailing pick-up / drop-off hotspot, and the attributes of the third node include the overall traffic status of the corresponding local area. The spatiotemporal traffic map is input into a spatiotemporal graph neural network to obtain the predicted probability distribution of travel time for each road in the road network within a future preset time period, as output by the spatiotemporal graph neural network. Receive user travel requests; The travel request and the predicted probability distribution are input into the reinforcement learning model to obtain the planned route output by the reinforcement learning model; wherein, the reinforcement learning model is obtained by modeling based on the route planning problem.

2. The method according to claim 1, characterized in that, The spatiotemporal traffic map also includes logical edges. The first node is connected to the third node where the first node is located through the logical edges. The second node is connected to the third node where the second node is located and to the fourth node adjacent to the third node through the logical edges. The logical edges are used to establish traffic state associations between the nodes at different granularities. The spatial convolutional layer of the spatiotemporal graph neural network includes a first branch, a second branch, a fusion unit, and a convolutional unit. The first branch is used to process the connection relationship of the physical edges, the second branch is used to process the connection relationship of the logical edges, the fusion unit is used to fuse the processing results of the two branches, and the convolutional unit is used to perform forward propagation based on the fusion result.

3. The method according to claim 1 or 2, characterized in that, The reinforcement learning model is a multi-agent model, which includes a central policy network shared by the agents in the model. The step of inputting the travel request and the predicted probability distribution into the reinforcement learning model to obtain the planned route output by the model includes: The reinforcement learning model establishes an intelligent agent based on the travel request; The agent uses a vector obtained based on the origin and destination of the travel request, the user's travel preference weights, and the predicted probability distribution as its state. It inputs the state into the central policy network to obtain the planned route output by the central policy network.

4. The method according to claim 3, characterized in that, The reward function of the agent includes a personal reward and a system penalty. The personal reward is obtained by weighting and summing the travel preference weights, the cost, time and quantification inappropriateness of the planned route. The system penalty is obtained by weighting and summing the changes in local congestion and vehicle distribution imbalance caused by executing the planned route.

5. The method according to claim 3, characterized in that, The training process for the central policy network includes: The central policy network is trained in stages in a simulated traffic environment through a course learning strategy. The difficulty of the training stages increases as the number of agents, traffic flow density, and probability of sudden event injection increase.

6. The method according to claim 4, characterized in that, The predicted probability distribution includes the prediction standard deviation, and the planned route includes K candidate routes; after inputting the travel request and the predicted probability distribution into the reinforcement learning model to obtain the planned route output by the reinforcement learning model, the method further includes: Calculate the total expected utility and total uncertainty of the candidate route; wherein the total expected utility is calculated based on the individual reward item at each step in executing the candidate route, and the total uncertainty is calculated based on the prediction standard deviation of all physical edges corresponding to the candidate route; The reliability score of the candidate route is calculated based on the total expected utility and the total uncertainty; The K candidate routes and their corresponding reliability scores are returned to the user's client.

7. The method according to claim 1 or 2, characterized in that, After inputting the spatiotemporal traffic map into a spatiotemporal graph neural network to obtain the predicted probability distribution of the travel time of the physical edge within a future preset time period output by the spatiotemporal graph neural network, the method further includes: Periodically acquire incremental data of travel time for each road in the road network; The incremental data is input into the spatiotemporal graph neural network to update the predicted probability distribution.

8. A travel route generation system, characterized in that, The system, applied to the method of any one of claims 1-7, comprises: The acquisition module is used to acquire geographic information and traffic information of the target city area. The geographic information includes bus stops and road networks, and the traffic information includes ride-hailing pick-up and drop-off hotspots and travel times on each road in the road network. A construction module is used to construct a spatiotemporal traffic map based on the geographic information and the traffic information. The nodes in the spatiotemporal traffic map include a first node corresponding to the bus stop, a second node corresponding to the ride-hailing pick-up / drop-off hotspot, and a third node corresponding to a traffic grid unit, which is obtained by dividing the target city area. The physical edges between two nodes correspond to roads between the actual locations of the two nodes, and the attributes of each physical edge include the travel time of the corresponding road. The attributes of the first node include the number of people waiting at the corresponding bus stop, the attributes of the second node include the number of available vehicles at the corresponding ride-hailing pick-up / drop-off hotspot, and the attributes of the third node include the overall traffic status of the corresponding local area. The prediction module is used to input the spatiotemporal traffic map into the spatiotemporal graph neural network to obtain the predicted probability distribution of the travel time of each road in the road network in the future preset time period, output by the spatiotemporal graph neural network. The receiving module is used to receive users' travel requests; The planning module is used to input the travel request and the predicted probability distribution into the reinforcement learning model to obtain the planned route output by the reinforcement learning model; wherein, the reinforcement learning model is obtained by modeling based on the route planning problem.

9. A computing device, characterized in that, include: Memory, used to store programs; A processor for loading the program to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of any one of claims 1-7.

Images