A method for constructing a route planning model for ride-sharing requests and service vehicles.

Abstract

This invention discloses a method for constructing a route planning model for ride-hailing requests and service vehicles, addressing the problems of high computational complexity and neglect of future requests in current ride-hailing request allocation and route planning methods. The solution of this invention is a reinforcement learning-based method for ride-hailing request allocation and route planning. This method considers the random generation of future requests when allocating requests to vehicles, thereby guiding vehicle movement, and simultaneously uses an efficient method to update vehicle routes in the ride-hailing service.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent transportation and spatiotemporal trajectory big data mining, specifically involving a method for planning the route of ride-sharing requests and service vehicles based on reinforcement learning. Background Technology

[0002] With the popularization of internet technology and the continuous improvement of urban residents' living standards, people's daily commuting needs have increased significantly, especially on holidays and weekends. However, the limited capacity of the road network often leads to traffic congestion, and the problems of parking difficulties and taxi shortages are becoming increasingly apparent. Currently, ride-hailing services are becoming increasingly popular with customers, as they can group passengers with similar travel routes together, greatly increasing vehicle utilization efficiency.

[0003] However, most existing algorithm research on ride-hailing services uses heuristic methods, whose computational complexity increases exponentially with the number of vehicles and requests, making them unsuitable for real-time changing application scenarios. Summary of the Invention

[0004] To address the shortcomings or deficiencies of existing technologies, this invention provides a method for constructing a route planning model for ride-sharing requests and service vehicles.

[0005] The method provided by this invention includes:

[0006] Step 1: Periodically collect vehicle information and carpooling passenger request information within the service area to obtain m vehicle information and n request information within the current period, where n is a natural number greater than or equal to 1 and m is a natural number greater than or equal to 1; the request information includes the request submission time, departure point coordinates, destination point coordinates, number of passengers, maximum acceptable detour time, and maximum acceptable waiting time for boarding; the vehicle information includes the vehicle's current location coordinates and vehicle capacity;

[0007] Construct departure and destination information for each request; the departure information includes the request submission time, departure coordinates, number of passengers, and maximum acceptable waiting time for boarding; the destination information includes the destination coordinates, number of disembarking passengers, and maximum acceptable detour time, wherein the number of disembarking passengers is a negative value of the number of passengers.

[0008] Step 2: Train the model using the data collected and constructed in Step 1. The model includes an encoder, a vehicle decoder, and a route decoder. The training includes:

[0009] Step 1: Input the current position coordinates of m vehicles, n departure point information and n destination point information into the encoder for processing; output a 128-dimensional embedding vector of the current position coordinates of each vehicle, a 128-dimensional embedding vector of each departure point information and a 128-dimensional embedding vector of each destination point information and a global embedding vector.

[0010] Step 2: Output the current planned routes of all vehicles, the remaining time of the assigned departure point and the remaining time of the assigned destination point of all vehicles to the vehicle decoder for processing, and output the vehicles to be assigned.

[0011] The current planned route for each vehicle is initially a 128-dimensional embedding vector of the vehicle's current location coordinates; the remaining time at the assigned departure point and the remaining time at the assigned destination point for each vehicle are initially set to 0.

[0012] The remaining time for each vehicle at one of its assigned departure points is the maximum acceptable waiting time for the corresponding request minus the first planned travel time; the first planned travel time is the shortest travel distance from the current location to the departure point via all points before the departure point in the current planned route divided by the vehicle's average travel speed.

[0013] The remaining time allocated to one of the destinations for each vehicle is the maximum acceptable detour time for the corresponding request minus the second planned travel time; the second planned travel time is the shortest travel distance from the current location to the destination via all points before the destination in the current planned route divided by the vehicle's average travel speed.

[0014] Step 3: Input the context embedding and all candidate routes of the vehicle to be assigned obtained in Step 2 into the route decoder for processing. The output is the updated planned route of the vehicle to be assigned. The updated planned route includes at least one unassigned request departure point or at least one requested destination point assigned to the vehicle to be assigned. The updated planned route is one of the candidate routes of the vehicle to be assigned.

[0015] The context embedding includes the current planned route and the global embedding vector for the current period of the vehicle to be assigned;

[0016] The method for obtaining candidate routes for vehicles to be assigned includes: inserting a 128-dimensional embedding vector of an unassigned request departure point or a 128-dimensional embedding vector of an assigned request destination point between any two adjacent points in the vehicle's current planned route to obtain an initial candidate route; filtering the initial candidate routes to ensure that the number of passengers carried by the vehicle to be assigned does not exceed the vehicle's capacity and that there is no requested destination point inserted before the assigned request departure point, to obtain at least one candidate route for the vehicle to be assigned; the initial candidate route for the vehicle to be assigned is a 128-dimensional embedding vector of the vehicle's current location coordinates;

[0017] Step 4: Repeat Step 2 and Step 3 until there is no output in Step 2; each vehicle with a request obtains a planned route; each assigned request obtains a planned route.

[0018] Step 5: Feedback the reward value to the model and update the parameters of the encoder, vehicle decoder and route decoder. The reward value is the sum of the shortest travel times of the planned routes of all vehicles with requests minus the sum of the travel times of the planned routes of all vehicles with requests. The shortest travel time of the planned route of any vehicle with a request is the travel time from the starting point to the destination of the planned route.

[0019] Step 3: Using the data collected and constructed in subsequent cycles, continue to train and update the model, following steps 1 and 2.

[0020] An optional approach is to set the cycle duration to 1-10 minutes.

[0021] An alternative approach is that the route decoder includes a 512-dimensional forward feedback layer, a multi-head attention mechanism layer, a single-layer attention mechanism layer, and a softmax layer connected in sequence.

[0022] The 512-dimensional forward feedback layer processes the context embedding and all candidate routes of the vehicle to be assigned to obtain the processed context embedding vector and the vector of each candidate route.

[0023] The multi-head attention mechanism layer generates an enhanced context vector based on the context embedding vector and all candidate route vectors, including: each single-head attention mechanism layer calculates an attention value based on the context embedding vector and all candidate route vectors, and then uses the attention value as a weight to generate single-head data of the context embedding vector; the multi-head attention mechanism layer processes the data to obtain single-head data of multiple context embedding vectors; then the single-head data of multiple context embedding vectors are averaged to obtain the enhanced context embedding vector.

[0024] The single-head attention mechanism layer calculates and generates a new vector for each candidate route based on the enhanced context embedding vector and multiple candidate route vectors;

[0025] The softmax layer selects the probability distribution of each candidate route based on the new vector of each candidate route, and the probability distributions of all candidate routes constitute a probability distribution vector.

[0026] The updated planned routes for vehicles to be assigned are selected from the probability distribution vector using a sampling selection method.

[0027] An alternative approach is to use a policy gradient method from reinforcement learning to update the parameters of the encoder, vehicle decoder, and route decoder in Step 5.

[0028] The present invention also provides a testing method for the above-mentioned model, characterized in that the method includes testing the model using the methods described in steps 1, 1-4, and 3 of claim 1. In a further embodiment, the route decoder described in steps 1-3 of claim 3 is used, and a greedy selection method is employed to select the updated planned route for the vehicle to be assigned from the probability distribution vector.

[0029] The model of this invention can take into account the random generation of future requests when allocating vehicle requests, thereby guiding vehicle travel, while using an efficient method to update vehicle travel routes in carpooling services. Attached Figure Description

[0030] Figure 1 This is a comparison of the runtime between the present invention and the PDTL method. Detailed Implementation

[0031] Unless otherwise specified, the scientific and technical terms used in this article are intended for understanding by those skilled in the art.

[0032] The encoder described in this invention is the one disclosed in the paper "Attention, learn to solve routing problems!" (published in 2018). This encoder obtains 128-dimensional high-level vectors corresponding to each input information, then captures the latent relationships between all high-level vectors, followed by feature extraction and smoothing of the vectors containing latent relationships, finally outputting m+2n+1 high-dimensional embedding vectors. The specific process is as follows:

[0033] 2.1 Input m vehicle information and n passenger requests into the linear projection layer to obtain m+2n high-level vectors with a dimension of 128;

[0034] 2.2 A multi-head attention mechanism is used to capture the potential relationships between m+n 128-dimensional high-level vectors, resulting in vectors containing potential relationships. These potential relationships actually contain information about the relationship between the requested location and the vehicle location, and between the number of requested passengers and the remaining capacity, which can help with subsequent vehicle request allocation.

[0035] 2.3 The vectors from 2.2 are input into the feedforward layer for feature extraction and smoothing, and the average of m+2n vectors is used as the global embedding vector.

[0036] The vehicle decoder of this invention is used to determine the order of vehicle allocation requests, selecting one vehicle to be allocated each time an allocation is performed. Specifically, the decoder disclosed in the paper "Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem" (published in 2021) is used. To meet the input requirements of the vehicle decoder of this invention, the decoder disclosed in that paper is reasonably adjusted, specifically replacing the max pooling layer with average pooling and deleting the vector concatenation layer before the feedforward layer.

[0037] The route decoder of this invention determines whether the vehicle to be assigned a request and the method of updating the travel route, ultimately generating a new travel path for the selected vehicle. The average travel speed of the vehicle is the average travel speed of vehicles in the historical data of the corresponding road segment. For example, the average travel speed of the vehicle in the first planned travel time is the average travel speed of vehicles in the historical data of the shortest travel distance road segment.

[0038] The updated planned route in the method of the present invention includes assigning a vehicle to a starting point that has not yet been assigned or a destination that has already been assigned. This can be understood as assigning a vehicle to a starting point that has not yet been assigned or a destination that has already been assigned to the vehicle to be assigned, thereby determining whether the vehicle to be assigned picks up a new passenger or drops off a passenger on board.

[0039] In the training process of this invention, the reward value is the sum of the shortest travel times of the planned paths of all vehicles that have been assigned requests, minus the sum of the travel times of the planned paths of all assigned requests. This value is negative. This value is fed back to the model as a reward value and the model parameters are updated. This method can calculate the long-term return of request assignment, thus providing predictability for the future.

[0040] Example:

[0041] This embodiment utilizes the road traffic network of San Francisco (service area) and passenger travel data from May 19, 2008 to June 5, 2008 to construct and train the model of the present invention; the route decoder in the model of this embodiment includes a 512-dimensional forward feedback layer, an eight-head attention mechanism layer, a single-layer attention mechanism layer and a softmax layer connected in sequence.

[0042] Then, a simulation test was conducted using the passenger travel dataset for this region from June 6, 2008 to June 9, 2008.

[0043] This data is used for request allocation and route planning for carpooling services. Based on this data, 500 vehicles are simulated. In the actual plan, the maximum capacity of each vehicle is different. In this embodiment, taxis are used as an example, and the maximum capacity of each vehicle is set to 4. The initial position coordinates of each vehicle are randomly generated within the data area, and all vehicles are dispersed within the area as much as possible. Requests and vehicle information within the area are collected every five minutes. The maximum acceptable detour time and the maximum acceptable waiting time for each request are both 600 seconds (ten minutes).

[0044] Meanwhile, the PDTL method (from the 2021 IEEE Transactions on Intelligent Transportation Systems, "A Dynamic Ridesplitting Method With Potential Pick-Up Probability Based on GPS Trajectories") was used for comparison (the data used was the passenger travel dataset from June 6th to June 9th, 2008, with 500 vehicles, each with a capacity of 4, a data collection cycle of 5 minutes, and a maximum acceptable detour time and maximum acceptable waiting time for boarding for each request of 600 seconds; the grid area size was 400 meters * 400 meters). The planning results of the two methods are shown in Table 1.

[0045] Table 1

[0046]

[0047]

[0048] Furthermore, the present invention is compared with the PDTL method in terms of runtime, and the results are as follows: Figure 1 As shown, the vertical axis represents the running time of the simulation implementation on each day's travel records, indicating that the computational time complexity of this invention is low.

Claims

1. A method for constructing a route planning model for ride-sharing requests and service vehicles, characterized in that, The methods include: Step 1: Periodically collect vehicle information and carpooling passenger request information within the service area to obtain m vehicle information and n request information within the current period, where n is a natural number greater than or equal to 1 and m is a natural number greater than or equal to 1; the request information includes the request submission time, departure point coordinates, destination point coordinates, number of passengers, maximum acceptable detour time, and maximum acceptable waiting time for boarding; the vehicle information includes the vehicle's current location coordinates and vehicle capacity; Construct departure and destination information for each request; the departure information includes the request submission time, departure coordinates, number of passengers, and maximum acceptable waiting time for boarding; the destination information includes the destination coordinates, number of disembarking passengers, and maximum acceptable detour time, wherein the number of disembarking passengers is a negative value of the number of passengers. Step 2: Train the model using the data collected and constructed in Step 1. The model includes an encoder, a vehicle decoder, and a route decoder. The training includes: Step 1: Input the current position coordinates of m vehicles, n departure point information and n destination point information into the encoder for processing; output a 128-dimensional embedding vector of the current position coordinates of each vehicle, a 128-dimensional embedding vector of each departure point information and a 128-dimensional embedding vector of each destination point information and a global embedding vector. Step 2: Output the current planned routes of all vehicles, the remaining time of the assigned departure point and the remaining time of the assigned destination point of all vehicles to the vehicle decoder for processing, and output the vehicles to be assigned. The current planned route for each vehicle is initially a 128-dimensional embedding vector of the vehicle's current location coordinates; the remaining time at the assigned departure point and the remaining time at the assigned destination point for each vehicle are initially set to 0. The remaining time for each vehicle at one of its assigned departure points is the maximum acceptable waiting time for the corresponding request minus the first planned travel time; the first planned travel time is the shortest travel distance from the current location to the departure point via all points before the departure point in the current planned route divided by the vehicle's average travel speed. The remaining time allocated to one of the destinations for each vehicle is the maximum acceptable detour time for the corresponding request minus the second planned travel time; the second planned travel time is the shortest travel distance from the current location to the destination via all points before the destination in the current planned route divided by the vehicle's average travel speed. Step 3: Input the context embedding and all candidate routes of the vehicle to be assigned obtained in Step 2 into the route decoder for processing. The output is the updated planned route of the vehicle to be assigned. The updated planned route includes at least one unassigned request departure point or at least one requested destination point assigned to the vehicle to be assigned. The updated planned route is one of the candidate routes of the vehicle to be assigned. The context embedding includes the current planned route and the global embedding vector for the current period of the vehicle to be assigned; The method for obtaining candidate routes for vehicles to be assigned includes: inserting a 128-dimensional embedding vector of an unassigned request departure point or a 128-dimensional embedding vector of an assigned request destination point between any two adjacent points in the vehicle's current planned route to obtain an initial candidate route; filtering the initial candidate routes to ensure that the number of passengers carried by the vehicle to be assigned does not exceed the vehicle's capacity and that there is no requested destination point inserted before the assigned request departure point, to obtain at least one candidate route for the vehicle to be assigned; the initial candidate route for the vehicle to be assigned is a 128-dimensional embedding vector of the vehicle's current location coordinates; Step 4: Repeat Step 2 and Step 3 until there is no output in Step 2; each vehicle with a request obtains a planned route; each assigned request obtains a planned route. Step 5: Feedback the reward value to the model and update the parameters of the encoder, vehicle decoder and route decoder. The reward value is the sum of the shortest travel times of the planned routes of all vehicles with requests minus the sum of the travel times of the planned routes of all vehicles with requests. The shortest travel time of the planned route of any vehicle with a request is the travel time from the starting point to the destination of the planned route. Step 3: Using the data collected and constructed in subsequent cycles, continue to train and update the model, following the steps 1 and 2. The route decoder includes a 512-dimensional forward feedback layer, a multi-head attention mechanism layer, a single-layer attention mechanism layer, and a softmax layer connected in sequence. The 512-dimensional forward feedback layer processes the context embedding and all candidate routes of the vehicle to be assigned to obtain the processed context embedding vector and the vector of each candidate route. The multi-head attention mechanism layer generates an enhanced context vector based on the context embedding vector and all candidate route vectors, including: each single-head attention mechanism layer calculates an attention value based on the context embedding vector and all candidate route vectors, and then uses the attention value as a weight to generate single-head data of the context embedding vector; the multi-head attention mechanism layer processes the data to obtain single-head data of multiple context embedding vectors; then the single-head data of multiple context embedding vectors are averaged to obtain the enhanced context embedding vector. The single-head attention mechanism layer calculates and generates a new vector for each candidate route based on the enhanced context embedding vector and multiple candidate route vectors; The softmax layer selects the probability distribution of each candidate route based on the new vector of each candidate route, and the probability distributions of all candidate routes constitute a probability distribution vector. The updated planned routes for vehicles to be assigned are selected from the probability distribution vector using a sampling selection method. Step 5 uses the policy gradient method in reinforcement learning to update the parameters of the encoder, vehicle decoder, and route decoder. The route decoder is used, and the updated planned route for the vehicle to be assigned is selected from the probability distribution vector using a greedy selection method.

2. The construction method according to claim 1, characterized in that, The cycle duration is 1-10 minutes.

3. A testing method for the model constructed by the method of claim 1, characterized in that, The method includes testing the model using the methods described in claim 1, Steps 1-4, and Step 3.

4. A method for route planning of ride-sharing requests and service vehicles, characterized in that, The methods include: Data is collected and constructed using the method described in step 1 of claim 1; Using the model constructed by the method described in claim 1, the methods described in Steps 1-4 and Step 3 are used to perform route planning for ride-sharing requests and service vehicles.

Images