A dynamic carpooling scheduling method for unmanned electric taxis

Abstract

The application discloses a dynamic carpooling scheduling method for unmanned electric taxis, which comprises the following steps: obtaining data and performing cleaning; mapping the data in each hexagonal grid; establishing a travel demand feature matrix and a spatial dependence matrix; constructing a spatio-temporal graph convolutional neural network based on an attention mechanism to obtain a travel demand prediction result; setting a constraint condition for single-car matching in a dynamic carpooling scenario; constructing a single-car matching decision model based on distributed deep reinforcement learning based on the travel demand prediction result and the single-car matching constraint condition; designing a Double DQN solving algorithm; performing real-time order matching and idle vehicle scheduling based on the result; notifying the matched vehicles to complete the corresponding orders; and notifying the idle vehicles that have not been matched to perform regional scheduling or charging energy supplementing. In the dynamic carpooling of multiple passengers and a single car, the electric vehicle endurance journey constraint and the vehicle charging energy supplementing demand are considered, so that the quality is improved and the actual operation requirements are met.

Description

TECHNICAL FIELD

[0001] The present application relates to unmanned technology, in particular to a dynamic carpooling dispatching method for unmanned electric taxis. BACKGROUND

[0002] With the rapid development of the sharing economy, autonomous driving technology and the electric vehicle industry, unmanned electric shared travel is expected to become an important part of future urban transportation systems. Compared with other shared travel modes, the dynamic carpooling mode of multi-passenger vehicles has outstanding advantages in reducing passenger travel costs, improving vehicle utilization and relieving traffic pressure, and has development potential. However, the existing car matching decision-making methods for shared electric vehicle fleets still have the shortcomings of short-sighted strategy, high computational complexity and poor algorithm adaptability. Therefore, it is necessary to develop a dynamic carpooling dispatching method that considers the characteristics of unmanned electric shared travel operation scenarios, integrates passenger travel demand prediction and fleet dispatching tasks, and maximizes the long-term benefits of fleet operation.

[0003] After searching, it is found that:

[0004] The Chinese patent application with application number 202410994378.1, "Intercity dynamic carpooling dispatching optimization method and device based on two-stage scheduling", constructs a two-stage scheduling model by obtaining carpooling order information and vehicle information. In the first stage, online vehicle scheduling and offline vehicle scheduling are performed based on the insertion algorithm, and in the second stage, all unlaunched vehicles are matched based on the large neighborhood search algorithm to achieve total running time reduction and total profit maximization. Disadvantages: Two-stage scheduling requires decision-making and adjustment at multiple time points, which may introduce uncertainty and volatility, and the system scalability is insufficient for large-scale intercity carpooling demand scenarios.

[0005] The Chinese patent application with application number 202311306118.2, "Real-time cross-platform carpooling route planning method", proposes an improved dynamic programming insertion method to plan new routes for workers of different crowdsourcing platforms, which can achieve optimal allocation of multiple different platform tasks and optimization of worker user routes. Disadvantages: It does not consider the vehicle range constraint, and does not take into account the vehicle energy supplement demand when setting the operation management scenario, affecting the actual dispatching effect.

[0006] The Chinese patent application with the application number 202310811799.1, titled "A point-to-point carpooling dynamic scheduling method and device for online car-hailing", uses clustering analysis to form point-to-point combined orders for orders with close departure and destination, and solves the optimal matching of car orders by setting a planning period, a multi-objective function, and a set of constraints, to effectively balance the interests of operators and passengers. Disadvantages: does not consider future user ride demand, cannot perform idle vehicle pre-deployment, and does not consider long-term rewards for car order matching, making the car order matching decision short-sighted.

[0007] The Chinese patent application with the application number 202211738830.5, titled "A vehicle scheduling management system based on heuristic algorithm", proposes a vehicle scheduling management system including a data acquisition module, a scheduling optimization calculation module, and a vehicle scheduling module, which can select the tabu search algorithm, the simulated annealing algorithm, or the ant colony search algorithm to obtain the optimal solution of vehicle scheduling based on the comparison result of the total amount of vehicle data and the preset threshold. Disadvantages: very sensitive to initial conditions, difficult to optimize algorithms, and insufficient scalability.

[0008] The Chinese patent application with the application number 202111433060.9, titled "A method and system for real-time order matching and idle vehicle scheduling for online car-hailing", proposes a method and system for real-time order matching and idle vehicle scheduling based on a dynamic programming-based value iteration algorithm for vehicle value function evaluation, achieving effective utilization of idle vehicles and maximization of long-term social welfare. Disadvantages: does not schedule idle vehicles based on user travel demand prediction results, affecting car order matching efficiency.

[0009] The Chinese patent application with the application number 202110477323.X, titled "A method and system for online car-hailing order matching based on vehicle conditions", matches order information and vehicle navigation routes to obtain candidate vehicles, establishes a driving prediction model based on weather, road surface, traffic, and other navigation route environmental information to obtain tire safety parameters, and then compares the tire conditions of candidate vehicles with the tire safety parameters to determine whether the candidate vehicles meet the car order matching requirements. Disadvantages: requires additional road condition information, including road flatness and number of road cracks, which is difficult to accurately and timely obtain in actual large-scale deployment.

[0010] The Chinese patent application with the application number 202110203068.X, titled "Vehicle scheduling method based on dynamic carpooling and related products", proposes a method and product for improving the accuracy of pushing target vehicles to target users by calculating the correlation degree of multiple cars and target user carpooling demand based on user carpooling information and vehicle information, and the carpooling cost of each car selected by the target user. Disadvantages: only considers current order demand, and does not consider long-term rewards, making the vehicle scheduling decision short-sighted.

[0011] In summary, the main deficiencies of the prior art are:

[0012] 1. The existing method does not fully consider the spatio-temporal characteristics of user travel demand, and does not perform vehicle scheduling deployment in advance based on travel demand prediction results, affecting the efficiency of vehicle matching;

[0013] 2. Some vehicle matching decision methods do not consider the long-term benefits after vehicle matching, and the vehicle carpool scheduling decision is short-sighted;

[0014] 3. Some methods do not consider the vehicle range constraint, and do not consider the vehicle energy supplement demand in the actual operation scenario, resulting in poor user actual carpool service experience. SUMMARY

[0015] The purpose of the application is to realize efficient decision-making of unmanned electric shared travel vehicle matching, and the application provides a dynamic carpool scheduling method for unmanned electric taxis.

[0016] Technical scheme: The application provides a dynamic carpool scheduling method for unmanned electric taxis, including the following steps:

[0017] (1) Obtain user historical travel orders, carpool orders and vehicle state position information data;

[0018] (2) Data cleaning is performed on the user historical travel orders;

[0019] (3) The operating area is divided into hexagonal grids, and the cleaned data is mapped in each hexagonal grid, reflecting the spatio-temporal distribution difference of user carpool travel demand;

[0020] (4) Based on the data cleaning and travel demand spatio-temporal distribution results in step (3), a travel demand feature matrix and a spatial dependence matrix are established to quantify the passenger travel time characteristics and spatial characteristics, respectively;

[0021] (5) Construct a spatio-temporal graph convolutional neural network based on attention mechanism to obtain travel demand prediction results;

[0022] (6) Set the constraint conditions of vehicle matching in the dynamic carpool scenario, including capacity constraint, distance constraint, time constraint, range constraint and sequence constraint;

[0023] (7) Based on the travel demand prediction results in step (5) and the vehicle matching constraint conditions in step (6), a vehicle matching decision model based on distributed deep reinforcement learning is constructed;

[0024] (8) Design Double DQN solving algorithm, use Double DQN network to dynamically generate optimization strategy, define the vehicle optimal action value function as the maximum expected reward that can be achieved, use the action that makes the neural network output the maximum value for each vehicle, and learn from scratch. According to the Epsilon-Greedy method, the greedy scheme is used to select the operation, under this strategy, the agent selects the action that leads to the highest Q value with a probability of 1-ε, otherwise selects a random action. For the nth vehicle, after selecting the action, update the Q value according to the reward r t , n, and the learning rate a;

[0025] (9) Based on the model solving results in step (8), real-time order matching and idle vehicle scheduling are carried out, and the vehicles that obtain matching are notified to complete the corresponding orders, and the idle vehicles that do not obtain matching are notified to carry out regional scheduling or charging energy supplement.

[0026] Further, the data of step (1) includes passenger trip origin and destination, number of passengers and departure time, vehicle remaining power and cruising range, vehicle position, and vehicle loadable passenger number.

[0027] Further, the data cleaning of step (2) includes: converting latitude and longitude to WGS84 coordinate system, converting timestamp to date format, deleting data outside the study area, and cleaning duplicate orders.

[0028] Further, the step (3) comprises:

[0029] 3.1 Use H3 spatial index system to divide the operation area into spatial grid, and use H3 hexagonal grid with resolution of 8 as the minimum action unit;

[0030] 3.2 Through H3 spatial index system, map the carpooling demand quantity, available vehicle supply quantity and supply-demand difference value cleaned out in step (2) to each hexagonal grid respectively, to reflect the spatial and temporal distribution difference of user carpooling demand.

[0031] Further, the step (4) comprises:

[0032] 4.1 In the spatial dimension, the spatial dependence characteristics of passenger trips between regions are characterized by Granger causality test, and a spatial dependence matrix A is established, the elements in the matrix only contain 0 and 1, representing whether there is a dependence between two regions: if there is no dependence relationship, it is recorded as 0; if there is a dependence relationship, it is recorded as 1, which is as follows,

[0033]

[0034] In the formula, is A adjmatrix elements; i, j are both divided regions; Lj is the arrival demand time series of region j; Li is the travel demand series of region i;

[0035] 4.2 In the time dimension, passenger travel demand data has natural periodicity, in addition, the travel demand of the adjacent prediction time will also have an impact on the prediction of the target time series, extract the travel demand data of the last time of the target prediction time, the travel demand data of the corresponding time of the previous day, and the travel demand data of the corresponding time of the previous week to construct the travel demand feature matrix X, and then construct the time series segment containing adjacent time, day cycle and week cycle.

[0036] Further, the step (5) comprises:

[0037] 5.1 Constructing a spatio-temporal graph convolutional neural network based on attention mechanism, inputting the time series segment in step 4 together with the spatial dependency matrix into the corresponding dependency analysis component to generate a normalized spatial attention matrix E' and a time attention matrix S';

[0038] 5.2 Time attention mechanism: first, construct a time dimension convolution module, operate along the time axis of the data to extract important features in the time series, and then capture the dynamic pattern and correlation of passenger travel demand with time, through one-dimensional convolution operation in the time dimension, the model updates the passenger travel demand along the time axis in real time according to the convolution kernel size, captures the correlation between times, as follows,

[0039]

[0040] In the formula, ReLU is an activation function, and Φ is a convolution and parameter in the time dimension,

[0041] Then, construct a time dimension attention module, through the self-attention mechanism, the model can automatically identify and extract key dependent features in the travel demand data, and dynamically adjust the correlation strength of the travel demand in the time dimension according to the specific situation of the input data, as follows,

[0042]

[0043] In the formula, V i , b i , Z1, Z2, Z3 are parameters learned by input data; E is a time correlation matrix automatically learned by the time dimension attention module according to the input data of the current time, S' is a time attention matrix obtained by normalizing E matrix using Softmax function; S m，n is the correlation degree between nodes m and n,

[0044] Further, by dynamically adjusting the input data, the features can be further extracted, and the normalized time attention matrix is applied to the current input data to obtain Using the adjusted data, the next stage of features can be further extracted;

[0045] 5.3 Spatial attention mechanism: first, construct a spatial dimension graph convolution module, use the spectral graph theory to interpret the data as a graph structure, use convolution technology to process these graph data, and through the spatial dimension ChebNet graph convolution analysis, extract spatial features from the graph information, thereby revealing the correlation of passenger travel demand in the spatial layer, as follows,

[0046]

[0047] wherein, is the graph signal matrix feature in the passenger travel data in the Mth region at the tth time; is the parameter to be learned; is the normalized Laplacian matrix,

[0048] Then build a spatial dimension attention module, use a self-attention mechanism to capture the dynamic changes of the dependence between regions in the spatial dimension, and automatically adjust the correlation between passenger travel demand data to adapt to these changes, as follows,

[0049]

[0050] wherein, is the input data of the kth spatio-temporal module; C k-1 is the number of channels of the input data of the kth spatio-temporal module; T k-1 is the time dimension length of the input data of the kth spatio-temporal module; V j , b j , Y1, Y2, Y3 are parameter sets learned through the attention mechanism; E is a spatial attention matrix learned based on the current input data; E m，n is the strength of the interaction between nodes m and n; E' is a normalized spatial attention matrix generated by standardizing E using the SoftmaX function, and each element value in the matrix represents the weight coefficient between regions,

[0051] Further, dynamically adjust the spatial attention matrix E' and the matrix of spatial dependence in the Chebyshev polynomial in the spatial graph convolution, and realize the reasonable distribution of attention between regions by affecting the weight;

[0052] 5.4 Train the model by stacking the spatio-temporal modules and applying the residual learning strategy. Adjust the outputs of the proximity dependency, daily periodicity, and weekly periodicity components using fully connected layers to match the dimension of the prediction target, and then fuse the outputs to derive the final travel demand prediction.

[0053] Further, the step (6) comprises:

[0054] 6.1 The mathematical expression of the capacity constraint is as follows,

[0055] N r ≤C max -n c

[0056] In the formula, N r is the number of passengers contained in the new order r, which must not exceed the current remaining available capacity of the vehicle; n c is the number of passengers already on the vehicle; C max is the total number of seats of the vehicle, set to 4 people;

[0057] 6.2 The mathematical expression of the pickup distance constraint is as follows,

[0058] d r，n (r loc , O r )≤ω

[0059] t r，n (n loc , O r )≤t q -t x

[0060] In the formula, O r is the pickup location of the new order r; n loc is the current position of the vehicle n; d r，n (n loc , O r ) is the pickup distance of the vehicle n from the current position n loc to the pickup location O r of the order r; t r，n (n loc , O r ) is the required pickup time according to the current route; t x is the order response time; t q is the maximum time a user can wait, set to 10 minutes; ω is 3 km,

[0061] 6.3 The mathematical expression of the detour distance constraint is as follows,

[0062]

[0063] where t is the time elapsed since the passenger trip l request was made; the new estimated trip time to the end of the passenger trip l due to detour or new carpool passenger added from time t onwards; the trip time taken if the vehicle only serves the passenger trip l; p is a constant set to 0.5,

[0064] 6.4 The mathematical expression of the range constraint is as follows,

[0065] r' n - l r ≤ range0

[0066] where range0 is the minimum range; l r is the updated driving path range after adding order r, set to 100km; r' n is the current range of vehicle n,

[0067] 6.5 The mathematical expression of the sequence constraint is as follows,

[0068]

[0069] where, is the pickup location O r of the passenger in the vehicle destination set; is the drop-off location D r of the passenger in the vehicle destination set.

[0070] Further, the step (7) comprises:

[0071] 7.1 Model the real-time order matching and vehicle dispatching process as a Markov decision process, and define the state, action, reward and state transition probability involved in the Markov decision process;

[0072] Define a triple to capture the system state update at time t: (X t , V t：t+T , D t：t+T ), and combine the triple state variables in a vector s t , where X t = {x t，1 , x t，2 ,..., x t,N} is the current state of all N vehicles at time t, V t：t+T is the number of vehicles in each region in the future T time periods, and D t：t+T is the future travel demand prediction of each region from time t to time t+T;

[0073] Define the action at,n The first part decides whether to serve existing passengers or new passengers if the vehicle is partially full. The second part needs to decide the area, denoted by u t，n，i , that the vehicle should go to at time slot t if there is no passenger on the vehicle or it decides to serve new passengers. If the vehicle decides to serve new users and decides to go to area i, u t，n，i = 1, otherwise 0. If the vehicle n has no empty seats, it cannot serve any other passengers.

[0074] The reward r t，n for vehicle n at time slot t is defined as follows,

[0075]

[0076] where β i is the weight of each part in the reward expression; b t，n is the number of orders served by vehicle n at time t; c t，n is the time spent by vehicle n to make a detour to pick up additional passengers if there are available seats; δ t，n，l is the additional time spent by vehicle n due to carpooling compared to directly serving the passenger trip l and making a carpooling case; e t，n is the vehicle occupancy status, which is set to 1 if the idle vehicle n is occupied, and 0 if the vehicle with existing passengers accepts a new customer;

[0077] The state transition probability matrix is defined as P(s t+1 |s t , a t，n ), which represents the probabilistic dependence between actions and reward functions. The present patent uses a neural network to determine the transition probabilities and reward distribution, and is used to find the state-action value Q value;

[0078] 7.2 Design the decision variables and objective function of multi-agent,

[0079] At each time step t, the model algorithm obtains the state s t of the environment and the reward r t given by the environment, according to the state of the environment and the reward, the model takes action a i to guide the idle vehicle to different locations, so as to maximize the expected future long-term reward. In this process, the agent has two decision variables, the first decision variable is At time t, the agent decides to dispatch vehicle n to area j, then otherwise 0. The second decision variable is ζ n，t, if the vehicle n is not full, the agent needs to decide whether the vehicle can serve a new customer at time period t, if the agent decides to serve the customer then n，t = 1, otherwise n， t = 0, thus, the objective function of the multi-agent at time t is as follows:

[0080]

[0081] In the formula, β1, β2, β3, β4 are weight factors of each target; the negative sign indicates that the ultimate goal actually wants to minimize the term in the parentheses; diff t (D) is the vehicle supply gap; is the total scheduling time; Δ t is the total additional travel time; e t is the total number of vehicles serving.

[0082] Further, the step (8) update process is as follows:

[0083]

[0084] Since the system space is large, a neural network needs to be used to maintain the system space, when updating these values, a loss function L i (θ i ) is used to calculate the difference between the predicted Q value and the target Q value, and the loss function expression is as follows:

[0085]

[0086] In the formula, θ i and are the weights of the training neural network and the target neural network respectively.

[0087] Compared with the prior art, the beneficial effects of the present application are:

[0088] 1. The present application focuses on unmanned electric taxis, in the dynamic carpooling scene of multiple passengers per vehicle, considering the endurance journey constraints of electric vehicles and the charging energy demand of vehicles, which is more in line with the actual operation requirements and can effectively improve the service quality.

[0089] 2. The present application establishes a spatio-temporal graph convolutional neural network based on attention mechanism, which can capture the spatio-temporal characteristics of passenger travel demand better than the baseline model, thereby realizing high-precision rolling prediction of multi-region passenger travel demand, and effectively improving vehicle operation efficiency and reducing resource waste.

[0090] 3. The Double DQN algorithm is improved, the convergence characteristics of the loss function during training are considered, and a dynamic adjustment mechanism for the learning rate and the greed coefficient is established, so that it changes nonlinearly according to the training process. The improvement improves the training stability, speeds up the convergence speed in complex dynamic scenarios, and improves the algorithm efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0091] Figure 1 The method flowchart of the present application is shown in the figure.

[0092] Figure 2 The spatio-temporal graph convolutional neural network structure based on the attention mechanism is shown in the figure, wherein (a) is the spatio-temporal graph convolutional neural network structure based on the attention mechanism, (b) is the spatio-temporal module structure, (c) is the time attention module, and (d) is the spatial attention module.

[0093] Figure 3 The improved Double DQN algorithm is proposed in the present application.

[0094] Figure 4 The pseudocode of the Double DQN algorithm is shown in the figure. DETAILED DESCRIPTION

[0095] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be further described below.

[0096] Figure 1 The dynamic carpooling dispatching method for unmanned electric taxis is shown in the figure. The following will be described in combination with the specific embodiments of the present application. Figure 1 The related steps are described in detail as follows:

[0097] Step 1, obtaining user historical travel orders, user real-time carpooling orders, vehicle real-time state and position information, including but not limited to: passenger travel starting point and ending point, travel number and departure time, vehicle remaining capacity and cruising range, vehicle position, vehicle loadable passenger number, etc.

[0098] Step 2, data cleaning of user historical travel orders, including but not limited to: converting latitude and longitude to WGS84 coordinate system, converting timestamp to date format, deleting data outside the research area, and cleaning duplicate orders.

[0099] Step 3, dividing the operation area into regular hexagonal grids, and mapping the cleaned data in each hexagonal grid, the specific process is as follows:

[0100] 3.1 Using H3 spatial indexing system to divide the operation area into spatial grids, and using H3 hexagonal grid with resolution of 8 as the minimum action unit.

[0101] 3.2 Through the H3 spatial index system, the carpooling demand quantity, the available vehicle supply quantity, and the supply-demand difference value cleaned up in step 2 are respectively mapped into each hexagonal grid to reflect the spatial and temporal distribution differences of the user carpooling demand.

[0102] Step 4, based on the data cleaning and the spatial and temporal distribution results of the travel demand in step 3, a travel demand feature matrix and a spatial dependence matrix are established to respectively quantify the passenger travel time characteristics and spatial characteristics, and the specific process is as follows:

[0103] 4.1 In the spatial dimension, the passenger travel demand of different regions will influence each other and there is a spatial dependence relationship. Granger causality test is a hypothesis testing method used to detect whether a group of variables is affected by the lag of another group of variables. If there is such an influence between two variables, there is a Granger causality. The spatial dependence characteristics of passenger travel between regions are characterized by Granger causality test, and a spatial dependence matrix A is established. The elements in the matrix only contain 0 and 1, representing whether there is a dependence between the travel demands of two regions: 0 if there is no dependence relationship; 1 if there is a dependence relationship. Specifically,

[0104]

[0105] In the formula, is the matrix element of A adj ; i, j are both divided regions; Lj is the arrival demand time series of region j; Li is the travel demand sequence of region i.

[0106] 4.2 In the time dimension, the passenger travel demand data has natural periodicity, such as daily periodicity and weekly periodicity, in addition, the travel demand of the adjacent prediction time will also have an impact on the prediction of the target time series. Therefore, the travel demand data of the previous time, the travel demand data of the corresponding time of the previous day, and the travel demand data of the corresponding time of the previous week are extracted to construct a travel demand feature matrix X, and then a time series segment containing adjacent time, daily periodicity and weekly periodicity is constructed.

[0107] Step 5, a spatio-temporal graph convolutional neural network based on attention mechanism is constructed, the travel demand feature matrix X and the spatial dependence matrix A in step 4 are input, and the travel demand prediction result is obtained, and the specific process is as follows:

[0108] 5.1 A spatio-temporal graph convolutional neural network based on attention mechanism is constructed, and the time series segment in step 4 is input into the corresponding dependence analysis component together with the spatial dependence matrix to generate a normalized spatial attention matrix E' and a time attention matrix S';

[0109] 5.2 The time attention mechanism in this patent is as follows Figure 2(c) shown. First, the time dimension convolution module is constructed to operate along the time axis of the data to extract important features in the time series and capture the dynamic patterns and correlations of passenger travel demand over time. Through one-dimensional convolution operation in the time dimension, the model updates the passenger travel demand along the time axis in real time according to the size of the convolution kernel, capturing the correlation between times. This time dimension analysis can increase the depth of the fully connected layer, ensuring that the output matches the dimension of the prediction target, optimizing the accuracy and efficiency of demand prediction. Specifically,

[0110]

[0111] where ReLU is an activation function, and Φ is the convolution and parameters in the time dimension.

[0112] Then, the time dimension attention module is constructed, and through the self-attention mechanism, the model can automatically identify and extract key dependent features in the travel demand data, while dynamically adjusting the correlation strength of the travel demand in the time dimension according to the specific circumstances of the input data. Specifically,

[0113]

[0114]

[0115] where V i , b i , Z1, Z2, Z3 are parameters learned from the input data; E is the time correlation matrix automatically learned by the time dimension attention module according to the input data at the current time. S' is the time attention matrix obtained by normalizing the E matrix using the Softmax function; S m，n is the correlation between nodes m and n.

[0116] Further, by dynamically adjusting the input data, its features can be extracted more deeply. The normalized time attention matrix is applied to the current input data to obtain Using the adjusted data, the features of the next stage can be further extracted;

[0117] 5.3 The spatial attention mechanism in this patent is shown in Figure 2 (d). First, the spatial dimension graph convolution module is constructed, which uses spectral graph theory to interpret data as a graph structure and uses convolution technology to process these graph data. Through spatial dimension ChebNet graph convolution analysis, spatial features are extracted from the graph information, revealing the spatial correlation of passenger travel demand. Specifically,

[0118]

[0119] wherein, is the graph signal matrix feature in the t-th time point of the M-th region of passenger trip data; is the parameter to be learned; is the normalized Laplacian matrix.

[0120] Then, a spatial dimension attention module is constructed, and a self-attention mechanism is adopted to capture the dynamic changes of the dependencies between regions in the spatial dimension and automatically adjust the relevance between passenger trip demand data to adapt to these changes.

[0121] Specifically,

[0122]

[0123] wherein, is the input data of the k-th spatio-temporal module; C k-1 is the number of channels of the input data of the k-th spatio-temporal module; T k-1 is the length of the time dimension of the input data of the k-th spatio-temporal module; V j , b j , Y1, Y2, Y3 are a set of parameters learned through an attention mechanism; E is a spatial attention matrix learned based on the current input data; E m，n is the strength of the interaction between nodes m and n; E' is a normalized spatial attention matrix generated by standardizing E using a Softmax function, and each element value in the matrix represents a weight coefficient between regions.

[0124] Further, the spatial attention matrix E' and the matrix of spatial dependencies in the Chebyshev polynomial in the spatial graph convolution are dynamically adjusted to realize the reasonable distribution of attention between regions by affecting the weights.

[0125] 5.4 The dynamic spatio-temporal features extracted in steps 5.2 and 5.3 are trained by stacking spatio-temporal modules and applying a residual learning strategy, the outputs of the proximity dependence, daily cycle and weekly cycle components are adjusted using a fully connected layer to match the dimension of the prediction target, and then the outputs are fused to obtain the final trip demand prediction.

[0126] Step 6, set the constraint conditions of car single matching in the dynamic carpooling scenario, including but not limited to: capacity constraint, distance constraint, time constraint, range constraint and sequence constraint, as follows:

[0127] 6.1 In the running process, the vehicle needs to provide enough seats and strictly control the number of passengers to be no more than the total number of seats. Therefore, for any new order, the number of passengers contained in the order must be no more than the current remaining available capacity of the vehicle. The mathematical expression of the capacity constraint is as follows,

[0128] N r ≤C max -n c

[0129] In the formula, N r The number of passengers included in the new order r, which must not exceed the vehicle's current remaining available capacity; n c The number of passengers already on the train; C max The total number of seats in the vehicle is set to 4.

[0130] 6.2 Order Acceptance Distance Constraint: By controlling the distance between the vehicle's current location and the pick-up point, the order acceptance distance is ensured to not exceed the maximum order acceptance distance. Simultaneously, the total passenger waiting time must not exceed the maximum waiting time. An excessively large order acceptance distance will result in an excessively large solution space and reduce order acceptance efficiency. The mathematical expression for the order acceptance distance constraint is as follows:

[0131] d r，n (n loc O r )≤ω

[0132] t r，n (n loc O r )≤t q -t x

[0133] In the formula, O r For the pick-up location of the new order r; n loc d represents the current position of vehicle n; r，n (n loc O r Let n be the vehicle starting from its current position. loc To the pick-up location of order r O r Order receiving distance; t r，n (n loc O r ) represents the order-taking time required according to the current route; t x For order response time; t q The maximum time a user can wait is set to 10 minutes; ω is 3km.

[0134] 6.3 Detour time constraints refer to the requirement that the additional travel time incurred by passengers due to carpooling must not exceed a certain multiple of the time spent traveling alone. This reduces detour time during carpooling, thereby improving passenger satisfaction and the efficiency of carpooling. A larger detour time constraint increases the likelihood of a successful carpool. The mathematical expression for the detour distance constraint is as follows:

[0135]

[0136] where t is the time elapsed since the passenger trip l request was made; the new estimated trip time to the end for the passenger trip due to detour or new carpool passenger added from time t; the trip time spent if the vehicle only serves the passenger trip l; p is a constant set to 0.5.

[0137] 6.4 The range constraint reflects the electric nature of the autonomous taxi. The range constraint can ensure that the vehicle can complete the received order task, and the range at the end of all trips cannot be too low, so as to reserve the power for the vehicle in low power state to drive to the charging facility. Assuming that the vehicle needs to be charged once every 12 hours, considering the actual situation of charging, the charging time is set to 2 hours. The mathematical expression of the range constraint is as follows,

[0138] r' n -l r ≤ range o

[0139] where range0 is the minimum range; l r is the updated driving path distance after adding order r, which is set to 100km; r' n is the range of the current vehicle n.

[0140] 6.5 The sequence constraint ensures that in the new driving route, the starting point of any order should be before the ending point, which is mainly used in the process of assigning new orders to vehicles and forming new driving paths. The mathematical expression of the sequence constraint is as follows,

[0141]

[0142] where, is the pickup location O r of the passenger in the sequence in the vehicle destination set; is the drop-off location D r of the passenger in the sequence in the vehicle destination set.

[0143] Step 7, based on the travel demand prediction results in step 5 and the vehicle-order matching constraint conditions in step 6, a vehicle-order matching decision model based on distributed deep reinforcement learning is constructed, and the specific process is as follows:

[0144] 7.1 The real-time order matching and vehicle scheduling process is modeled as a Markov decision process, and the state, action, reward and state transition probability involved in the Markov decision process are defined.

[0145] State variables are defined to map the state of the environment, which will affect the reward feedback from different actions. This patent defines a triple to capture the system state update at time t: (X t , V t：t+T , D t：t+T ), and combines the triple state variables in a vector s t . Where X t = {x t,1 , x t，2 ,..., x t,N} is the current state of all N vehicles at time t, V t：t+T is the number of vehicles in each zone in the future T time periods, and D t：t+T is the future demand forecast for each zone from time t to time t+T.

[0146] The action a t,n of vehicle n is composed of two parts: the first part is whether the vehicle decides to serve existing passengers or new passengers if the vehicle is partially full. The second part is to decide the zone that the vehicle should go to at time slot t, denoted by u t，n，i . If the vehicle decides to serve new passengers and decides to go to zone i, u t，n，i = 1, otherwise 0. If vehicle n has no empty seats, it cannot serve any other passengers. If the vehicle only serves existing passengers, it may not drop off passengers in the order they boarded. In order to minimize travel distance and quickly cover all destinations, the vehicle will choose the best route, thereby reaching each destination more quickly.

[0147] If the vehicle chooses to only serve existing passengers in the vehicle, it has no impact on demand imbalance, dispatch time, detour time, and number of vehicle uses, so its reward is 0. Therefore, this study focuses on whether to make a carpooling decision in car matching decision making, and turns its attention to the scenario where the vehicle decides to serve new passengers and is willing to detour at time t. Therefore, the reward r t of vehicle n at time slot t, n, is defined as follows,

[0148]

[0149] where β i is the weight of each part in the reward expression; b t，n is the number of orders served by vehicle n at time t; c t，n is the time spent by vehicle n to detour to pick up additional passengers if there are available seats; δ t，n，l is the additional time spent by vehicle n due to carpooling compared to directly serving passenger trip l; and et，n For the vehicle occupancy state, it is set to 1 if the idle vehicle n is occupied, and 0 if the vehicle with existing passengers accepts a new customer.

[0150] Define the state transition probability matrix as P(s t+1 |s t , a t，n ), which represents the probabilistic dependence between actions and reward functions. This patent uses neural networks to determine transition probabilities and reward distributions, and is used to find state-action values Q. At time t, the agent observes the environment state s t , which consists of vehicle states, future demand, and future available vehicles, and takes action a t according to the reward r t . After selecting the action, the environment matches the vehicle with the order according to the action, and updates the vehicle state X t in a timely manner after the matching is completed.

[0151] 7.2 Design decision variables and objective functions for multi-agent.

[0152] At each time step t, the model algorithm obtains the state s t of the environment and the reward r t given by the environment. According to the state of the environment and the reward, the model takes action a i to guide idle vehicles to different locations in order to maximize the expected future long-term reward. In this process, the agent has two decision variables. The first decision variable is At time t, the agent decides to dispatch vehicle n to region j, then otherwise 0. The second decision variable is ζ n，t , if the vehicle n is not full, the agent needs to decide whether the vehicle can provide service to new customers at time period t. If it decides to provide service to customers, ζ n，t = 1, otherwise ζ n，t = 0. Thus, the objective function of the multi-agent at time t is as follows:

[0153]

[0154]

[0155] In the formula, β1, β2, β3, β4 are weight factors of each objective; the negative sign indicates that the ultimate goal actually wants to minimize the term in the parentheses; diff t (D) is the vehicle supply gap; is the total dispatch time; Δ t is the total additional travel time; e t is the total number of vehicles serving.

[0156] Step 8, design Double DQN solving algorithm. The improved Double DQN algorithm pseudocode is shown in Figure 3 , which shows the detailed steps of the training phase. First, initialize the experience pool D, train the Q network parameters θ, and the target Q network parameters After that, at the beginning of each training, initialize the simulator, execute the scheduling command at each time step, and update the entire environment state Ω t . Then, the agent obtains the reward corresponding to the action according to the environment information. Then, for all available vehicles, the following loop operation is performed, first get the state vector of the vehicle Ω t，n , then record the state at the last time, the action at the last time, the reward obtained, the state vector of the vehicle n at this time, and the scheduling time c t，n required to go to the new destination, and store them in the tuple (Ω t-1，n , a t-1，n， r t，n , Ω t，n , c t，n ). Then select tuples from the experience pool D as samples for training, and constantly update the network while updating the set of available vehicles A t . After obtaining the set of available vehicles, use the Epsilon-Greedy method to select actions for all available vehicles, with a probability of ε to select a random action, and with a probability of 1-ε to select the action with the maximum action value in the trained neural network. After selecting the action, the vehicle is transferred to the new destination according to the selected action, and the new vehicle state is obtained. For the nth vehicle, after selecting the action, update the Q value according to the reward r t，n with learning rate a, the update process is as follows:

[0157]

[0158] Due to the large system space, a neural network is used to maintain the system space. When updating these values, the loss function L i (θ i ) is used to calculate the difference between the predicted Q value and the target Q value, and the loss function expression is as follows:

[0159]

[0160] where θ i and are the weights of the trained neural network and the target neural network, respectively.

[0161] Based on the use of experience return and the estimation of action value using two independently trained neural networks, the learning rate a and the greed coefficient ε are dynamically adjusted according to the training process and the loss function during training, making them dynamic and variable parameters.

[0162] In the initial training phase, a warm-up phase is established to prevent model instability caused by starting with a large learning rate. During this phase, the learning rate gradually increases from a small value to a predetermined initial learning rate. After the warm-up phase, the training transitions to the normal training phase, which begins with a higher learning rate. This facilitates the agent's exploration and learning of the environment, allowing the model to converge faster. Subsequently, to ensure training stability, the learning rate is gradually reduced. To make the learning rate change smoother, this patent uses exponential warm-up and exponential decay during the warm-up phase. The learning rate formulas for the exponential warm-up phase and the exponential decay phase are as follows:

[0163]

[0164] In the formula, α(t) is the learning rate at training step t; α s α is the initial value of the learning rate. max The maximum learning rate at the end of the warm-up period; α min T represents the minimum learning rate. w This represents the total number of steps during the warm-up period.

[0165] Building upon this foundation, to enhance the model's generalization ability and adaptability, this patent employs a loss feedback mechanism, dynamically adjusting the learning rate based on the rate of change in loss. This mechanism can promptly adjust the learning rate when fluctuations or instability occur during training, preventing overfitting at local optima. The principle of the learning rate loss feedback mechanism is as follows: if the model's loss continuously decreases, it indicates that the current parameter update direction is correct, and the learning rate can be appropriately reduced for fine-tuning; if the loss suddenly increases, the learning rate needs to be increased to escape local minima. Therefore, based on the basic learning rate scheduling α(t), α′(t) can be dynamically adjusted according to changes in the loss value, as expressed below:

[0166] α′ (t) =α(t)·f(ΔL) i (θ i ))

[0167] In the formula, ΔL i (θ i ) represents the change in the loss function, which is the weighted average sum of the changes in the loss function over the most recent 10 steps; f(·) is the dynamic adjustment function, defined as follows:

[0168]

[0169] In the formula, k and k′ are positive adjustment coefficients used to adjust the learning rate. If the loss value decreases, then f(ΔL) i (θ i)) will make the learning rate slightly smaller; if the loss increases, the learning rate will increase to try to correct the direction or jump out of the local optimum.

[0170] Step 9, based on the model solution result in step 8, real-time order matching and idle vehicle scheduling are performed, the vehicle obtaining the matching is informed to complete the corresponding order, and the idle vehicle not obtaining the matching is informed to perform regional scheduling or charging energy supplement.

[0171] The above are only preferred embodiments of the present application, and do not have any limiting effect on the present application. Any person skilled in the art can make any form of equivalent replacement, modification or change of the technical solutions and technical contents disclosed by the present application without departing from the scope of the technical solutions of the present application, and still belongs to the protection scope of the present application.

Claims

1. A dynamic carpooling dispatching method for driverless electric taxis, characterized in that, Comprising the following steps: (1) obtaining user historical travel orders, carpooling orders and vehicle state position information data; (2) data cleaning of user historical travel orders; (3) dividing the operation area into regular hexagonal grids, and mapping the cleaned data in each hexagonal grid to reflect the spatial and temporal distribution differences of user carpooling travel demand; (4) based on the data cleaning and travel demand spatial and temporal distribution results in step (3), establishing a travel demand feature matrix and a spatial dependence matrix to quantify passenger travel time characteristics and spatial characteristics respectively; (5) constructing a spatio-temporal graph convolutional neural network based on attention mechanism to obtain travel demand prediction results; (6) setting the constraint conditions of car-single matching in dynamic carpooling scenarios, including capacity constraint, distance constraint, time constraint, endurance mileage constraint and sequence constraint; (7) based on the travel demand prediction results in step (5) and the car-single matching constraint conditions in step (6), constructing a car-single matching decision model based on distributed deep reinforcement learning; (8) Design Double DQN algorithm, adopt Double DQN network to dynamically generate optimization strategy, define the vehicle best action value function as the maximum expected reward that can be achieved, adopt the action that makes the neural network output the maximum value for each vehicle, and learn from zero. According to the Epsilon-Greedy method, the greedy scheme is used to select the operation. Under this strategy, the intelligent agent selects the action with the highest Q value with a probability of , otherwise selects a random action. For the first vehicle, after selecting the action, the reward is used to update the Q value with a learning rate . (9) based on the model solution results in step (8), real-time order matching and idle vehicle scheduling are carried out, and the matched vehicles are notified to complete the corresponding orders, and the idle vehicles that have not been matched are notified to carry out regional scheduling or charging energy supplement, The data in step (1) includes passenger travel origin and destination, number of passengers and departure time, vehicle remaining power and endurance mileage, vehicle position, and number of passengers that can be carried by the vehicle, The step (4) comprises: 4.1 In the spatial dimension, the spatial dependence of passenger trips between regions is characterized by Granger causality test, and a spatial dependence matrix is established The elements in the matrix only contain 0 and 1, representing whether there is a dependence of trip demand between two regions: 0 if there is no dependence relationship; 1 if there is a dependence relationship, as follows, ； In the formula, for Matrix elements; Both are areas to be divided; For the region Arrival demand time series; For the region The travel demand sequence; 4.2 In the time dimension, the passenger trip demand data has natural periodicity, in addition, the trip demand close to the prediction time will also have an impact on the prediction of the target time series, extract the trip demand data of the last time of the target prediction time, the trip demand data of the corresponding time of the previous day, and the trip demand data of the corresponding time of the previous week to construct the trip demand feature matrix , and then construct the time series segment containing adjacent time, day cycle and week cycle, The step (5) comprises: 5.1 Build a spatio-temporal graph convolutional neural network based on attention mechanism, input the time series segment in step 4 into the corresponding dependency analysis component together with the spatial dependency matrix to generate a normalized spatial attention matrix and a time attention matrix ; 5.2 Time attention mechanism: first, construct a time dimension convolution module to operate along the time axis of the data to extract important features in the time series, and then capture the dynamic patterns and correlations of passenger travel demand over time. Through one-dimensional convolution operation in the time dimension, the model updates passenger travel demand along the time axis in real time according to the convolution kernel size, captures the correlation between each time, and the specific process is as follows, ； wherein is an activation function, is a convolution over the time dimension and parameters, Then, construct a time dimension attention module. Through the self-attention mechanism, the model can automatically identify and extract key dependent features in the travel demand data, and dynamically adjust the correlation strength of the travel demand in the time dimension according to the specific situation of the input data. The specific process is as follows, ； ； In the formula, , , , , These are parameters learned from the input data; This is a temporal correlation matrix automatically learned from the input data at the current moment through a time-dimensional attention module. To utilize the Softmax function for The time attention matrix is ​​obtained by normalizing the matrix. For nodes and The degree of correlation between them Further, by dynamically adjusting the input data, the features can be further extracted, and the normalized time attention matrix is applied to the current input data to obtain , and using the adjusted data, the features of the next stage can be further extracted; 5.3 Space attention mechanism: first, construct a space dimension graph convolution module. The spectral graph theory is used to interpret the data as a graph structure, and the convolution technology is used to process these graph data. Through ChebNet graph convolution analysis in the space dimension, spatial features are extracted from the graph information to reveal the spatial correlation of passenger travel demand. The specific process is as follows, ； In the formula, is the graph signal matrix feature in the passenger travel data at the Mth region and the th time point; is the to-be-learned parameter; is the normalized Laplacian matrix, Then, construct a space dimension attention module. The self-attention mechanism is used to capture the dynamic changes of the dependence between regions in the space dimension, and automatically adjust the correlation between passenger travel demand data to adapt to these changes. The specific process is as follows, ； ； In the formula, is the input data of the first layer spatio-temporal module; is the number of channels of the input data of the first layer spatio-temporal module; is the length of the time dimension of the input data of the first layer spatio-temporal module; , , , , is a parameter set obtained through attention mechanism learning; is a spatial attention matrix learned based on current input data learning; is the intensity of interaction between nodes and ; is a normalized spatial attention matrix generated by normalizing using a Softmax function, wherein each element value in the spatial attention matrix represents a weight coefficient between regions, Further, the spatial attention matrix is dynamically adjusted and the spatial dependence in Chebyshev polynomials in spatial graph convolution, by affecting the weights, realize the reasonable distribution of attention between regions; 5.4 Train by stacking the space-time modules and applying the residual learning strategy. Use the fully connected layer to adjust the output of the proximity dependence, daily cycle and weekly cycle components to match the dimension of the prediction target, and then fuse these outputs to obtain the final travel demand prediction, The step (6) comprises: 6.1 The mathematical expression of the capacity constraint is as follows, ； wherein is a new order contains the number of passengers, which must not exceed the current remaining capacity of the vehicle; is the number of passengers already on the vehicle; is the total number of seats in the vehicle, set to 4 people; 6.2 The mathematical expression of order distance constraint is as follows, ； ； In the formula, For new orders The pick-up location; For vehicles The current location; For vehicles From current location Orders Pick-up location Order acceptance distance; To determine the order acceptance time required for the current route; For order response time; The maximum time a user can wait is set to 10 minutes; It is 3km long. 6.3 The mathematical expression of detour distance constraint is as follows, ； wherein is the passenger trip time elapsed since the request to pick up the passenger; is the trip time spent due to detours or from time the start to add a new ride-share passenger to the passenger trip is the new estimated trip time from the start to the end; is the trip time spent if the vehicle only serves the passenger trip ; and is a constant, set to 0.5, 6.4 The mathematical expression of endurance distance constraint is as follows, ； In the formula, is the lowest range; is the order added The driving path mileage after updating is set to 100km; is the current vehicle range, 6.5 The mathematical expression of sequence constraint is as follows, ； wherein is the pick-up location of the passenger is the order in the set of vehicle destinations is the drop-off location of the passenger is the order in the set of vehicle destinations The step (7) comprises: 7.1 Model the real-time order matching and vehicle scheduling process as a Markov decision process, and define the state, action, reward and state transition probability involved in the Markov decision process; Define a triple to capture time System status update: And combine the triplet state variables into a vector. In, among them, In time All of the time The current status of each vehicle. For the future Number of vehicles in each area within a given time period From time to each region Time Forecast of future travel demand; Define vehicle action It consists of two parts: the first part determines whether to serve existing passengers or new passengers if the vehicle is partially full; the second part determines the time slot the vehicle should be in if the vehicle is empty or if it decides to serve new passengers. The area to be visited, using It indicates that if the vehicle decides to provide service to a new user and decides to travel to the area... ,but Otherwise, it is 0; if the vehicle If there are no available seats, it cannot serve any other passengers; Defining a vehicle In a time slot A reward As follows, ； wherein is the weight of each part of the reward expression; is the vehicle is the time is the number of orders served; is the vehicle is the time spent detouring to pick up additional passengers if there are available seats; is the passenger trip is the vehicle is the additional time spent due to carpooling; is the vehicle occupancy status, which is 1 if an idle vehicle is set to 1 if the vehicle is occupied, and 0 if a vehicle with an existing passenger accepts a new customer; The state transition probability matrix is defined as , representing the probability dependence between actions and reward functions, using a neural network to determine transition probabilities and reward distributions, and used to find state-action values Q values; 7.2 Design the decision variables and objective function of multi-agent, At each time step , the model algorithm gets the state of the environment and the reward given by the environment , according to the state of the environment and the reward, the model takes action to guide the idle vehicles to different locations to maximize the expected future long-term reward, in the process, the agent has two decision variables, the first decision variable is , at time , the agent decides to dispatch the vehicle to the area , then , otherwise 0, the second decision variable is , if the vehicle n passengers are not full, the agent needs to decide whether the vehicle can provide service for new customers in the time period t, if it decides to provide service for customers , otherwise , thus, the objective function of the multi-agent at time is as follows: ； ； ； ； ； wherein is a weight factor for each objective; the negative sign indicates that the ultimate objective actually wants to minimize the term inside the parentheses; is the supply gap for vehicles; is the total dispatch time; is the total extra travel time; is the total number of vehicles serving, The step (8) updates the process as follows: ； Because the system space is large, a neural network is needed to maintain the system space. When updating these values, a loss function is used. To calculate the difference between the predicted Q-value and the target Q-value, the loss function expression is as follows: ； wherein and are the weights of the training neural network and the target neural network, respectively.

2. The dynamic carpool dispatching method for driverless electric taxis of claim 1, wherein, The step (2) data cleaning comprises: converting latitude and longitude into WGS84 coordinate system, converting timestamp into date format, deleting data outside the research area, and cleaning repeated orders. 3.The dynamic carpooling dispatching method for driverless electric taxis of claim 2, wherein, The step (3) comprises: 3.1 Use the H3 spatial indexing system to divide the operation area into a spatial grid, and use the H3 hexagonal grid with a resolution of 8 as the minimum action unit; 3.2 Through the H3 spatial indexing system, map the carpool travel demand, available vehicle supply and supply-demand difference values cleaned out in step (2) into each hexagonal grid to reflect the spatial and temporal distribution differences of user carpool travel demand.

Images