Intelligent scheduling method of connected vehicle system

Abstract

The application discloses an intelligent scheduling method of a network-connected vehicle system and belongs to the field of intelligent traffic; specifically comprising the following steps: firstly, real road network information is acquired to construct a traffic network graph; the traffic network graph is divided into a plurality of disjoint regions based on a partition model; then, real-time vehicle density information is acquired and input into a neural network, and the estimated driving time of each pair of adjacent regions is output, and the weight of each region to the adjacent region and itself is further calculated; real-time supply and demand information in the service range of each region is acquired and input into a matching model, and the matching result of the vehicle and the passenger is output and executed; the remaining supply and demand information and the weight information of each region are input into a reinforcement learning scheduling model, and the scheduling strategy of the vehicle is output and executed; the application considers the real-time inter-regional driving time to improve the application performance of the scheduling model in the real environment, including the vehicle service level and the income thereof.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, specifically an intelligent scheduling method for connected vehicle systems. Background Technology

[0002] In the field of intelligent transportation, online ride-hailing has become the preferred mode of transportation, offering advantages such as convenience and high service quality. However, the imbalance between vehicles and passengers in different regions significantly impacts passenger experience and vehicle utilization efficiency. For example, in high-demand areas like shopping malls or schools, passengers often have to wait, reducing their travel experience; while in remote areas, passenger demand is low, and vehicles have to wait or drive randomly, increasing the probability of traffic congestion and energy consumption. As vehicles gradually acquire certain computing and communication capabilities, intelligent decision-making can be made between vehicles, between vehicles and passengers, and between vehicles and base stations, providing the foundation for solving the problem of supply and demand imbalance between regions.

[0003] To address the imbalance between supply and demand in different regions, existing solutions include grid / region-based scheduling methods and road-based scheduling methods. Grid-based scheduling methods simplify the problem and are suitable for large-scale cities; however, they do not consider the actual road network, which may lead to ineffective vehicle scheduling strategies. Road-based scheduling methods achieve precise vehicle scheduling; however, they require significant computation and are not suitable for large-scale cities.

[0004] Furthermore, if a grid-based scheduling method is used, it is considered equivalent for a vehicle to be scheduled to any adjacent area, without taking into account the connectivity between adjacent areas. If a large number of surplus vehicles in one area are scheduled to adjacent areas connected by only one road, it may cause additional road congestion and energy consumption, and may even prevent some vehicles from completing the scheduling strategy. Summary of the Invention

[0005] To address the problem that existing grid-based scheduling methods fail to consider the impact of actual roads on scheduling strategies, leading to supply-demand imbalances between regions, this invention provides an intelligent scheduling method for connected vehicle systems that considers real-time inter-regional travel time to improve the performance of the scheduling model in real-world environments, including vehicle service levels and their benefits.

[0006] The intelligent scheduling method for the connected vehicle system includes the following steps:

[0007] Step 1: For the areas where connected vehicles are to be served, obtain the road network information of the area and construct a traffic network map;

[0008] The road network information includes: the location information of intersections, the connection relationship between roads and intersections, the length of roads, and the number of lanes on roads;

[0009] The traffic network graph is constructed as follows: roads are used as nodes, and the midpoint of a road is used as the node location; two adjacent intersections are connected as edges, and the length of the road and the number of lanes are used as node attributes.

[0010] Step 2: Divide the traffic network map into multiple regions based on the partitioning model to obtain the clustering matrix;

[0011] The partitioning model is as follows:

[0012] Based on the actual service area size, construct several hexagonal regions with adjustable radii, ensuring that the total area of ​​all hexagonal regions completely encompasses the traffic network map; calculate the distance between each node in the traffic network map and the center of all hexagonal regions; assign each node to the nearest hexagonal region; the inclusion relationship between nodes and hexagonal regions is defined as a clustering matrix.

[0013] Step 3: Obtain real-time vehicle density information, input it into the neural network, and combine it with the clustering matrix to output the estimated travel time for a vehicle to travel from the current hexagonal region to the adjacent hexagonal region;

[0014] Neural networks include: temporal graph convolutional networks composed of graph convolutional networks and gated recurrent units, graph pooling, and multilayer perceptrons;

[0015] Specifically:

[0016] First, real-time vehicle density information is input into a time-map convolutional network to extract latent features that characterize vehicle density information.

[0017] The vehicle density information is the ratio of the number of vehicles on each road to the length of the road.

[0018] Then, the latent features and clustering matrix are input into the graph pooling function, which outputs the latent features of each hexagonal region.

[0019] Finally, the latent features of each hexagonal region are input into the multilayer perceptron, and the estimated travel time for each pair of adjacent hexagonal regions is output.

[0020] Step 4: Based on the estimated travel time of adjacent hexagonal regions, calculate the weights of the vehicle from each region to its adjacent regions and itself.

[0021] First, for the vehicle's current location, collect the estimated travel time to each adjacent area, and select the minimum value eta. min And multiply it by a city- and business-specific coefficient α to get the travel time for the vehicle to travel from the current area to itself;

[0022] Then, taking the travel time to each region and its neighboring regions as input, a cost-based normalization model is used to calculate the weights of the vehicle's journey from the current region to each neighboring region and its own region.

[0023] Step 5: Obtain real-time supply and demand information within the service area of ​​each hexagonal region, input it into the matching model, output the vehicle and passenger matching results, and execute the process.

[0024] The supply and demand information includes: the location and number of vehicles and passengers in each area of ​​the current time slot;

[0025] The matching model is any model suitable for matching vehicles and passengers.

[0026] Step 6: Input the remaining supply and demand information and weight information after matching vehicles and passengers in each hexagonal region into the reinforcement learning scheduling model, output the vehicle scheduling strategy and execute it.

[0027] The remaining supply and demand information includes the number of unmatched vehicles and passengers in each hexagonal region and its adjacent regions in the current time slot.

[0028] Specifically:

[0029] First, the remaining supply and demand information of each hexagonal region is input into the estimation Q network, and the estimated value of each hexagonal region reaching its neighboring regions and its own region is output.

[0030] Then, the weight information of each hexagonal region reaching its neighboring regions and its own region is multiplied by its corresponding estimated value to output the estimated weighted value of each region reaching its neighboring regions and its own region.

[0031] Finally, the estimated weighted value of each region reaching its neighboring regions and its own region is input into the Softmax function to obtain the probability of each region reaching its neighboring regions and its own region. Based on this probability, a scheduling strategy is assigned to the remaining vehicles in each region.

[0032] The advantages of this invention are:

[0033] (1) An intelligent scheduling method for connected vehicle system, which combines grid-based scheduling method and actual road network to achieve fine scheduling with low computational complexity.

[0034] (2) An intelligent scheduling method for connected vehicle system, which achieves joint optimization of scheduling strategies in different regions by mixing the optimal value of each region.

[0035] (3) An intelligent scheduling method for connected vehicle systems, which improves the application performance of the scheduling model in real environment by considering the impact of connectivity between adjacent areas on the scheduling strategy, including vehicle service level and its benefits. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the intelligent scheduling method in the connected vehicle system proposed in this invention.

[0037] Figure 2 This is a schematic diagram of the neural network structure of the present invention;

[0038] Figure 3 This is a schematic diagram illustrating the process of estimating travel time between regions according to the present invention.

[0039] Figure 4 This is a schematic diagram of the structure of the convolutional network of the present invention;

[0040] Figure 5 This is a schematic diagram of the recurrent neural network structure of the present invention;

[0041] Figure 6 This is a schematic diagram of the weight calculation process of the present invention;

[0042] Figure 7 This is a flowchart illustrating the reinforcement learning scheduling model of the present invention.

[0043] Figure 8 This is a schematic diagram of the reinforcement learning scheduling model of the present invention;

[0044] Figure 9 This is a schematic diagram of the structure of related devices in the connected vehicle system provided in the embodiments of the present invention.

[0045] Figure 10 This diagram illustrates how reinforcement learning can be used to calculate the value of a grid reaching its neighboring grids in existing vehicle scheduling methods. Detailed Implementation

[0046] The present invention will be further described below with reference to the embodiments and accompanying drawings. It should be understood that these embodiments are only used to illustrate the technical solutions and implementation processes of the present invention, and are not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any inventive modifications are within the protection scope of the present invention.

[0047] This invention discloses an intelligent scheduling method for connected vehicles. By considering inter-regional travel time, it integrates the actual road network into a grid-based method. Specifically, it includes: acquiring real road network information and constructing a traffic network map; dividing the traffic network map into multiple non-overlapping regions based on a partitioning model; acquiring real-time vehicle density information and inputting it into a trained neural network model to output the estimated travel time for each pair of adjacent regions; acquiring real-time supply and demand information within the service area of ​​each region and inputting it into a matching model to output and execute the vehicle-passenger matching results; calculating the weights of each region's arrival at adjacent regions and its own weights based on the estimated inter-regional travel time; and inputting the remaining supply and demand information and weight information of each region into a trained reinforcement learning scheduling model to output and execute the vehicle scheduling strategy. This invention solves the problem of vehicle supply and demand imbalance and the problem of homogeneity between adjacent grids in existing vehicle scheduling methods, thus improving the application performance of vehicle scheduling methods in real-world environments.

[0048] The intelligent scheduling method of the connected vehicle system, such as Figure 1 As shown, the specific steps include:

[0049] Step 1: For the areas where connected vehicles are to be served, obtain the road network information of the area and construct a traffic network map;

[0050] The road network information includes: the location information of intersections, the connection relationship between roads and intersections, the length of roads, and the number of lanes on roads;

[0051] The traffic network graph is constructed as follows: roads are used as nodes, and the midpoint of a road is used as the node location; two adjacent intersections are connected as edges, and the length of the road and the number of lanes are used as node attributes.

[0052] Step 2: Divide the traffic network map into multiple regions based on the partitioning model to obtain the clustering matrix;

[0053] For the vehicle dispatching problem in a city, due to the large number of vehicles, selecting a different dispatching strategy for each vehicle is not feasible in real time. Therefore, current technical solutions assume that if two vehicles are close to each other, their dispatching strategies are often the same. This leads to the concept of a region / grid, which divides a city into multiple non-overlapping regions. During each dispatch, all vehicles in a region are dispatched, and the dispatching strategies for the vehicles can be the same or different.

[0054] The partitioning model is as follows:

[0055] Based on the actual service area size, construct several hexagonal regions with adjustable radii, for example, a radius of 2.2km, so that the total area of ​​all hexagonal regions completely encompasses the traffic network map; calculate the distance between each node in the traffic network map and the center of all hexagonal regions; divide each node into the nearest hexagonal region; the inclusion relationship between the node and the hexagonal region is defined as the clustering matrix.

[0056] Step 3: Obtain real-time vehicle density information, input it into the trained neural network, and combine it with the clustering matrix to output the estimated travel time of a vehicle from the current hexagonal region to the adjacent hexagonal region;

[0057] In order to convert traffic road network information into grid information and realize fine-grained scheduling of vehicle scheduling algorithms in the grid environment, this invention proposes to predict the travel time of adjacent areas in the grid environment by road vehicle density.

[0058] like Figure 2 As shown, the neural network includes: a temporal graph convolutional network composed of graph convolutional networks and gated recurrent units, graph pooling, and a multilayer perceptron;

[0059] like Figure 3 As shown, specifically:

[0060] First, real-time vehicle density information is obtained through communication between vehicles and base stations. This information is then input into a time-map convolutional network to extract latent features representing vehicle density information from the traffic network graph.

[0061] The vehicle density information is the ratio of the number of vehicles on each road to the length of the road.

[0062] The temporal graph convolutional network consists of two layers of graph convolutional networks and one layer of recurrent neural networks, such as... Figure 4 and Figure 5 As shown. The calculation formula for the convolutional network is as follows:

[0063]

[0064] Among them, X origin The node characteristics of the traffic network graph consist of road length, number of lanes, and real-time vehicle density; X graph This represents the feature vector output by the graph convolutional network; A represents the adjacency matrix of the traffic network graph, I represents the m-order identity matrix, and D represents... The degree matrix; ReLU and σ represent the ReLU activation function and the sigmoid activation function, respectively; W0 and W1 represent the network parameters of the first and second layers.

[0065] The calculation formula for a recurrent neural network is as follows:

[0066] zt =σ(W z [h t-1 ,X graph ])

[0067] r t =σ(W r [h t-1 ,X graph ])

[0068]

[0069]

[0070] X recurrent =W o h t

[0071] Among them, h t-1 X represents the historical features of the recurrent neural network output at the previous time step. recurrent The latent features representing the vehicle density information output by the recurrent neural network, [·,·] denote the concatenation of two matrices, * denotes element-wise multiplication of the matrices, and W z W r W h and W o These represent different network parameters.

[0072] Then, the latent features and clustering matrix are input into the graph pooling function, which outputs the latent features of each hexagonal region.

[0073] The graph pooling function is as follows:

[0074] X pool =S T X recurrent

[0075] A pool =S T AS

[0076] d represents the dimension of the latent feature; S represents the clustering matrix. S T X represents the transpose of the clustering matrix S; pool The latent feature representing regional vehicle density information, A pool This represents the adjacency matrix between regions.

[0077] Finally, the latent features of each hexagonal region are input into the multilayer perceptron, and the estimated travel time for each pair of adjacent hexagonal regions is output.

[0078] This invention trains a neural network by obtaining experience samples from a historical travel time sample pool of the region to be served; the process of obtaining experience samples is as follows:

[0079] First, obtain vehicle density and location information in the actual road network. The time intervals for these information are of equal length and are related to the size of the area, for example, 10 minutes.

[0080] Then, calculate the travel time of all vehicles traveling from one region to another adjacent region between two time slots; and average the travel time of all vehicles occurring in each pair of adjacent regions to obtain the true inter-region travel time of the previous time slot.

[0081] Finally, the vehicle density information and inter-regional travel time of each historical time slot are stored in the sample pool; the sample pool is sampled with replacement to obtain empirical samples.

[0082] A loss function is constructed based on minimizing the difference between the actual and estimated travel times of a sample as the training objective. For example, the loss function can be calculated using mean squared error.

[0083]

[0084] Where, θ W eta represents the parameters mentioned in the aforementioned neural network. i,j and real i,j Representing region Z i With region Z j The estimated travel time and the actual travel time.

[0085] Step 4: Based on the estimated travel time of adjacent hexagonal regions, calculate the weights of the vehicle from each region to its adjacent regions and itself.

[0086] First, for the vehicle's current location, collect the estimated travel time to each adjacent area, and select the minimum value eta. min And multiply it by a city- and business-specific coefficient α to get the travel time for the vehicle to travel from the current area to itself;

[0087] Since the region is hexagonal in shape, each region has six neighboring regions (ignoring edge regions). Therefore, grid-based scheduling methods generally have seven scheduling strategies: scheduling to six neighboring regions, or scheduling to itself (no scheduling). To consider the strategy of scheduling to itself, it is also necessary to estimate the travel time for each region to reach itself. Intuitively, the time required for each region to reach itself is always shorter than the time required to reach other regions; therefore, the time required to reach itself is determined by the minimum travel time to reach its neighboring regions.

[0088] Then, taking the travel time to each region and its neighboring regions as input, a cost-based normalization model, such as a linear normalization model, is used to calculate the weights of the vehicle's journey from the current region to each neighboring region and its own region.

[0089] This embodiment includes a weight calculation module and a value weighting module;

[0090] The weight calculation module calculates the scheduling weight of vehicles between regions based on characteristic data between certain regions, such as travel time between regions.

[0091] Specifically, the process involves: First, acquiring feature data between regions, such as travel time between adjacent regions; for any given region, collecting feature data from other regions; for its own feature data, determining the maximum or minimum value based on the feature data from other regions, where the maximum or minimum value depends on the nature of the feature data, for example, taking the minimum value when the feature data is travel time between adjacent regions; and finally, based on the benefit / cost normalization principle, obtaining the weights for each region to reach other regions and itself.

[0092] Value weighting module: Obtain the scheduling value between regions through a certain scheduling model, such as a reinforcement learning scheduling model; then, based on the weights obtained by the weighting module, weight the scheduling value of vehicles between regions.

[0093] Step 5: Obtain real-time supply and demand information within the service area of ​​each hexagonal region, input it into the matching model, output the vehicle and passenger matching results, and execute the process.

[0094] The supply and demand information includes: the location and number of vehicles and passengers in each area of ​​the current time slot;

[0095] The matching model is any model suitable for matching vehicles and passengers.

[0096] Step 6: Input the remaining supply and demand information and weight information after matching vehicles and passengers in each hexagonal region into the trained reinforcement learning scheduling model, output the vehicle scheduling strategy and execute it.

[0097] The remaining supply and demand information includes the number of unmatched vehicles and passengers in each hexagonal region and its adjacent regions in the current time slot.

[0098] Specifically:

[0099] First, the remaining supply and demand information of each hexagonal region is input into the estimation Q network, and the estimated value of each hexagonal region reaching its neighboring regions and its own region is output.

[0100] Then, the weight information of each hexagonal region reaching its neighboring regions and its own region is multiplied by its corresponding estimated value to output the estimated weighted value of each region reaching its neighboring regions and its own region.

[0101] In reinforcement learning, the decision-making agent is the intelligent agent. In each round of training or testing, the agent calculates the value of each possible action based on the current state, then selects the action with the highest value, and so on. In vehicle scheduling methods, the intelligent agent often refers to an "agent" representing all vehicles in a grid / area, rather than an actual vehicle. An "agent" implies that all vehicles in a region are homogeneous, have the same state, and have the same value for each action. Therefore, to some extent, the existence of an "agent" simplifies the computation.

[0102] For the "agent," its optional actions include: not scheduling (itself) and scheduling to a neighboring region. Meanwhile, since the weights for each region's reach to neighboring regions and itself are obtained, the value of the "agent's" optional actions is multiplied by the weights to obtain an estimated weighted value, thus enabling the agent's path selectivity.

[0103] Finally, the estimated weighted value of each region reaching its neighboring regions and its own region is input into the Softmax function to obtain the probability of each region reaching its neighboring regions and its own region. Based on this probability, a scheduling strategy is assigned to the remaining vehicles in each region.

[0104] In current reinforcement learning models, each region represents an agent, and an agent's "optimal weighted value" represents the maximum of the estimated weighted values ​​for reaching neighboring regions and for itself. This invention inputs the optimal weighted values ​​of all regions into an estimation hybrid network, outputting the estimated value of the overall scheduling policy, which is then used to train the reinforcement learning model. Through multi-agent reinforcement learning, the Q-values ​​of multiple agents are fused into a single overall Q-value, enabling the agents to learn towards the overall optimal direction.

[0105] In training the reinforcement learning scheduling model, this invention trains by acquiring experience samples of vehicle scheduling. The specific acquisition process is as follows: First, the remaining supply and demand information, overall scheduling strategy and its estimated value at each historical moment are stored in the sample pool; then, the sample pool is sampled with replacement to obtain experience samples.

[0106] Example:

[0107] Step S11: Obtain road network information and construct a traffic network map.

[0108] Based on the city's road network, construct an urban traffic network graph G = (V, E), where v ∈ V represents a road and e ∈ E represents an intersection.

[0109] Suppose graph G has m vertices and n edges. Each vertex of graph G contains a set of feature vectors, consisting of static features and dynamic features.

[0110] Static features include: vertex position, road length, and number of lanes, where the vertex position is determined by the position of the road midpoint. Dynamic features include: real-time vehicle density of the road.

[0111] Step S12: Divide the traffic network map into multiple regions based on the zoning model.

[0112] Divide the city into k non-overlapping hexagonal regions, Z = {Z1, Z2, ..., Zn}. k} where k is related to the city's area and the size of a single hexagonal region. Calculate the distance between nodes in the traffic network graph and the centers of all hexagonal regions; assign nodes to the nearest hexagonal regions; the inclusion relationship between nodes and hexagonal regions is defined as a clustering matrix.

[0113] Step S13: Obtain real-time vehicle density information, input it into the trained neural network model, and output the estimated travel time for each pair of adjacent areas.

[0114] Divide each day into T time slots, i.e., T = {1, 2, ..., T}. Calculate the ratio of the number of vehicles at each vertex v in time slot t to the road length, denoted as den. t,v Therefore, the real-time vehicle density is expressed as densities. t,v .

[0115] Step S14: Obtain real-time supply and demand information within the service area of ​​each region, input it into the matching model, output the vehicle and passenger matching results, and execute the process.

[0116] The system calculates the number of vehicles and passengers in each area, as well as the location of each vehicle and passenger. This information is then input into a matching model that minimizes the number of vehicles and passengers, and the system outputs the matching results.

[0117] Specifically, for any region, a bipartite graph is constructed using vehicles and passengers within the region as nodes, matching vehicles and passengers as edges, and the distance between vehicles and passengers as the weight of the edges. Then, the maximum weight matching of the bipartite graph within each region is calculated, and the matching results of vehicles and passengers are output.

[0118] Step S15: Based on the estimated travel time between regions, calculate the weight of each region to its neighboring regions and itself.

[0119] Specific steps are as follows Figure 6 As shown:

[0120] First, for any given region, collect the estimated travel time to reach neighboring regions;

[0121] Let U(i) represent region Z i Adjacent regions: U(i)={Z j A pool (i,j)=1,i≠j}; then, region Z i The estimated travel time to the adjacent area can be expressed as: {eta i,j ;j∈U(i)}.

[0122] The travel time to reach a region itself is determined by the minimum estimated travel time to reach an adjacent region:

[0123] eta i,i =αmin({eta i,j ;j∈U(i))

[0124] Here, α∈(0,1] is a hyperparameter that is selected based on the city and business of the service.

[0125] Then, based on the cost-based normalization principle, the weights of each region reaching its neighboring regions and itself are obtained.

[0126] For example, using a linear normalized model:

[0127]

[0128] Among them, w i,j Representing region Z i Arrival in area Z j The weights; max(eta) i,* ) and min(eta i,* ) represent eta respectively i,* The maximum and minimum values ​​of eta; i,* ={eta i,j ;j∈(U(i)∪{i})}, representing the region Z i The set of estimated travel times to neighboring regions and to itself.

[0129] Step S16: Input the remaining supply and demand information and weight information of each region into the trained reinforcement learning scheduling model, output the vehicle scheduling strategy and execute it.

[0130] Statistical time slot t region Z i The number of unmatched vehicles and passengers is denoted as .

[0131] The training steps of the reinforcement learning scheduling model are as follows: Figure 7 As shown, its model structure is as follows Figure 8 As shown:

[0132] Step 1: Obtain experience samples from the vehicle scheduling sample pool; input the remaining supply and demand information of each region at the current historical moment into the estimation Q network, and output the estimated value of each region reaching its neighboring regions and itself;

[0133] Each region has an estimated Q-network, and they share the same parameters, calculated as follows: Q i Representing region Z i The estimated Q network, Represents the time slot region Z of time slot t. i The remaining supply and demand information of region Z and its adjacent regions, (·) represents region Z i Arbitrary scheduling strategies for the remaining vehicles, including scheduling to region Z i The adjacent regions and itself, θ q This indicates the estimated parameters of the Q-network.

[0134] Step II: Multiply the weight information with the corresponding estimated value to output the estimated weighted value of each region reaching its neighboring regions and itself; based on the estimated weighted value of each region reaching its neighboring regions and itself, obtain the scheduling strategy for the remaining vehicles in each region.

[0135] The specific implementation steps are as follows:

[0136] a) Input the estimated weighted value of each region into the Softmax function, and output the probability distribution of each region reaching its neighboring regions and itself;

[0137] The formula for the Softmax function is expressed as follows:

[0138] Among them, z i,j Indicates that the vehicle is from region Z i Dispatch to region Z j The probability, Z represents i To reach region Z j The estimated weighted value.

[0139] b) Select a scheduling strategy for the remaining vehicles in each region according to the above probability distribution.

[0140] Step III: Input the optimal weighted estimated value of all regions into the estimation hybrid network, and output the estimated value of the overall scheduling strategy;

[0141] The hybrid network is used to calculate the joint value of the remaining vehicle scheduling strategies across all regions:

[0142] Q mix =mixing(Q w1Q w2 ,...,Q wk ;θ mix )

[0143] Q w1 Representing region Z i The optimal weighted value, Q mix θ represents the estimated value of the overall scheduling strategy after mixing. mix This represents the parameters of the hybrid network.

[0144] Step IV: Input the remaining supply and demand information of each region at the next historical moment into the target Q network, and output the target value of each region reaching its neighboring regions and itself; multiply the weight information with the corresponding target value, and output the target weighted value of each region reaching its neighboring regions and itself.

[0145] Step V: Input the optimal target weighted value of all regions into the target hybrid network, and output the target value of the overall scheduling policy; based on minimizing the difference between the estimated value and the target value of the overall scheduling policy as the training objective, construct the loss function as follows:

[0146]

[0147] Among them, s t This represents the remaining supply and demand information for all regions in time slot t, a t Q represents the optimal scheduling strategy for the remaining vehicles in all regions. tot This represents the integration of the aforementioned neural networks. Let r represent the network parameters of the target Q-network and the target hybrid network, respectively. t This represents the reward value of the overall scheduling strategy for time slot t.

[0148] r t The specific calculations are as follows:

[0149] Representing the time slot region Z respectively i The distribution of vehicles and passengers.

[0150] In fact, r t This measures the difference in the KL distance between the supply and demand distributions in time slot t and time slot t+1. This means r t As the value approaches 0, the supply and demand distributions of time slot t and time slot t+1 become nearly identical, and the overall scheduling strategy approaches convergence.

[0151] Step VI: Update the parameters of the estimated Q-network and the estimated hybrid network; update the parameters of the target Q-network and the target hybrid network at regular intervals; until the required number of iterations is reached, the estimated Q-network and the hybrid Q-network are identified as trained reinforcement learning scheduling models.

[0152] The above-mentioned empirical samples were obtained through the following steps:

[0153] The remaining supply and demand information, overall scheduling strategy, and their estimated value at each historical moment are used to form an empirical sample <s>. t ,a t ,r t ,s t+1 >, store in the sample pool;

[0154] By sampling the sample pool with replacement, empirical samples are obtained, and the correlation between data is broken.

[0155] The following describes the related device based on this embodiment, and its first structural schematic diagram is shown below. Figure 9 As shown, it may include the following modules:

[0156] The zoning module constructs a traffic network map based on the current city's road network information and divides the traffic network map into multiple regions based on the zoning model;

[0157] The inter-regional travel time estimation module obtains real-time vehicle density information through communication between the vehicle and the base station, inputs it into a trained neural network model, and outputs the estimated travel time between each pair of adjacent regions; wherein, the trained neural network model is obtained based on the inter-regional travel time training module;

[0158] The matching module communicates with the base station to obtain real-time supply and demand information within the service area of ​​each region, inputs it into the matching model, outputs the matching results between vehicles and passengers, and executes the matching.

[0159] The weighting module calculates the weights of each region's arrival at neighboring regions and itself, based on a normalized model of estimated travel time and cost between regions, and applies these weights to the corresponding values ​​in the scheduling module.

[0160] The scheduling module inputs the remaining supply and demand information of each region into the trained reinforcement learning scheduling model, outputs the vehicle scheduling strategy, and executes it; the trained reinforcement learning scheduling model is obtained through the vehicle scheduling training module.

[0161] Optionally, the related apparatus provided in the embodiments of the present invention further includes:

[0162] The storage module stores vehicle density information and inter-regional travel time for each historical time slot, as well as remaining supply and demand information, overall scheduling strategy and its estimated value for each historical moment, into the sample pool.

[0163] The sampling module performs sampling with replacement on the sample pool to obtain empirical samples.

[0164] Meanwhile, embodiments of the present invention provide an electronic device, including a processor, a memory, a communication interface, and a communication bus, wherein the communication bus is used to complete mutual communication between the processor and the memory; the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory and implement any of the method steps in the above-described intelligent scheduling method.

[0165] By comparing with existing vehicle scheduling methods, it can be seen that existing methods use grids as agents, employing reinforcement learning to calculate the value of each grid reaching its neighboring grids (including itself), and then scheduling vehicles based on this value. For example... Figure 10 As shown: the value of grids A through B, C, and D is 30, 30, 30, 30, 30. Therefore, 25% of the remaining vehicles in grid A remain stationary (scheduled to grid A, i.e., scheduled to themselves); 25% are scheduled to grid B; 25% are scheduled to grid C; and 25% are scheduled to grid D.

[0166] However, existing vehicle scheduling methods have a problem: they do not consider the differences between grids, meaning they assume that the time required for a vehicle to be scheduled from grid A to grids B, C, and D is the same. But due to the actual traffic network, the time required for a vehicle to travel from grid A to grids B, C, and D will definitely differ.

[0167] This invention addresses the heterogeneity of vehicle scheduling between different grids by integrating traffic road networks, meaning that the considerations for a vehicle traveling from grid A to grids B, C, and D differ. Specifically, it designs a multi-agent reinforcement learning framework that combines traffic road network information with the grid environment. This framework involves converting traffic road network information into weights for adjacent grids and then applying these weights to the reinforcement learning value, thus solving the problem of homogeneity in scheduling between adjacent grids. Simultaneously, it implements a model that converts traffic road network information into grid information, i.e., an inter-regional travel time estimation model, which is used within the aforementioned framework.

[0168] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement any of the method steps in the above-described intelligent scheduling method.

[0169] This invention proposes an intelligent scheduling method and apparatus for connected vehicle systems to achieve supply and demand balance between different regions. By jointly considering grid-based scheduling methods and actual road networks, the proposed intelligent scheduling method achieves refined scheduling with low computational complexity; by combining the optimal value of each region, it achieves joint optimization of scheduling strategies for different regions; and by considering the impact of connectivity between adjacent regions on scheduling strategies, it improves the application performance of the scheduling model in real-world environments. Therefore, in practical applications, this invention can effectively improve vehicle service levels and their benefits.

Claims

1. An intelligent scheduling method for a connected vehicle system, characterized in that, By considering inter-regional travel time, the actual road network is integrated into a grid-based method, specifically including: First, for the areas where connected vehicles are waiting to be served, real road network information is obtained to construct a traffic network map; and based on a partitioning model, the traffic network map is divided into multiple non-overlapping regions to obtain a clustering matrix. The process of obtaining the clustering matrix is ​​as follows: Based on the actual service area size, construct several hexagonal regions with adjustable radii, ensuring that the total area of ​​all hexagonal regions completely encompasses the traffic network map; calculate the distance between each node in the traffic network map and the center of all hexagonal regions; assign each node to the nearest hexagonal region; the inclusion relationship between nodes and hexagonal regions is defined as a clustering matrix; Then, real-time vehicle density information is obtained and input into the trained neural network model. Combined with the clustering matrix, the estimated travel time of a vehicle from the current hexagonal region to the adjacent hexagonal region is output. Based on the estimated travel time between adjacent hexagonal regions, the weights of the vehicle's journey from each region to its adjacent regions and its own region are calculated. Next, real-time supply and demand information within the service area of ​​each hexagonal region is obtained, input into the matching model, and the results of vehicle-passenger matching are output and executed. Finally, after matching vehicles and passengers in each hexagonal region, the remaining supply and demand information and weight information are input into the trained reinforcement learning scheduling model, which outputs and executes the vehicle scheduling strategy.

2. The intelligent scheduling method for a connected vehicle system as described in claim 1, characterized in that, The road network information includes: the location information of intersections, the connection relationship between roads and intersections, the length of roads, and the number of lanes on roads; The traffic network graph is constructed as follows: roads are used as nodes, and the midpoint of a road is used as the node location; two adjacent intersections are connected as edges, and the length of the road and the number of lanes are used as node attributes.

3. The intelligent scheduling method for a connected vehicle system as described in claim 1, characterized in that, The neural network model includes: a temporal graph convolutional network composed of a graph convolutional network and gated recurrent units, graph pooling, and a multilayer perceptron; The specific process for outputting the estimated travel time is as follows: First, real-time vehicle density information is input into a time-map convolutional network to extract latent features that characterize vehicle density information. The vehicle density information is the ratio of the number of vehicles on each road to the length of the road. Then, the latent features and clustering matrix are input into the graph pooling function, which outputs the latent features of each hexagonal region. Finally, the latent features of each hexagonal region are input into the multilayer perceptron, and the estimated travel time for each pair of adjacent hexagonal regions is output.

4. The intelligent scheduling method for a connected vehicle system as described in claim 1, characterized in that, The calculation of the weights of vehicles arriving at adjacent areas and their own areas from each area is specifically as follows: First, for the vehicle's current location, collect the estimated travel time to each adjacent area, and select the minimum value from these. and with city- and business-specific coefficients. Multiply by the vehicle to get the travel time from the current area to itself; Then, taking the travel time to each region and its neighboring regions as input, a cost-based normalization model is used to calculate the weights of the vehicle's journey from the current region to each neighboring region and its own region.

5. The intelligent scheduling method for a connected vehicle system as described in claim 1, characterized in that, The supply and demand information includes: the location and number of vehicles and passengers in each area of ​​the current time slot; The matching model is any model suitable for matching vehicles and passengers.

6. The intelligent scheduling method for a connected vehicle system as described in claim 1, characterized in that, The remaining supply and demand information includes: the number of unmatched vehicles and passengers in each hexagonal region and its adjacent regions in the current time slot; The specific process of outputting the vehicle scheduling strategy is as follows: First, the remaining supply and demand information of each hexagonal region is input into the estimation Q network, and the estimated value of each hexagonal region reaching its neighboring regions and its own region is output. Then, the weight information of each hexagonal region reaching its neighboring regions and its own region is multiplied by its corresponding estimated value to output the estimated weighted value of each region reaching its neighboring regions and its own region. Finally, the estimated weighted value of each region reaching its neighboring regions and its own region is input into the Softmax function to obtain the probability of each region reaching its neighboring regions and its own region. Based on this probability, a scheduling strategy is assigned to the remaining vehicles in each region.

Images