Vehicle scheduling optimization method and system based on fusion of multi-scale spatiotemporal graph and hierarchical reinforcement learning, terminal and storage medium

Abstract

The application relates to the technical field of scheduling optimization, and discloses a vehicle scheduling optimization method and system fusing a multi-scale space-time graph and hierarchical reinforcement learning, a terminal and a storage medium.The method comprises the following steps: constructing a global space-time graph and vehicle local representation; constructing a macro-control signal and a micro-action intention under different time scales respectively; generating a limited candidate set based on the micro-action intention, screening the limited candidate set under the feasibility rules consistent with the low-layer decision, and constructing a three-part graph; based on the three-part graph, the limited candidate set in the same decision cycle is subjected to conflict resolution and global correction to generate an executable scheduling instruction at the current time. According to the application, the regional situation and vehicle local interaction information are obtained through cross-scale representation, the macro-control signal is generated by the high layer to guide the low layer to generate the action intention, and the centralized correction mechanism is introduced to eliminate the multi-vehicle conflict and correct the feasibility, so that the invalid driving and charging waiting are reduced, and the scheduling task completion efficiency and service balance are improved.

Description

Technical Field

[0001] This invention relates to the field of scheduling optimization technology, and in particular to a vehicle scheduling optimization method, system, terminal, and computer-readable storage medium that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. Background Technology

[0002] Limited by range and refueling conditions, electric vehicles are prone to problems such as supply-demand mismatch, clustering in hot spots, ineffective driving and waiting for refueling in a dynamically changing demand environment, which leads to a decrease in overall system efficiency and uneven distribution of service space.

[0003] However, in existing scheduling methods, the macro-level regional situation modeling and the micro-level real-time decision-making of individual vehicles are often disconnected. Macro-level predictions or rules are difficult to reliably map into individual executable actions, while individual learning strategies are difficult to fully utilize global periodic patterns and cross-regional control signals. Furthermore, under constraints such as energy constraints, capacity and availability of replenishment resources, task timeliness constraints, and multi-vehicle conflict constraints, pure learning strategies are prone to generating unexecutable or conflicting instructions. While pure optimization methods can explicitly handle constraints, they face significant solution pressure in rolling real-time decision-making.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a vehicle scheduling optimization method, system, terminal, and computer-readable storage medium that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. This aims to solve the problem that existing technologies are unable to simultaneously utilize macroscopic situation and individual real-time decision-making, and are unable to ensure the execution of instructions under multiple constraints, resulting in low efficiency in vehicle scheduling.

[0006] To achieve the above objectives, this invention provides a vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. The vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes the following steps:

[0007] A global spatiotemporal map is constructed based on multiple supply and demand features and multiple correlation features within the target area. Regional spatiotemporal embedding vectors are extracted from the global spatiotemporal map to obtain all schedulable vehicles within the target area, and a local vehicle representation of each schedulable vehicle is constructed.

[0008] For the first preset time scale, a macro-control signal is generated using the regional spatiotemporal embedding vector, operation information and historical high-level instruction summary. Then, at the second preset time scale, the corresponding micro-action intention is obtained based on the local representation of each vehicle and the macro-control signal.

[0009] Multiple finite candidate sets are generated based on schedulable vehicles, tasks, refueling point information, and micro-action intentions. All macro-control signals, micro-action intentions, and finite candidate sets are input into the centralized planner, and the finite candidate sets are filtered using feasibility rules consistent with low-level decision-making. A tripartite graph within the same decision cycle is constructed based on the filtered candidate relationships.

[0010] Based on the aforementioned tripartite graph, a centralized planner is used to perform unified conflict resolution and global correction on the finite candidate set within the same decision cycle, generating a set of executable scheduling instructions for the current moment.

[0011] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein the step of constructing a global spatiotemporal graph based on multiple supply and demand features and multiple correlation features within the target area, extracting regional spatiotemporal embedding vectors from the global spatiotemporal graph, obtaining all schedulable vehicles within the target area, and constructing a local vehicle representation for each schedulable vehicle specifically includes:

[0012] Multiple supply and demand features and multiple association features of the target region are obtained, all the supply and demand features are defined as node attributes, and all the association features are defined as edge attributes, so as to construct a global spatiotemporal graph of the target region;

[0013] Feature extraction is performed on the global spatiotemporal graph in both spatial and temporal dimensions to obtain regional spatiotemporal embedding vectors.

[0014] Each schedulable vehicle within the target area is identified, and a corresponding local heterogeneous graph is constructed for each schedulable vehicle to extract the local vehicle representation of each schedulable vehicle.

[0015] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein the step of extracting features from the global spatiotemporal graph in both spatial and temporal dimensions to obtain regional spatiotemporal embedding vectors specifically includes:

[0016] The global spatiotemporal graph is input into a spatiotemporal graph neural network for feature extraction to obtain global region embeddings.

[0017] A gating fusion mechanism and a smoothing mechanism are introduced into the global spatiotemporal graph to suppress jitter in the global region embedding, and the output region spatiotemporal embedding vector is:

[0018] ;

[0019] ;

[0020] in, Indicates the first Global region embedding for each region Indicates the gating coefficient. Indicating regional embeddings of short-term fluctuations Indicates the region embedding of cyclical fluctuations. Indicates the first The spatiotemporal embedding vector of each region This represents the smoothing coefficient.

[0021] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein determining each schedulable vehicle within the target region and constructing a corresponding local heterogeneous graph for each schedulable vehicle to extract local vehicle representations for each schedulable vehicle specifically includes:

[0022] Obtain vehicle information, candidate task information, and reachable refueling point information for each schedulable vehicle within the target area;

[0023] The vehicle information, the candidate task information, and the reachable refueling point information are defined as vehicle nodes, task nodes, and refueling point nodes, respectively. The reachability, estimated arrival time, energy consumption, and waiting information of each schedulable vehicle to all the refueling points and task nodes are defined as corresponding edges. A local heterogeneous graph of each schedulable vehicle is constructed based on all the vehicle nodes, task nodes, refueling point nodes, and all the corresponding edges.

[0024] For each of the local heterogeneous graphs, the local heterogeneous graphs are input into a heterogeneous graph neural network for feature extraction, and the local vehicle representation of the schedulable vehicle is output. The local vehicle representation is used as input to a low-level decision state to generate the corresponding micro-action intention.

[0025] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein, for a first preset time scale, macro-control signals are generated using the regional spatiotemporal embedding vectors, operational information, and historical high-level instruction summaries, and at a second preset time scale, corresponding micro-action intentions are obtained based on the local representation of each vehicle and the macro-control signals, specifically including:

[0026] A decision-making process model is established to obtain historical instruction summaries and operational information of each of the schedulable vehicles in a first preset time scale, wherein the historical instruction summary represents a summary of macro-control signals in the previous macro-decision window;

[0027] The regional spatiotemporal embedding vector, the operational information, and the historical instruction summary are input into the decision-making process model for macro-control, and the macro-control signal corresponding to the current macro-decision window is output.

[0028] The macro-control signal remains stable within a preset window and is used by both the low-level decision-making unit and the centralized planner.

[0029] A low-level graph network at a second preset time scale is obtained. A feature-level linear generator network is used to map the macroscopic control signal into continuous modulation parameters. The intermediate features of the low-level graph network are then fused with the continuous modulation parameters to obtain the microscopic action intent of each schedulable vehicle.

[0030] ;

[0031] ;

[0032] in, Indicates continuous modulation parameters, This indicates scaling per channel. Indicates translation along each channel. Indicates a mapping operation. This indicates a macroeconomic control signal. Indicates microscopic intentions. Indicates intermediate features. This indicates channel-by-channel multiplication. This represents the index of dispatchable vehicles.

[0033] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes generating multiple finite candidate sets based on schedulable vehicles, tasks, refueling point information, and micro-action intentions. All macro-control signals, all micro-action intentions, and all finite candidate sets are input into a centralized planner. The finite candidate sets are then filtered using feasibility rules consistent with low-level decision-making. A tripartite graph within the same decision cycle is constructed based on the filtered candidate relationships. Specifically, this includes:

[0034] All macro-control signals and all micro-action intentions are input into a centralized planner for analysis, and a finite candidate set for each schedulable vehicle is output, wherein the finite candidate set includes candidate relationships between the schedulable vehicle and the task, refueling point and migration area respectively;

[0035] The candidate relationships are uniformly screened using feasibility rules consistent with low-level decision-making, and a ternary graph is constructed based on schedulable vehicles, tasks, and refueling points within the same decision-making cycle.

[0036] Based on the regional quota, cost weight template, and destination prior template in the macroeconomic control signal, the candidate edges in the tripartite graph are weighted, and the regional quota is mapped as a constraint condition for unified optimization.

[0037] The centralized planner performs feasibility screening, conflict resolution, and global consistency correction on candidate relationships to generate executable scheduling instructions for the current moment:

[0038] ;

[0039] in, This represents the weighted result of the candidate edges. Indicates candidate edges, Indicates the weighting coefficient. Representing edge feature terms, Indicates the bias weight. This represents the bias term indicating the prior knowledge of the destination.

[0040] Optionally, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein the step of using a centralized planner to perform unified conflict resolution and global correction on the finite candidate set within the same decision period based on the tripartite graph to generate a set of executable scheduling instructions for the current moment specifically includes:

[0041] The continuous operation data of the target vehicle is obtained through a rolling time window. The continuous operation data and the three-part graph of the target vehicle are input into the centralized planner. The centralized planner updates the three-part graph in real time according to the continuous operation data and the constraints.

[0042] For the ternary graph at the current moment, the candidate edges in the ternary graph are checked for state consistency and updated for constraints. Based on the updated candidate edges, the set of executable scheduling instructions for the target vehicle at the current moment is generated.

[0043] Furthermore, to achieve the above objectives, the present invention also provides a vehicle scheduling optimization system that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein the vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes:

[0044] The spatiotemporal graph construction module is used to construct a global spatiotemporal graph based on multiple supply and demand features and multiple related features within the target area, extract regional spatiotemporal embedding vectors from the global spatiotemporal graph, obtain all schedulable vehicles within the target area, and construct a local vehicle representation for each schedulable vehicle.

[0045] The hierarchical collaborative decision-making module is used to generate macro-control signals for a first preset time scale by utilizing the regional spatiotemporal embedding vector, operation information and historical high-level instruction summaries, and to obtain the corresponding micro-action intentions based on the local representation of each vehicle and the macro-control signals at a second preset time scale.

[0046] The global correction module is used to generate multiple finite candidate sets based on schedulable vehicles, tasks, refueling point information and micro-action intentions. All macro-control signals, all micro-action intentions and all finite candidate sets are input into the centralized planner, and the finite candidate sets are filtered using feasibility rules consistent with low-level decision-making. Based on the filtered candidate relationships, a tripartite graph within the same decision cycle is constructed.

[0047] The scheduling generation module, based on the tripartite graph, uses a centralized planner to perform unified conflict resolution and global correction on the finite candidate set within the same decision cycle, generating a set of executable scheduling instructions for the current moment.

[0048] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, stored in the memory and executable on the processor. When the vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning is executed by the processor, it implements the steps of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described above.

[0049] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, and when the vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning is executed by a processor, it implements the steps of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described above.

[0050] In this invention, a global spatiotemporal graph is constructed based on multiple supply and demand features and multiple related features within a target area. A regional spatiotemporal embedding vector is extracted from the global spatiotemporal graph to obtain all schedulable vehicles within the target area, and a local vehicle representation is constructed for each schedulable vehicle. For a first preset time scale, a macro-control signal is generated using the regional spatiotemporal embedding vector, operational information, and historical high-level instruction summaries. At a second preset time scale, corresponding micro-action intentions are obtained based on each vehicle's local representation and the macro-control signal. Multiple finite candidate sets are generated based on schedulable vehicles, tasks, refueling point information, and micro-action intentions. All macro-control signals, all micro-action intentions, and all finite candidate sets are input into a centralized planner, and the finite candidate sets are filtered using feasibility rules consistent with low-level decision-making. A ternary graph is constructed within the same decision period based on the filtered candidate relationships. Based on the ternary graph, the centralized planner performs unified conflict resolution and global correction on the finite candidate sets within the same decision period to generate a set of executable scheduling instructions for the current moment. This invention acquires regional situation and vehicle local interaction information through cross-scale representation, generates macro-control signals from higher levels to guide lower levels to generate action intentions, and introduces a centralized correction mechanism to resolve and correct the feasibility of multi-vehicle conflicts, thereby improving the efficiency of scheduling tasks and the balance of services. Attached Figure Description

[0051] Figure 1 This is a flowchart of a preferred embodiment of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to the present invention;

[0052] Figure 2 This is a system flowchart of a preferred embodiment of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to the present invention;

[0053] Figure 3 This is a structural diagram of a preferred embodiment of the vehicle scheduling optimization system that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to the present invention;

[0054] Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] The vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning described in the preferred embodiment of the present invention, such as... Figure 1As shown, the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes the following steps:

[0057] Step S10: Construct a global spatiotemporal map based on multiple supply and demand features and multiple related features within the target area; extract regional spatiotemporal embedding vectors from the global spatiotemporal map; obtain all schedulable vehicles within the target area; and construct a local vehicle representation for each schedulable vehicle.

[0058] In the embodiments disclosed in this invention, service types such as "orders, service requests, and delivery requests" are collectively referred to as tasks, and service areas such as "charging stations, battery swapping points, and parking / recharging points" are collectively referred to as recharging points. In existing scheduling methods, macro-level regional situational modeling and micro-level real-time decision-making for individual vehicles are often disconnected. Macro-level predictions or rules are difficult to stably map into individual executable actions, while individual learning strategies struggle to fully utilize global periodic patterns and cross-regional control signals. Therefore, this invention discloses a cross-scale joint representation method of a spatiotemporal map of the entire region and a heterogeneous local map of the vehicle center to address this problem.

[0059] Specifically, multiple supply and demand features and multiple association features of the target region are obtained, all the supply and demand features are defined as node attributes, and all the association features are defined as edge attributes, so as to construct a global spatiotemporal graph of the target region;

[0060] Feature extraction is performed on the global spatiotemporal graph in both spatial and temporal dimensions to obtain regional spatiotemporal embedding vectors.

[0061] Each schedulable vehicle within the target area is identified, and a corresponding local heterogeneous graph is constructed for each schedulable vehicle to extract the local vehicle representation of each schedulable vehicle.

[0062] When analyzing a target area (or a city), based on the spatial division of the city (e.g., a hexagonal grid), each sub-grid is defined as a node. The node features (i.e., node attributes) are information such as the number of tasks arriving, the number of tasks completed, the number of available vehicles, the average waiting time, the number of available charging piles at the charging points and their occupancy rate. The average travel time and distance between areas are defined as edge features, thus constructing a regional-level global spatiotemporal map of the target area.

[0063] Furthermore, the global spatiotemporal graph is input into a spatiotemporal graph neural network for feature extraction to obtain global region embeddings;

[0064] A gating fusion mechanism and a smoothing mechanism are introduced into the global spatiotemporal graph to suppress jitter in the global region embedding, and the output region spatiotemporal embedding vector is:

[0065] ;

[0066] ;

[0067] in, Indicates the first Global region embedding for each region Indicates the gating coefficient. Indicating regional embeddings of short-term fluctuations Indicates the region embedding of cyclical fluctuations. Indicates the first The spatiotemporal embedding vector of each region This represents the smoothing coefficient.

[0068] In the spatial dimension, the global region embedding extracted by the spatiotemporal graph neural network can reflect the changing trend of regional supply and demand. In the time dimension, the dual-channel input of short-term and periodic sequences is adopted, and the global region embedding is further processed by gating fusion and smoothing mechanisms to enhance the dimensional matching degree and the subsequent decision expression power.

[0069] Furthermore, vehicle information, candidate task information, and reachable refueling point information for each schedulable vehicle within the target area are obtained;

[0070] The vehicle information, the candidate task information, and the reachable refueling point information are defined as vehicle nodes, task nodes, and refueling point nodes, respectively. The reachability, estimated arrival time, energy consumption, and waiting information of each schedulable vehicle to all the refueling points and task nodes are defined as corresponding edges. A local heterogeneous graph of each schedulable vehicle is constructed based on all the vehicle nodes, task nodes, refueling point nodes, and all the corresponding edges.

[0071] For each of the local heterogeneous graphs, the local heterogeneous graphs are input into a heterogeneous graph neural network for feature extraction, and the local vehicle representation of the schedulable vehicle is output. The local vehicle representation is used as input to a low-level decision state to generate the corresponding micro-action intention.

[0072] Furthermore, in the process of modeling multi-scale spatiotemporal graph fusion, a local heterogeneous graph is constructed within the neighborhood of each schedulable vehicle. The local heterogeneous graph is centered on each schedulable vehicle, and the nodes include vehicle nodes, candidate task nodes, and reachable charging point nodes. The edges explain the relationship between schedulable vehicles and tasks and charging points, such as the time and energy consumption of schedulable vehicles to reach charging points, and the time information of waiting for charging piles. Then, the local heterogeneous graph is analyzed by a heterogeneous graph neural network to construct a local vehicle representation for each schedulable vehicle, which is used as input for low-level decision state to generate corresponding micro-action intentions.

[0073] Specifically, by combining regional-level global spatiotemporal maps with vehicle-centric local heterogeneous maps across scales, macroscopic situations and microscopic interactions can be simultaneously characterized within the same framework. Regional-level global spatiotemporal maps capture large-scale traffic flows, inter-regional connectivity, and the structural characteristics of the overall network, providing rich information for understanding the macroscopic situation of the entire transportation system. Meanwhile, vehicle-centric local heterogeneous maps focus on the interactions between individual vehicles or small groups of vehicles, revealing microscopic behavioral patterns and heterogeneity among vehicles. Through cross-scale joint representation, information at both macroscopic and microscopic levels can be obtained simultaneously, providing a more comprehensive understanding of the transportation system and improving the accuracy and rationality of subsequent scheduling decisions.

[0074] Step S20: For the first preset time scale, a macro-control signal is generated using the regional spatiotemporal embedding vector, operation information, and historical high-level instruction summary. Then, at the second preset time scale, the corresponding micro-action intention is obtained based on the local representation of each vehicle and the macro-control signal.

[0075] Among them, such as Figure 2 As shown, in the embodiments disclosed in this invention, unified scheduling decisions that are both macroscopically controllable and microscopically executable are achieved through different levels of reinforcement learning collaborative decision-making and implicit hierarchical communication.

[0076] Specifically, a decision-making process model is established to obtain historical instruction summaries and operational information of each of the schedulable vehicles in a first preset time scale, wherein the historical instruction summary represents a summary of macro-control signals from the previous macro-decision window;

[0077] The regional spatiotemporal embedding vector, the operational information, and the historical instruction summary are input into the decision-making process model for macro-control, and the macro-control signal corresponding to the current macro-decision window is output.

[0078] The macro-control signal remains stable within a preset window and is used by both the low-level decision-making unit and the centralized planner.

[0079] A low-level graph network at a second preset time scale is obtained. A feature-level linear generator network is used to map the macroscopic control signal into continuous modulation parameters. The intermediate features of the low-level graph network are then fused with the continuous modulation parameters to obtain the microscopic action intent of each schedulable vehicle.

[0080] ;

[0081] ;

[0082] in, Indicates continuous modulation parameters, This indicates scaling per channel. Indicates translation along each channel. Indicates a mapping operation. This indicates a macroeconomic control signal. Indicates microscopic intentions. Indicates intermediate features. This indicates channel-by-channel multiplication. This represents the index of dispatchable vehicles.

[0083] In the embodiments disclosed in this invention, within a first preset timescale (i.e., a rolling decision dimension over a long timescale), global region embedding, operational information of schedulable vehicles (e.g., task completion rate, overall empty-run rate, energy consumption level, etc.), and historical instruction summaries (i.e., the macro-control signal from the previous moment) are used as state inputs. A semi-Markov decision process model is established using the DuelingDouble DQN (Convection Double Deep Q-Network) method for macro-control, outputting a macro-control signal:

[0084] ;

[0085] Where g represents the regional quota (e.g., order acceptance, migration, charging capacity level). represents the cost weight template (e.g., discrete levels of efficiency, revenue, fairness, and energy consumption), and p represents the destination prior template (e.g., removing hotspots, removing low supply-demand ratios, achieving local equilibrium, charging friendliness, and conservatism). The macro-control signal will remain stable in the first preset time scale and will be used by both the lower layer (i.e., the second preset time scale) and the centralized planner.

[0086] Furthermore, in the second preset time scale (i.e., the rolling decision dimension of the short time scale), the local vehicle representation and macro-control signals of the schedulable vehicle are used as state inputs to obtain the micro-action intention of the schedulable vehicle.

[0087] In order to enhance the ability of macro-control signals to transmit information to micro-action intentions, the embodiments disclosed in this invention adopt an implicit communication method. The macro-control signals are mapped to continuous modulation parameters using a feature-level linear generation network, and the intermediate features of the low-level graph network are scaled and biased to obtain a fused representation, so that macro-control can influence the selection of micro-actions in a soft constraint manner.

[0088] The lower layer employs the Dueling Double DQN method to establish a Markov decision process model, with the action set including accepting orders, heading to charging stations, inter-regional migration, and waiting in place. To ensure consistency between training and execution, a feasible domain mask is introduced to uniformly mask actions that violate hard constraints such as the task's SLA (Service-Level Agreement), service radius, SOC (State of Charge) reachability, and station operating rules. Rewards comprehensively consider per-vehicle revenue, passenger waiting time, empty runs, and energy consumption, and penalize competitive empty runs caused by prolonged cruising in hotspot areas without accepting orders to reduce ineffective competition.

[0089] Step S30: Generate multiple finite candidate sets based on schedulable vehicles, tasks, refueling point information and micro-action intentions. Input all the macro-control signals, all the micro-action intentions and all the finite candidate sets into the centralized planner, and use feasibility rules consistent with low-level decision-making to filter the finite candidate sets. Construct a tripartite graph within the same decision cycle based on the filtered candidate relationships.

[0090] Under conditions such as energy constraints, energy replenishment resource capacity and availability constraints, task timeliness constraints, and multi-vehicle conflict constraints, pure learning strategies are prone to generating unexecutable or conflicting instructions. While pure optimization methods can explicitly handle constraints, they face significant solution pressure in rolling real-time decision-making. Therefore, this invention uses a centralized correction module to perform feasibility checks and conflict resolution on multi-vehicle intentions, and outputs directly executable scheduling instructions.

[0091] Specifically, all the macro-control signals and all the micro-action intentions are input into the centralized planner for analysis, and a finite candidate set for each schedulable vehicle is output, wherein the finite candidate set includes the candidate relationships between the schedulable vehicle and the task, refueling point and migration area respectively.

[0092] The candidate relationships are uniformly screened using feasibility rules consistent with low-level decision-making, and a ternary graph is constructed based on schedulable vehicles, tasks, and refueling points within the same decision-making cycle.

[0093] Based on the regional quota, cost weight template, and destination prior template in the macroeconomic control signal, the candidate edges in the tripartite graph are weighted, and the regional quota is mapped as a constraint condition for unified optimization.

[0094] The centralized planner performs feasibility screening, conflict resolution, and global consistency correction on candidate relationships to generate executable scheduling instructions for the current moment:

[0095] ;

[0096] in, This represents the weighted result of the candidate edges. Indicates candidate edges, Indicates the weighting coefficient. Representing edge feature terms, Indicates the bias weight. This represents the bias term indicating the prior knowledge of the destination.

[0097] Among them, the edge characteristics for different types of candidate edges include: the expected arrival time, arrival waiting time, timeout risk, energy consumption, average hourly revenue, and fairness penalty for the order-receiving edge; the expected arrival time to the charging station, charging and queuing time, energy consumption, electricity price, and power for the charging edge; and the expected arrival time, energy consumption, and prior information on the demand intensity and supply-demand gap in the destination area for the migration edge.

[0098] Driven by both macro-level control signals from higher levels and micro-level action intentions from lower levels, the centralized planner generates a finite candidate set for each schedulable vehicle (including the relationships between schedulable vehicles and tasks, replenishment points, and migration regions). It further utilizes a masking function library consistent with the lower-level graph network for feasibility screening, avoiding training-execution drift. For example, in a certain lower-level decision step, for a schedulable vehicle, multiple accessible tasks, reachable replenishment points, and adjacent regions corresponding to the lower-level vehicle's intention are selected as migration candidates within its neighborhood. Then, the same set of feasibility rules as in the training phase are used to filter out unexecutable candidates, thereby avoiding inconsistencies between the training and execution action sets.

[0099] Furthermore, the dispatchable vehicles, tasks, and refueling points within the same decision-making cycle are aggregated into a three-part graph, and the regional quota g and cost weight template in the macro-control signals are utilized. Optimize with the prior template p of the destination, so as to The weighted cost is the main factor, with the destination prior template superimposed. The regional quota is mapped to the regional capacity constraint, and then the candidate edges in the tripartite graph (where the candidate edges connect three types of relationships: vehicle-task, vehicle-refueling point, and vehicle-region migration) are optimized by weighting.

[0100] In the first phase, the centralized planner performs minimum cost flow for order acceptance or assigns tasks to complete vehicle matching. In the second phase, it assigns charging and migration options to unassigned vehicles to meet regional quotas and site capacity. Finally, it performs fine-tuning of conflict subsets using small-scale integer planning and provides timeout rollback (maintaining the state of unassigned vehicles and using the previous feasible plan) to meet the time delay budget of rolling decision-making. Based on the hierarchical strategy output intent, this invention introduces a centralized feasibility correction and conflict resolution module to perform consistent filtering, conflict resolution, and global consistency correction on multi-vehicle candidate intents. This ensures that the output instructions meet the executable conditions such as unique execution for a single vehicle, availability of energy and recharging resources, and task timeliness, thereby reducing invalid migrations and recharging waits at the execution level and improving overall operational efficiency.

[0101] Step S40: Based on the tripartite graph, the centralized planner is used to perform unified conflict resolution and global correction on the finite candidate set within the same decision cycle, generating the executable scheduling instruction for the current moment.

[0102] Specifically, the continuous operation data of the target vehicle is obtained through a rolling time window, and the continuous operation data and the three-part graph of the target vehicle are input into the centralized planner. The centralized planner updates the three-part graph in real time according to the continuous operation data and the constraints.

[0103] For the ternary graph at the current moment, the candidate edges in the ternary graph are checked for state consistency and updated for constraints. Based on the updated candidate edges, the set of executable scheduling instructions for the target vehicle at the current moment is generated.

[0104] During the deployment phase, when the vehicle is running, macro-control signals and micro-action intentions are periodically generated according to the rolling time window, and then issued for execution after centralized correction; and the operation data is continuously recorded for periodic evaluation and model updates to adapt to the time-varying characteristics of demand and energy supply.

[0105] Furthermore, based on all the historical operational data that has been generated, a task revisit sequence can be constructed and simulated. A global representation model and a hierarchical reinforcement learning strategy can be jointly trained, and relevant parameters can be centrally corrected. These training and optimization processes can be verified through simulation during the verification phase.

[0106] During the verification phase, the effectiveness of the method is evaluated through multi-scenario simulation or offline playback. Comparisons and ablation analyses can be conducted from dimensions such as task completion rate, revenue, efficiency, ineffective driving and waiting, refueling congestion, and spatial service balance. Through continuous iterative optimization, the ineffective driving and waiting time of schedulable vehicles can be reduced, and the task completion efficiency and service balance can be improved.

[0107] This invention acquires regional situation and vehicle local interaction information through cross-scale representation, generates macro-control signals from higher levels to guide lower levels to generate action intentions, and introduces a centralized correction mechanism to resolve and correct the feasibility of multi-vehicle conflicts, thereby improving the efficiency of scheduling tasks and the balance of services.

[0108] Furthermore, such as Figure 3 As shown, based on the above-mentioned vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, this invention also provides a vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, wherein the vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes:

[0109] The spatiotemporal graph construction module 51 is used to construct a global spatiotemporal graph based on multiple supply and demand features and multiple related features in the target area, extract regional spatiotemporal embedding vectors from the global spatiotemporal graph, obtain all schedulable vehicles in the target area, and construct a vehicle local representation for each schedulable vehicle.

[0110] The hierarchical collaborative decision-making module 52 is used to generate macro-control signals for a first preset time scale using the regional spatiotemporal embedding vector, operation information and historical high-level instruction summary, and to obtain the corresponding micro-action intentions based on the local representation of each vehicle and the macro-control signals at a second preset time scale.

[0111] The global correction module 53 generates multiple finite candidate sets based on schedulable vehicles, tasks, refueling point information and micro-action intentions. It inputs all the macro-control signals, all the micro-action intentions and all the finite candidate sets into the centralized planner, and uses feasibility rules consistent with low-level decision-making to filter the finite candidate sets. Based on the filtered candidate relationships, it constructs a tripartite graph within the same decision cycle.

[0112] The scheduling generation module 54, based on the tripartite graph, uses a centralized planner to perform unified conflict resolution and global correction on the finite candidate set within the same decision cycle, and generates an executable scheduling instruction for the current moment.

[0113] Furthermore, such as Figure 4 As shown, based on the above-mentioned vehicle scheduling optimization method and system that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0114] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a vehicle scheduling optimization program 40 that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. This vehicle scheduling optimization program 40 can be executed by the processor 10, thereby implementing the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning in this application.

[0115] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning.

[0116] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0117] In one embodiment, when the processor 10 executes the vehicle scheduling optimization program 40 that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning in the memory 20, it implements the steps of the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described above.

[0118] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, and when the vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning is executed by a processor, it implements the steps of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described above.

[0119] In summary, this invention provides a vehicle scheduling optimization method and related equipment that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. The method includes: constructing a global spatiotemporal graph based on multiple supply and demand features and multiple correlation features within a target area; extracting regional spatiotemporal embedding vectors from the global spatiotemporal graph; obtaining all schedulable vehicles within the target area; and constructing a local vehicle representation for each schedulable vehicle. For a first preset time scale, a macro-control signal is generated using the regional spatiotemporal embedding vectors, operational information, and historical high-level instruction summaries. At a second preset time scale, the corresponding micro-action intent is obtained based on the local vehicle representation and the macro-control signal. Based on schedulable vehicles, tasks, and refueling point information... Multiple finite candidate sets are generated from macro-control signals and micro-action intentions. All macro-control signals, micro-action intentions, and finite candidate sets are input into a centralized planner, and the finite candidate sets are filtered using feasibility rules consistent with low-level decision-making. A tripartite graph is constructed based on the filtered candidate relationships within the same decision-making period. Based on the tripartite graph, the centralized planner performs unified conflict resolution and global correction on the finite candidate sets within the same decision-making period to generate executable scheduling instructions for the current moment. The continuous operation data and the tripartite graph are input into the centralized planner, and the centralized planner, according to a rolling time window, selects the set of executable scheduling instructions for the target vehicle at the current moment from the tripartite graph. This invention obtains regional situation and local vehicle interaction information through cross-scale representation, uses macro-control signals generated by a high-level layer to guide low-level action intentions, and introduces a centralized correction mechanism to resolve and correct multi-vehicle conflicts, thereby improving the efficiency of scheduling task completion and service balance.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0121] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0122] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, characterized in that, The vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes: A global spatiotemporal map is constructed based on multiple supply and demand features and multiple correlation features within the target area. Regional spatiotemporal embedding vectors are extracted from the global spatiotemporal map to obtain all schedulable vehicles within the target area, and a local vehicle representation of each schedulable vehicle is constructed. For a first preset time scale, a macro-control signal is generated using the regional spatiotemporal embedding vector, operational information, and historical high-level instruction summaries. Then, at a second preset time scale, the corresponding micro-action intent is obtained based on the local representation of each vehicle and the macro-control signal, specifically including: A decision-making process model is established to obtain historical high-level instruction summaries and operational information of each schedulable vehicle within a first preset time scale, wherein the historical high-level instruction summaries represent macro-control signal summaries from the previous macro-decision window; The regional spatiotemporal embedding vector, the operational information, and the historical high-level instruction summary are input into the decision-making process model for macro-control, and the macro-control signal corresponding to the current macro-decision window is output. The macro-control signal remains stable within a preset window and is used by both the low-level decision-making and centralized planners. A low-level graph network at a second preset time scale is obtained. A feature-level linear generator network is used to map the macroscopic control signal into continuous modulation parameters. The intermediate features of the low-level graph network are then fused with the continuous modulation parameters to obtain the microscopic action intent of each schedulable vehicle. ； ； in, Indicates continuous modulation parameters, This indicates scaling per channel. Indicates translation along each channel. Indicates a mapping operation. This indicates a macroeconomic control signal. Indicates microscopic intentions. Indicates intermediate features. This indicates channel-by-channel multiplication. v An index representing the available vehicles; Based on information on schedulable vehicles, tasks, refueling points, and micro-level action intentions, multiple finite candidate sets are generated. All macro-level control signals, all micro-level action intentions, and all finite candidate sets are input into a centralized planner. Feasibility rules consistent with lower-level decision-making are used to filter the finite candidate sets. Based on the filtered candidate relationships, a tripartite graph is constructed for the same decision cycle, specifically including: All macro-control signals and all micro-action intentions are input into a centralized planner for analysis, and a finite candidate set for each schedulable vehicle is output, wherein the finite candidate set includes candidate relationships between the schedulable vehicle and the task, refueling point and migration area respectively; The candidate relationships are uniformly screened using feasibility rules consistent with low-level decision-making, and a ternary graph is constructed based on schedulable vehicles, tasks, and refueling points within the same decision-making cycle. Based on the regional quota, cost weight template, and destination prior template in the macroeconomic control signal, the candidate edges in the tripartite graph are weighted, and the regional quota is mapped as a constraint condition for unified optimization. The centralized planner performs feasibility screening, conflict resolution, and global consistency correction on candidate relationships to generate executable scheduling instructions for the current moment: ； in, This represents the weighted result of the candidate edges. e Indicates candidate edges, Indicates the weighting coefficient. Representing edge feature terms, Indicates the bias weight. The bias term represents the prior information about the destination; Based on the aforementioned tripartite graph, a centralized planner is used to perform unified conflict resolution and global correction on the finite candidate set within the same decision cycle, generating a set of executable scheduling instructions for the current moment.

2. The vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to claim 1, characterized in that, The process of constructing a global spatiotemporal map based on multiple supply and demand features and multiple correlation features within the target area, extracting regional spatiotemporal embedding vectors from the global spatiotemporal map, obtaining all schedulable vehicles within the target area, and constructing a local vehicle representation for each schedulable vehicle specifically includes: Multiple supply and demand features and multiple association features of the target region are obtained, all the supply and demand features are defined as node attributes, and all the association features are defined as edge attributes, so as to construct a global spatiotemporal graph of the target region; Feature extraction is performed on the global spatiotemporal graph in both spatial and temporal dimensions to obtain regional spatiotemporal embedding vectors. Each schedulable vehicle within the target area is identified, and a corresponding local heterogeneous graph is constructed for each schedulable vehicle to extract the local vehicle representation of each schedulable vehicle.

3. The vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to claim 2, characterized in that, The step of extracting features from the global spatiotemporal map in both spatial and temporal dimensions to obtain a regional spatiotemporal embedding vector specifically includes: The global spatiotemporal graph is input into a spatiotemporal graph neural network for feature extraction to obtain global region embeddings. A gating fusion mechanism and a smoothing mechanism are introduced into the global spatiotemporal graph to suppress jitter in the global region embedding, and the output region spatiotemporal embedding vector is: ； ； in, Indicates the first i Global region embedding for each region Indicates the gating coefficient. Indicating regional embeddings of short-term fluctuations Indicates the region embedding of cyclical fluctuations. Indicates the first i The spatiotemporal embedding vector of each region This represents the smoothing coefficient.

4. The vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to claim 2, characterized in that, The step of determining each schedulable vehicle within the target area and constructing a corresponding local heterogeneous graph for each schedulable vehicle to extract the local vehicle representation for each schedulable vehicle specifically includes: Obtain vehicle information, candidate task information, and reachable refueling point information for each schedulable vehicle within the target area; The vehicle information, the candidate task information, and the reachable refueling point information are defined as vehicle nodes, task nodes, and refueling point nodes, respectively. The reachability, estimated arrival time, energy consumption, and waiting information of each schedulable vehicle to all the refueling points and task nodes are defined as corresponding edges. A local heterogeneous graph of each schedulable vehicle is constructed based on all the vehicle nodes, task nodes, refueling point nodes, and all the corresponding edges. For each of the local heterogeneous graphs, the local heterogeneous graphs are input into a heterogeneous graph neural network for feature extraction, and the local vehicle representation of the schedulable vehicle is output. The local vehicle representation is used as input to a low-level decision state to generate the corresponding micro-action intention.

5. The vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning according to claim 1, characterized in that, Based on the tripartite graph, a centralized planner is used to uniformly resolve conflicts and globally correct the finite candidate set within the same decision cycle, generating a set of executable scheduling instructions for the current moment. Specifically, this includes: The continuous operation data of the target vehicle is obtained through a rolling time window. The continuous operation data and the three-part graph of the target vehicle are input into the centralized planner. The centralized planner updates the three-part graph in real time according to the continuous operation data and the constraints. For the ternary graph at the current moment, the candidate edges in the ternary graph are checked for state consistency and updated for constraints. Based on the updated candidate edges, the set of executable scheduling instructions for the target vehicle at the current moment is generated.

6. A vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning, characterized in that, The vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning is used to implement the vehicle scheduling optimization method integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described in any one of claims 1-5. The vehicle scheduling optimization system integrating multi-scale spatiotemporal graphs and hierarchical reinforcement learning includes: The spatiotemporal graph construction module is used to construct a global spatiotemporal graph based on multiple supply and demand features and multiple related features within the target area, extract regional spatiotemporal embedding vectors from the global spatiotemporal graph, obtain all schedulable vehicles within the target area, and construct a local vehicle representation for each schedulable vehicle. The hierarchical collaborative decision-making module is used to generate macro-control signals for a first preset time scale by utilizing the regional spatiotemporal embedding vector, operation information and historical high-level instruction summaries, and to obtain the corresponding micro-action intentions based on the local representation of each vehicle and the macro-control signals at a second preset time scale. The global correction module is used to generate multiple finite candidate sets based on schedulable vehicles, tasks, refueling point information and micro-action intentions. All macro-control signals, all micro-action intentions and all finite candidate sets are input into the centralized planner, and the finite candidate sets are filtered using feasibility rules consistent with low-level decision-making. Based on the filtered candidate relationships, a tripartite graph within the same decision cycle is constructed. The scheduling generation module is used to generate a set of executable scheduling instructions for the current moment by using a centralized planner to perform unified conflict resolution and global correction on the finite candidate set within the same decision period based on the tripartite graph.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning, stored in the memory and executable on the processor. When the vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning is executed by the processor, it implements the steps of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning. When the vehicle scheduling optimization program that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning is executed by a processor, it implements the steps of the vehicle scheduling optimization method that integrates multi-scale spatiotemporal graphs and hierarchical reinforcement learning as described in any one of claims 1-5.

Images