Multi-Agent Based Multi-Hub Dynamic Dispatching System for PBV Platoon Shuttles and Dispatching Method Thereof

Abstract

A multi-base dynamic dispatch system for a cluster shuttle according to one embodiment comprises: an agent unit including a plurality of base agents corresponding to each of a plurality of traffic bases and calculating the number of vehicles to be dispatched from a traffic base to another traffic base based on local observation information related to vehicle dispatch at the respective traffic base; wherein, during inference, each of the plurality of base agents independently calculates the number of vehicles to be dispatched based on the local observation information of the respective traffic base.

Description

Technology Field

[0001] The present invention relates to a vehicle dispatching technology in a cluster shuttle network comprising a plurality of transportation hubs, and more specifically, to a multi-agent-based multi-hub dynamic dispatching system for a PBV cluster shuttle and a dispatching method, wherein a hub agent corresponding to each of the plurality of transportation hubs determines vehicle dispatching between transportation hubs based on local observation information and organizes empty vehicles into a cluster formation for return. Background Technology

[0002] In the field of Autonomous Mobility-on-Demand (AMoD) services utilizing autonomous vehicles, fleet imbalance caused by the asymmetry between passengers' origins and destinations is a key factor degrading service quality. Over time, a phenomenon inevitably occurs where vehicles become overconcentrated at specific locations while others become undercapacitated, requiring rebalancing technology to resolve this issue.

[0003] In this regard, existing vehicle relocation technologies have the following limitations.

[0004] First, existing AMoD redeployment research focuses on redeployment at the individual vehicle level, such as for taxis and ride-sharing vehicles. However, in shuttle networks operating in swarm formations, performing redeployment at the swarm formation level is advantageous in terms of energy efficiency and road occupancy. When multiple empty vehicles are returned individually, energy consumption and road occupancy increase because each vehicle must independently plan and drive its route. On the other hand, if vehicles are organized into swarm formations and returned collectively, energy savings and minimized road occupancy can be achieved by having only the lead vehicle plan the route while subsequent vehicles follow. However, existing technologies do not address redeployment at the swarm formation level.

[0005] Second, existing research deals only with vehicles of the same type and does not consider the vehicle's module type during redeployment. Since modular vehicles can be converted for various uses, such as passenger or cargo transport, by replacing the upper modules, it is necessary to predict the module type required for the next train set at the destination hub and prioritize the return of vehicles of that type during redeployment. For example, if three passenger modules and one cargo module are required at the destination hub, four vehicles should not be sent randomly but must be sent with a matching module type configuration. However, existing technology treats vehicles as homogeneous, so this module type matching is not considered.

[0006] Third, existing relocation algorithms are designed for centralized optimization, which leads to a rapid increase in the scale of decision variables as the number of bases increases, thereby degrading scalability. Since the centralized approach handles relocation decisions for all base pairs simultaneously, the scale of decision variables grows rapidly with the increase in the number of bases. In contrast, in a distributed structure where agents at each base perform relocation decisions based on their own bases, the scale of decisions directly handled by each agent can be mitigated. Additionally, in a centralized structure, information from all bases must be collected centrally during execution, resulting in communication load, and the entire system may be paralyzed in the event of a communication failure.

[0007] Fourth, existing redeployment decisions do not sufficiently consider remaining battery capacity. In electric vehicle-based shuttle networks, redeploying a vehicle with insufficient remaining battery capacity relative to the return distance creates a risk of discharge along the route. While filtering is required to determine availability by combining remaining battery capacity and return distance, existing technologies either do not consider remaining battery capacity or apply only simple fixed criteria.

[0008] Accordingly, there is a need for a dynamic dispatch technology that overcomes the limitations of the aforementioned conventional technology and can efficiently resolve vehicle concentration between hubs in a cluster shuttle network composed of multiple transportation hubs. The problem to be solved

[0009] The present invention aims to solve the above-mentioned problems by providing a multi-agent-based multi-hub dynamic dispatch system for PBV cluster shuttles and a dispatch method thereof, wherein, in a cluster shuttle network composed of multiple transportation hubs, a hub agent corresponding to each transportation hub calculates the number of vehicles to be dispatched to other transportation hubs based on local observation information, and organizes empty vehicles into cluster formations for return, thereby efficiently resolving vehicle imbalance between hubs. means of solving the problem

[0010] A multi-base dynamic dispatch system for a cluster shuttle according to one embodiment comprises: an agent unit including a plurality of base agents corresponding to each of a plurality of traffic bases and calculating the number of vehicles to be dispatched from a traffic base to another traffic base based on local observation information related to vehicle dispatch at the respective traffic base; wherein, during inference, each of the plurality of base agents independently calculates the number of vehicles to be dispatched based on the local observation information of the respective traffic base.

[0011] The above local observation information includes at least two of the following: idle vehicle information of the relevant transportation hub, demand forecast information for vehicles of the relevant transportation hub, access information to other transportation hubs, and current time information.

[0012] In addition, the above idle vehicle information includes the number of idle vehicles at the relevant transportation hub, the number of passenger module vehicles among the idle vehicles, the number of cargo module vehicles among the idle vehicles, information on the remaining battery levels of the idle vehicles, the number of dispatched vehicles moving to the relevant hub, and the number of dispatched vehicles departing from the relevant hub and moving.

[0013] In addition, each of the aforementioned multiple base agents is configured such that the total number of vehicles to be dispatched is less than or equal to the value obtained by subtracting a preset minimum number of reserve vehicles from the number of idle vehicles at the corresponding transportation base. Effects of the invention

[0014] According to the multi-agent-based multi-base dynamic dispatch system and dispatch method of the PBV cluster shuttle of the present invention, in a cluster shuttle network composed of a plurality of traffic bases, a base agent corresponding to each traffic base calculates the number of vehicles to be dispatched to other traffic bases based on local observation information, and organizes empty vehicles into a cluster formation and returns them, thereby efficiently resolving vehicle imbalance between bases. Brief explanation of the drawing

[0015] FIG. 1 is a configuration diagram of a dynamic dispatch system according to one embodiment. Figure 2 is a configuration table of local observation information for each base agent. Figure 3 is an explanatory diagram of the structure and constraints of the action space. Figure 4 is an explanatory diagram of the cluster formation unit return process. Figure 5 is an explanatory diagram of the network structure during learning. Figure 6 is an explanatory diagram of the composition of the compensation function. Figure 7 is an explanatory diagram of the dispatch decision cycle. Figure 8 is an explanatory diagram for comparing scalability with increasing number of bases. FIG. 9 is an explanatory diagram for comparing dispatching problems in a two-base structure and a multi-base structure. Figure 10 is an explanatory diagram of an example dispatch scenario in a four-base network. FIG. 11 is a flowchart of a dynamic dispatch method according to one embodiment. Specific details for implementing the invention

[0016] Hereinafter, a multi-agent-based multi-base dynamic dispatch system for a PBV cluster shuttle and a dispatch method according to an embodiment of the present invention will be described in detail with reference to the attached drawings. It should be understood that the following embodiments of the present invention are merely intended to embody the present invention and do not limit or restrict the scope of the rights of the present invention. Anything that can be easily inferred by a person skilled in the art to which this disclosure pertains from the detailed description and embodiments of the present invention is interpreted as falling within the scope of the rights of the present invention.

[0018] The key terms used in this specification are defined as follows.

[0019] A transportation hub refers to a physical location where a swarm of shuttles departs, arrives, or waits, and may include stations, bus stops, depots, etc.

[0020] A platoon refers to a group of vehicles organized such that follow vehicles follow the driving path of a lead vehicle.

[0021] An empty vehicle refers to a vehicle that is not loaded with passengers or cargo.

[0022] Return transport refers to moving an empty vehicle from a starting point to a destination point.

[0023] A Hub Agent refers to a software decision-making entity that calculates the number of vehicles to be dispatched based on local observation information of each traffic hub. The Hub Agent may be executed on a device located at the traffic hub or on a remote server.

[0024] The demand forecasting unit refers to a configuration that forecasts future demand for each transportation hub and provides the forecasted demand information to multiple hub agents.

[0025] Local observation information refers to information used for dispatching decisions at specific transportation hubs.

[0026] Idle vehicles refer to vehicles that are waiting at a specific transportation hub without being put into operation.

[0027] A module type refers to the type of module mounted on the upper part of a modular vehicle according to its intended use. A passenger module is a module configured for the boarding and transport of passengers, and a cargo module is a module configured for the loading and transport of cargo. In addition to passenger and cargo modules, a module type may include modules for other purposes.

[0028] Dispatch refers to the decision and execution of moving vehicles from one traffic hub to another, and includes the rebalancing of empty vehicles.

[0029] Dispatch decision refers to calculating the number of vehicles to be dispatched from a specific transportation hub to another transportation hub.

[0030] Global status information refers to integrated status information at the entire network level, including vehicle distribution at each of multiple transportation hubs, vehicle status by module type, battery status, future demand forecasts, and ongoing dispatch status, and can be utilized during training. In other words, global status information signifies comprehensive information regarding the current and projected operational status of the entire hub network.

[0031] The expected reward value for each action refers to the estimate of the reward predicted to be obtained cumulatively in the future when a specific action is taken at a specific point in time.

[0032] The behavioral expected reward calculation network refers to a neural network in which each base agent receives local observation information as input and calculates a behavioral expected reward for each possible dispatch vector. It is used during both training and inference.

[0033] A mixing network refers to a neural network that calculates an expected reward value from a network perspective by combining the behavioral expected reward values ​​calculated by each of multiple base agents.

[0034] A hypernetwork refers to a neural network that takes global state information as input and dynamically generates weights or biases for a blended network.

[0035] The reward function serves as an evaluation criterion for a base agent's dispatch decision; it refers to a value calculated retrospectively from the environment after the dispatch decision is executed and is utilized during training.

[0036] A fleet refers to the entire fleet of vehicles deployed at a specific transportation hub.

[0038] A dynamic dispatch system (100) according to one embodiment may be configured to include at least one computing device comprising at least one processor, memory, and communication device. The memory contains instructions executable by at least one processor, and at least one processor executes the instructions.

[0040] Components expressed as '~parts' in this specification may be implemented in hardware, in software, or in a combination thereof. When implemented in software, the components are implemented as a set of instructions or a program executable by at least one processor included in the dynamic dispatch system, which may be stored on a non-transitory computer-readable recording medium. Additionally, these components may be implemented in the form of routines, subroutines, objects, threads, or a combination thereof.

[0042] FIG. 1 shows a configuration diagram of a dynamic dispatch system (100) according to one embodiment.

[0043] Referring to FIG. 1, a dynamic dispatch system (100) according to one embodiment may be configured to include a demand forecasting unit (110), an agent unit (120), and a dispatch execution unit (130).

[0045] The agent unit (120), demand forecasting unit (110), and dispatch execution unit (130) constituting the dynamic dispatch system (100) may be executed on a single computing device or distributed across multiple computing devices. For example, in one embodiment, multiple base agents (121(1) to 121(K)), demand forecasting unit (110), and dispatch execution unit (130) may be executed collectively on a single central server. In another embodiment, each base agent (121(1) to 121(K)) may be executed on an edge computing device located at the corresponding transportation hub, while the demand forecasting unit (110) and dispatch execution unit (130) may be executed on a central server. In yet another embodiment, a hybrid structure may be applied in which some base agents (121(1) to 121(K)) are executed on local devices, and the remaining base agents (121(1) to 121(K)) are executed on a central server. In the present invention, the fact that each base agent (121(1) to 121(K)) independently performs dispatch decisions based on local observation information of its own traffic base during inference means that, in terms of the algorithm structure, each agent (121(1) to 121(K)) makes decisions based only on its own base information without referring to the information of other agents (121(1) to 121(K)), and does not necessarily mean that it must be executed on a physically separate device.

[0047] The demand forecasting unit (110) forecasts future demand for each transportation hub and provides the forecasted demand forecasting information to multiple hub agents (121(1) to 121(K)).

[0048] The demand forecasting unit (110) can predict future demand by using at least one of the following as input: past operation history data of each transportation hub, demand patterns by time of day, information on days of the week and holidays, weather information, and surrounding traffic condition information. In one embodiment, the demand forecasting unit (110) can use past boarding and alighting history, reservation information, and arrival schedules of public transportation (KTX, subway, etc.) near each transportation hub as input.

[0049] The demand forecasting information calculated by the demand forecasting unit (110) may include demand forecast values ​​for multiple future time steps for each transportation hub, as well as demand for passenger modules and demand for cargo modules. In one embodiment, the demand forecasting unit (110) calculates demand forecast values ​​for six future time steps (one future hour) for each transportation hub at 10-minute intervals.

[0050] In one embodiment, the demand forecasting unit (110) can forecast demand using a machine learning-based time series forecasting model with inputs such as past operation history data, demand patterns by time period, and weather information. The demand forecasting method of the demand forecasting unit (110) is not limited to a specific model.

[0051] By including demand forecast information in the local observation information of base agents (121(1) to 121(K)), each base agent (121(1) to 121(K)) can make proactive dispatch decisions that consider not only the vehicle status at the current time but also future demand fluctuations. For example, even if there are enough idle vehicles at G1 at the current time, if it is predicted that demand will surge due to the arrival of the KTX in 30 minutes, this can be reflected in advance to predetermine dispatch from other bases. This is differentiated from a reactive dispatch method that responds after demand becomes insufficient.

[0053] The agent unit (120) includes a plurality of base agents (121(1) to 121(K)) corresponding to each of the plurality of traffic bases. Each base agent (121(1) to 121(K)) calculates the number of vehicles to be dispatched from the corresponding traffic base to another traffic base based on local observation information related to vehicle dispatch at the corresponding traffic base.

[0054] In one embodiment, when the dynamic dispatch system (100) is composed of K traffic hubs (G1, G2, ..., GK), the agent unit (120) includes K hub agents (121(1) to 121(K)). For example, when K=4, four hub agents (121(1) to 121(4)) corresponding to each traffic hub of G1, G2, G3, and G4 are included in the agent unit (120).

[0055] Each base agent (121(1) to 121(K)) independently makes dispatch decisions based on local observation information of its own traffic base during inference. Here, "independent" means that each base agent (121(1) to 121(K)) calculates the number of vehicles to be dispatched using only local observation information obtained from its own traffic base, without referring to the current observation information or dispatch decision results of other base agents.

[0057] Figure 2 shows a configuration table of local observation information for each base agent.

[0058] Referring to FIG. 2, local observation information may include idle vehicle information at the traffic hub, demand forecast information for vehicles at the traffic hub, access information to other traffic hubs, and current time information.

[0059] Idle vehicle information is the number of idle vehicles (n) at the relevant transportation hub. idle,k ), number of passenger module vehicles among idle vehicles (n pax,k ) and number of cargo module vehicles (n cargo,k ), average battery level of idle vehicles (bat mean,k) and minimum battery level (bat min,k ), number of dispatched vehicles en route to the corresponding base (n incoming,k ), and the number of dispatched vehicles (n) moving from the corresponding base outgoing,k It may include ).

[0060] Demand forecast information includes demand forecast values ​​for future T timesteps of the relevant transportation hub ( ), and demand for passenger modules among future forecasted demand ( ) and demand for cargo modules( It may include ). In one embodiment, T is set to 6 to predict demand for the next hour (6 intervals of 10 minutes). Demand prediction information may be received from the demand prediction unit (110).

[0061] Access information to other transportation hubs includes travel time from that hub to each other transportation hub (travel time,k→j It can include ). When K=4, since the travel time to 3 bases excluding one's own base is included, K-1 becomes 3-dimensional.

[0062] The current time information can be expressed in two dimensions of sin(2π·h / 24) and cos(2π·h / 24) as a cyclic encoded value of the current time. This reflects the periodic characteristics of demand patterns by time of day, such as commuting hours and late-night hours.

[0063] In one embodiment, when K=4 bases and T=6 prediction intervals, the dimension of the local observation information of each base agent (121(1)–121(K)) becomes 11 + T + (K-1) = 11 + 6 + 3 = 20 dimensions. This local observation information constitutes the state space of each base agent. Each base agent receives an observation vector belonging to the state space as input and selects a dispatch vector in the action space. The selected dispatch vector is the action a of the base agent at that time point k(t) is the number of vehicles to be dispatched from the relevant hub to other transportation hubs. .

[0065] Figure 3 shows an explanatory diagram of the structure and constraints of the action space.

[0066] Referring to FIG. 3, the number of vehicles to be dispatched calculated by each base agent (121(1) to 121(K)) consists of a vector (hereinafter referred to as a dispatch vector) representing the number of vehicles to be dispatched from the corresponding base to each other transportation base. In one embodiment, the dispatch vector of base agent k is a k = [r k→1 , r k→2 , ..., r k→K It is expressed as ], and the number of dispatched vehicles to the base (r k→k ) is always 0. Each r k→j is an integer greater than or equal to 0 and represents the number of vehicles to be dispatched from base k to base j.

[0067] In one embodiment, each r k→j is from 0 to a predetermined maximum value (n_ max_rebal Discrete values ​​up to , e.g., 5) are taken. When K=4, each agent calculates a value from 0 to 5 for each of the three directions excluding its own base, so the number of possible dispatch vector combinations is 6³ = 216. All of these possible dispatch vector combinations constitute the action space of each base agent. Each base agent selects one of the 216 dispatch vectors belonging to the action space to perform a dispatch decision.

[0069] Each base agent (121(1) to 121(K)) applies the following constraints when calculating the number of vehicles to be dispatched.

[0070] The first constraint is the total quantity constraint. The total number of vehicles to be dispatched is the minimum number of reserve vehicles (n) pre-set from the number of idle vehicles at the corresponding transportation hub. reserveThe value excluding ) must be less than or equal to. The minimum number of reserve vehicles is the minimum number of vehicles that must be maintained in preparation for the occurrence of emergency demand at the base after dispatch. For example, if G2 has 5 idle vehicles and the minimum number of reserve vehicles is 1, the total number of vehicles that can be dispatched from G2 to other bases is limited to 4 or less.

[0071] The second restriction is the battery restriction. Vehicles with a battery level below the standard required for travel to the destination transportation hub are excluded from dispatch. Standard battery level (bat threshold ) is calculated based on the travel distance to the destination transportation hub. In one embodiment, bat threshold (d) = d × e km / bat capacity It can be calculated as + margin. Here, d is the travel distance, e km energy consumption per km, BATC apacity is the total battery capacity, and margin is the safety margin. For example, if the distance from G2 to G1 is 10km and the standard battery level is 15%, vehicles with a battery level of less than 15% are excluded from dispatching from G2 to G1 and are switched to waiting for charging. Since the standard battery level varies depending on the destination hub even for the same vehicle, there may be cases where dispatching to a nearby hub is possible but dispatching to a distant hub is not.

[0072] When a constraint is violated, each base agent (121(1) to 121(K)) performs a correction. In one embodiment, clipping may be applied to reduce the number of dispatched vehicles in order of the lowest expected reward value for each target base direction. For example, if a G2 agent (121(2)) intends to dispatch a total of 5 vehicles, including 3 to G1 and 2 to G4, but can only dispatch up to 4 vehicles due to a total quantity constraint, the expected reward values ​​for the G1 direction and the G4 direction are compared, and the number of vehicles is reduced starting from the direction with the lower expected reward value. In another embodiment, action masking may be applied to block the selection itself in advance by setting the expected reward value of a dispatch vector that violates the constraint to negative infinity.

[0074] In one embodiment, each base agent (121(1) to 121(K)) may include a network for calculating expected reward values ​​for each action (121a(1) to 121a(K)). The network for calculating expected reward values ​​for each action (121a(1) to 121a(K)) may be configured as a recurrent neural network structure, for example, as a GRU (Gated Recurrent Unit)-based recurrent network, and may calculate the number of vehicles to be dispatched at the current time point by using local observation information at the current time point together with the number of vehicles to be dispatched calculated at the previous time point. By using the dispatch results from the previous time point together as input, duplicate dispatching can be prevented by considering vehicles that were already dispatched immediately prior. In one embodiment, the hidden dimension of the recurrent network is 64, and the output dimension is the number of combinations of dispatch vectors (e.g., 216). Each base agent (121(1) to 121(K)) calculates the expected reward value for each action for each of the 216 possible dispatch vectors and selects the dispatch vector with the highest expected reward value.

[0076] In one embodiment, a plurality of base agents (121(1) to 121(K)) may share the same learning model parameters. That is, all base agents (121(1) to 121(K)) use a learning model having the same structure and weights, and each base agent (121(1) to 121(K)) has base identification information (hub) representing the characteristics of the corresponding transportation base. type Embeddings are included in the input. This allows learned knowledge to be shared among base agents, while also enabling differentiated dispatch decisions based on the unique characteristics of each transportation base (e.g., G1 is a transportation hub, G4 is an industrial complex, etc.).

[0078] Figure 4 shows an explanatory diagram of the cluster formation unit return process.

[0079] Referring to FIG. 4, the dispatch execution unit (130) returns empty vehicles from the departure traffic base to the destination traffic base based on the dispatch details determined by each base agent (121(1) to 121(K)) of the agent unit (120). If there are multiple vehicles to be dispatched, they are organized into a cluster formation and returned.

[0080] (1) Select vehicle

[0081] The dispatch execution unit (130) can prioritize the vehicles to be dispatched based on the remaining battery level when selecting a vehicle to be dispatched. By prioritizing the selection of a vehicle with a high remaining battery level, the remaining battery is sufficient after arrival at the destination base, allowing it to be immediately deployed to the next formation.

[0082] Additionally, the dispatch execution unit (130) can select vehicles to be dispatched by considering the ratio of passenger module vehicles and cargo module vehicles based on the predicted demand of the destination transportation hub. For example, if it is predicted that three passenger vehicles and one cargo vehicle are needed for the next train set in G1, the dispatch execution unit (130) selects three passenger module vehicles and one cargo module vehicle in order of highest remaining battery charge.

[0083] (2) Formation and return

[0084] If there is only one vehicle to be dispatched, that vehicle is returned to the destination transportation hub by autonomous driving on its own.

[0085] When there are multiple vehicles to be dispatched, the dispatch execution unit (130) organizes the vehicles into a platoon. The platoon is organized in a structure where the lead vehicle plans the autonomous driving path and the subsequent vehicles follow the lead vehicle. In one embodiment, the autonomous driving kit (ADS Kit) of the lead vehicle operates in master mode to perform path planning and surrounding environment recognition, and the autonomous driving kit of the subsequent vehicle operates in slave mode to follow the lead vehicle. The control of the spacing between vehicles during platoon return can be performed by a separate platoon driving control system.

[0086] Vehicles being transported in a convoy are empty, so there is no need for passenger boarding or alighting; they proceed directly to the destination transportation hub without intermediate stops. This minimizes transport time.

[0087] Convoy transport offers the following advantages compared to individual vehicle transport. In the case of individual transport, each vehicle plans and drives its route independently, resulting in energy consumption and increased road occupancy proportional to the number of vehicles. In contrast, in convoy transport, only the lead vehicle plans the entire route while subsequent vehicles perform low-power follow-up driving; consequently, energy consumption is reduced and road occupancy is minimized due to dense driving between vehicles.

[0088] (3) Transfer after arrival

[0089] Upon arrival at the destination transport hub, if a swarm formation has been formed, the formation is disbanded, and each vehicle is incorporated into the standby fleet of that hub. The incorporated vehicles will be utilized as available vehicles for the next formation.

[0091] Figure 5 shows an explanatory diagram of the network structure during learning.

[0092] Referring to FIG. 5, multiple base agents (121(1) to 121(K)) learn a cooperative dispatch policy using global state information regarding multiple traffic bases by using Multi-Agent Reinforcement Learning (MARL).

[0093] Learning of cooperative dispatching policies refers to the process of adjusting the learning model parameters of each base agent (121(1)–121(K)) so that multiple base agents achieve optimal dispatching results from the perspective of the entire network through their respective dispatching decisions. After learning is completed, the goal of the learning is for each base agent (121(1)–121(K)) to independently make dispatching decisions based solely on local observation information from their own traffic base during inference, while ensuring that such decisions lead to desirable results from the perspective of the entire network.

[0094] To this end, a structure is adopted in which global state information across multiple traffic hubs is used during training, and each hub agent (121(1) to 121(K)) makes decisions based only on local observation information during execution (inference). This structure is called CTDE (Centralized Training with Decentralized Execution). During training, the optimal dispatch policy is learned by utilizing information from the entire network, but during execution, each agent can operate independently without communication between hubs.

[0096] Figure 6 shows an explanatory diagram of the composition of the compensation function.

[0097] Referring to Fig. 6, learning of the cooperative dispatch policy is performed using a reward function. The reward function is an evaluation value calculated retrospectively from the environment after the dispatch decision of the base agent (121(1) to 121(K)) is executed, and indicates how desirable the dispatch decision was from the perspective of the entire network.

[0098] In one embodiment, a compensation function (R t ) can be calculated as a weighted sum of four terms as shown in the following [Equation 1].

[0099]

[0100] Here, w₁~w₄ are relative weights between each compensation term and can be adjusted according to the operating environment or service policy. In one embodiment, w₁=0.4, w₂=0.2, w₃=0.2, and w₄=0.2 can be used as default values.

[0102] Below, the variables used in the compensation function are defined as follows.

[0103] ▶ n idle,k (t): Number of idle vehicles at base k. This is the number of vehicles waiting and not deployed for operation at the current time.

[0104] ▶ n available,k (t): The number of available vehicles at base k. This is the sum of the number of idle vehicles and the number of dispatched vehicles currently scheduled to arrive. That is, n available,k (t) = n idle,k (t) + n incoming,_k (t) is.

[0105] ▶ n pax,k (t): Number of available passenger module vehicles at base k. This is the sum of the number of idle passenger vehicles and the number of passenger dispatch vehicles scheduled to arrive.

[0106] ▶ n cargo,k (t): Number of available cargo module vehicles at base k. This is the sum of the number of idle cargo vehicles and the number of cargo dispatch vehicles scheduled to arrive.

[0107] ▶ : This is the number of personnel predicted for base k.

[0108] ▶ : This is the predicted demand for passenger modules at base k.

[0109] ▶ : This is the predicted demand for cargo modules at base k.

[0110] ▶ C: Capacity per vehicle.

[0111] ▶ r k→j : This is the number of dispatched vehicles from base k to base j.

[0112] ▶ dist k→j : This is the travel distance (km) from base k to base j. Travel included in local observation information. time,k→j It is distinct from (travel time, minutes). While travel time may vary depending on traffic conditions, travel distance is a relatively fixed value determined by the route.

[0113] ▶ e km : This is the energy consumption coefficient per km.

[0115] (i) Unmet demand penalty (R service )

[0116] R service is a term that reflects the overall demand satisfaction of multiple transportation hubs and can be calculated as shown in the following [Equation 2].

[0117]

[0118] If available vehicles are insufficient compared to demand at each base, a negative compensation is generated for the shortfall; if demand is met, it becomes zero. If demand is perfectly met at all bases, R service = 0, and this is the maximum value of this term.

[0120] (ii) Idle cost penalty (R idle )

[0121] R idle is a term that reflects the degree of excess idle vehicles across multiple transportation hubs, and can be calculated as shown in the following [Equation 3].

[0122]

[0123] Here, α is the physical scaling factor of idle costs. If there are an excessive number of vehicles waiting at each location relative to demand, negative compensation occurs to suppress over-dispatch.

[0124] R service and R idle They act in opposite directions. R service If you send fewer vehicles, the penalty increases, and R idle If too many vehicles are sent, the penalty increases. Through the balance of these two terms, the agent learns the trade-off between under-dispatch (unmet demand) and over-dispatch (vehicle idleness) to derive the appropriate dispatch quantity.

[0126] (iii) Energy cost penalty (R energy )

[0127] R energy is a term that reflects the energy consumption associated with empty state return, and can be calculated as shown in the following [Equation 4].

[0128]

[0129] Here, β is the physical scaling factor of energy costs. As the number of dispatched vehicles increases and the travel distance increases, the energy cost penalty becomes larger, so a policy that prefers dispatching between nearby bases is learned.

[0131] (iv) Module matching reward (R module )

[0132] R module ... is a term that reflects the degree of matching between the module type of the dispatched vehicle and the demand by module type predicted at the destination traffic hub, and can be calculated as shown in the following [Equation 5].

[0133]

[0134] Here, γ is the physical scaling factor for module matching. Since the smaller value between the available number of units and the demand is taken, only the number of units for which a match is actually successful is reflected as compensation. The more the module type matches the demand, the more positive the compensation received.

[0136] R module is the only positive reward among the four terms. R service , R idle , R energy While is a penalty (deduction) for undesirable results, R module It is designed as an incentive (bonus points) for desirable results, encouraging agents (121(1) to 121(K)) to actively learn dispatching that takes into account the module type.

[0137] α, β, and γ are physical scaling factors for each compensation term, serving to normalize the raw values ​​of each term to the same scale. Since the four compensation terms each have different physical units and value ranges (e.g., R_service is the shortage of logs, R_energy is logs × km × energy factor), adjusting solely by weight w without α, β, and γ could result in a specific term dominating the overall compensation. It is a two-stage structure where α, β, and γ align the size of each term to a similar scale, and then w₁–w₄ determine policy importance on top of that. α, β, and γ are set according to the operating environment, such as vehicle specifications and distances between bases, while w₁–w₄ can be adjusted according to service policies (e.g., prioritizing demand fulfillment vs. prioritizing energy conservation).

[0139] Below, the compensation function (R t Explains the relationship between ) and expected reward values ​​by behavior, as well as the learning mechanism.

[0140] Compensation function (R t ) is an actual evaluation value calculated retrospectively from the environment after the dispatch decision has been executed. In contrast, the expected reward value (Q-value) for each action is an estimate of the future cumulative reward that each base agent (121(1)–121(K)) predicts in advance for each action in the current observation state.

[0141] Specifically, base agents (121(1) to 121(K)) have local observation information at the current time o kThe expected reward value calculated for each possible dispatch vector based on (t) is Q k It is. The agent is Q k By selecting the highest dispatch vector, it takes the action predicted to maximize future cumulative rewards.

[0142] During training, the blended network (122) has individual expected reward values ​​(Q₁, Q₂, ..., Q₁) of K base agents. K By combining ) the expected reward value from a network perspective (Q tot Produces ). Q tot is the cumulative reward predicted from the perspective of the entire network when multiple agents (121(1) to 121(K)) each take their respective actions.

[0143] The key to learning is Q tot Wow, actual reward R t It is a comparison of Q. tot is the aggregated value of the predictions of the agents (121(1)–121(K)), and R t is the value actually generated in the environment. The parameters of the entire network (the behavior-specific expected reward calculation network (121a(1)–121a(K)), the mixed network (122), and the hypernetwork (123)) are updated in a direction that reduces the difference (error) between these two. If this process is repeated tens of thousands of times, Q tot Ga R t It gradually approaches, and as a result, the individual expected reward value (Q_k) of each agent also becomes accurate.

[0144] When executing (inference) after learning is complete, the blended network (122) and the hypernetwork (123) are not used. Each base agent (121(1) to 121(K)) uses only the behavior-specific expected reward value calculation network (121a(1) to 121a(K)) and inputs the behavior-specific expected reward value (Q) using local observation information of its own traffic base as input. kCalculate the value and select the dispatch vector with the highest expected reward. This is possible because, during the learning process, the learning model of each agent (121(1) to 121(K)) has already been trained to make "desirable judgments from the perspective of the entire network based only on local observations."

[0146] In one embodiment, the blending network (122) may be configured based on a QMIX (Monotonic Value Function Factorization) structure. In the QMIX structure, the blending network (122) is composed of individual expected reward values ​​(Q) of each base agent (121(1) to 121(K)). k As ) increases, the expected reward value from a network perspective (Q tot A monotonic relation (∂Q) that does not decrease tot / ∂Q k It is configured to maintain (≥ 0). This monotonic constraint is achieved by restricting the weights of the mixed network (122) to positive (non-negative). Due to the monotonic constraint, each agent (121(1)–121(K)) has its expected reward value (Q k If you choose the action that maximizes ), the expected reward value from a network perspective (Q tot It is guaranteed that ) is also maximized (Individual-Global-Max, IGM principle). According to this principle, at execution, without a mixed network (122), each agent (121(1)–121(K)) independently Q k Even if you choose an action that maximizes it, the result is optimal from the perspective of the entire network.

[0148] In one embodiment, the weights or biases of the blending network (122) may be dynamically generated by a hypernetwork (123) that receives global state information as input. By the hypernetwork (123) dynamically generating the weights of the blending network (122) according to the global state, the blending network (122) can adjust the combination method of the expected reward values ​​for each agent according to the situation. For example, in a situation where the demand for a specific base increases rapidly, the combination weight for the expected reward value of the agent at that base may increase dynamically.

[0150] Figure 7 shows an explanatory diagram of the dispatch decision cycle.

[0151] Referring to FIG. 7, the dispatch decision is repeated at a predetermined period (Δt). In one embodiment, Δt = 10 minutes.

[0152] The following process is performed in each cycle.

[0153] ▶ Step 1: The demand forecasting unit (110) calculates the demand forecast value for each transportation hub for the future T time step (e.g., T=6, future 1 hour).

[0154] ▶ Step 2: Each base agent (121(1) to 121(K)) constructs local observation information and calculates the number of vehicles to be dispatched through the behavioral expected reward value calculation network (121a(1) to 121a(K)).

[0155] ▶ Step 3: Each base agent (121(1) to 121(K)) verifies the constraints (total amount constraint, battery constraint) and corrects for violations by applying clipping or behavior masking.

[0156] ▶ Step 4: The dispatch execution unit (130) selects a vehicle to be dispatched by considering the battery remaining capacity sorting and module type matching. For the selected vehicle, if there are multiple vehicles to be dispatched, the dispatch execution unit (130) organizes them into a cluster formation and performs the return.

[0157] Steps 1 through 4 above are performed in common during learning and inference. During learning, the following steps are additionally performed.

[0158] ▶ Step 5: In the next cycle (after Δt elapsed), compensation R for the dispatch decision of the previous cycle t It calculates the reward. The calculated reward consists of experience data including the state, action, reward, and next state at that point in time, and is stored in the Replay Buffer. The learning model is updated by randomly sampling from the experience data accumulated in the Replay Buffer. The reason for using random sampling is that experiences that are continuous in chronological order have high cross-correlation, which can lead to biased learning.

[0160] Figure 8 shows an explanatory diagram for comparing scalability according to an increase in the number of bases.

[0161] Referring to Fig. 8, learning is performed in a simulation environment.

[0162] (1) Simulation environment

[0163] Learning is performed in a simulation environment that simulates a network of K transportation hubs using software. In the simulation environment, the demand pattern of each transportation hub, travel time and distance between hubs, and vehicle battery consumption and charging time are modeled. The demand for each hub can be generated based on the prediction model of the demand forecasting unit (110) or actual driving history data.

[0164] (2) Episode

[0165] One episode corresponds to one operation day. In one embodiment, one operation day is approximately 16 hours (06:00–22:00) and consists of approximately 96 time steps with 10-minute decision-making. When one day's operation ends, the simulation environment is reset to its initial state and a new episode begins. By repeating this process for thousands to tens of thousands of episodes, the learning model parameters of the base agents (121(1)–121(K)) are gradually improved.

[0166] (3) Curriculum Learning

[0167] In the early stages of learning, a curriculum learning approach can be applied that starts in an environment with a small number of bases (e.g., K=2) and gradually expands to an environment with a large number of bases (K=3, 4, ...).

[0168] In the K=2 stage, dispatching is relatively simple, allowing agents to learn basic principles such as "rewards increase when sending to underserved bases" and "idle costs are incurred when sending overloaded." In the K=3 stage, the choice of "which of multiple bases to send from" is added, enabling agents to learn the role of energy costs (considering distance). In stages K=4 and above, agents learn more complex interdependencies between bases.

[0169] Since multiple base agents (121(1) to 121(K)) share the same learning model parameters, parameters learned at K=2 can be brought directly into the K=3 environment to perform additional learning. Each base agent (121(1) to 121(K)) has base identification information (hub type Since embeddings are included in the input, differentiated dispatch decisions based on base characteristics are possible even with the same training model. This allows for flexible expansion through additional training or fine-tuning even when bases are added.

[0171] Figure 9 shows an explanatory diagram for comparing dispatching problems in a two-base structure and a multi-base structure.

[0172] Referring to Fig. 9, the dispatching problem in a two-base structure and a multi-base structure is essentially different.

[0173] In a two-base (G1, G2) structure, dispatching is resolved by simple subtraction. If G1 requires 6 vehicles and currently has 1, the shortage of 5 vehicles can be covered by sending them from G2. Since the decision-making is unidirectional (G2→G1 or G1→G2), the optimal dispatch is derived solely through the arithmetic operation max(number of required vehicles - number of current vehicles, 0). In this case, multi-agent reinforcement learning is not required.

[0174] However, in networks consisting of three or more transportation hubs, dispatching transforms into a combinatorial optimization problem. For example, in a situation where G1 is short 5 vehicles, G2 has 3 spares, and G3 has 4 spares, the following interdependent decisions are required: From where should the vehicles be sent, G2 or G3? If G2 also needs 3 vehicles in 20 minutes, is it appropriate to send the spares from G2? Should they be sent from G3, but since the distance from G3 to G1 is 30 minutes, can they arrive on time? Is it better to split the dispatch, sending 2 vehicles from G2 and 3 from G3?

[0175] Such decision-making must simultaneously consider the current status of each base, future demand forecasts, distance between bases, battery status, and module type configuration, and the number of combinations to be considered increases rapidly as the number of bases increases.

[0176] In a centralized method, since a single controller handles dispatch decisions for all base pairs simultaneously, the scale of decision variables increases rapidly as the number of bases increases. On the other hand, in the multi-agent structure of the present invention, since each base agent (121(1) to 121(K)) performs dispatch decisions based only on its own base, the increase in the scale of decisions directly handled by each agent (121(1) to 121(K)) is mitigated.

[0177] For example, at K=4 bases, the centralized method must simultaneously determine the number of dispatches for 12 routes (K×(K-1)), whereas each base agent (121(1)–121(K)) determines the number of dispatches for only 3 directions (K-1). When extended to K=10, this difference becomes even greater.

[0179] Figure 10 shows an explanatory diagram of an example of a dispatch scenario in a base network.

[0180] Referring to Fig. 10, the dispatch scenario in a network consisting of four traffic hubs (G1, G2, G3, G4) is as follows.

[0181] The current status of each base is as follows: G1 has 2 idle vehicles (insufficient), G2 has 5 idle vehicles (spare), G3 has 4 idle vehicles (spare), and G4 has 3 idle vehicles (adequate).

[0182] The future demand forecast calculated by the demand forecasting unit (110) is as follows: 6 units needed for G1 (surge in demand due to KTX arrival), 2 units needed for G2, 3 units needed for G3, and 4 units needed for G4 (after work hours).

[0183] Each base agent (121(1) to 121(4)) independently makes dispatch decisions based solely on local observation information from their own traffic base.

[0184] ▶ G1 Agent(121(1)): 2 idle, 6 needed, 4 insufficient. Since there is no capacity to send on its own, dispatch vector a₁ = [0, 0, 0, 0].

[0185] ▶ G2 Agent (121(2)): 5 idle, 2 needed, 3 spare. Travel time to G1 is 10 minutes (close), to G3 is 20 minutes, to G4 is 25 minutes. Since the expected reward value in the direction of G1 is the highest, dispatch vector a₂ = [3, 0, 0, 0]. Return 3 vehicles to G1.

[0186] ▶ G3 Agent (121(3)): 4 idle, 3 required, 1 spare. Travel time to G4 is 15 minutes (moderate), to G1 is 30 minutes (long distance). The expected reward value for the G4 direction is higher than for the G1 direction (predicted increase in G4's commute demand + large energy cost penalty due to the long distance to G1). Dispatch vector a₃ = [0, 0, 0, 1]. Return 1 vehicle to G4.

[0187] ▶ G4 Agent(121(4)): 3 idle, 4 needed, 1 shortage. Since there is no capacity to send on its own, dispatch vector a₄ = [0, 0, 0, 0].

[0188] As a result of this decision, three vehicles are sent from G2 to G1 in a platoon, and one vehicle is sent from G3 to G4 alone. Dispatch from G3 to G1 is not made because the distance from G3 to G1 (30 minutes) is three times the distance from G2 to G1 (10 minutes), so the energy cost penalty is much greater. The agents (121(1) to 121(4)) automatically learned this comprehensive judgment of distance, demand, and surplus during the learning process.

[0189] Although each base agent (121(1)–121(4)) made decisions independently without knowing the decisions of other base agents (121(1)–121(4)), reasonable dispatching was achieved from the perspective of the entire network. This is because local judgments corresponding to the global optimal were internalized in each agent through the CTDE structure during learning.

[0191] A multi-agent-based multi-base dynamic dispatch method for a PBV cluster shuttle according to one embodiment will be described below. Since the dynamic dispatch method according to one embodiment is implemented by the dynamic dispatch system (100) described above, it is obvious that it includes all features of the dynamic dispatch system (100) without further explanation. In addition, each step of the dynamic dispatch method can be performed by at least one processor. That is, the dynamic dispatch method can be implemented in the form of a computer program performed by at least one processor, and each step included in the dynamic dispatch method can be performed by one of at least one processor.

[0193] FIG. 11 shows a flowchart of a multi-agent-based multi-base dynamic dispatch method for a PBV cluster shuttle according to one embodiment.

[0194] Referring to FIG. 11, a dynamic dispatch method according to one embodiment includes: a step (S110) of predicting future demand for each traffic hub and providing the predicted demand prediction information to a plurality of hub agents (121(1) to 121(K)); a step (S120) in which a hub agent (121(1) to 121(K)) corresponding to each of the plurality of traffic hubs calculates the number of vehicles to be dispatched from the traffic hub to each other traffic hub based on local observation information related to vehicle dispatch at the traffic hub; and a step (S130) of returning empty vehicles from the starting traffic hub to the destination traffic hub based on the dispatch content determined by each hub agent (121(1) to 121(K)).

[0195] Specifically, in step S120, each base agent (121(1) to 121(K)) configures local observation information including idle vehicle information of the corresponding traffic base, demand forecast information for vehicles of the corresponding traffic base, access information to other traffic bases, and current time information.

[0196] In step S120, each base agent (121(1) to 121(K)) calculates the expected reward value for each action for each possible dispatch vector using the behavioral expected reward value calculation network (121a(1) to 121a(K)), and selects the dispatch vector with the highest expected reward value. At this time, each base agent (121(1) to 121(K)) can calculate the number of dispatched vehicles at the current time by using the local observation information at the current time together with the number of vehicles to be dispatched calculated at the previous time.

[0197] Each base agent (121(1) to 121(K)) calculates the number of vehicles to be dispatched independently based on local observation information of the corresponding traffic base during inference at step S120. That is, without referring to observation information or dispatch decision results of other base agents, they make dispatch decisions solely based on the local observation information of their own traffic base.

[0198] In step S120, each base agent (121(1) to 121(K)) verifies the pre-set constraints on the calculated number of vehicles to be dispatched and corrects for any violations.

[0199] Specifically, it is verified whether the total number of vehicles to be dispatched is less than or equal to the value obtained by subtracting the preset minimum number of reserve vehicles (n_reserve) from the number of idle vehicles at the corresponding transportation hub (total quantity constraint). Additionally, it is verified whether any vehicles with a battery level below the standard required for travel to the destination transportation hub are included in the dispatch target (battery constraint).

[0200] If a constraint is violated, each base agent (121(1) to 121(K)) corrects by applying at least one of clipping to reduce the number of vehicles to be dispatched in response to the violation, or action masking to block the selection of actions in response to the violation.

[0201] In addition, at step S130, vehicles to be dispatched are prioritized based on the remaining battery charge, and vehicles to be dispatched are selected by considering the ratio of passenger modular vehicles and cargo modular vehicles based on the predicted demand of the destination transportation hub.

[0202] In step S130, if there are multiple vehicles to be dispatched, the vehicles are organized into a swarm formation and returned. In step S130, if there is only one vehicle to be dispatched, the vehicle is returned to the destination transportation hub by autonomous driving alone.

[0203] The platoon is organized in a structure where the lead vehicle plans the autonomous driving route and the subsequent vehicles follow the lead vehicle; since it is empty, it proceeds directly to the destination transportation hub without intermediate stops.

[0204] Upon arrival at the destination transport hub, if a swarm formation is formed, disband the swarm and incorporate each vehicle into the standby fleet at the corresponding hub.

[0205] Steps S110 through S130 are repeated at a predetermined period (Δt). In one embodiment, Δt = 10 minutes.

[0207] In one embodiment, the dynamic dispatch method may further include a step (S200) in which a plurality of base agents (121(1) to 121(K)) learn a cooperative dispatch policy.

[0208] In step S200, multiple base agents (121(1) to 121(K)) learn a cooperative dispatch policy using global status information, which is information that aggregates the operational status of multiple traffic bases.

[0209] Learning of the cooperative dispatch policy can be performed using a compensation function that reflects the overall demand satisfaction of multiple transportation hubs, the degree of excess idle vehicles, the energy consumption associated with empty return, and the degree of matching between the module type of the dispatched vehicle and the predicted demand by module type at the destination transportation hub.

[0210] Additionally, learning of the cooperative dispatch policy can be performed through a blended network (122) that calculates an expected reward value from a network perspective by combining the expected reward values ​​for each action calculated by each of the multiple base agents (121(1) to 121(K)) using global state information. The weights or biases of the blended network (122) can be dynamically generated by a hypernetwork (123) that receives global state information as input.

[0211] After the learning is complete, the mixed network (122) and the hypernetwork (123) are not used, and each base agent (121(1) to 121(K)) performs step S120 using only the behavior-specific expected reward value calculation network (121a(1) to 121a(K)).

[0213] As described above, according to the dynamic dispatch system (100) and dynamic dispatch method according to one embodiment of the present invention, the following effects can be obtained.

[0214] A base agent (121(1) to 121(K)) corresponding to each of the multiple traffic bases calculates the number of vehicles to be dispatched based on local observation information, and by organizing empty vehicles into a cluster formation and returning them, the vehicle concentration between bases can be efficiently resolved.

[0215] Since each base agent (121(1) to 121(K)) independently makes dispatch decisions based on local observation information of the corresponding traffic base during inference, dispatch is possible without communication between bases and can operate stably even in the event of a communication failure.

[0216] Scalability can be ensured by a distributed structure in which agents (121(1) to 121(K)) at each base make dispatch decisions based on their respective bases, thereby mitigating the increase in the scale of decision-making handled by each agent even as the number of bases increases.

[0217] By performing dispatching by considering the matching between the module type of the dispatched vehicle and the demand by module type predicted at the destination transportation hub, it is possible to secure in advance a vehicle that matches the module configuration required for the next formation at the destination hub.

[0218] By determining the availability of vehicles to be dispatched by combining battery charge and return distance, it is possible to prevent the risk of discharge during return and derive an efficient dispatch policy that prioritizes dispatching between nearby bases.

[0219] By including demand forecast information in the local observation information of base agents (121(1)~121(K)), dispatching can be performed to proactively respond to demand fluctuations.

[0220] During platoon transport, only the lead vehicle plans the route while subsequent vehicles follow, resulting in reduced energy consumption and minimized road occupancy compared to individual vehicle transport.

[0221] By utilizing global state information during training to learn the optimal dispatch policy from the perspective of the entire network and operating independently using only local observation information during execution, the CTDE structure can simultaneously ensure training quality and execution efficiency. Explanation of the symbols

[0222] 100 : Dynamic dispatch system 110 : Demand Forecasting Unit 120 : Agent Department 121(1)~121(K): Base Agent 121a(1)~121a(K): Network for calculating expected rewards per action 122 : Mixed Network 123 : Hypernetwork 130 : Dispatch Execution Unit

Claims

Claim 1 In a multi-base dynamic dispatch system for a cluster shuttle, the system comprises: an agent unit including multiple base agents corresponding to each of multiple traffic bases and calculating the number of vehicles to be dispatched from a given traffic base to another traffic base based on local observation information related to vehicle dispatch at the given traffic base; wherein each of the multiple base agents independently calculates the number of vehicles to be dispatched based on the local observation information of the given traffic base during inference; wherein the multiple base agents learn a cooperative dispatch policy using global state information, which is information that integrates the operational status of the multiple traffic bases; wherein the learning of the cooperative dispatch policy is performed using a compensation function that reflects the degree of demand satisfaction for the entire plurality of traffic bases, the degree of excess idle vehicles, the energy consumption associated with empty state return, and the degree of matching between the module type of the dispatched vehicle and the demand by module type predicted at the destination traffic base; wherein the learning of the cooperative dispatch policy is performed through a hybrid network that calculates an expected reward value from a network perspective by combining the behavioral expected reward values ​​calculated by each of the multiple base agents using the global state information; and wherein the weights of the hybrid network or The bias is a dynamic dispatch system that is dynamically generated by a hypernetwork receiving the above global state information as input. Claim 2 A dynamic dispatch system according to claim 1, wherein the local observation information comprises at least two of idle vehicle information of the relevant transportation hub, demand forecast information for vehicles of the relevant transportation hub, access information to other transportation hubs, and current time information. Claim 3 In paragraph 2, the idle vehicle information comprises the number of idle vehicles at the relevant transportation hub, the number of passenger module vehicles among the idle vehicles, the number of cargo module vehicles among the idle vehicles, information on the remaining battery level of the idle vehicles, the number of dispatched vehicles moving to the relevant hub, and the number of dispatched vehicles departing from the relevant hub and moving, in a dynamic dispatch system. Claim 4 A dynamic dispatch system according to claim 1, wherein each of the plurality of base agents is configured such that the total sum of the number of vehicles to be dispatched is less than or equal to the value obtained by subtracting the number of idle vehicles at the corresponding traffic base from the number of pre-set minimum reserve vehicles. Claim 5 A dynamic dispatch system according to claim 1, wherein each of the plurality of base agents corrects by applying at least one of clipping, which reduces the number of vehicles to be dispatched corresponding to the violation, or action masking, which blocks the selection of an action corresponding to the violation in advance, when the number of vehicles to be dispatched violates a preset constraint. Claim 6 A dynamic dispatch system according to claim 1, wherein each of the plurality of base agents calculates the number of vehicles to be dispatched at the current time using the local observation information at the current time and the number of vehicles to be dispatched calculated at the previous time. Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 In a multi-base dynamic dispatch method for a cluster shuttle, a base agent corresponding to each of a plurality of traffic bases calculates the number of vehicles to be dispatched from each of the traffic bases to each of the other traffic bases based on local observation information related to vehicle dispatch at the respective traffic base; A dynamic dispatch method comprising: a step in which multiple base agents learn a cooperative dispatch policy using global state information, which is information that aggregates the operational status of multiple traffic bases; wherein each of the multiple base agents calculates the number of vehicles to be dispatched independently based on local observation information of the corresponding traffic base during inference; the step of learning the cooperative dispatch policy is performed using a compensation function that reflects the degree of demand satisfaction for all of the multiple traffic bases, the degree of excess idle vehicles, the energy consumption associated with empty state return, and the degree of matching between the module type of the dispatched vehicle and the demand by module type predicted at the destination traffic base; and the step of learning the cooperative dispatch policy is performed through a hybrid network that calculates an expected reward value from a network perspective by combining the expected reward values ​​for each action calculated by each of the multiple base agents using the global state information, and the weights or biases of the hybrid network are dynamically generated by a hypernetwork that receives the global state information as input. Claim 12 A dynamic dispatch method according to claim 11, wherein the local observation information comprises at least two of idle vehicle information of the relevant transportation hub, demand forecast information for vehicles of the relevant transportation hub, access information to other transportation hubs, and current time information. Claim 13 A dynamic dispatch method according to claim 12, wherein the idle vehicle information comprises the number of idle vehicles at the relevant transportation hub, the number of passenger module vehicles among the idle vehicles, the number of cargo module vehicles among the idle vehicles, information on the remaining battery level of the idle vehicles, the number of dispatch vehicles moving to the relevant hub, and the number of dispatch vehicles moving from the relevant hub. Claim 14 A dynamic dispatch method according to claim 11, wherein in the step of calculating the number of vehicles to be dispatched, the total sum of the number of vehicles to be dispatched is less than or equal to the value obtained by subtracting the number of pre-set minimum reserve vehicles from the number of idle vehicles at the corresponding traffic hub. Claim 15 A dynamic dispatch method according to claim 11, wherein in the step of calculating the number of vehicles to be dispatched, if the number of vehicles to be dispatched violates a preset constraint, at least one of clipping to reduce the number of vehicles to be dispatched corresponding to the violation, or action masking to block the selection of an action corresponding to the violation in advance is applied to correct the result. Claim 16 A dynamic dispatch method according to claim 11, wherein in the step of calculating the number of vehicles to be dispatched, the number of vehicles to be dispatched at the current time is calculated using the local observation information at the current time and the number of vehicles to be dispatched calculated at the previous time. Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images