Delivery planning apparatus, delivery planning method, and program

Abstract

To provide a technique for properly formulating a delivery plan for a group of mobile bodies having different characteristics.SOLUTION: A delivery planning apparatus includes an algorithm calculation unit which determines paths of mobile bodies so as to minimize moving costs, for providing a service to a plurality of nodes using a mobile body group consisting of a plurality of mobile bodies that start from a certain place. The algorithm calculation unit includes: an encoder which calculates, for each node, node characteristics having the minimum arrival time at the node of each of the mobile bodies and relative demand of each of the mobile bodies with respect to the capacity of the mobile body at the node, based on the state of the mobile body group; and a decoder which calculates the probability with respect to a target mobile body for each destination node, based on the node characteristics.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for solving a vehicle delivery planning problem. [Background technology]

[0002] The vehicle routing problem (VRP) is an optimization problem that considers the optimal order (lowest cost) for which service vehicles should visit which customers when delivering packages from a package collection point (service center) to each customer (each node) using service vehicles. Note that the "vehicle routing problem" can also be called the "vehicle allocation problem." Another problem similar to VRP is the Traveling Salesman Problem (TSP). VRP was proposed as a generalization of TSP.

[0003] In classical VRP, the problem is set up such that a group of vehicles with homogeneous capacity (such as the amount of cargo a vehicle can carry) departs from the same depot (such as a parcel collection and distribution center) to provide service to customers with demand.

[0004] VRP is known to be an NP-hard problem. Therefore, exact solutions cannot be applied to large-scale problems due to the large computational complexity. To address this issue, many existing techniques apply deep reinforcement learning (DRL) to VRP. Early methods using DRL simplify the problem by assuming a single capacity of vehicles traveling multiple routes (called a single VRP). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Li, J. et al., 2022. Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem. IEEE Transactions on Cybernetics, pp. 52(12):13572 - 13585.. Summary of the Invention [Problem to be solved by the invention]

[0006] Recently, in order to match real-world settings, HVRP (Heterogenous VRP) has been proposed, which targets a group of vehicles with heterogeneous characteristics (for example, Non-Patent Document 1).

[0007] However, in the prior art, it was difficult to apply the above-mentioned single VRP method to HVRP, meaning that the prior art was unable to properly perform delivery planning for a group of vehicles with heterogeneous characteristics.

[0008] The present invention has been made in view of the above points, and has an object to provide a technique for appropriately carrying out a delivery plan for a group of moving objects having heterogeneous characteristics. [Means for solving the problem]

[0009] According to the disclosed technology, in order to provide services to a plurality of nodes by a group of moving objects consisting of a plurality of moving objects departing from a certain location, an algorithm calculation unit is provided that determines a route for each moving object so as to minimize a movement cost, The algorithm calculation unit an encoder for computing, for each node based on the state of the population of mobiles, node features comprising, for each mobile, a minimum arrival time of the mobile to the node and, for each mobile, a relative demand for the mobile's capacity at the node; a decoder that calculates a probability for each destination node for the target moving object based on the node features; A delivery planning device is provided, comprising: [Effects of the Invention]

[0010] The disclosed technology provides a technology for appropriately performing delivery planning for a group of moving objects with heterogeneous characteristics. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a configuration of an apparatus according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram for explaining interpretational node features. [Figure 3] FIG. 1 is a diagram illustrating the configuration of a vehicle group encoder and a decoder. [Figure 4] FIG. 1 illustrates Algorithm 1. [Figure 5] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0013] In this embodiment, for example, the vehicle is a vehicle that delivers luggage, and the nodes that the vehicle passes through are customer locations (such as homes). However, this is just an example. Anything that moves (which will be called a "mobile body") may perform the delivery. For example, the mobile body may be a person, a drone, a ship, a bicycle, a motorcycle, an airplane, a spaceship, etc. Note that fuel for operating a generator is also an example of "baggage." Baggage may also be called a load.

[0014] Furthermore, the delivery service is not limited to the delivery of parcels, etc. For example, the vehicle may be an EV (electric vehicle), and the service destination node may be a base station. In this case, the EV travels to each base station and charges each base station. In this case, the power required by the base station corresponds to the demand, and the power held by the EV corresponds to the load (parcel).

[0015] In addition, in this embodiment, a delivery plan is created with the aim of minimizing the longest travel time among all vehicles, but this is just one example, and the target to be minimized may be a travel cost other than the "longest travel time among all vehicles."

[0016] In the following, first, the conventional technology and problems with respect to the technology according to the present embodiment will be described in detail, and then the technology according to the present embodiment will be described.

[0017] (Regarding prior art, etc.) As mentioned above, in classical VRP, the problem is posed as a fleet of vehicles with homogeneous capacity departing from the same depot to serve customers with demand. On the other hand, in recent years, in order to fit the real-world setting, HVRP (Heterogenous VRP) has been proposed, which targets fleets of vehicles with heterogeneous characteristics.

[0018] The technology according to this embodiment belongs to HVRP, in which the size of a vehicle fleet is fixed and each vehicle is characterized by its capacity and speed.

[0019] Generally, HVRP has two objectives: min-sum and min-max. The former objective is to minimize the total travel time of all vehicles, and the latter objective is to minimize the longest travel time of all vehicles. In this embodiment, we adopt the min-max objective, aiming to reproduce realistic situations where the last delivery is important, such as route selection for disaster relief or newspaper delivery.

[0020] As mentioned above, many methods have been proposed that apply Deep Reinforcement Learning (DRL) to VRP. Early methods using DRL simplify the problem setting by assuming a single-capacity vehicle traveling multiple routes (single VRP).

[0021] In a single VRP, the route generation process is modeled as a Markov decision process (MDP), where a policy constructs a solution by iteratively assigning the next destination node (customer or depot) to a vehicle.

[0022] DRL utilizes a self-driven learning procedure, which frees us from the need for manually created labels and rules. Furthermore, by training a policy network in advance, it is possible to generate solutions with fast computations during the inference stage. Due to these features, many DRL-based methods for a single VRP have been proposed, achieving high performance in a short time compared to classical heuristic methods.

[0023] (About the assignment) In the prior art, applying the above-described single VRP technique to HVRP poses the following problems.

[0024] One of the challenges is the difficulty of cooperation between vehicles. In HVRP, when assigning a target vehicle to its next destination node, the current location, current capacity, and speed of other vehicles relative to the fleet must be taken into account. Therefore, an effective representation of the current fleet state and a specific architecture for cooperation are required.

[0025] However, prior art is limited to simply extending the policy network for a single VRP and directly inputting the fleet state. Given the complexity of HVRPs, such approaches cannot capture sufficient contextual information to generate high-quality solutions.

[0026] Another challenge arises from the formulation of MDP: it is necessary to select target vehicles that are assigned destination nodes at a certain step. However, in conventional techniques, target vehicles are selected alternately or in a probabilistic order.

[0027] However, such a conventional formulation ignores the travel time differences between all vehicles to reach the destination node, making it impossible to manage the entire fleet on a unified timeline. Therefore, in the conventional technology, the input of the fleet state to the policy network is the "current" state of each vehicle in its asynchronous timeline, which makes it impossible to perform cooperative node selection.

[0028] (Overview of the technology according to the present embodiment) As a technique for solving the above-mentioned problems in HVRP, this embodiment proposes a new DRL-based method called Chronological and Cooperative Route Optimization Method (C2ROM).

[0029] The core idea of C2ROM is to manage the entire fleet on a unified timeline and achieve cooperative node selection based on the current fleet state.

[0030] More specifically, we employ a novel time-series MDP (cMDP) for HVRP, which considers the travel time of each vehicle and assigns the next destination continously to each vehicle that arrives at a destination. Thus, the current fleet state in a unified timeline is available.

[0031] Furthermore, in this embodiment, the policy network employs an attention-based fleet encoder, which uses the current fleet state to calculate how each node is interpreted by all vehicles in the fleet, such as the "time it takes to reach the node" and "the magnitude of demand for that node." Based on this node-wise interpretation by the entire fleet, the policy network can achieve cooperative node selection.

[0032] In summary, the technology according to this embodiment has at least the following three features.

[0033] (i) This embodiment provides a cMDP for formulating an HVRP, which allows the entire fleet of vehicles to be managed on a unified timeline.

[0034] (ii) We provide a fleet encoder in the policy network that can compute node-wise interpretations by all vehicles to facilitate cooperative node selection.

[0035] (iii) Extensive experiments demonstrate that C2ROM outperforms other DRL-based methods in problem-solving quality for any number of customers and vehicles.

[0036] An example of the device configuration and device operation in this embodiment will be described in detail below.

[0037] (Device configuration example) Fig. 1 shows a configuration diagram of a delivery planning device 100 according to the present embodiment. As shown in Fig. 1, the delivery planning device 100 includes a user information collection unit 110, a service vehicle information collection unit 120, an algorithm calculation unit 130, a map API unit 140, and a vehicle dispatch unit 150. The algorithm calculation unit may also be called a "calculation unit."

[0038] The delivery planning device 100 may be implemented in one device (computer) or in multiple devices. For example, the algorithm calculation unit 130 may be implemented in one computer, and the other functional units may be implemented in another computer. An overview of the operation of the delivery planning device 100 is as follows.

[0039] The user information collection unit 110 acquires information on each user (which may also be called a customer, a node, etc.) The information on each user includes a location, an initial demand, and the like.

[0040] The service vehicle information collection unit 120 collects information about each service vehicle. The information about each service vehicle includes maximum capacity, speed, etc. Note that the "service vehicle" may also be called a "vehicle."

[0041] The algorithm calculation unit 130 outputs a delivery plan by solving the HVRP problem based on the information of each node and each vehicle. Details of the algorithm calculation unit 130 will be described later.

[0042] The map API unit 140 performs a route search based on the delivery plan information output from the algorithm calculation unit 130, and, for example, draws the route of the delivery plan for each vehicle on a map. The vehicle allocation unit 150 distributes service route information to each vehicle (or a terminal at a service center) via a network based on the output result of the map API unit 140. The vehicle allocation unit 150 may also be called an "output unit."

[0043] The map API unit 140 may perform route searches by, for example, accessing an external map server. Alternatively, the map API unit 140 may itself store a map database and perform route searches using the map database.

[0044] As an example, assume that the delivery plan "0 → 2 → 3 → 0" is obtained by the algorithm calculation unit 130. Here, 0 indicates the depot (service center, starting point), and 2 and 3 indicate the node numbers, respectively. In this case, the map API unit 140 draws the actual road route of "service center → node 2 → node 3 → service center" on the map, and the dispatch unit 150 outputs map information with the route drawn.

[0045] (Operation of the algorithm calculation unit 130) The operation and configuration of the algorithm calculation unit 130 will be described in detail below. In the following description, in the text of the specification, a symbol (e.g., ~) that is placed at the beginning of a character is placed at the upper left of the character. Also, in the text of the specification, a symbol that is the upside-down version of "^" (a mountain shape with an upward convexity) is represented by "-" (a valley shape with a downward convexity).

[0046] As will be described later, the algorithm calculation unit 130 calculates a neural network model (specifically, a policy network π θ ) to solve the HVRP. Below, the problem setting, model configuration, and operation of the model will be described in detail. The operation of the model described below corresponds to the operation of the algorithm calculation unit 130. Furthermore, since the operation of the model is determined by a policy, in the following description, the term "policy" may be used to refer to the subject of the operation. More specifically, the expression that the subject of the operation is a "policy" means that the algorithm calculation unit 130 (or the model) operates in accordance with the policy.

[0047] <Problem setting: cMDP (temporal MDP) for HVRP> In this embodiment, the HVRP (Heterogeneous Vehicle Routing Planning Problem) to be solved is defined using N+1 nodes (customers and depots) and a vehicle group consisting of M vehicles. The vehicle group may also be called a fleet.

[0048] The set of nodes is X={x i} i=0 N ={(s i,D i )} i=0 N where s i and D i are the nodes x i The two-dimensional position coordinates and initial demand are shown. 0 indicates the depot, and X´=X\{x 0} represents a set of customer nodes. Depots have no demand. That is, D 0 =0.

[0049] The vehicle fleet is V={v k} k=1 M ={(f k ,Q k )} k=1 M where f k and Q k are vehicle v k It represents the speed (speed) and maximum capacity of a vehicle. For example, in the case where a vehicle carries cargo, the maximum capacity is the maximum amount of cargo (which may be weight or volume) that can be loaded onto the vehicle. Also, in the case where a vehicle carries electricity, the maximum capacity is the maximum amount of electricity that can be charged into the vehicle.

[0050] Each vehicle travels one or more routes that start and end at a depot. When a vehicle travels one route, the total demand of the nodes satisfied by that route does not exceed the vehicle's maximum capacity.

[0051] The objective of HVRP in this embodiment is to generate a set of routes that meets the demands of all nodes and minimizes the longest travel time among all vehicle travel times.

[0052] In this embodiment, we formulate this problem as a cMDP, which manages the entire fleet of vehicles on a unified timeline. In this formulation, each decision step is triggered when a vehicle reaches its current destination node based on a chronological strategy for vehicle selection. At each step, a policy assigns the next destination node to this vehicle (called the target vehicle) that has reached the destination node.

[0053] In some cases, it may be optimal for a target vehicle at a depot to remain there and leave service to the remaining nodes to other vehicles. To enable such a decision, in this embodiment, the policy can declare the target vehicle out-of-service. A vehicle that arrives at a depot and is declared out-of-service will remain at the depot in subsequent steps. The essential elements of cMDP, "states, actions, and transitions," are defined as follows:

[0054] situation: The state at step t is s t =(X t ,V t ,V t OSS ,m t ) is defined as X t is X t ={x t i} i=0 N ={(s i ,d t i )} i=0 N The node state is expressed as, where s i is the node x i represents the two-dimensional position coordinate of t i ∈{0,D i} is the node x at step t i In other words, in this embodiment, the demand of node x i The demand for is either satisfied (0) or remains at the initial demand (D i ) or

[0055] V t is V t ={v t k} k=1 M ={(f k ,q t k ,g t k ,r t k )} k=1 M is the state of the fleet at step t, denoted as t k Each element in represents the vehicle v at step t. k The speed (f k ), vehicle v k The remaining capacity (q t k ), vehicle v k The current destination node (g t k ), and vehicle v k The remaining time to the destination node (r t k )

[0056] V t OOS represents the set of non-operating vehicles. m t is the index of the target vehicle that has just arrived at the current destination node. In other words, the remaining time to the destination node is 0, so r t m_t =0.

[0057] The initial state s0 is X0={(s i ,D i )} i=0 N , V0={(f k ,Q k ,x 0 ,0)} k=1 M , V t OOS =φ (empty set), m0 = 1. The terminal state sτ is X τ ={(s i ,0)} i=0 N , V τ ={(f k ,Q k ,x 0 ,0)} k=1 M , V τ OOS = V, m τ =φ.

[0058] Action: The policy determines the action. The action is the target vehicle v m_t corresponds to the next destination node of a t =x n ∈A t where A t is the set of valid actions that includes the depot and the unvisited nodes whose demands are satisfied by the remaining capacity.

[0059] Selecting a depot for the same target vehicle in two consecutive steps is considered as declaring it out of service. t prohibits declaring the last in-service vehicle out of service when unmet demand remains.

[0060] transition: Current Status t and current behavior a t Given , the deterministic transition rule τ moves the state forward by one step. That is, s t+1 =τ(s t ,a t ) In the algorithm calculation unit 130, first, the node state X t is updated as shown in the following equation (1).

[0061]

number

[0062] Vehicle-related conditions V t ,V t OOS , and m t The transition can be described in two stages.

[0063] In the first stage, V t and V t OOS is the intermediate remaining time r' t+1 is updated using

[0064]

number

[0065]

number

[0066]

number

[0067]

number

[0068] Equation (3) indicates that a target vehicle arriving at a destination node supplies its load (such as luggage), and also indicates that a target vehicle arriving at a depot is replenished with load.

[0069] Equation (4) represents that the target vehicle calculates the time to the next destination node based on the distance from the current node to the next destination node.

[0070] Equation (5) shows that when the policy declares the target vehicle as “non-operating,” the target vehicle is V t+1 OOS indicates that the value is to be added to the

[0071] In the second stage, we plot the unified timeline ΔT until the in-service vehicle reaches the destination node. t+1 Shift by the remaining time r t and target vehicle index m t is updated as follows:

[0072]

number

[0073]

number

[0074]

number

[0075] Equation (8) indicates that the vehicle that has reached the destination node is selected as the next target vehicle. Specifically, if there are multiple vehicles that have reached the destination node, the vehicle with the smallest index is selected as the next target vehicle. When all vehicles are out of service, the final state s τ to reach.

[0076] Rewards: The objective of the HVRP in this embodiment is to minimize the longest travel time of any vehicle in the fleet. In other words, the objective is to minimize the total time shift (ΔT t The goal is to minimize the total reward R for the entire step, R=-Σ t=1 τ-1 ΔT t The reward R is used in reinforcement learning, which will be described later.

[0077] <Policy Network> Next, a policy network in the model according to this embodiment will be described. In this embodiment, a policy π θ is implemented, where θ denotes a learnable parameter.

[0078] Policy network π θ is the current state s t Given a, the probability that the target vehicle will take action a (selection probability for each node) is output. This is π θ (a|s t ) Action a t is probabilistic, i.e., a t ~π θ (|s t ) or deterministically, i.e., a t =argmax a π θ (|s t ) is selected.

[0079] In this embodiment, an encoder-decoder structure is adopted as the policy network structure. The encoder includes a node encoder and a vehicle group encoder. The node encoder is executed in the initial step, and d h = 128-dimensional static node embeddings are computed, which represent the 2D position coordinates and initial demands of the nodes.

[0080] In this embodiment, we use two attention layers to take the graph structure into account. A fleet encoder and its decoder are executed at each step. These are the core structures for cooperative node selection. Below, we explain how they achieve cooperation and their architecture.

[0081] <About cooperation between vehicles> Before going into a detailed description of the architecture, we will explain how cooperative node selection is performed by the algorithm calculation unit 130. Here, we will explain using a simple example.

[0082] As shown in Figure 2, suppose there are two vehicles α and β and four nodes A to D. Vehicle α is heading to node A, and vehicle β has just arrived at node D. The node demand is shown in parentheses next to each node. The initial capacity of vehicles α and β is both 10. After vehicle α meets the demand of node A, the remaining capacity of vehicle α is 6, which is not enough to meet the demand of node B. Therefore, node B is interpreted as not being a candidate for vehicle α's next destination node. The policy network uses this interpretation to select the next node for vehicle β.

[0083] That is, the policy network selects the next destination node of the target vehicle β from node B or C. For simplicity of explanation, returning to the depot is not considered here.

[0084] The location and demand of two nodes obtained by static node embedding are important information. Furthermore, to achieve cooperation, it is important to consider how each node is interpreted by other vehicles in the fleet.

[0085] In this example, as mentioned above, the remaining capacity after vehicle α supplies node A is sufficient to supply node C, but not to supply node B. Therefore, from vehicle α's perspective, only node C is interpreted as a candidate for vehicle α's next destination node.

[0086] The strategy needs to grasp such interpretations of each node from the perspective of all vehicles in order to cooperatively select the next destination node for a target vehicle. Intuitively, a node close to a vehicle whose demand can be satisfied by the vehicle's remaining capacity is interpreted as a preferred candidate for that vehicle.

[0087] Based on this idea, in this embodiment, we introduce an interpretative node feature for each node, which consists of two elements: minimum reachable time and relative demand. These two elements represent the "time required for each vehicle to reach the node" and the "relative magnitude of demand to the remaining capacity of each vehicle."

[0088] The role of the fleet encoder is to compute interpretive node features based on the current fleet state and to compute interpretive node embeddings at each step.

[0089] The decoder can combine static and interpretive node embeddings to evaluate each node from multiple perspectives. Finally, based on the rich contextual node-wise embeddings and target vehicle context, the decoder achieves collaborative node selection.

[0090] <About Architecture> The vehicle group encoder and decoder will be described in detail below. Figure 3 shows the configuration of each of the vehicle group encoder 10 and decoder 20. The configuration shown in Figure 3 mainly shows the internal processing content (contents described below).

[0091] <Vehicle Group Encoder 10> First, the vehicle group encoder 10 will be described. At step t, the algorithm calculation unit 130 calculates the node state X t ={(s i ,d t i )} i=0 N and vehicle fleet state V t ={(f k ,q t k ,g t k ,r t k )} k=1 M Given, the fleet encoder 10 first computes the elements of the interpretational node features.

[0092] Vehicle V k From the perspective of node x i Minimum reachable time to ~ r t k,i and node x i Relative demand for ~ d t k,i is calculated as follows:

[0093]

number

[0094]

number

[0095] Next, we connect all the vehicle elements to the node x as follows: i Interpretive node features of - x t i ∈R 2×M get.

[0096]

number

[0097] Next, the vehicle group encoder 10 - x t i , linearly - h´ t i ∈R d_h and inputs it to a single attention layer 11 for processing. The purpose of the attention layer is to improve the interpretability of each node by taking into account the relationships between nodes. Finally, the fleet encoder 10 generates the interpretive node embedding at step t. - h t i ∈R d_h Output.

[0098] <Decoder 20> Next, we will explain the decoder 20. For the decoder 20, we will use static node embedding ^h i ,Interpretive Node Embedding - h t i , and the state of the target vehicle v t m =(f m,q t m ,g t m ,r t m ), the decoder 20 calculates, at each step t, the node selection probability p t Calculate.

[0099] According to the decoder architecture in the attention model, the decoder 20 includes a multi-head attention (MHA) module 21 and a compatibility module 22. The MHA module 21 is configured to detect the vehicle v m The glimpse of the nodes from the viewpoint of the node is output. The glimpse may be called a feature, a glimpse, a glimpse, or a symptom.

[0100] First, the decoder 20 t i =^h i + - h t i As the node embedding h t i ∈R d_h The decoder 20 also obtains the target vehicle context vector H'. t m ∈R d_h is calculated as follows:

[0101]

number

[0102]

number

[0103] The decoder 20 then uses the MHA module 21 to generate the Glimpse H t m ∈R d_h Here, the MHA module 21 receives a query and a key-value pair. The query is calculated as a context vector H' t m and the key-value pairs are stored in the node embedding h t =(h t 0 ,h t 1 ,···,h t N ) T ∈R N×d_h The node embedding h t i represents not only static node features but also the interpretation by the entire fleet of vehicles, so H t m As such, the collaborative context can be obtained.

[0104] The compatibility module 22 calculates the Glimpse H t m and node embedding h t Based on the compatibility between t Calculate.

[0105]

number

[0106]

number

[0107] <About learning> model (specifically, the policy network π θ In this embodiment, for example, the reinforcement learning algorithm shown in FIG. 4 is used to learn the function for learning. Here, the algorithm calculation unit 130 has a function for learning. However, the algorithm calculation unit 130 may not have a function for learning. In this case, a model learned outside the algorithm calculation unit 130 is stored in the algorithm calculation unit 130 and used.

[0108] Furthermore, when the algorithm calculation unit 130 has a function for learning, the learning stage and the operation stage may be separated, or the model (parameters) may be updated while performing reinforcement learning in actual operation.

[0109] Algorithm 1 shown in Figure 4 is an algorithm of an existing method called "REINFORCE with Greedy Rollout Baseline."

[0110] In this algorithm, the baseline policy π, which is the frozen policy of the previous epoch, θBL Use to accelerate your learning.

[0111] For each batch, a random problem is given and the strategy π is applied until the final state is reached. θThe algorithm repeats the action probabilistically and performs sampling (lines 4 to 9). Next, it calculates the joint probability and reward of the generated solution (lines 10 and 11).

[0112] On the other hand, the baseline policy π θBL By deterministically selecting an action, we generate a solution for the same instance and obtain a baseline reward. This value reduces the variance of the gradient (line 13), and the gradient is used to update the parameter θ (line 14). At the end of each epoch, a paired t-test (α=0.05) was conducted on the 10240 instances to determine the best fit for the policy π θ is the baseline policy π θBL If the improvement is large, the parameter θ BL is updated (lines 16 to 18).

[0113] (Example of hardware configuration) The delivery planning device 100 described in this embodiment can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0114] That is, the device can be realized by executing a program corresponding to the processing performed by the device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0115] Fig. 5 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 5 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus BS. The computer may further include a GPU.

[0116] A program for realizing processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0117] The memory device 1003 reads and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the delivery planning device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the calculation results.

[0118] (Summary of implementation form, effects, etc.) As described above, the technology described in this embodiment makes it possible to manage the entire vehicle on a unified timeline and realize cooperative node selection based on the current vehicle state.

[0119] Specifically, a novel time-series MDP (cMDP) for HVRP considers the travel time of each vehicle and continuously assigns the next destination to a vehicle that has reached its destination node, making the current vehicle state available in a unified timeline.

[0120] The attention-based policy network in this embodiment employs a fleet encoder and decoder as its core structure. The fleet encoder uses the current fleet state to calculate how each node is interpreted by all vehicles in the fleet, such as the "time it takes to reach a node" and "the magnitude of its demand." Based on this interpretation of each node by all vehicles, the decoder can achieve cooperative node selection.

[0121] The following additional notes are provided regarding the above-described embodiments.

[0122] <Additional Notes> (Additional note 1) The system includes an algorithm calculation unit that determines a route for each mobile object so as to minimize a travel cost in order to provide services to a plurality of nodes by a group of mobile objects consisting of a plurality of mobile objects departing from a certain location, The algorithm calculation unit an encoder for computing, for each node, node features comprising, for each mobile, a minimum arrival time of the mobile to the node and, for each mobile, a relative demand for the mobile's capacity at the node based on the state of the mobiles; a decoder that calculates a probability for each destination node for the target moving object based on the node features; A delivery planning device comprising: (Additional note 2) When a moving object reaches a destination node, the algorithm calculation unit determines the moving object as the target moving object and determines a destination node for the target moving object. Item 1. A delivery planning device according to claim 1. (Additional note 3) The algorithm calculation unit shifts the unified timeline of the group of moving objects by the time of a moving object that takes the shortest time to reach a destination node among the plurality of moving objects. 3. The delivery planning device according to claim 1 or 2. (Additional note 4) The algorithm computation unit deactivates a mobile unit by selecting a starting point as a destination node for the mobile unit in two successive steps. 4. A delivery planning device according to any one of appendixes 1 to 3. (Additional note 5) A delivery planning method executed by a delivery planning device having an algorithm calculation unit including an encoder and a decoder, The method includes a planning step in which the algorithm calculation unit determines a route for each of a plurality of moving objects so as to minimize a travel cost in order to provide services to a plurality of nodes by a group of moving objects consisting of a plurality of moving objects departing from a certain location, In the planning step, the encoder calculates, for each node based on the state of the fleet of mobiles, node characteristics comprising, for each mobile, a minimum arrival time of the mobile to the node and, for each mobile, a relative demand for the mobile's capacity at the node; The decoder calculates a probability for each destination node for the target moving object based on the node features. Delivery planning methods. (Additional note 6) A non-transitory storage medium storing a program for causing a computer to function as an algorithm calculation unit in the delivery planning device according to any one of appendixes 1 to 4.

[0123] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0124] 10 Vehicle Fleet Encoder 20 Decoder 100 Delivery planning device 110 User Information Collection Department 120 Service Vehicle Information Collection Department 130 Algorithm Calculation Unit 140 Map API Section 150 Dispatch Department 1000 Drive Device 1001 Recording media 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output Device

Claims

1. The system includes an algorithm calculation unit that determines a route for each mobile object so as to minimize a travel cost in order to provide services to a plurality of nodes by a group of mobile objects consisting of a plurality of mobile objects departing from a certain location, The algorithm calculation unit an encoder for computing, for each node based on the state of the population of mobiles, node features comprising, for each mobile, a minimum arrival time of the mobile to the node and, for each mobile, a relative demand for the mobile's capacity at the node; a decoder that calculates a probability for each destination node for the target moving object based on the node features; A delivery planning device comprising:

2. When a moving object reaches a destination node, the algorithm calculation unit determines the moving object as the target moving object and determines a destination node for the target moving object. The delivery planning device according to claim 1 .

3. The algorithm calculation unit shifts the unified timeline of the group of moving objects by the time of a moving object that takes the shortest time to reach a destination node among the plurality of moving objects. The delivery planning device according to claim 1 .

4. The algorithm computation unit takes a mobile unit out of service by selecting a starting point as a destination node for the mobile unit in two successive steps. The delivery planning device according to claim 1 .

5. A delivery planning method executed by a delivery planning device having an algorithm calculation unit including an encoder and a decoder, The algorithm calculation unit includes a planning step of determining a route for each of a plurality of moving bodies so as to minimize a travel cost in order to provide services to a plurality of nodes by a group of moving bodies consisting of a plurality of moving bodies departing from a certain location, In the planning step, the encoder calculates, for each node based on the state of the fleet of mobiles, node characteristics comprising, for each mobile, a minimum arrival time of the mobile to the node and, for each mobile, a relative demand for the mobile's capacity at the node; The decoder calculates a probability for each destination node for the target moving object based on the node features. Delivery planning methods.

6. A program for causing a computer to function as an algorithm calculation unit in the delivery planning device according to any one of claims 1 to 4.

Images