Delivery planning device, delivery planning method, and program

Abstract

To realize a delivery planning in consideration of a status of an edge between nodes.SOLUTION: A delivery planning device includes: an edge calculation part which calculates edge information, which is a travel time or travel distance between nodes, for each pair of nodes in a plurality of nodes; and an algorithm calculation part for solving a delivery planning problem determining a route to provide a service to the plurality of nodes by a moving object departing from a certain location using a neural network that performs reinforcement learning using an actor-critic method. The algorithm calculation part solves the delivery planning problem using a feature quantity related to the edge information obtained by the edge calculation part.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for solving a vehicle delivery planning problem. [Background technology]

[0002] The vehicle routing problem (VRP) is an optimization problem that considers the optimal order (lowest cost) for which service vehicles (deliverers) should visit which customers when delivering packages from a package collection point (service center) to each customer (each node). Note that the "vehicle routing problem" may also be called the "vehicle allocation problem."

[0003] In real-world applications, there are many practical business scenarios where distribution and service costs can be optimized through VRP solutions, such as drone route generation, just-in-time delivery in e-commerce, cold chain delivery, and store replenishment.

[0004] Therefore, various variations of VRP have been proposed to meet different practical requirements. One variation of VRP is the time-windowed VRP (VRPTW). In VRPTW, a time window is set for the delivery of goods to customers. Another VRP is the multi-depot vehicle routing problem (MDVRP). In MDVRP, there are multiple depots (service centers) where vehicles can start and end their trips.

[0005] Since VRP and its variations have been proven to be NP-hard problems, various operations research (OR)-based methods that return approximate solutions have been investigated for many years.

[0006] However, for practical-scale VRP problems (with 100 or more customers), OR-based algorithms have the drawback of requiring days or even years of computation to obtain an optimal or approximate solution.

[0007] For example, Non-Patent Document 1 discloses a solution to VRP based on actor-critic reinforcement learning, which solves the drawbacks of OR-based algorithms. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Nazari, Mohammadreza, Afshin Oroojlooy, Lawrence V. Snyder, and Martin Takac, "Reinforcement Learning for Solving the Vehicle Routing Problem", NIPS, 2018. Summary of the Invention [Problem to be solved by the invention]

[0009] However, the reinforcement learning-based method disclosed in Non-Patent Document 1 and the like approximates the edges between nodes by the straight-line distance, and therefore cannot take into account situations such as impassable areas or traffic congestion.Furthermore, it cannot handle cases where the edges change (such as when traffic congestion suddenly occurs).

[0010] The present invention has been made in view of the above points, and has an object to provide a technique for realizing a delivery plan that takes into account the status of edges between nodes. [Means for solving the problem]

[0011] According to the disclosed technology, an edge calculation unit calculates edge information, which is a travel time or a travel distance between nodes, for each pair of nodes in a plurality of nodes; an algorithm calculation unit that solves a delivery planning problem by using a neural network that performs reinforcement learning using an actor-critic method to determine a route for a mobile object departing from a certain location to provide service to the plurality of nodes, The algorithm calculation unit calculates the edge information obtained by the edge calculation unit. A feature amount relating to the edge information is calculated using a graph convolutional network from the Solve the above delivery planning problem using the features A delivery planning device, The delivery planning device further includes a map API unit, The edge calculation unit calculates the edge information based on road conditions between nodes for each pair of nodes in the plurality of nodes, the road conditions being acquired by the map API unit. A dispatch planning device is provided. [Effects of the Invention]

[0012] According to the disclosed technology, it is possible to realize a delivery plan that takes into account the status of edges between nodes. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating a configuration of an apparatus according to an embodiment of the present invention. [Figure 2] FIG. 1 is a configuration diagram of an algorithm calculation unit 130. [Figure 3] FIG. 1 is a diagram illustrating a problem setting. [Figure 4] FIG. 1 is a diagram for explaining graph folding. [Figure 5] FIG. 10 is a diagram illustrating the processing of the attention mechanism. [Figure 6] FIG. 1 is a diagram illustrating a reinforcement learning algorithm (processing procedure). [Figure 7] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0015] For example, in this embodiment, a GCN (Graph Convolutional Network) is used to obtain edge-related feature quantities, but edge-related feature quantities may be obtained using a method other than a GCN.

[0016] In addition, in this embodiment, delivery is performed by a vehicle, but this is an example. Anything that moves (hereinafter referred to as a "mobile body") may perform delivery. For example, the mobile body may be a person, a drone, a ship, a bicycle, a motorcycle, an airplane, a spaceship, etc.

[0017] (Device configuration example) Fig. 1 shows a configuration diagram of a delivery planning device 100 according to the present embodiment. As shown in Fig. 1, the delivery planning device 100 includes a node information collection unit 110, a service vehicle information collection unit 120, an algorithm calculation unit 130, a map API unit 140, a vehicle dispatch unit 150, and an edge calculation unit 160. Note that the map API unit 140 is used for both output from the node information collection unit 110 and output from the algorithm calculation unit 130, and is therefore shown in two places for convenience.

[0018] The delivery planning device 100 may be implemented in one device (computer) or in multiple devices. For example, the algorithm calculation unit 130 may be implemented in one computer, and the other functional units may be implemented in another computer. An overview of the operation of the delivery planning device 100 is as follows.

[0019] The node information collection unit 110 acquires information about each node, such as the location and demand of each node.

[0020] The edge calculation unit 160 uses the map API unit 140 to calculate the actual travel time (or travel distance) between nodes and outputs the calculation result in the form of a distance matrix. Note that travel distance and travel time may be collectively referred to as travel cost. Note that an element in the distance matrix may be travel time.

[0021] The service vehicle information collection unit 120 collects information about each service vehicle. The information about each service vehicle is, for example, the departure position of each service vehicle. In this embodiment, the service vehicle may also be called a delivery vehicle. The load (amount of luggage) of the service vehicle may be initialized to 1 when each service vehicle departs.

[0022] The algorithm calculation unit 130 outputs a delivery plan by solving a VRP problem based on information about each node and each service vehicle. That is, the algorithm calculation unit 130 converts the Distance Matrix calculated by the edge calculation unit 160 into features and generates a delivery plan using a Pointer Network. In addition, the algorithm calculation unit 130 uses an ActorCritic reinforcement learning model to perform learning so that an optimal delivery plan can be output. Details of the algorithm calculation unit 130 will be described later.

[0023] The map API unit 140 can acquire map information as well as actual road conditions (e.g., road congestion status, travel time when congested, and information on road closures due to accidents or construction) in real time. The map API unit 140 also performs a route search for the output (delivery plan) from the algorithm calculation unit 130, and draws, for example, the routes of the delivery plan for each service vehicle on a map. The dispatch unit 150 distributes service route information to each service vehicle (or a terminal at the service center) via a network based on the output result of the map API unit 140. The dispatch unit 150 may also be called an "output unit."

[0024] The map API unit 140 may perform route searches by, for example, accessing an external map server. Alternatively, the map API unit 140 may itself store a map database and perform route searches using the map database.

[0025] As an example, assume that the algorithm calculation unit 130 has obtained a delivery plan of "0 → 2 → 3 → 0." Here, 0 indicates the service center, and 2 and 3 indicate the node numbers, respectively. In this case, the map API unit 140 draws the actual road route of "service center → customer 2 → customer 3 → service center" on the map, and the dispatch unit 150 outputs map information with the route drawn.

[0026] (Configuration example of algorithm calculation unit 130) 2 shows an example of the configuration of the algorithm calculation unit 130. The algorithm calculation unit 130 is a neural network model that performs actor-critic reinforcement learning.

[0027] As shown in Figure 2, this model includes two neural networks: an actor network 131 and a critic network 132.

[0028] The actor network 131 also includes an encoder 133 for embedding (feature quantification) input data, and the encoder 133 includes a graph convolutional layer 133. The graph convolutional layer calculates the features of nodes and edges.

[0029] 2 constitutes a Pointer Network. The Pointer Network has an encoder 133 and a decoder 134. The encoder 133 converts the input sequence (node ​​information and edge information) into features, which are input to the decoder 134 (the decoder uses the features). The decoder 134 uses an attention mechanism to determine which input to select as the next delivery destination.

[0030] The encoder 133 in the actor network 131 in Fig. 2 has an embedding layer that embeds input data into an h-dimensional vector, and a graph convolutional layer. As described above, the graph convolutional layer calculates the feature quantities of nodes and edges.

[0031] The graph convolutional layer may also be called a GCN (Graph Convolutional Network). A GCN is a network that convolves a graph structure, inputs data with a graph structure, and outputs the features of that data.

[0032] Furthermore, since the order of input data is irrelevant in VRP, the RNN encoder is omitted from the configuration shown in FIG.

[0033] The learning method is the same as that disclosed in Non-Patent Document 1, and uses a policy gradient method. In this policy gradient method, an actor network 131 predicts the probability distribution of the next action using a pointer network, and a critic network 132 estimates the reward for the problem instance.

[0034] The critic network 132 has a dense embedding layer. In the critic network 132, a loss function is obtained based on the features obtained from the input data by the dense embedding layer and the reward, and learning is performed to reduce the loss.

[0035] (Details of the processing by the algorithm calculation unit 130) The following describes in detail the processing performed by the algorithm calculation unit 130. In addition, the operations of the edge calculation unit 160 and the map API unit 140 in relation to the input to the algorithm calculation unit 130 will also be described.

[0036] <Question setting> The problem setting in this embodiment will be described with reference to Fig. 3. Note that the nodes described below may also be called customers. As shown in Fig. 3, a set of nodes is located within a certain range on a map. Each black circle in Fig. 3 indicates a node that requires a service (delivery of luggage in this embodiment). There is also a service center (luggage collection point) where luggage is loaded for providing the service.

[0037] First, a set of delivery vehicles is arranged at a service center. In the optimization problem of this embodiment, when delivering packages from the service center to each node using delivery vehicles, it is considered which delivery vehicle should visit which nodes and in what order is optimal (to minimize cost). The above cost is, for example, the driving distance of the delivery vehicle or the driving time of the delivery person. Note that the departure point may be different for each delivery vehicle. Furthermore, nodes that can be visited (or nodes that cannot be visited) may be determined for each delivery vehicle.

[0038] Each node is serviced only once by one of the delivery vehicles. After visiting all planned nodes, the delivery vehicle returns to the service center. Figure 3 shows an example where there are three delivery vehicles, each making deliveries along a different route.

[0039] As a condition for a service vehicle to provide service, a condition (constraint) is used, such as "when the load of the service vehicle is close to 0 and the capacity (remaining load) to provide service to the remaining customers is insufficient," in which the service vehicle must return to the service center.

[0040] The algorithm calculation unit 130 finds a solution ζ for the VRP. The solution ζ is a sequence in the set of nodes χ, which can be interpreted as a service route or service order. For example, if the solution sequence ζ={0, 3, 2, 0, 4, 1, 0} is obtained, this sequence corresponds to two routes. One route follows the order 0→3→2→0, and the other route follows the order 0→4→1→0, which implies the use of two service vehicles. This can also be interpreted as a case where a service vehicle returns to a service center.

[0041] <About input data> Next, the input data to the algorithm calculation unit 130 will be explained. The state (node ​​information) of node i is expressed as x i If you write it as x i is expressed as follows, where t indicates each time in the time step.

[0042] x i :{x t i =(s i ,d t i ), t=0,....,T} The meaning of each symbol is as follows:

[0043] x i : The state of node i s i : 2D coordinates (address) of node i d t i : Demand at node i at step t (assuming demand varies by node) The above demand is the same as the demand characteristics in the classical VRP problem. Also, assume that there are n nodes.

[0044] In addition, the algorithm calculation unit 130 receives edge information e between two nodes. ij The edge information e for node i is input. ij More specifically, it is expressed as follows:

[0045] e ij :{e t ij ,t=0,....,T,j=0,....,n} where e ij is the travel time of the service vehicle from node i to node j. However, using travel time is an example. e ij The travel distance of the service vehicle from node i to node j may be used as

[0046] e ij Regarding the calculation method of e, for example, the map API unit 140 acquires the actual road conditions between each node or traffic information photographed from the air, and the edge calculation unit 160 calculates each e using information such as the actual road conditions. ij Calculate e ij and e ji In addition, the actual road conditions change depending on the time t, so e t ij can change depending on the time t. ij e ij It may be expressed as:

[0047] <Actor Network 131> The solution ζ of the VRP is a Markov decision process (MDP) for a sequence, which is the process that chooses the next action in the sequence (i.e., which node to service next).

[0048] In this embodiment, a pointer network (PointerNet) is used to formulate the MDP process. Note that the pointer network (PointerNet) itself is an existing technology.

[0049] First, the encoder 133 performs feature quantification on the node information and edge information. Then, the decoder 134 restores the behavior of the MDP by using RNN cells connected one by one. The RNN cells are supplied with node position information (s i ) is input. The encoder 133 and the pointer network will be described in more detail below.

[0050] First, the embedding layer in the encoder 133 uses the two-dimensional coordinates s of each node as the feature of each node i. i ∈[0,1] 2 The following embedding is performed into h-dimensional features:

[0051] α i =A1s i +b1 Also, the embedding layer uses the edge value e ij The following embedding is performed into h-dimensional features:

[0052] β ij =A2e ij +b2 where A1∈R h×2 , A2∈R h×1 , is.

[0053] Next, the graph embedding layer in the encoder 133 calculates the node feature value in layer l (l is a lowercase letter L) using the following formula: - x i l and edge features - e ij l In the text of this specification, for convenience of description, the symbol at the beginning of a letter is written at the upper left of the letter. - "x" is an example.

[0054] Depending on the number of layers, the following formula is repeatedly applied to obtain the node features. - x i l and edge features - e ij l The number of layers can be a predetermined value.

[0055]

number

[0056]

number

[0057] The decoder 134 contains a sequence of RNN cells (e.g., LSTM cells). In the decoder 134, the sequence of RNN cells is used to model actions in the MDP. At each step t∈(0, 1, ..., T) of the decoder, the hidden state of the RNN cell is expressed as h t where T is the total number of decoder steps.

[0058] In this embodiment, the attention mechanism t+1 The service order is modeled by calculating . That is, it determines which node is pointed to at each step t of the decoder 134. Note that the operation of the attention mechanism here is basically the same as that of the attention mechanism disclosed in Non-Patent Document 1, except for the difference in the features used.

[0059] Specifically, in step t, a tis calculated using softmax as follows: a t indicates how relevant each input data (each node) is (whether it is suitable as a delivery destination) at step t.

[0060] a t =a t ( - e t i ,h t )=softmax(u t ) u t i =v a T tanh(W a [ - e t i ;h t ]) - e t i is the feature value after embedding by GCN, and is the feature value of the adjacent edge of node i. - e t i is mentioned above. - e ij l is the value of h at step t. t is the hidden state of the RNN cell at step t, as mentioned above.

[0061] Next, in the decoder 134, the context vector c is calculated by the following formula: t Calculate.

[0062]

number

[0063] P(y t+1 |Y t ,E t )=softmax( - u t i ) - u t i =v c T tanh(W c [ - e t i ;c t ]) In the above formula, Y t is the sequence of selected nodes up to t, and E t is the edge feature of the input data at t after GCN ( - v is a sequence of a , v c , W a , W c are all parameters that can be learned. Figure 5 shows an image of the calculation process described above.

[0064] Furthermore, the actor network 131 of the algorithm calculation unit 130 may be provided with a masking unit that uses, for example, the masking technology disclosed in Non-Patent Document 1. For example, if the remaining load of a delivery truck is 0, the masking unit masks all nodes (i.e., prevents delivery to any node). The masking unit also masks, for example, nodes with demand greater than the current load of the delivery truck (i.e., prevents delivery to that node). Note that the load of a delivery truck is reduced by the amount of cargo each time it delivers to a node.

[0065] <Actor-Critic> In this embodiment, actor-critic based deep reinforcement learning is used to simultaneously learn both the policy (measure) and the value function. Note that using an actor-critic based method for VRP is an existing technology, as described in Non-Patent Document 1, for example. However, the reward calculation in this embodiment differs from that disclosed in Non-Patent Document 1.

[0066] Let θ be the learnable parameters (weights) in the actor network 131. actor We will express this as follows.

[0067] In this embodiment, θ actor We use π to parametrize a stochastic policy π, which generates a probability distribution over the next action (which node to visit) at any given decoder step t.

[0068] On the other hand, the learnable parameter θ critic The critic network 132 with the following function estimates the gradient for any problem instance from a given state in reinforcement learning.

[0069] The critic network 132, for example, consists of three dense layers and predicts rewards. In this embodiment, the output probability of the actor network 131 is used as a weight, and a weighted sum of the embedded inputs (outputs from the dense layers) is calculated to output a single value. This can be interpreted as the output of the value function predicted by the critic network 132.

[0070] FIG. 6 shows an example of the actor-critic process (algorithm) executed by the algorithm calculation unit 130.

[0071] In the first line, we divide the actor network into random weights θ actor and the critic network is initialized with random weights θ criticLines 2 and 17 mean that lines 3 to 16 are repeated for each epoch.

[0072] In the third line, the gradient of the parameter dθ actor and dθ critic In the fourth line, we reset the current θ actor We sample B instances according to the actor network with B. Lines 5 and 14 mean that we repeat lines 6 to 13 for each sample in B.

[0073] In line 6, we perform embedding (graph embedding, etc.) based on the current weights to obtain features. Lines 7 and 12 mean that lines 8 to 11 are repeated for each decoder step t∈(1, 2, ..., T). Line 8 means that lines 9 to 11 are repeated as long as the termination condition is met.

[0074] In line 9, the distribution p(y t ) based on y t Calculate y t indicates the node to be serviced (visited) in the t-th step.

[0075] In line 10, the columns y1,…,y t-1 ,y t In line 11, t is updated to t+1.

[0076] In line 13, we calculate the reward R. In line 15, we calculate the policy gradient ∇θ using the following formula: actor Calculate θ actor In line 16, the gradient ∇θ critic Calculate θ critic Update.

[0077]

number

[0078] The actor-critic algorithm in this embodiment shown in Figure 6 shows a training process. After this training process, a test (actual delivery plan output) may be performed, or a test may be performed while the learning progresses. The process shown in Figure 6 can be summarized as follows:

[0079] The current weight θ of the actor network actor At each training iteration, we take B samples and use Monte Carlo simulation to generate a sequence of possible outcomes based on the current policy. This means that at each step of the decoder, we generate a distribution p(y t ) based on the pointer y t This means that the probability is calculated probabilistically.

[0080] Once sampling is complete, the reward and policy gradient are calculated and the actor network is updated in line 15.

[0081] Also, in line 16, we update the critic network in a direction that reduces the difference between the observed reward and the expected reward. Finally, we use the same learning speed in the end-to-end method to improve the gradient dθ actor and gradient dθ critic Using θ actor and θ critic Update.

[0082] Next, the reward will be described. In this embodiment, the value e calculated by the edge calculation unit 160 is ijA reward function based on the cost of travel (total distance traveled or total time taken to travel) is used using the above. The reward function may also be customized depending on the specific use case. For example, if there is a time limit (time frame) for travel, a penalty term may be included that adds a penalty value if the travel takes longer than the time limit.

[0083] (Example of hardware configuration) The delivery planning device 100 can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0084] That is, the delivery planning device 100 can be realized by executing a program corresponding to the processing performed by the delivery planning device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0085] Fig. 7 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 7 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected by a bus BS.

[0086] A program for realizing processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0087] When an instruction to start a program is received, the memory device 1003 reads and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes functions related to the delivery planning device 100 in accordance with the program stored in the memory device 1003. Specifically, for example, the CPU 1004 performs calculations for updating the state in the procedure shown in Fig. 6, stores the updated data in the memory device 1003, reads the updated data for the next time step from the memory device 1003, and performs weight update calculations.

[0088] The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. according to a program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the calculation results.

[0089] (Summary of the embodiment and its effects) As described above, the technology according to this embodiment not only calculates the positions (coordinates) of nodes, but also structures graph data including edge information, and performs feature quantification on the structured data (e.g., distance matrix) using a GCN. Then, the VRP problem is solved using an actor-critic neural network model with a pointer network.

[0090] As described above, by using edge information, it is possible to obtain an optimal solution to the VRP problem by taking into account the actual travel time or distance (flight trajectory in the air and traffic congestion information on roads).In addition, it is possible to respond to re-planning even if the external situation changes during travel (e.g., if the flight path or road is suddenly closed).

[0091] Furthermore, compared to conventional techniques (e.g., Non-Patent Document 1) that use only node coordinates as features, the technology according to this embodiment can effectively convert node information and edge divisions into features using GCN, improving learning efficiency and accelerating convergence, thereby reducing learning time.

[0092] The technology according to this embodiment makes it possible to instantly find an approximate solution even when taking into account actual traffic conditions, etc., during the inference phase and re-planning, thereby significantly improving the execution efficiency in practical business applications.

[0093] (Addendum) This specification discloses at least the following delivery planning device, delivery planning method, and program. (Additional note 1) an edge calculation unit that calculates edge information, which is a travel time or a travel distance between the nodes, for each pair of nodes among the plurality of nodes; an algorithm calculation unit that solves a delivery planning problem by using a neural network that performs reinforcement learning using an actor-critic method to determine a route for a mobile object departing from a certain location to provide service to the plurality of nodes, The algorithm calculation unit solves the delivery planning problem using feature quantities related to the edge information obtained by the edge calculation unit. Delivery planning device. (Additional note 2) The delivery planning device further includes a map API unit, The edge calculation unit calculates the edge information based on the road conditions acquired by the map API unit. Item 1. A delivery planning device according to claim 1. (Additional note 3) The algorithm calculation unit has a graph convolution layer that calculates the feature amount using the edge information. 3. The delivery planning device according to claim 1 or 2. (Additional note 4) The algorithm calculation unit includes an attention mechanism, and the attention mechanism calculates the probability that each node will be selected as a delivery destination using the feature amount. 4. A delivery planning device according to any one of claims 1 to 3. (Additional note 5) A computer-implemented method for dispatch planning, comprising: an edge calculation step of calculating edge information, which is a travel time or a travel distance between the nodes, for each pair of nodes among the plurality of nodes; and an algorithm calculation step of solving a delivery planning problem that determines a route for a mobile object departing from a certain location to provide a service to the plurality of nodes, using a neural network that performs reinforcement learning by an actor-critic method, In the algorithm calculation step, the delivery planning problem is solved using the feature quantity related to the edge information obtained in the edge calculation step. Delivery planning methods. (Additional note 6) A non-transitory storage medium storing a program for causing a computer to function as each unit in the delivery planning device according to any one of appendixes 1 to 4.

[0094] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0095] 100 Delivery planning device 110 Node Information Collection Unit 120 Service Vehicle Information Collection Department 130 Algorithm Calculation Unit 131 Actor Network 132 Critic Network 133 Encoder 134 decoder 140 Map API Section 150 Dispatch Department 1000 Drive Device 1001 Recording media 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output Device

Claims

1. an edge calculation unit that calculates edge information, which is a travel time or a travel distance between the nodes, for each pair of nodes among the plurality of nodes; an algorithm calculation unit that solves a delivery planning problem by using a neural network that performs reinforcement learning using an actor-critic method to determine a route for a mobile object departing from a certain location to provide service to the plurality of nodes, the algorithm calculation unit calculates a feature amount related to the edge information obtained by the edge calculation unit using a graph convolutional network, and solves the delivery planning problem using the feature amount, The delivery planning device further includes a map API unit, The edge calculation unit calculates the edge information based on road conditions between nodes for each pair of nodes in the plurality of nodes, the road conditions being acquired by the map API unit. Delivery planning device.

2. The algorithm calculation unit includes an attention mechanism, and the attention mechanism calculates the probability that each node will be selected as a delivery destination using the feature amount. The delivery planning device according to claim 1 .

3. A computer-implemented method for dispatch planning, comprising: an edge calculation step of calculating edge information, which is a travel time or a travel distance between the nodes, for each pair of nodes among the plurality of nodes; and an algorithm calculation step of solving a delivery planning problem that determines a route for a mobile object departing from a certain location to provide a service to the plurality of nodes, using a neural network that performs reinforcement learning using an actor-critic method, In the algorithm calculation step, a feature amount related to the edge information obtained in the edge calculation step is calculated using a graph convolutional network, and the feature amount is used to solve the delivery planning problem, the computer includes a map API unit, In the edge calculation step, the computer calculates the edge information based on road conditions between nodes for each pair of nodes in the plurality of nodes, which are road conditions acquired by the map API unit. Delivery planning methods.

4. A program for causing a computer to function as each unit in the delivery planning device according to claim 1 or 2.

Images