Unmanned delivery vehicle path optimization method based on attention mechanism and graph neural network

Abstract

The invention discloses an unmanned distribution vehicle path optimization method based on an attention mechanism and a graph neural network. The method comprises the following steps: 1, constructing an unmanned distribution vehicle distribution task model; and 2, constructing an algorithm solving model based on an attention mechanism and a graph neural network, thereby obtaining an unmanned distribution vehicle path optimization model corresponding to the optimal parameters, and outputting the optimal distribution path of the unmanned distribution vehicle. According to the method, offline training is carried out on the model through deep reinforcement learning and a pseudo-label method, so that the method is applied to unmanned distribution vehicle path online optimization considering battery capacity, unmanned distribution vehicle load and customer satisfaction, the transportation efficiency is effectively improved, and the total transportation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle path planning, and specifically to a path optimization method for an unmanned delivery vehicle based on an attention mechanism and a graph neural network. Background Art

[0002] The vehicle routing problem (VRP) was first proposed by Dantzig and Ramser in 1959. The problem aims to solve the problem of how to achieve the goal of minimum cost, shortest delivery time or shortest distance when a distribution center delivers the required goods to multiple nodes under the conditions of meeting customer demand and other constraints, such as the load of the delivery vehicle cannot be greater than the maximum capacity of the vehicle. Each node has a different demand for goods, and the number of vehicles that can deliver goods is set according to the number of nodes and demand, and the delivery vehicles all depart from the distribution center and finally return to the distribution center.

[0003] Scholars at home and abroad have carried out a lot of research on the application of artificial intelligence in logistics systems. For example: Wang Xiuhong et al. (see Xiuhong W, Xuehao L, Yongcheng W. A Research on Task Scheduling and Path Planning of Mobile Robot in Warehouse Logistics Based on Improved A*algorithm[J]. Industrial Engineering Journal, 2019, 22: 34-39.) Liu et al. (see Liu XH, Zhang D, Zhang J, et al. A path planning method based on the particle swarm optimization trained fuzzy neural network algorithm[J]. Cluster Computing, 2021: 1-15.) combined heuristic algorithms and neural network algorithms, proposed a path planning method based on a fuzzy neural network algorithm trained with particle swarm optimization, designed training rules for the improved particle swarm algorithm to train the fuzzy neural network, and used a hybrid algorithm to solve the path planning problem of intelligent driving cars. Kool et al. (see Kool W, Van Hoof H, Welling M. Attention, Learn to Solve Routing Problems![J]. 2018.) introduced a multi-head attention mechanism to solve various path optimization problems, which is significantly improved compared to the pointer network. Zou et al. (see Zou Y, Wu H, Yin Y, et al. An improved transformer model with multi-attention and attention to attention for low-carbon multi-depot vehicle routing problem[J]. Annals of Operations Research, 2022: 1-20.) proposed an improved multi-head attention and attention Transformer model to solve the low-carbon multi-depot vehicle routing problem.

[0004] When the above algorithm is used to solve the vehicle routing problem, although the algorithm converges quickly in the early stage and has a faster calculation speed when solving instances with larger data scales, the update of particles in the later stage is related to the candidate solution. Therefore, the algorithm also has the disadvantages of poor global search ability and easy to fall into local optimality and then stagnate.

[0005] The last mile of tram delivery is currently a specific scenario of logistics transportation. In the tram delivery route planning, the existing work does not consider the constraints of customer satisfaction and the compensation costs caused by delivery not within the customer's tolerance limit, resulting in the inability to implement it in the actual delivery scenario. The present invention considers the actual application of last-mile logistics tram delivery, proposes a path planning problem that considers tram charging strategy and customer satisfaction, and proposes a new context vector-based attention mechanism and graph neural network model for solution.

[0006] When using the attention mechanism to solve the vehicle routing problem, although the initial training of the algorithm takes a certain amount of time, the speed at which the trained model processes instances in the later stage is very impressive, and results can often be obtained in a few milliseconds, which is very consistent with the requirements of reaction speed in transportation scenarios. In addition, the introduction of the attention mechanism and graph neural network can improve the algorithm's ability to search for the optimal solution and the accuracy of the solution. Therefore, the algorithm can avoid the shortcomings of traditional algorithms, such as poor global search capabilities, easy to fall into local optimality and then stagnate, thus causing problems and disadvantages in path planning. Summary of the invention

[0007] The present invention aims to address the deficiencies of the above-mentioned prior art and proposes a path optimization method for unmanned delivery vehicles based on attention mechanism and graph neural network, in order to meet the reaction speed requirements of unmanned delivery vehicle transportation scenarios while improving customer satisfaction, thereby improving the delivery efficiency of enterprises and reducing the delivery costs of logistics enterprises, so as to solve the problems of slow speed, low accuracy and no consideration of actual customer needs when using existing algorithms for multi-node path planning.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical scheme:

[0009] The unmanned delivery vehicle path optimization method based on attention mechanism and graph neural network is characterized in that it is applied to a single warehouse and n delivery customer nodes with delivery time constraints. In the vehicle delivery task consisting of K and K unmanned delivery vehicles with limited capacity, represents the i'th delivery customer node, The unmanned delivery vehicle path optimization method is carried out according to the following steps:

[0010] S1. Establish a delivery task model for unmanned delivery vehicles;

[0011] S1.1. Define a warehouse node The node set consisting of n delivery customer nodes is denoted as V , taking a vertex of the circumscribed rectangle of the delivery area as the origin and the two adjacent sides of the vertex as the x-axis and y-axis respectively, thus establishing a rectangular coordinate system, where: Indicates nodes, let the i-th node The node coordinates are marked as ( );

[0012] Defining a set of paths ,in, Represents the i-th node To the jth node Path;

[0013] S1.2. Use formula (1) to establish the objective function of the unmanned delivery vehicle delivery task model :

[0014] (1)

[0015] In formula (1), Represents the total number of unmanned delivery vehicles; Represents the i-th node and the jth node Path between distance; Indicates Whether the unmanned delivery vehicle passes the i-th node To the jth node If the path between , then let , otherwise, let ; v represents the specific fuel consumption per unit time; Indicates that the unmanned delivery vehicle starts from the i-th node To the jth node The time used; h represents the unit energy consumption; Indicates unit transportation consumption; Indicates 'Delivery customer nodes The penalty function is Indicates that the kth vehicle leaves the ith node Load at the time;

[0016] S1.3, using formula (2) to construct 'Delivery customer nodes Satisfaction :

[0017] (2)

[0018] In formula (1), Indicates An unmanned delivery vehicle arrived at 'Delivery customer nodes time, and For the 'Delivery customer nodes The upper and lower limits of the tolerance time window; and For the 'Delivery customer nodes The upper and lower limits of the service time window;

[0019] S1.4. Use equations (3) to (15) to construct the constraints of the unmanned delivery vehicle delivery task model:

[0020] (3)

[0021] n(4)

[0022] (5)

[0023] (6)

[0024] (7)

[0025] (8)

[0026] (9)

[0027] (10)

[0028] (11)

[0029] (12)

[0030] (13)

[0031] (14)

[0032] (15)

[0033] In formula (3) to formula (15), No. Nodes Quantity of demand; Indicates that the unmanned delivery vehicle has arrived at Nodes time node, Represents the i-th node The time of the required service, D represents the maximum travel distance of each vehicle; represents the maximum loading capacity of each vehicle, Represents the i-th node The loading volume, Represents the i-th node The unloading volume, Represents the i-th node Whether Unmanned delivery vehicle service, and Represents 2 penalty coefficients;

[0034] S2. Warehouse nodes The upper limit of the service time window is initialized to positive infinity, the lower limit of the service time window is initialized to negative infinity, and the warehouse node The upper limit of the time tolerance window is initialized to positive infinity, and the lower limit of the time tolerance window is initialized to negative infinity, so that the i-th node The node coordinates , the i-th node The upper and lower limits of the tolerance time window, the upper and lower limits of the service time window, and the demand Composed of the i-th node The eigenvector of ; Thus we get the feature vector set ;

[0035] S3. Construct an attention mechanism and a graph neural network, including an encoder, a first decoder, and a second decoder, and solve the unmanned delivery vehicle delivery task model to obtain the unmanned delivery vehicle path;

[0036] S3.1. Construct an encoder based on the attention mechanism, including: linear mapping layer, multi-head attention layer and feedforward layer, and use it to After processing, we get the embedding vector h= ; and average the embedding vector h to get ;in, Represents the i-th node The embedding vector of

[0037] S3.2. Construct the first decoder based on the attention mechanism, including: context decoding layer, multi-head attention layer, single-head attention layer, linear layer and normalization layer, and perform Processing is performed to obtain the distribution plan of K unmanned delivery vehicles at the tth time step and the prediction matrix of K unmanned delivery vehicles at the tth time step ;

[0038] S3.3, the second decoder uses formula (20) to Processing is performed to obtain the graph embedding vector of the tth time step , so using formula (21) Processing is performed to obtain the probability of the existence of edges between nodes at the tth time step ;

[0039] (20)

[0040] (twenty one)

[0041] In formulas (20) and (21), Represents the i-th node A neighbor node of represents the i-th node at time step t The set of neighbor nodes of and represents two normalization factors, and Represents the weight matrix and bias vector to be learned; represents a multi-layer perceptron, and softmax represents an activation operation;

[0042] S3.4. Use integrated reinforcement learning and pseudo-label learning training methods to train the decoder to output the optimal delivery path solution for the unmanned delivery vehicle.

[0043] The unmanned delivery vehicle path optimization method based on attention mechanism and graph neural network described in the present invention is also characterized in that S3.1 includes:

[0044] S3.1.1, define the current number of coding layers as , and initialize ;

[0045] The linear mapping layer uses formula (16) to obtain the i-th node In the Layer Embedding , so that all nodes in Embedding vector of the layer ;

[0046] (16)

[0047] In formula (16), and For the The weights and biases of the layer to be trained;

[0048] S3.1.2, the multi-head attention layer Process and get the i-th node In the Attention vector of the layer , so that all nodes in Embedding vector of the layer ;

[0049] S3.1.3, the feed-forward layer Process and get the i-th node In the +1 layer of embedding vector ;

[0050] S3.1.4. +1 assigned to Then, return to step 3.1.2 until >N, get the embedding vector h= of all nodes in the last layer , where N is the total number of coding layers.

[0051] Furthermore, S3.2 includes:

[0052] S3.2.1. Define the total number of time steps as T, define the current time step as t, and initialize t=1;

[0053] S3.2.2, when t=1, the following decoding layer uses formula (17a) to obtain the first The context feature vector of an unmanned delivery vehicle at the tth time step :

[0054] (17a)

[0055] In formula (17a), and Represents two placeholder parameters to be trained. For horizontal splicing operation;

[0056] When t 1, the following decoding layer uses formula (17b) to obtain the first The context feature vector of an unmanned delivery vehicle at the tth time step :

[0057] (17b)

[0058] In formula (17b), Indicates The context feature vector of an unmanned delivery vehicle at the t-1th time step;

[0059] S3.2.3, the multi-head attention layer will Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ; Thus, the multi-head attention mechanism is used to calculate and The similarity between , and after inputting the softmax function, we get The updated context node embedding of an unmanned delivery vehicle at time step t ;

[0060] S3.2.4, the single-head attention layer will Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ; Thus, the single-head attention mechanism is used to calculate and The similarity between them is corrected using the tanh function to obtain Similarity in range ; Wherein, F represents the correction factor;

[0061] S3.2.5. Use formula (18) to construct the i-th node The mask condition of If equation (18) is satisfied, then The i-th node in Mask all nodes that satisfy equation (18) and get The unmanned delivery vehicle is masked at the tth time step. ;

[0062] (18)

[0063] In formula (18), is the i-th node The demand at the tth time step is Indicates The remaining capacity of the unmanned delivery vehicles arriving at the current location at the tth time step;

[0064] S3.2.6, the linear layer uses Function pair Process it and get The probability of selecting each node at the tth time step for an unmanned delivery vehicle , and then get the probability of each node selection of K unmanned delivery vehicles at the tth time step , and select the node corresponding to the maximum probability value as the The nodes visited by the unmanned delivery vehicle at the tth time step, thus forming the The delivery plan of an unmanned delivery vehicle at the tth time step ; Then we get the distribution plan of K unmanned delivery vehicles at the tth time step ;

[0065] S3.2.7. The delivery plan based on K unmanned delivery vehicles at the tth time step ,right After 0-1 encoding and matrix processing of the encoded path set, the prediction matrix of K unmanned delivery vehicles at the tth time step is obtained. .

[0066] Furthermore, S3.4 includes:

[0067] S3.4.1. Calculate the objective function value of the unmanned delivery vehicle delivery task model at time step t-1 using formula (1): , and then use formula (22) to calculate Rewards :

[0068] (twenty two)

[0069] S3.4.2. Use formula (23) to obtain the status information of all vehicles at time step t :

[0070] (twenty three)

[0071] In formula (23), represents the node solution that the k-th vehicle has visited before time step t, is the cumulative travel time of the kth vehicle at time step t+1; represents the remaining capacity of the kth vehicle at time step t;

[0072] S3.4.3. Construct the state information of the i-th node at the t-th time step , thus obtaining the state information of all nodes at the tth time step , and then construct the state information of all unmanned delivery vehicles at all nodes in the tth time step and record it as ;

[0073] S3.4.3, according to , use the first decoder to obtain the distribution plan of K unmanned delivery vehicles at the tth time step , and then use formula (22) to calculate the reward value of the t-th time step ;

[0074] S3.4.4. Use formula (24) to construct the loss of the first decoder at the tth time step ;

[0075] (twenty four)

[0076] In formula (24), represents the parameters to be trained of the first decoder, is the baseline, indicating the minimum reward value at the tth time. When t=0, it is initialized =0, Indicates that the distribution plan is obtained at time step t-1 Solve the distribution plan for the tth time step under the condition The selection probability distribution of each node at the time;

[0077] S3.4.5. Use equation (25) to construct the loss of the second decoder at the tth time step ;

[0078] (25)

[0079] In formula (25), represents the cross entropy loss function, represents the parameters to be trained of the second decoder;

[0080] S3.4.6. Use formula (26) to construct the total loss function at the tth time step ;

[0081] (26)

[0082] S3.4.7. Total loss function With minimization as the goal, the back propagation algorithm is used to optimize and update the parameters in the attention mechanism and graph neural network until After convergence or reaching the maximum number of iterations, the training is stopped to generate an unmanned delivery vehicle path solution model corresponding to the optimal parameters, which is used to output the optimal delivery path solution for the unmanned delivery vehicle.

[0083] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the unmanned delivery vehicle path optimization method, and the processor is configured to execute the program stored in the memory.

[0084] The present invention provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, which is characterized in that when the computer program is run by a processor, the steps of the unmanned delivery vehicle path optimization method are executed.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] 1. The present invention proposes a function for calculating customer satisfaction based on node arrival time, and then proposes a new penalty function calculation method to enhance the practicality of cost calculation and make the model more suitable for actual delivery scenarios. Customers have different satisfaction evaluations for different delivery times, so as to reduce the probability of not delivering within the customer node time window when solving the path, thereby improving customer satisfaction.

[0087] 2. The present invention proposes a method based on attention mechanism and graph neural network to solve the delivery task model of unmanned delivery vehicles. This method uses the probability matrix output by the first decoder as the pseudo-label of the second decoder, effectively combining the features that the attention mechanism focuses on and the features that the graph neural network focuses on. This idea can effectively focus on the parameters that minimize the objective function, thereby improving the efficiency and accuracy of the algorithm in solving the delivery task model of unmanned delivery vehicles, and thus improving the practicability of the algorithm in actual delivery scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is an algorithm framework diagram of the algorithm in the embodiment of the present invention. DETAILED DESCRIPTION

[0089] In this embodiment, Figure 1 As shown in Figure 1, a method for unmanned delivery vehicle path optimization based on attention mechanism and graph neural network is applied to a single warehouse and n delivery customer nodes with delivery time constraints. In the vehicle delivery task consisting of K and K unmanned delivery vehicles with limited capacity, represents the i'th delivery customer node, ; The unmanned delivery vehicle path optimization method is carried out according to the following steps:

[0090] S1. Establish a delivery task model for unmanned delivery vehicles;

[0091] S1.1. Define a warehouse node The node set consisting of n delivery customer nodes is denoted as V , taking a vertex of the circumscribed rectangle of the delivery area as the origin and the two adjacent sides of the vertex as the x-axis and y-axis respectively, thus establishing a rectangular coordinate system, where: Indicates nodes, let the i-th node The node coordinates are marked as ( );

[0092] Defining a set of paths ,in, Represents the i-th node To the jth node Path.

[0093] S1.2. Use formula (1) to establish the objective function of the unmanned delivery vehicle delivery task model :

[0094] (1)

[0095] In formula (1), Represents the total number of unmanned delivery vehicles; Represents the i-th node and the jth node Path between distance; Indicates Whether the unmanned delivery vehicle passes the i-th node To the jth node If the path between , then let , otherwise, let ; v represents the specific fuel consumption per unit time; Indicates that the unmanned delivery vehicle starts from the i-th node To the jth node The time used; h represents the unit energy consumption; Indicates unit transportation consumption; Indicates 'Delivery customer nodes The penalty function is Indicates that the kth vehicle leaves the ith node Load at the time.

[0096] S1.3, using formula (2) to construct 'Delivery customer nodes Satisfaction :

[0097] (2)

[0098] In formula (1), Indicates An unmanned delivery vehicle arrived at 'Delivery customer nodes time, and For the 'Delivery customer nodes The upper and lower limits of the tolerance time window; and For the 'Delivery customer nodes The upper and lower bounds of the service time window.

[0099] S1.4. Use equations (3) to (15) to construct the constraints of the unmanned delivery vehicle delivery task model:

[0100] (3)

[0101] n(4)

[0102] (5)

[0103] (6)

[0104] (7)

[0105] (8)

[0106] (9)

[0107] (10)

[0108] (11)

[0109] (12)

[0110] (13)

[0111] (14)

[0112] (15)

[0113] In formula (3) to formula (15), No. Nodes Quantity of demand; Indicates that the unmanned delivery vehicle has arrived at Nodes time node, Represents the i-th node The time of the required service, D represents the maximum travel distance of each vehicle; represents the maximum loading capacity of each vehicle, Represents the i-th node The loading volume, Represents the i-th node The unloading volume, Represents the i-th node Whether Unmanned delivery vehicle service, and Represents 2 penalty coefficients.

[0114] S2. Warehouse nodes The upper limit of the service time window is initialized to positive infinity, the lower limit of the service time window is initialized to negative infinity, and the warehouse node The upper limit of the time tolerance window is initialized to positive infinity, and the lower limit of the time tolerance window is initialized to negative infinity, so that the i-th node The node coordinates , the i-th node The upper and lower limits of the tolerance time window, the upper and lower limits of the service time window, and the demand Composed of the i-th node The eigenvector of ; Thus we get the feature vector set .

[0115] S3. Construct an attention mechanism and a graph neural network, including an encoder, a first decoder, and a second decoder, and solve the unmanned delivery vehicle delivery task model to obtain the unmanned delivery vehicle path;

[0116] S3.1. Construct an encoder based on the attention mechanism, including: linear mapping layer, multi-head attention layer and feedforward layer, and use it to Processing to obtain the embedding vector ;in, Represents the i-th node The embedding vector of .

[0117] S3.1.1, define the current number of coding layers as , and initialize ;

[0118] The linear mapping layer uses formula (16) to get the i-th node In the Layer Embedding , so that all nodes in Embedding vector of the layer ;

[0119] (16)

[0120] In formula (16), and For the The weights and biases of the layer to be trained.

[0121] S3.1.2, Multi-head attention layer uses formula (17) to get the i-th node The attention vector , so that all nodes are Embedding vector of the layer ;

[0122] (17)

[0123] In formula (17), represents the batch normalization layer, Represents a multi-head attention layer;

[0124] S3.1.3, the feed-forward layer uses (18) Perform update processing to obtain the i-th node In the +1 layer of embedding vector , where each sub-layer adds residual connections and batch normalization;

[0125] (18)

[0126] In formula (17) and formula (18), represents the multi-head attention layer, represents the fully connected layer, is the output of the multi-head attention layer, Represents a batch normalization layer.

[0127] S3.1.3.1. Larger hidden layer dimensions help the model capture richer and higher-level features. For complex tasks such as unmanned delivery vehicles, higher dimensions and nonlinear activation functions can provide more space to represent complex input information while avoiding the problem of gradient disappearance. Therefore, the fully connected layer uses formula (19) to calculate the output of the multi-head attention layer. Transform using a hidden layer with a dimension of 512 and a ReLU activation function:

[0128] (19)

[0129] In formula (19), , , and Represents four learned parameters.

[0130] S3.1.3.2. When training a deep neural network, the input data distribution between different layers may change. Therefore, batch normalization with learnable parameters is implemented after each fully connected layer using formula (20). By standardizing the input of each layer, this change is reduced and the instability of gradient propagation between layers is avoided.

[0131] (20)

[0132] In formula (20), is batch normalization without affine transformation, is the Hadamard product. and are learnable affine parameters.

[0133] S3.1.4. +1 assigned to Then, return to step 3.1.2 until >N, get the embedding vector h= of all nodes in the last layer , where N is the total number of coding layers;

[0134] S3.1.5. Average the embedding vector h to obtain ;

[0135] S3.2. Construct the first decoder based on the attention mechanism, including: context decoding layer, multi-head attention layer, single-head attention layer, linear layer and normalization layer, and perform Processing is performed to obtain the distribution plan of K unmanned delivery vehicles at the tth time step and the prediction matrix of K unmanned delivery vehicles at the tth time step .

[0136] S3.2.1. Define the total number of time steps as T, define the current time step as t, and initialize t=1;

[0137] S3.2.2, when t = 1, the following decoding layer uses formula (21a) to obtain The context feature vector of an unmanned delivery vehicle at the tth time step :

[0138] (21a)

[0139] In formula (21a), and Represents two placeholder parameters to be trained. For horizontal splicing operation;

[0140] When t 1, the following decoding layer uses formula (22b) to obtain the first The context feature vector of an unmanned delivery vehicle at the tth time step :

[0141] (21b)

[0142] In formula (17b), Indicates The context feature vector of an unmanned delivery vehicle at the t-1th time step.

[0143] S3.2.3, the multi-head attention layer uses formula (22) to Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ;

[0144] (twenty two)

[0145] Thus, equation (23) is used to calculate the node context feature vector of vehicle k at time step t: With the embedding vector The similarity between ;

[0146] (twenty three)

[0147] Then the similarity Perform softmax calculation, and then use formula (24) to update the context node embedding to obtain ;

[0148] (twenty four)

[0149] S3.2.4, single-head attention layer Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ; Then use formula (25) to calculate and embedding vector Similarity, at this time the query matrix ,in is the parameter to be trained in the single-head attention layer, and then the tanh function is used to correct the result to Get similarity in range :

[0150] (25)

[0151] Wherein, F represents the correction factor.

[0152] S3.2.5. Use formula (26) to construct the i-th node The mask condition of If equation (18) is satisfied, then The i-th node in Mask all nodes that satisfy equation (18) and get The unmanned delivery vehicle is masked at the tth time step. ;

[0153] (26)

[0154] In formula (26), is the i-th node The demand at the tth time step is Indicates The remaining capacity of the unmanned delivery vehicle arriving at the current location at the tth time step.

[0155] S3.2.6, linear layer uses formula (27) to Process it and get The probability of selecting each node in t time steps for an unmanned delivery vehicle , and then get the probability of each node selection of K unmanned delivery vehicles at the tth time step , and select the node corresponding to the maximum probability value as the The nodes visited by the unmanned delivery vehicle at the tth time step, thus forming the The delivery plan of an unmanned delivery vehicle at the tth time step ; Then we get the distribution plan of K unmanned delivery vehicles at the tth time step ;

[0156] (27)

[0157] S3.2.7. The delivery plan based on K unmanned delivery vehicles at the tth time step ,right Perform 0-1 encoding, where the i-th node With the jth node When there is an edge connection , the i-th node With the jth node When there is no edge connection , and the encoded path set is matrixed to obtain the prediction matrix of K unmanned delivery vehicles at the tth time step ;

[0158] S3.3, the second decoder uses formula (28) to Processing is performed to obtain the graph embedding vector of the tth time step , so using formula (29) Processing is performed to obtain the probability of the existence of edges between nodes at the tth time step ;

[0159] (28)

[0160] (29)

[0161] In formulas (28) and (29), Represents the i-th node A neighbor node of represents the i-th node at time step t The set of neighbor nodes of and represents two normalization factors, and Represents the weight matrix and bias vector to be learned; represents a multi-layer perceptron, and softmax represents an activation operation.

[0162] S3.4. Use integrated reinforcement learning and pseudo-label learning training methods to train the decoder, and finally output the optimal delivery path plan for the unmanned delivery vehicle.

[0163] S3.4.1. Calculate the objective function value of the unmanned delivery vehicle delivery task model at time step t-1 using formula (1): , so formula (30) calculates Rewards :

[0164] (30)

[0165] S3.4.2. Use formula (31) to obtain the status information of all vehicles at time step t :

[0166] (31)

[0167] In formula (31), represents the node solution that the k-th vehicle has visited before time step t, is the cumulative travel time of the kth vehicle at time step t+1; represents the remaining capacity of the kth vehicle at time step t;

[0168] S3.4.3. Construct the state information of the i-th node at the t-th time step , thus obtaining the state information of all nodes at the tth time step , and then construct the state information of all unmanned delivery vehicles at all nodes in the tth time step and record it as .

[0169] S3.4.4, according to , use the first decoder to get the distribution plan of K unmanned delivery vehicles at the tth time step , and then use formula (30) to calculate the reward value of the t-th time step ;

[0170] S3.4.5. Use the stochastic gradient descent method to train the network. The gradient calculation formula used is formula (32):

[0171] (32)

[0172] In formula (32), represents the reward value at time step t, represents the parameters to be trained of the first decoder, is the baseline, indicating the minimum reward value at the tth time. When t=0, it is initialized is 0.

[0173] Use equation (33) to construct the loss of the first decoder at the tth time step ;

[0174] (33)

[0175] In formula (33), Indicates that the distribution plan is obtained at time step t-1 Solve the distribution plan for the tth time step under the condition The selection probability distribution of each node at the time.

[0176] S3.4.5. Use equation (34) to construct the loss of the second decoder at the tth time step ;

[0177] (34)

[0178] In formula (25), represents the cross entropy loss function, represents the parameters to be trained of the second decoder;

[0179] S3.4.6. Use formula (35) to construct the total loss function at the tth time step ;

[0180] (35)

[0181] S3.4.7. Total loss function With minimization as the goal, the back propagation algorithm is used to optimize and update the parameters in the attention mechanism and graph neural network until After convergence or reaching the maximum number of iterations, the training is stopped to generate an unmanned delivery vehicle path solution model corresponding to the optimal parameters, which is used to output the optimal delivery path solution for the unmanned delivery vehicle.

[0182] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0183] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

Claims

1. A method for unmanned delivery vehicle path optimization based on attention mechanism and graph neural network, characterized in that: It is applied to a single warehouse and n delivery customer nodes with delivery time constraints. In the vehicle delivery task consisting of K and K unmanned delivery vehicles with limited capacity, represents the i'th delivery customer node, The unmanned delivery vehicle path optimization method is carried out according to the following steps: S1. Establish a delivery task model for unmanned delivery vehicles; S1.

1. Define a warehouse node The node set consisting of n delivery customer nodes is denoted as V , taking a vertex of the circumscribed rectangle of the delivery area as the origin and the two adjacent sides of the vertex as the x-axis and y-axis respectively, thus establishing a rectangular coordinate system, where: Indicates nodes, let the i-th node The node coordinates are marked as ( ); Defining a set of paths ,in, Represents the i-th node To the jth node Path; S1.

2. Use formula (1) to establish the objective function of the unmanned delivery vehicle delivery task model : (1) In formula (1), Represents the total number of unmanned delivery vehicles; Represents the i-th node and the jth node Path between distance; Indicates Whether the unmanned delivery vehicle passes the i-th node To the jth node If the path between , then let , otherwise, let ; v represents the specific fuel consumption per unit time; Indicates that the unmanned delivery vehicle starts from the i-th node To the jth node The time used; h represents the unit energy consumption; Indicates unit transportation consumption; Indicates 'Delivery customer nodes The penalty function is Indicates that the kth vehicle leaves the ith node Load at the time; S1.3, using formula (2) to construct 'Delivery customer nodes Satisfaction : (2) In formula (1), Indicates An unmanned delivery vehicle arrived at 'Delivery customer nodes time, and For the 'Delivery customer nodes The upper and lower limits of the tolerance time window; and For the 'Delivery customer nodes The upper and lower limits of the service time window; S1.

4. Use equations (3) to (15) to construct the constraints of the unmanned delivery vehicle delivery task model: (3) n(4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) In formula (3) to formula (15), No. Nodes Quantity of demand; Indicates that the unmanned delivery vehicle has arrived at Nodes time node, Represents the i-th node The time of the required service, D represents the maximum travel distance of each vehicle; represents the maximum loading capacity of each vehicle, Represents the i-th node The loading volume, Represents the i-th node The unloading volume, Represents the i-th node Whether Unmanned delivery vehicle service, and Represents 2 penalty coefficients; S2. Warehouse nodes The upper limit of the service time window is initialized to positive infinity, the lower limit of the service time window is initialized to negative infinity, and the warehouse node The upper limit of the time tolerance window is initialized to positive infinity, and the lower limit of the time tolerance window is initialized to negative infinity, so that the i-th node The node coordinates , the i-th node The upper and lower limits of the tolerance time window, the upper and lower limits of the service time window, and the demand Composed of the i-th node The eigenvector of ; Thus we get the feature vector set ; S3. Construct an attention mechanism and a graph neural network, including an encoder, a first decoder, and a second decoder, and solve the unmanned delivery vehicle delivery task model to obtain the unmanned delivery vehicle path; S3.

1. Construct an encoder based on the attention mechanism, including: linear mapping layer, multi-head attention layer and feedforward layer, and use it to After processing, we get the embedding vector h= ; and average the embedding vector h to get ;in, Represents the i-th node The embedding vector of S3.

2. Construct the first decoder based on the attention mechanism, including: context decoding layer, multi-head attention layer, single-head attention layer, linear layer and normalization layer, and perform Processing is performed to obtain the distribution plan of K unmanned delivery vehicles at the tth time step and the prediction matrix of K unmanned delivery vehicles at the tth time step ; S3.3, the second decoder uses formula (20) to Processing is performed to obtain the graph embedding vector of the tth time step , so using formula (21) Processing is performed to obtain the probability of the existence of edges between nodes at the tth time step ; (20) (21) In formulas (20) and (21), Represents the i-th node A neighbor node of represents the i-th node at time step t The set of neighbor nodes of and represents two normalization factors, and Represents the weight matrix and bias vector to be learned; represents a multi-layer perceptron, and softmax represents an activation operation; S3.

4. Use integrated reinforcement learning and pseudo-label learning training methods to train the decoder to output the optimal delivery path solution for the unmanned delivery vehicle.

2. According to claim 1, a method for optimizing the path of an unmanned delivery vehicle based on an attention mechanism and a graph neural network is characterized in that: S3.1 includes: S3.1.1, define the current number of coding layers as , and initialize ; The linear mapping layer uses formula (16) to obtain the i-th node In the Layer Embedding , so that all nodes in Embedding vector of the layer ; (16) In formula (16), and For the The weights and biases of the layer to be trained; S3.1.2, the multi-head attention layer Process and get the i-th node In the Attention vector of the layer , so that all nodes in Embedding vector of the layer ; S3.1.3, the feed-forward layer Process and get the i-th node In the +1 layer of embedding vector ; S3.1.

4. +1 assigned to Then, return to step 3.1.2 until >N, get the embedding vector h= of all nodes in the last layer , where N is the total number of coding layers.

3. According to claim 2, a method for optimizing the path of an unmanned delivery vehicle based on an attention mechanism and a graph neural network is characterized in that: S3.2 includes: S3.2.

1. Define the total number of time steps as T, define the current time step as t, and initialize t=1; S3.2.2, when t=1, the following decoding layer uses formula (17a) to obtain the first The context feature vector of an unmanned delivery vehicle at the tth time step : (17a) In formula (17a), and Represents two placeholder parameters to be trained. For horizontal splicing operation; When t 1, the following decoding layer uses formula (17b) to obtain the first The context feature vector of an unmanned delivery vehicle at the tth time step : (17b) In formula (17b), Indicates The context feature vector of an unmanned delivery vehicle at the t-1th time step; S3.2.3, the multi-head attention layer will Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ; Thus, the multi-head attention mechanism is used to calculate and The similarity between , and after inputting the softmax function, we get The updated context node embedding of an unmanned delivery vehicle at time step t ; S3.2.4, the single-head attention layer will Mapping to key matrix Sum Matrix ,Will Mapped to the query matrix of the kth vehicle at the tth time step ; Thus, the single-head attention mechanism is used to calculate and The similarity between them is corrected using the tanh function to obtain Similarity in range ; Wherein, F represents the correction factor; S3.2.

5. Use formula (18) to construct the i-th node The mask condition of If equation (18) is satisfied, then The i-th node in Mask all nodes that satisfy equation (18) and get The unmanned delivery vehicle is masked at the tth time step. ; (18) In formula (18), is the i-th node The demand at the tth time step is Indicates The remaining capacity of the unmanned delivery vehicles arriving at the current location at the tth time step; S3.2.6, the linear layer uses Function pair Process it and get The probability of selecting each node at the tth time step for an unmanned delivery vehicle , and then get the probability of each node selection of K unmanned delivery vehicles at the tth time step , and select the node corresponding to the maximum probability value as the The nodes visited by the unmanned delivery vehicle at the tth time step, thus forming the The delivery plan of an unmanned delivery vehicle at the tth time step ; Then we get the distribution plan of K unmanned delivery vehicles at the tth time step ; S3.2.

7. The delivery plan based on K unmanned delivery vehicles at the tth time step ,right After 0-1 encoding and matrix processing of the encoded path set, the prediction matrix of K unmanned delivery vehicles at the tth time step is obtained. .

4. According to claim 3, a method for optimizing the path of an unmanned delivery vehicle based on an attention mechanism and a graph neural network is characterized in that: S3.4 includes: S3.4.

1. Calculate the objective function value of the unmanned delivery vehicle delivery task model at time step t-1 using formula (1): , and then use formula (22) to calculate Rewards : (22) S3.4.

2. Use formula (23) to obtain the status information of all vehicles at time step t : (23) In formula (23), represents the node solution that the kth vehicle has visited before time step t, is the cumulative travel time of the kth vehicle at time step t+1; represents the remaining capacity of the kth vehicle at time step t; S3.4.

3. Construct the state information of the i-th node at the t-th time step , thus obtaining the state information of all nodes at the tth time step , and then construct the state information of all unmanned delivery vehicles at all nodes in the tth time step and record it as ; S3.4.3, according to , use the first decoder to obtain the distribution plan of K unmanned delivery vehicles at the tth time step , and then use formula (22) to calculate the reward value of the t-th time step ; S3.4.

4. Use formula (24) to construct the loss of the first decoder at the tth time step ; (24) In formula (24), represents the parameters to be trained of the first decoder, is the baseline, indicating the minimum reward value at the tth time. When t=0, it is initialized =0, Indicates that the distribution plan is obtained at time step t-1 Solve the distribution plan for the tth time step under the condition The selection probability distribution of each node at the time; S3.4.

5. Use equation (25) to construct the loss of the second decoder at the tth time step ; (25) In formula (25), represents the cross entropy loss function, represents the parameters to be trained of the second decoder; S3.4.

6. Use formula (26) to construct the total loss function at the tth time step ; (26) S3.4.

7. Total loss function With minimization as the goal, the back propagation algorithm is used to optimize and update the parameters in the attention mechanism and graph neural network until After convergence or reaching the maximum number of iterations, the training is stopped to generate an unmanned delivery vehicle path solution model corresponding to the optimal parameters, which is used to output the optimal delivery path solution for the unmanned delivery vehicle.

5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the unmanned delivery vehicle path optimization method described in any one of claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the unmanned delivery vehicle path optimization method described in any one of claims 1 to 4 are executed.

Images