A learning-based logistics fleet route optimization method and system based on edge weight features

Abstract

The invention relates to the field of intelligent transportation technology, and discloses a learning-based logistics fleet route optimization method and system based on edge weight features, including the following steps: constructing a logistics fleet route optimization problem model; designing a learning-based decision-making method based on edge weight features, including: generating a training data set containing an edge weight matrix, designing a strategy network based on a graph neural network, and constructing a reinforcement learning training algorithm; based on the learning-based decision-making method, constructing a hierarchical solution architecture to optimize the logistics fleet route. The learning-based logistics fleet route optimization method and system based on edge weight features, the method of the invention has high real-time and robustness in solving the route planning problem of the logistics fleet, and realizes the overall efficiency improvement of logistics transportation.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation, and specifically relates to a learning-based logistics fleet route optimization method and system based on edge weight features. Background Art

[0002] With the rapid development of the logistics industry, optimizing the driving routes of logistics fleets has become the key to improving transportation efficiency and reducing costs.

[0003] The route optimization problem of logistics fleets is usually modeled as a vehicle routing problem. Traditional solutions to vehicle routing problems include exact algorithms and heuristic algorithms, but these algorithms are difficult to balance solution quality and calculation speed. With the continuous development of machine learning, learning algorithms are gradually being used to solve various combinatorial optimization problems. These methods use the parameter update process of neural networks to establish data-driven search rules, thereby achieving a more efficient solution process. In recent years, many studies have found that learning algorithms have shown better performance than traditional methods in vehicle routing problems. Therefore, applying learning algorithms to route optimization of logistics fleets is of great significance for improving logistics efficiency and reducing logistics costs.

[0004] However, in the prior art, the study of learning algorithms for logistics vehicle route optimization usually only encodes the coordinates of the vertices as features, while ignoring the features of the edges. In actual urban traffic networks, the driving edge weight between two points is often inconsistent with the straight-line distance between the vertices. Therefore, in order to improve the application effect of learning algorithms in real urban road networks, edge weight features must be considered. The current model has limitations in practical applications because it fails to fully consider these factors. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the shortcomings of the prior art, the present invention provides a learning logistics fleet route optimization method and system based on edge weight features, which fully considers the actual situation and the existing technology level, constructs a logistics fleet route optimization problem model, designs a learning decision method based on edge weight features, and finally optimizes the driving routes of logistics vehicles in combination with a hierarchical solution architecture.

[0007] (II) Technical solution

[0008] In order to solve the above problems, the present invention provides the following technical solutions:

[0009] The present invention first provides a learning-based logistics fleet route optimization method and system based on edge weight features, comprising the following steps:

[0010] Step 1: Define the logistics fleet path optimization problem based on the complete directed graph, set constraints, determine the objective function, and complete the mathematical model construction;

[0011] Step 2: Generate a training data set containing an edge weight matrix, design a policy network based on a graph neural network, build a reinforcement learning algorithm to train the policy network, and complete the design of a learning decision algorithm;

[0012] Step 3: Design a hierarchical solution architecture. The lower layer obtains the optimal path between any two points in the road network based on the shortest path algorithm to form an edge weight matrix. The upper layer uses a learning decision algorithm based on the edge weight matrix to obtain the optimal driving path for the logistics fleet.

[0013] In step 1, the complete directed graph is represented as G = (V, E), where warehouses and customers are constructed as vertices, and the vertices are connected by directed edges; V represents the set of all vertices on the graph, which consists of two different subsets: the warehouse set D = {0}, and the customer set C = {1,2,...,N C +1} indicates that the client vertex is combined, so the total number of vertices in the graph N = 1 + N C E = {(i,j)|i,j∈V,i≠j} represents the edge set connecting the points;

[0014] The logistics vehicle routing optimization problem can be described in the figure: the number is N C The customers of are distributed in the complete directed graph G, and each customer (i∈C) has his own demand for goods q i ; A certain number of logistics fleets K = {1,2,…,k}, starting from a unified warehouse, visit these customer points, and return to the warehouse after completing the delivery; The optimization goal is to design a series of driving routes for the logistics fleet, which can minimize the given objective function value while satisfying all constraints; The mathematical model of the logistics fleet route optimization problem can be expressed as follows:

[0015] Objective function:

[0016]

[0017] Constraints:

[0018]

[0019] u0=U (5)

[0020]

[0021] Other constraints(8)

[0022] Among them, x ij is a binary decision variable. When a vehicle leaves node i and arrives at node j, xij is 1, otherwise it is 0; U represents the maximum load of the vehicle; u i represents the load of the vehicle when it arrives at node i; c ij Represents the edge weight of edge (i,j).

[0023] Constraint (2) indicates that each customer needs to be visited by the service and only once; constraint (3) indicates the flow conservation constraint; constraint (4) tracks the cargo volume of the vehicle; constraint (5) indicates that the vehicle departs from the warehouse in a fully loaded state; constraint (6) indicates that the number of vehicles used does not exceed the maximum number K; constraint (7) indicates that the decision variables are subject to the {0,1} constraint; finally, corresponding constraints can be added according to other constraints in actual logistics transportation, such as time windows.

[0024] The above logistics fleet path optimization problem model is a generalized modeling method. ij It can be set according to actual problems, such as driving distance, driving time, driving cost, etc., and other constraints can be added according to actual logistics needs; the learning decision method in step 2 only needs to obtain the edge weight matrix between nodes and the corresponding constraints to solve the generalized logistics vehicle route optimization problem.

[0025] Step 2: Based on the problem model constructed in step 1, a learning decision-making method is constructed, including: generating a training data set containing an edge weight matrix, designing a strategy network based on a graph neural network, and constructing a reinforcement learning training algorithm.

[0026] The learning decision method belongs to the data-driven algorithm. First, the data set required for training needs to be generated. The present invention generates D logistics route optimization problem examples as data sets. In each logistics route optimization problem example, the two-dimensional coordinates of the customer and the warehouse are randomly sampled and generated in a uniformly distributed manner within [0,1]. The cargo demand of each customer is regarded as discrete and is in [0,q max ] is randomly generated by uniform sampling, q max represents the maximum value of the set demand, and the demand of each customer is normalized by the following formula:

[0027]

[0028] For each instance of the edge weight matrix, the edge weight c between two points ij Determined by the Euclidean distance; at the same time, in order to simulate the actual urban road network c ij With c ji Usually not equal, c is determined according to the following formula ji Values:

[0029]

[0030] In the above formula, Gaussian noise is a random variable with a state distribution, σ represents the standard deviation of the distribution; finally, the obtained edge weight matrix is ​​normalized to the maximum and minimum values:

[0031]

[0032] c min is the minimum value in the edge weight matrix, because when i=j, c ij =0, so all values ​​of the edge weight matrix are equal to c max The proportion is scaled to the interval [0,1], and the edge weights are proportionally scaled without affecting the composition of the final solution. In the designed learning decision algorithm, each node feature where d i represents the average distance between node i and all its vertices, and the characteristics of edge (i, j) are Finally, according to the size of the training set D and the corresponding feature information in the example, a data set for logistics fleet route optimization problem is generated.

[0033] After obtaining the data set, the present invention adopts a reinforcement learning algorithm set policy network to make decisions; the designed policy network consists of an encoder and a decoder; the encoder is composed of a graph attention network, which is responsible for encoding the vertex and edge weight matrices; the decoder is based on the attention mechanism, and selects a vertex in each time step to gradually build a solution; specifically, the encoder first encodes the vertices and edges through the embedding layer, as shown in the following formula:

[0034]

[0035] In the above formula, W E1 , W E2 , b E1 , b E2 is a learnable parameter, and BN represents batch normalization operation; then, the encoder uses L layers of graph attention layers to further encode vertices and edges. In the lth layer (1≤l≤L), the vertex information update rule is as follows:

[0036]

[0037] represents the attention weight of node i to node j in the lth layer, ξ is the activation function, W l is the learnable parameter of the lth layer. The edge update rule of the lth layer can be expressed as follows:

[0038]

[0039] in is a trainable parameter; for the update of vertex and edge information, residual connection and batch normalization operations are added to each graph attention layer; after L layers of graph attention layers, the encoding information of each vertex is obtained At the same time, define the graph encoding information Embed information h for the vertex L The average value of

[0040] After obtaining the encoded information of vertices and graphs, a decoder is used to decode them; the decoder is based on the attention layer. At decoding step t, the attention layer is based on the current context information The correlation between the information encoded by each vertex and the probability p of each node being visited at the current moment is calculated t , context The definition is as follows:

[0041]

[0042] Above is the encoding value of the vertex visited at the last moment, S t Represents the vehicle status, which can be set according to the specific problem, including vehicle load status or vehicle driving time, etc.; the attention embedding value is calculated based on this context information

[0043]

[0044] The above formula MHA represents the calculation of the multi-head attention mechanism. are all learnable parameters; then the attention score u of each vertex and the value is calculated through the embedded value t :

[0045]

[0046] C in the above formula is the scaling constant, q t and k t The values ​​of b k is the corresponding hidden layer dimension; finally, the probability p of each vertex being visited is determined by masked softmax t :

[0047] p t =softmax(u t +Z Mask t ) (20)

[0048] Mask t Represents a mask rule, through which points that cannot be accessed at the current moment are shielded. Z represents a large negative number. The mask rule is determined according to the constraints in the actual problem.

[0049] The present invention adopts reinforcement learning algorithm to Train and optimize the policy network parameters to obtain better decision-making solutions; train the gradient of the loss function The setup is as follows:

[0050]

[0051] Above represents the total reward obtained by the policy network, represents the total reward obtained by the baseline network, Construct advantage function, Represents the logarithmic distribution of the probability of the policy network selecting a vertex; in each training round, the parameters of the policy network are updated according to the gradient;

[0052] Step 3: Based on the learning decision-making method in step 2, a hierarchical solution architecture is designed to obtain a solution to the logistics fleet path optimization problem.

[0053] In the logistics fleet path optimization problem, some special circumstances may lead to the appearance of negative edge weights; therefore, the present invention adopts the SPFA algorithm that can effectively handle negative edge weights as the lower-level algorithm; for the logistics fleet path optimization problem in the urban road network, firstly, based on the set objective function, the edge weights between any two points are obtained by the SPFA algorithm, and then the edge weight matrix is ​​obtained, and the edge weight matrix is ​​normalized using formula (11). The edge weight matrix is ​​used as the input of the upper-level algorithm;

[0054] The upper-level algorithm is based on the designed learning decision-making algorithm and the edge weight matrix obtained by the lower-level algorithm to make decisions and obtain the logistics fleet routing solution;

[0055] Since the lower-level learning decision algorithm in this architecture can make decisions based only on the edge weight matrix, this architecture can decouple the lower-level path planning problem and the upper-level order decision problem, and has high practical value. At the same time, since this method makes decisions based on neural networks, the actual complexity of the algorithm is low and the calculation speed is fast.

[0056] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. Innovatively design a learning decision algorithm based on edge weight features, which improves the practicality of the learning decision algorithm in real urban road networks.

[0058] 2. The proposed dataset generation method and strategy network structure take the actual situation into consideration, and the algorithm has a wide range of applications.

[0059] 3. Based on the learning decision-making method, a hierarchical solution architecture is constructed to optimize the logistics fleet route. The proposed learning decision-making algorithm has high real-time performance and high solution accuracy.

[0060] 4. The hierarchical solution architecture can decouple the routing problem and the order decision problem, and has high practicality.

[0061] 5. The method of the present invention has high real-time and robustness in solving the route planning problem of logistics fleets, thereby improving the overall efficiency of logistics transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is the logistics vehicle path optimization problem of an embodiment of the present invention;

[0063] Figure 2 It is a learning-based logistics fleet route optimization method and system based on edge weight features in an embodiment of the present invention;

[0064] Figure 3 is a policy network according to an embodiment of the present invention;

[0065] Figure 4 is a schematic diagram of a hierarchical solution architecture according to an embodiment of the present invention;

[0066] Figure 5 It is a visualization effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with the accompanying drawings:

[0068] The logistics vehicle routing optimization problem mentioned above can be found in Figure 1 This embodiment provides a learning-based logistics fleet route optimization method and system based on edge weight features, see Figure 2 , including the following steps:

[0069] The first step is to define the logistics fleet path optimization problem based on a completely directed graph, set constraints, determine the objective function, and complete the mathematical model construction.

[0070] The complete directed graph is represented by G = (V, E), where warehouses and customers are constructed as vertices, and the vertices are connected by directed edges; V represents the set of all vertices on the graph, which consists of two different subsets: the warehouse set D = {0}, and the customer set C = {1,2,...,N C +1} indicates that the client vertex is combined, so the total number of vertices in the graph N = 1 + N C . E={(i,j)|i,j∈V,i≠j} represents the edge set connecting the points.

[0071] The logistics vehicle routing optimization problem can be described in the figure: the number is NC The customers of are distributed in the complete directed graph G, and each customer (i∈C) has his own demand for goods q i ; A certain number of logistics fleets K = {1,2,…,k}, starting from a unified warehouse, visit these customer points, and return to the warehouse after completing the delivery; The optimization goal is to design a series of driving routes for the logistics fleet, which can minimize the given objective function value while satisfying all constraints; The mathematical model of the logistics fleet route optimization problem can be expressed as follows:

[0072] Objective function:

[0073]

[0074] Constraints:

[0075]

[0076] u0=U (5)

[0077]

[0078] Other constraints(8)

[0079] Among them, x ij is a binary decision variable. When a vehicle leaves node i and arrives at node j, x ij is 1, otherwise it is 0; U represents the maximum load of the vehicle; u i represents the load of the vehicle when it arrives at node i; c ij represents the edge weight of edge (i, j);

[0080] Constraint (2) indicates that each customer needs to be visited by the service only once; constraint (3) indicates the flow conservation constraint; constraint (4) tracks the cargo volume of the vehicle; constraint (5) indicates that the vehicle leaves the warehouse fully loaded; constraint (6) indicates that the number of vehicles used does not exceed the maximum number K; constraint (7) indicates that the decision variable obeys the {0,1} constraint; finally, corresponding constraints can be added according to other constraints in actual logistics transportation, such as time windows;

[0081] The above logistics fleet path optimization problem model is a generalized modeling method. ij It can be set according to the actual problem, such as driving distance, driving time, driving cost, etc., and other constraints can be added according to the actual logistics needs; the learning decision method in step 2 only needs to obtain the edge weight matrix between nodes and the corresponding constraints to solve the generalized logistics vehicle route optimization problem;

[0082] The second step is to build a learning decision-making method, including: generating a training data set containing edge weight matrices, designing a strategy network based on graph neural networks, and building a reinforcement learning training algorithm.

[0083] The learning decision method belongs to the data-driven algorithm. First, the data set required for training needs to be generated. The present invention generates D logistics route optimization problem examples as data sets. In each logistics route optimization problem example, the two-dimensional coordinates of the customer and the warehouse are randomly sampled and generated in a uniformly distributed manner within [0,1]. The cargo demand of each customer is regarded as discrete and is in [0,q max ] is randomly generated by uniform sampling, q max represents the maximum value of the set demand, and the demand of each customer is normalized by the following formula:

[0084]

[0085] For each instance of the edge weight matrix, the edge weight c between two points ij Determined by the Euclidean distance; at the same time, in order to simulate the actual urban road network c ij With c ji Usually not equal, c is determined according to the following formula ji Values:

[0086]

[0087] In the above formula, Gaussian noise is a random variable with a state distribution, σ represents the standard deviation of the distribution; finally, the obtained edge weight matrix is ​​normalized to the maximum or minimum value:

[0088]

[0089] c min is the minimum value in the edge weight matrix, because when i=j, c ij =0, so all values ​​of the edge weight matrix are equal to c max The proportion is scaled to the interval [0,1], and the edge weights are proportionally scaled without affecting the composition of the final solution; in the designed learning decision algorithm, each node feature where d i represents the average distance between node i and all its vertices, and the characteristics of edge (i, j) are Finally, according to the size of the training set D and the corresponding feature information in the example, a data set for logistics fleet route optimization problem is generated.

[0090] After obtaining the data set, the present invention adopts the reinforcement learning algorithm set strategy network to make decisions; the designed strategy network consists of an encoder and a decoder. The network structure is shown in Figure 3; The encoder is composed of a graph attention network, which is responsible for encoding the vertex and edge weight matrices; the decoder is based on the attention mechanism, selecting a vertex in each time step to gradually build a solution; specifically, the encoder first encodes the vertices and edges through the embedding layer, as shown in the following formula:

[0091]

[0092] In the above formula, W E1 , W E2 , b E1 , b E2 is a learnable parameter, and BN represents a batch normalization operation. Next, the encoder uses L layers of graph attention layers to further encode vertices and edges. In the lth layer (1≤l≤L), the vertex information update rule is as follows:

[0093]

[0094] represents the attention weight of node i to node j in the lth layer, ξ is the activation function, W l is the learnable parameter of the lth layer; the edge update rule of the lth layer can be expressed as follows:

[0095]

[0096] in is a trainable parameter; for the update of vertex and edge information, residual connection and batch normalization operations are added to each graph attention layer; after L layers of graph attention layers, the encoding information of each vertex is obtained At the same time, define the graph encoding information Embed information h for the vertex L The average value of .

[0097] After obtaining the encoded information of vertices and graphs, a decoder is used to decode them; the decoder is based on the attention layer. At decoding step t, the attention layer is based on the current context information The correlation between the information encoded by each vertex and the probability p of each node being visited at the current moment is calculated t , context The definition is as follows:

[0098]

[0099] Above is the encoding value of the vertex visited at the last moment, S t Represents the vehicle status, which can be set according to the specific problem, including vehicle load status or vehicle driving time, etc.; the attention embedding value is calculated based on this context information

[0100]

[0101] The above formula MHA represents the calculation of the multi-head attention mechanism. are all learnable parameters. Then the attention score u of each vertex and this value is calculated by the embedded value t :

[0102]

[0103] C in the above formula is the scaling constant, q t and k t The values ​​of b k is the corresponding hidden layer dimension; finally, the probability p of each vertex being visited is determined by masked softmax t :

[0104] p t =softmax(u t +Z Mask t ) (20)

[0105] Mask t Represents a mask rule, through which points that cannot be accessed at the current moment are shielded, Z represents a large negative number, and the mask rule is determined according to the constraints in the actual problem.

[0106] The present invention adopts reinforcement learning algorithm to Train and optimize the policy network parameters to obtain better decision-making solutions; train the gradient of the loss function The setup is as follows:

[0107]

[0108] Above represents the total reward obtained by the policy network, represents the total reward obtained by the baseline network, Construct advantage function, Represents the logarithmic distribution of the probability of the policy network selecting a vertex; in each training round, the parameters of the policy network are updated according to the gradient.

[0109] The third step is to design a hierarchical solution architecture based on the learning decision-making method to obtain a solution to the logistics fleet path optimization problem. For a schematic diagram of the architecture, see Figure 4 .

[0110] In the logistics fleet path optimization problem, some special circumstances may lead to the appearance of negative edge weights. Therefore, the present invention adopts the SPFA algorithm that can effectively handle negative edge weights as the lower-level algorithm. For the logistics fleet path optimization problem in the urban road network, firstly, based on the set objective function, the edge weights between any two points are obtained by the SPFA algorithm, and then the edge weight matrix is ​​obtained, and the edge weight matrix is ​​normalized using formula (11). The edge weight matrix is ​​used as the input of the upper-level algorithm.

[0111] The upper-level algorithm is based on the designed learning decision algorithm, and the decision is solved based on the edge weight matrix obtained by the lower-level algorithm to obtain the logistics fleet path plan; the present invention is based on the real map and adopts the designed hierarchical solution architecture to optimize the logistics vehicle path. The implementation effect can be seen in Figure 5 .

[0112] This embodiment provides an electronic device or a terminal, including:

[0113] one or more processors;

[0114] A memory for storing one or more programs;

[0115] When the one or more programs are executed by the one or more processors, the one or more processors implement a learning-based logistics fleet route optimization method based on edge weight features as in the embodiment.

[0116] The processing result is sent to the logistics fleet through the electronic device or terminal, so that the logistics fleet can drive the vehicle route according to the received control signal.

[0117] This embodiment provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the learning-based logistics fleet route optimization method based on edge weight features of the embodiment.

Claims

1. A learning-based logistics fleet route optimization method and system based on edge weight features, characterized by: The following steps are involved: Step 1: Define the logistics fleet path optimization problem based on the complete directed graph, set constraints, determine the objective function, and complete the mathematical model construction; Step 2: Generate a training data set containing an edge weight matrix, design a policy network based on a graph neural network, build a reinforcement learning algorithm to train the policy network, and complete the design of a learning decision-making algorithm; Step 3: Design a hierarchical solution architecture. The lower layer obtains the optimal path between any two points in the road network based on the shortest path algorithm to form an edge weight matrix. The upper layer uses a learning decision algorithm based on the edge weight matrix to obtain the optimal driving path for the logistics fleet. In step 1, the logistics fleet path optimization problem is expressed as: Complete directed graph G = (V, E), where warehouses and customers are constructed as vertices, and vertices are connected by directed edges; V represents the set of all vertices on the graph, which consists of two different subsets: warehouse set D = {0}, customer set C = {1,2,...,N C +1} indicates that the client vertex is combined, so the total number of vertices in the graph N = 1 + N C ; E = {(i,j)|i,j∈V,i≠j} represents the edge set connecting the points; The logistics vehicle routing optimization problem can be described in the figure: the number is N C The customers of are distributed in the complete directed graph G, and each customer (i∈C) has his own demand for goods q i A certain number of logistics fleets K = {1, 2, ..., k} start from a unified warehouse, visit these customer points, and return to the warehouse after completing the delivery. The optimization goal is to design a series of driving routes for the logistics fleet, which can minimize the given objective function value while satisfying all constraints. In step 1, the mathematical model is expressed as: Objective function: Constraints: u0=U (5) Other constraints(8) Among them, x ij is a binary decision variable. When a vehicle leaves node i and arrives at node j, x ij is 1, otherwise it is 0; U represents the maximum load of the vehicle; u i represents the load of the vehicle when it arrives at node i; c ij represents the edge weight of edge (i, j); Constraint (2) indicates that each customer needs to be visited by the service only once; constraint (3) indicates the flow conservation constraint; constraint (4) tracks the cargo volume of the vehicle; constraint (5) indicates that the vehicle leaves the warehouse fully loaded; constraint (6) indicates that the number of vehicles used does not exceed the maximum number K; constraint (7) indicates that the decision variables are subject to the {0,1} constraint; finally, according to other constraints in actual logistics transportation, corresponding constraints are added; The above logistics fleet path optimization problem model is a generalized modeling method. ij It can be set according to the actual problem, and other constraints can be added according to the actual logistics needs; the learning decision method in step 2 only needs to obtain the edge weight matrix between nodes and the corresponding constraints to solve the generalized logistics vehicle route optimization problem; In step 2, the training data set containing the edge weight matrix is ​​generated as: By generating D logistics route optimization problem examples as data sets; in each logistics route optimization problem example, the two-dimensional coordinates of the customer and the warehouse are randomly sampled in [0,1] in a uniformly distributed manner; the cargo demand of each customer is regarded as discrete and is in [0,q max ] is randomly generated by uniform sampling, q max represents the maximum value of the set demand, and the demand of each customer is normalized by the following formula: For each instance of the edge weight matrix, the edge weight c between two points ij Determined by the Euclidean distance; at the same time, in order to simulate the actual urban road network c ij With c ji Usually not equal, c is determined according to the following formula ji Values: In the above formula, Gaussian noise is a random variable with a state distribution, σ represents the standard deviation of the distribution; finally, the obtained edge weight matrix is ​​normalized to the maximum and minimum values: c min is the minimum value in the edge weight matrix, because when i=j, c ij =0, so all values ​​of the edge weight matrix are equal to c max The proportion is scaled to the interval [0,1], and the edge weights are proportionally scaled without affecting the composition of the final solution; in the designed learning decision algorithm, each node feature where d i represents the average distance between node i and all its vertices, and the characteristics of edge (i, j) are Finally, based on the training set size D and the corresponding feature information in the example, a data set for logistics fleet route optimization problem is generated; In step 2, the policy network based on graph neural network is expressed as: The designed strategy network consists of an encoder and a decoder. The encoder consists of a graph attention network, which is responsible for encoding the vertex and edge weight matrices. The decoder selects a vertex in each time step based on the attention mechanism to gradually build a solution. Specifically, the encoder first encodes the vertices and edges through the embedding layer, as shown in the following formula: In the above formula, W E1 , W E2 , b E1 , b E2 is a learnable parameter, and BN represents a batch normalization operation. Next, the encoder uses L layers of graph attention layers to further encode vertices and edges. In the lth layer (1≤l≤L), the vertex information update rule is as follows: represents the attention weight of node i to node j in the lth layer, ξ is the activation function, W l is the learnable parameter of the lth layer; the edge update rule of the lth layer can be expressed as follows: in is a trainable parameter; for the update of vertex and edge information, residual connection and batch normalization operations are added to each graph attention layer; after L layers of graph attention layers, the encoding information of each vertex is obtained At the same time, define the graph encoding information Embed information h for the vertex L The average value of After obtaining the encoded information of vertices and graphs, a decoder is used to decode them; the decoder is based on the attention layer. At decoding step t, the attention layer is based on the current context information The correlation between the information encoded by each vertex and the probability p of each node being visited at the current moment is calculated t , context The definition is as follows: Above is the encoding value of the vertex visited at the last moment, S t Represents the vehicle status, which can be set according to the specific problem, including vehicle load status and vehicle travel time; the attention embedding value is calculated through this context information The above formula MHA represents the calculation of the multi-head attention mechanism. are all learnable parameters; then embed the value Calculate each vertex and embedding value The attention score u t : C in the above formula is the scaling constant, q t and k t The values ​​of b k is the corresponding hidden layer dimension; finally, the probability p of each vertex being visited is determined by masked softmax t : p t =softmax(u t +Z·Mask t ) (20) Mask t Indicates mask rules, through Mask t The mask rule shields the points that cannot be accessed at the current moment. Z represents a large negative number. The mask rule is determined according to the constraints in the actual problem.

2. According to claim 1, a learning-based logistics fleet route optimization method and system based on edge weight features is characterized by: In step 2, the process of building a reinforcement learning algorithm to train the policy network is described as: Using reinforcement learning algorithm to improve the policy network Conduct training and optimize policy network parameters to obtain better decision-making solutions; Gradient of the training loss function The settings are as follows: Above represents the total reward obtained by the policy network, represents the total reward obtained by the baseline network, Construct advantage function, Represents the logarithmic distribution of the probability of the policy network selecting a vertex; in each training round, the parameters of the policy network are updated according to the gradient.

3. According to the method and system for learning-based logistics fleet route optimization based on edge weight features as described in claim 1, it is characterized by: In step 3, the hierarchical solution architecture is expressed as: In the logistics fleet path optimization problem, some special cases may lead to the appearance of negative edge weights; therefore, the present invention adopts the SPFA algorithm that can effectively handle negative edge weights as the lower-level algorithm; for the logistics fleet path optimization problem in the urban road network, firstly, based on the set objective function, the edge weight between any two points is obtained by the SPFA algorithm, and then the edge weight matrix is ​​obtained, and the edge weight matrix is ​​normalized by using formula (11), and the edge weight matrix is ​​used as the input of the upper-level algorithm; The upper-level algorithm is based on the designed learning decision-making algorithm and the edge weight matrix obtained by the lower-level algorithm to make a decision and obtain a logistics fleet routing solution.

Images