Path planning method for processing multi-warehouse limited arc path problem and related equipment

Abstract

The invention discloses a path planning method for processing a multi-warehouse limited arc path problem and related equipment. The method comprises the following steps: abstracting the problem into an undirected graph structure; converting the undirected graph into a line graph, converting edges in the undirected graph into points in the line graph, expressing the points in the line graph in a feature vector form, and extracting an adjacent matrix; inputting the feature vector and the adjacent matrix into a graph convolutional network to generate an initial embedded vector so as to accurately express information of edges in an original undirected graph; deep information in the initial embedded vector is learned through an encoder based on edge filtering attention, and a coding embedded vector is obtained; a plurality of starting points are selected to be input into an edge-filtering-based attention decoder to construct a plurality of solution sequences at the same time; the optimal direction of each edge in the multiple solution sequences is planned through a dynamic algorithm, and the minimum total cost is calculated; and taking a directional sequence with the minimum cost in all solution sequences as a final output scheme. According to the method, the time required for solving the problem can be greatly shortened, and the vehicle scheduling efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the fields of deep reinforcement learning and neural combinatorial optimization, and in particular to a path planning method and related equipment for processing multi-warehouse limited arc path problems. Background Art

[0002] The Multi-Depot Capacitated Arc Routing Problem (MDCARP) is an extension of the Capacitated Arc Routing Problem (CARP), characterized by vehicles departing from multiple service stations to serve desired edges. The goal is to determine a set of vehicle routes that serve all desired edges while minimizing total cost and satisfying vehicle capacity constraints. The MDCARP typically involves real-world scenarios such as logistics distribution, urban services, or network maintenance, where multiple warehouses need to coordinate limited resources to serve edges distributed across the network.

[0003] CARP has been proven to be NP-hard. Since single-warehouse CARP is a special case of MDCARP, MDCARP is also NP-hard. The complexity of the problem shows that exact algorithms are only applicable to small-scale instances. As the problem size increases, the solution space expands exponentially, and it becomes increasingly difficult to obtain an optimal solution in a reasonable time. Therefore, research has mainly focused on heuristic methods that can provide near-optimal solutions with significantly reduced computation time. However, despite significant advances in algorithmic efficiency, existing methods still struggle to meet the strict time requirements of time-sensitive applications, especially large-scale instances.

[0004] In recent years, deep reinforcement learning (DRL) has emerged as a powerful paradigm for solving routing problems, achieving impressive results in the Traveling Salesman Problem (TSP) and various variants of the Vehicle Routing Problem (VRP). Several researchers have successfully extended DRL approaches to the Multi-Depot Vehicle Routing Problem (MDVRP). However, the application of DRL to MDCARP has remained largely unexplored. DRL offers unique advantages for MDCARP, particularly in handling the computational complexity that arises with increasing problem size. By leveraging parallel processing, the problem can be solved quickly and with high-quality solutions after model training is complete. Summary of the Invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a path planning method and related equipment for processing the multi-warehouse limited arc path problem.

[0006] The first technical solution adopted by the present invention is:

[0007] A path planning method for processing a multi-warehouse limited arc path problem includes the following steps:

[0008] Obtain the instance information of the problem to be solved, express the instance information as an undirected graph, obtain multiple isomorphic graphs through the instance enhancement method, and calculate the shortest distance matrix between points in the graph using the Floyd algorithm;

[0009] Convert the undirected graph into a line graph, obtain the adjacency matrix of the line graph, and express each vertex in the line graph with a eigenvector;

[0010] The adjacency matrix and feature vector of the line graph are input into the graph convolutional network, and the initial embedding vector of each edge is generated by the graph convolutional network;

[0011] The obtained initial embedding vector is input into the edge-filtered attention-based encoder to obtain the encoded embedding vector;

[0012] Use the nearest neighbor strategy to construct multiple solution sequences. Starting from each warehouse, select the n edges closest to the warehouse as the first moment through the shortest distance matrix, and construct m×n solution sequences; m is the number of warehouses.

[0013] Use dynamic programming to calculate the optimal direction of each edge in each solution sequence and find the minimum total cost of the solution sequence;

[0014] The sequence with the smallest total cost among all solution sequences is taken as the final output.

[0015] Furthermore, the problem instance information that needs to be solved is obtained, the instance information is expressed as an undirected graph, and multiple isomorphic graphs are obtained through an instance enhancement method, including:

[0016] The instance information is expressed as an undirected graph G = (V, E) with a vertex set V and an edge set E; each edge e in the edge set E has a requirement dem e and driving cost c e Two features: the vertex set V contains a subset of warehouses D; in each warehouse there is a fleet of identical vehicles with a maximum capacity of Q;

[0017] On the basis of retaining the original undirected graph topology, the sequence numbers of the vertices in the graph are reordered to obtain multiple isomorphic graphs to avoid the interference of vertex sequence numbers on model solution.

[0018] Furthermore, converting the undirected graph into a line graph, obtaining the adjacency matrix of the line graph, and expressing each vertex in the line graph with a feature vector includes:

[0019] Consider each warehouse as a virtual edge, where both endpoints are warehouse vertices, and both demand and cost are 0;

[0020] Convert the original undirected graph G to a line graph L(G). Each vertex of the line graph L(G) corresponds to an edge of the original graph G. When two edges in the original undirected graph share a common vertex, an edge is established between the two corresponding vertices in the line graph. Specifically, if the edges e1 = (u, v) and e2 = (w, x) in the original undirected graph are Then there is an edge (e1,e2) in L(G);

[0021] Convert virtual edges and edges in the original undirected graph into vertices in the line graph;

[0022] Correspond the vertex i in the line graph to the original undirected graph e i =(u,v) is expressed by a ten-dimensional eigenvector.

[0023] Furthermore, the expression of the characteristic vector is:

[0024]

[0025] Where Y(u) and Y(v) are two-dimensional variables used to determine whether the vertex is a warehouse. If it is, it is 1, otherwise it is 0; d u ,d v are the warehouses closest to the two vertices, and sp is the shortest distance between the two vertices; avg(u) and avg(v) are the average values ​​of the shortest distances from the two vertices to all warehouses. is the normalized cost of the edge, is the normalized demand of the edge.

[0026] Furthermore, the adjacency matrix and eigenvector of the line graph are input into the graph convolutional network, and the initial embedding vector of each edge is generated by the graph convolutional network, including:

[0027] The adjacency matrix and eigenvector of the line graph are input into the graph convolutional network, and a three-layer graph convolutional network is used for information aggregation, which can be expressed as:

[0028]

[0029] Where X represents the feature matrix composed of the feature vectors expressed by each vertex; W (0) 、W (1) and W (2) are the learnable weight matrices of each layer; It is a normalized adjacency matrix with self-loops; the first two layers use the ReLU activation function, and the third layer uses the tanh function to limit the embedded value to a bounded range; the final output H (0) as the initial embedding vector.

[0030] Furthermore, the obtained initial embedding vector is input into an encoder based on edge filtering attention to obtain an encoded embedding vector, including:

[0031] The initial embedding vector is input into an encoder of an N-layer encoding layer. The structure of each encoding layer is the same, but the parameters between layers are independent of each other. The encoding layer consists of a multi-head edge filter attention layer (MHEFA), a feedforward layer (FFN), and an instance normalization layer (IN). The sub-layers are connected by residual connections. The encoding layer of layer l is expressed as:

[0032]

[0033] Where, represents the intermediate embedding vector of layer l, h (l) Represents the final embedding vector of layer l, IN (l) Represents the lth layer instance normalization layer, FFN (l) Represents the lth feed-forward layer.

[0034] Furthermore, the edge filter attention in the l-th encoding layer is expressed as:

[0035] Q=W Q h (l-1) ,K=W K h (l-1) ,V=W V h (l-1)

[0036]

[0037] MHEFA(Q,K,V)=Concat(EFA (1) ,EFA (2) ,…,EFA (H) )W O

[0038] Where, is a learnable parameter, α H It is a hyperparameter to prevent α from being initialized to 0. Through two independent multi-head attention mechanisms, different information is learned. For MHA1, information about edges with a large relationship to the current edge is learned. For MHA2, information about edges with a small relationship to the current edge is learned and excluded. W Q ,W K is the projection matrix of query and key, W V is the projection matrix of the value; the embedding vector is divided into different subspaces, that is, different heads, the superscript (i) represents the i-th head; H is the total number of subspaces.

[0039] Furthermore, the nearest neighbor strategy is used to construct multiple solution sequences, starting from each warehouse, and selecting the n edges closest to the warehouse as the first moment selection through the shortest distance matrix to construct m×n solution sequences, including:

[0040] Construct multiple solution sequences based on the nearest neighbor strategy. At time t=0, a warehouse is selected as the starting point. At time t=1, the n edges with the closest distance are sorted by distance and selected as the choice at time t=1, thereby constructing n different solution sequences.

[0041] Starting from time t = 2, the warehouse where the current vehicle departs, the encoded embedding vector corresponding to the road selected at the previous moment, and the remaining capacity of the current vehicle are concatenated into a context vector as the input of the decoder;

[0042] Through the random sampling strategy, the edge of the current moment is selected, and the state is updated to make the selection at the next moment, so that the requirements of all edges are met.

[0043] The second technical solution adopted by the present invention is:

[0044] An electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a path planning method for processing a multi-warehouse limited arc path problem as described above.

[0045] The third technical solution adopted by the present invention is:

[0046] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement a path planning method for processing a multi-warehouse limited arc path problem as described above.

[0047] The fourth technical solution adopted by the present invention is:

[0048] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned path planning method for a multi-warehouse limited arc path problem.

[0049] The present invention has the following advantages: It designs an edge-filtered attention algorithm for better learning graph information and, based on the characteristics of MDCARP, a neighbor decoding strategy, which significantly reduces the time required to solve the problem and improves vehicle dispatch efficiency. Furthermore, the present invention proposes an example enhancement method to diversify input. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 It is a technical solution flow chart of the path planning method for processing the multi-warehouse limited arc path problem in an embodiment of the present invention.

[0052] Figure 2 is a schematic diagram of an encoder based on edge filtering attention in an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of the neighbor strategy at different times in an embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of an example enhancement technology in an embodiment of the present invention;

[0055] Figure 5 This is a flow chart of a path planning method for processing a multi-warehouse limited arc path problem in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0057] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise clearly defined, words such as setting, installing, and connecting should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0058] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0059] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0060] In the description of this application, "and / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0061] Explanation of terms:

[0062] Floyd algorithm: also known as the insertion point method, is an algorithm that uses the idea of ​​dynamic programming to find the shortest path between multiple source points in a given weighted graph.

[0063] To address existing technical issues and leveraging the unique advantages of DRL methods, this paper proposes a deep reinforcement learning-based solution for the multi-warehouse limited arc path problem. This paper employs an edge-filtered attention algorithm to better learn graph information. Based on the characteristics of MDCARP, this paper designs a neighbor decoding strategy and proposes an example augmentation method to diversify input.

[0064] Example 1:

[0065] See also Figure 1 and Figure 5This embodiment provides a path planning method for processing the multi-warehouse limited arc path problem. The specific implementation steps are as follows:

[0066] Step 1: Define the process of solving the multi-warehouse limited arc path problem as a Markov decision process (MDP). The rewards in this process are sparse, and the reward value is not returned until the solution sequence is fully constructed. The model parameters are updated through backpropagation.

[0067] Specifically, the process of solving MDCARP through deep reinforcement learning in step 1 can be effectively modeled as an MDP. The components of an MDP include state space, action space, state transition, and reward function, which are formally defined as follows:

[0068] a) State; State s is defined as the representation of the current solution and the remaining capacity of the vehicle. The specific expression is: s = (S t ,C t ). Where S t represents the partially constructed solution sequence up to time step t, C t represents the remaining capacity of the vehicle at time t.

[0069] b) Action: In the process of constructing the solution sequence, action a t Expressed as (e j ), that is, select the edge to be served or the warehouse virtual edge e at time step t j . When e j When it is a warehouse virtual edge, two situations may occur:

[0070] 1) If the edge selected at time t-1 is also a warehouse virtual edge, it means that the new car departs from the warehouse selected at time t;

[0071] 2) If the edge selected at time t-1 is not a warehouse virtual edge, it indicates that the current vehicle returns to the warehouse.

[0072] c) State transition: The state of the next time step is determined by the current state and the current action. The specific transition rule is expressed as:

[0073] If e j is the edge that needs service: S t+1 =S t ∪{e j}, C t+1 =C t -d(e j )

[0074] If e j is the warehouse virtual edge: S t+1 =S t ∪{e j}, Ct+1 =Q.

[0075] d) Reward function: To minimize the total cost of the solution, the reward R is defined as:

[0076] R=-DP(S)

[0077] Where S is a complete solution sequence that serves all edges that need to be served. Dynamic programming (DP) is used to determine the optimal driving direction for each edge in the solution sequence S to minimize the total cost and calculate the minimum total cost.

[0078] e) Strategy: Random strategy π θ Assign selection probabilities to the available necessary edges at each time step t, and select the edges or warehouses that need to be served at time step t through probabilistic sampling. In this way, the solution sequence is gradually constructed until all edges that need to be served are served, and finally a complete solution sequence S is formed:

[0079]

[0080] where π refers to the policy with trainable parameters θ, and T is the number of time steps required to build a complete solution.

[0081] Step 2: Obtain problem instance information and abstract it into an undirected graph. Using instance enhancement technology, we perform instance enhancement while preserving the undirected graph structure to diversify the model input.

[0082] For example, the input instance information is expressed as an undirected graph G = (V, E) with a vertex set V and an edge set E, where each edge e in the edge set E has a requirement dem e and driving cost c e Two features, the vertex set contains a subset of warehouses D. In each warehouse there is a fleet of identical vehicles with a maximum capacity of Q.

[0083] Among them, see Figure 4 Instance enhancement is achieved by shuffling the vertices in the undirected graph while maintaining the original undirected graph topology. This prevents vertex sequence numbers from interfering with the model solution, thereby enhancing the model's solution performance and robustness.

[0084] Step 3: Convert the undirected graph into a line graph, obtain the adjacency matrix of the line graph, and express each vertex in the line graph with an eigenvector.

[0085] As an implementation method, each warehouse is considered as a virtual edge, with both endpoints being warehouse vertices, and both demand and cost being 0. The original undirected graph G is converted into a line graph L(G), where each vertex of the line graph L(G) corresponds to an edge of the original graph G. When two edges in the original undirected graph share a common vertex, an edge is established between the two corresponding vertices in the line graph. This can be expressed as follows: in the original undirected graph, edges e1 = (u, v) and e2 = (w, x), if Then there is an edge (e1,e2) in L(G).

[0086] Convert the virtual edges and the edges in the original undirected graph into vertices in the line graph. Correspond the vertex i in the line graph to the original undirected graph e. i =(u,v) is expressed as a ten-dimensional eigenvector:

[0087]

[0088] Among them, Y(u) and Y(v) are two-dimensional variables used to determine whether the vertex is a warehouse. If it is, it is 1, otherwise it is 0. u ,d v u and v are the closest warehouses to the two vertices, respectively, and sp is the shortest distance between the two vertices. avg(u) and avg(v) are the averages of the shortest distances from the two vertices to all warehouses. is the normalized cost of the edge, is the normalized demand of the edge.

[0089] Step 4: Input the adjacency matrix of the line graph and the feature vector of each point in the graph into the graph convolutional network to generate the initial embedding vector corresponding to each point.

[0090] In some embodiments, an initial embedding vector is generated by a graph convolutional network, and a three-layer graph convolutional network is used to perform information aggregation operations, which can be expressed as:

[0091]

[0092] where X∈R (|E|+|D|)×10 Represents the feature matrix composed of the feature vectors expressed by each vertex in step 3, W (0) ∈R 10×32 、W (1) ∈R 32×64 and W (2) ∈R 64×128 are the learnable weight matrices of each layer, It is a normalized adjacency matrix with self-loops. The first two layers use the ReLU activation function, and the third layer uses the tanh function to limit the embedded value to a bounded range. The final output H 0 ∈R (|E|+|D|)×128 will be used as the initial embedding vector.

[0093] Step 5: Use the edge-filtered attention-based encoder to learn the relationship between the initial embedding vectors and generate the encoded embedding vector.

[0094] In some embodiments, see Figure 2 , the encoder based on edge filter attention has a total of N identical encoding layers, where the edge filter attention in the l-th encoding layer can be expressed as:

[0095]

[0096] V=W V h (1-1)

[0097]

[0098] in and are the query and key projection matrices, and the two sets of projection matrices are independent of each other. They are both trainable parameter matrices, and the dimensions of the two sets of projection matrices are the same, 128×128. α is used to adjust the relationship between the two independent attention scores, where is a learnable parameter, α H It is a hyperparameter with an initialization range of (0,1) to avoid α being initialized to 0.

[0099] Step 6: Use the nearest neighbor strategy to start from each warehouse and select the n edges closest to the warehouse.

[0100] Specifically, see Figure 3 , use the nearest neighbor strategy to select the warehouse and the selection at time t=1 after the warehouse. Starting from each warehouse (number is m), select the n edges closest to the warehouse. At the same time, construct n×m solution sequences to achieve multi-starting point decoding. Warehouse d i With edge e i =(u,v) is defined as:

[0101] dist=min(sp(d i ,u),sp(d i ,v))

[0102] The shortest distance between the warehouse point and the two vertices of the edge is taken as the distance from the warehouse point to the edge.

[0103] Step 7: Update the current state and input the current state into the decoder, output the probability distribution of each edge at the current moment, use the probability sampling strategy to select the edge at the current moment in the training phase, and use the greedy strategy to select the edge at the current moment in the testing phase.

[0104] For the edge selection after time t=1, the vehicle capacity is updated based on the previous selection. The encoding embedding vector corresponding to the warehouse where the current vehicle departs, the encoding embedding vector corresponding to the previous selection, and the vehicle's remaining capacity at the current moment are concatenated together as the decoding vector and input into the decoder for decoding. The decoding vector can be expressed as:

[0105]

[0106] in The encoding embedding vector corresponding to the warehouse where the current vehicle departs, is the embedding vector of the edge of the solution sequence at the previous moment, C t is the remaining capacity of the current vehicle.

[0107] The context vector corresponding to the decoded vector is calculated through a masked edge filtering attention. When the edge selected at time t-1 is a warehouse, the current warehouse and the served edges are filtered out through the mask. Unserved edges or other warehouses can be selected. When other warehouses are selected, it means that the vehicle departs from the last selected warehouse. When the edge selected at time t-1 is an edge that needs to be served, warehouses other than the warehouse where the current vehicle departs, served edges, and edges with demand greater than the vehicle's remaining capacity are filtered out through the mask. When the vehicle has sufficient remaining capacity, the vehicle can choose to continue serving or return to the departure warehouse. Masked edge filtering attention is expressed as:

[0108]

[0109] Where M represents the mask vector, which is used to exclude the interference of unselectable edges when calculating edge filtering attention.

[0110] In calculating the context vector After that, the attention score of each side is calculated through a single-head attention, which is expressed as:

[0111] K=W K H (L)

[0112]

[0113] u represents the masked attention score for all edges. Edges that cannot be selected at the current moment are assigned a negative infinity value. The attention scores are normalized using Softmax to obtain the probability distribution of each edge. Different probability selection strategies are chosen based on the task. During the training phase, a probabilistic sampling strategy is used for edge selection, while during the testing phase, a greedy strategy is used for edge selection at the current moment.

[0114] Step 8: Repeat step 7 until all edges that need to be served are served, and obtain a solution sequence with a determined service order and undetermined edge direction.

[0115] Step 9: Using the solution sequence obtained in Step 8, a dynamic algorithm is used to plan the optimal direction for each edge to minimize the total cost. The service order with the minimum cost is determined, and the solution sequence with the determined edge directions is used as the final output.

[0116] As an optional implementation, the model in the above steps is optimized using the REINFORCE algorithm in reinforcement learning. The costs of multiple solution sequences constructed using the dynamic algorithm are averaged and used as a baseline to reduce the variance of the policy gradient estimate. It can be expressed as:

[0117]

[0118] The number of warehouses is |D|, and each warehouse selects the M nearest edges as the first moment selection, and constructs the solution sequence respectively. The gradient update formula in the REINFORCE algorithm can be expressed as:

[0119]

[0120] The Adam optimizer is used to optimize the model parameters using a gradient ascent strategy.

[0121] In summary, the path planning method for processing the multi-warehouse limited arc path problem proposed in the embodiment of the present invention has at least the following advantages:

[0122] (1) The method described in this example belongs to the field of deep reinforcement learning. Through end-to-end training, the model automatically learns the optimal strategy, reducing its reliance on domain knowledge. This method significantly reduces the time required to solve the problem, can solve multiple instances of the problem in parallel, and can obtain a near-optimal solution in a very short computation time.

[0123] (2) This embodiment proposes an instance enhancement technique for the multi-warehouse limited arc path problem. The performance of the model is improved by modifying the training data. By scrambling the vertex numbers in the original undirected graph, diverse training instances are generated while maintaining the original graph topology. This reduces the impact of changes in vertex numbers on the model solution. This allows the model to reduce the impact of local features of non-specific instances.

[0124] (3) This embodiment introduces an edge filtering attention mechanism. Through two independent attention mechanisms, different feature information is learned, and the current optimal choice and the current worst choice are learned simultaneously. By filtering out poor choices, the model suppresses gradient updates of inefficient actions during training. This enhances the learning stability of the model, improves the convergence effect of the model, and improves the solution effect of the model.

[0125] (4) This embodiment uses a nearest neighbor strategy for decoding, constructing multiple different solution sequences by selecting the edges closest to the warehouse. This improves the training efficiency and performance of the model. This strategy also effectively improves the generalization performance of the model, making it more robust.

[0126] Example 2:

[0127] See also Figure 1 and Figure 5 This embodiment provides a path planning method for solving the multi-warehouse limited arc path problem based on deep reinforcement learning. The problem solving includes the following steps:

[0128] S1: Obtain problem instance information, represented by an undirected graph G. The graph includes: the vertex set V containing the warehouse subset D, the edge set E, the demand dem(e) corresponding to the edge, and the travel cost c(e) of the edge.

[0129] S2: Use instance enhancement technology to perform instance enhancement on the original undirected graph G to obtain multiple instance enhanced undirected graphs G N .

[0130] S3: Process the original undirected graph and the instance-enhanced graph simultaneously, convert them into corresponding line graphs, extract the adjacency matrix of the corresponding line graph, express the vertices in the line graph as eigenvectors, and compose the eigenvectors of all vertices into an eigenvector matrix.

[0131] S4: Through the graph convolutional network, the adjacency matrix and eigenvector matrix of multiple graphs are processed in parallel to generate the corresponding initial embedding matrix.

[0132] S5: Multiple initial embedding matrices are input into the encoder and processed in parallel by the encoder of the 6 coding layers to generate corresponding encoded embedding vectors.

[0133] S6: Use the nearest neighbor strategy to construct a warehouse of solution sequences for all original instances and enhanced instances and select time t = 1. For times after t = 1, the decoder decodes the corresponding probability distribution. According to the greedy strategy, the edge with the highest probability is selected as the choice at the current moment. The state is updated and re-decoded until all edges that need to be served are served.

[0134] S7: The solution sequences of all original instances and enhanced instances are used to determine the optimal direction of each edge in the solution sequence through a dynamic programming algorithm, and the solution sequence with the minimum total cost is used as the output of this instance.

[0135] S8: Compare the minimum cost of the best solution sequence of all original instances and enhanced instances, and take the one with the minimum cost as the final output.

[0136] S9: Determine whether the final output solution sequence corresponds to the original instance. If so, output it directly. If not, remap the vertex sequence to the original vertex sequence before outputting it.

[0137] The constructed solution sequence must satisfy the problem definition restrictions of MDCARP. The constraints that need to be satisfied are:

[0138] 1) The departure warehouse and return warehouse of a vehicle must be the same.

[0139] 2) The sum of the demands of the edges served by a vehicle cannot be greater than the maximum capacity of the vehicle.

[0140] 3) Each edge can only be served by one vehicle at a time and cannot be served by multiple vehicles separately.

[0141] To verify the feasibility of this method, this example compares the heuristic algorithm RoCaSH (route clustering and search heuristic), which is specifically designed to solve the MDCARP problem. For the specific algorithm, please refer to (Zhang Y, Mei Y, Huang S, et al. A route clustering and search heuristic for large-scale multidepot-capacitated arc routing problem [J]. IEEE Transactions on Cybernetics, 2021, 52(8): 8286-8299.).

[0142] This example compares the solution quality and speed of the two methods on the classic MDCARP test set MAVAL. Table 1 shows the comparison of the test results of the two methods on the MAVAL test set.

[0143] Table 1: Test results of the two methods on the maval test set

[0144]

[0145]

[0146] In the table, |V| represents the number of vertices, |E| represents the number of edges, |T| represents the number of edges to be served, and Q represents the maximum capacity of the vehicle. The number of warehouses in each test set is 3.

[0147] The results in Table 1 show that the method proposed in the embodiment of the present invention has a better solution effect than the heuristic algorithm and requires less solution time, which verifies the feasibility and effectiveness of this method.

[0148] Example 3:

[0149] This embodiment provides a method for planning a multi-warehouse limited arc path. Taking the multi-station sprinkler truck path planning problem in a city as an example, the method includes:

[0150] Step 1: Obtain the city road network information and represent it as an undirected graph. The undirected graph includes the set of roads E, the set of road endpoints V, the set of sprinkler station locations D, the required watering volume dem(e) for each road, and the travel cost c(e) for each road. Obtain the maximum capacity Q of the sprinkler trucks.

[0151] Step 2: Enhance the acquired urban road network information through instance enhancement technology. Without changing the road structure information, the sequence numbers of the road endpoints are shuffled and rearranged to achieve input diversification.

[0152] Step 3: Convert the undirected urban road network graph into a line graph, obtain the corresponding adjacency matrix of the line graph, and express each vertex of the line graph as a feature vector. Aggregate the information of the line graph using a three-layer graph convolutional network to obtain the initial embedding vector.

[0153] Step 4: The encoder learns the relationship between the initial embedding vectors and expresses the deep relationship with the encoded embedding vector.

[0154] Step 5: Decode using the nearest neighbor strategy. Repeat the decoding until all roads are served to obtain an initial solution sequence with a determined service order but an undetermined road service direction.

[0155] Step 6: Use a dynamic programming algorithm to determine the optimal direction of road services in the initial solution sequence to minimize the total cost. The minimum cost solution sequence for the original instance and the enhanced instance is used as the final output.

[0156] This example randomly generates 1,000 instances of different sizes to simulate the multi-site sprinkler planning problem, as shown in Table 2.

[0157] Table 2: Settings for different scale examples of urban multi-site sprinkler path planning

[0158]

[0159]

[0160] Similarly, this example was compared with RoCaSH to verify the feasibility of the model. The test results are shown in Table 3:

[0161] Table 3

[0162]

[0163] The experimental results further demonstrate the feasibility of our method, which not only achieves excellent solution quality but also significantly reduces the solution time. This also verifies the feasibility and effectiveness of our method for the multi-station sprinkler truck problem.

[0164] Example 4:

[0165] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 5 A path planning method for dealing with the multi-warehouse limited arc path problem is shown.

[0166] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0167] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0168] Since the electronic device is an electronic device corresponding to a path planning method for processing a multi-warehouse limited arc path problem in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0169] Embodiment 5:

[0170] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 5 A path planning method for dealing with the multi-warehouse limited arc path problem is shown.

[0171] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0172] Since the storage medium is a storage medium corresponding to a path planning method for processing a multi-warehouse limited arc path problem in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0173] Example 6:

[0174] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to execute the steps of the path planning method for processing a multi-warehouse limited arc path problem according to various exemplary embodiments of the present application described above in this specification. The executable computer program code or "code" used to execute the various embodiments may be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0175] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0176] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0177] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A path planning method for processing the multi-warehouse limited arc path problem, characterized in that: The following steps are involved: Obtain instance information of the problem to be solved, express the instance information as an undirected graph, obtain multiple isomorphic graphs through instance enhancement methods, and calculate the shortest distance matrix between points in the graph; Convert the undirected graph into a line graph, obtain the adjacency matrix of the line graph, and express each vertex in the line graph with a eigenvector; The adjacency matrix and feature vector of the line graph are input into the graph convolutional network, and the initial embedding vector of each edge is generated by the graph convolutional network; The obtained initial embedding vector is input into the encoder based on edge filtering attention to obtain the encoded embedding vector; multiple solution sequences are constructed using the nearest neighbor strategy. Starting from each warehouse, the n edges closest to the warehouse are selected as the first moment through the shortest distance matrix to construct m×n solution sequences; m is the number of warehouses; Calculate the best direction of each edge in each solution sequence and find the minimum total cost of the solution sequence; The sequence with the smallest total cost among all solution sequences is taken as the final output.

2. A path planning method for processing a multi-warehouse limited arc path problem according to claim 1, characterized in that: The problem instance information to be solved is obtained, the instance information is expressed as an undirected graph, and multiple isomorphic graphs are obtained through the instance enhancement method, including: The instance information is expressed as an undirected graph G = (V, E) with a vertex set V and an edge set E; each edge e in the edge set E has a requirement dem e and driving cost c e Two features, the vertex set V contains a warehouse subset D; In each warehouse there is a fleet of identical vehicles with a maximum capacity of Q; On the basis of retaining the original undirected graph topology, the sequence numbers of the vertices in the graph are reordered to obtain multiple isomorphic graphs to avoid the interference of vertex sequence numbers on model solution.

3. A path planning method for processing a multi-warehouse limited arc path problem according to claim 1, characterized in that: The step of converting the undirected graph into a line graph, obtaining the adjacency matrix of the line graph, and expressing each vertex in the line graph with a feature vector includes: Consider each warehouse as a virtual edge, where both endpoints are warehouse vertices, and both demand and cost are 0; Convert the original undirected graph G to a line graph L(G). Each vertex of the line graph L(G) corresponds to an edge of the original graph G. When two edges in the original undirected graph share a common vertex, an edge is established between the two corresponding vertices in the line graph. Specifically, if the edges e1 = (u, v) and e2 = (w, x) in the original undirected graph are Then there is an edge (e1,e2) in L(G); Convert virtual edges and edges in the original undirected graph into vertices in the line graph; Correspond the vertex i in the line graph to the original undirected graph e i =(u,v) is expressed by an eigenvector.

4. A path planning method for processing a multi-warehouse limited arc path problem according to claim 3, characterized in that: The expression of the characteristic vector is: Where Y(u) and Y(v) are two-dimensional variables used to determine whether the vertex is a warehouse. If it is, it is 1, otherwise it is 0; d u ,d v are the warehouses closest to the two vertices, and sp is the shortest distance between the two vertices; avg(u) and avg(v) are the average values ​​of the shortest distances from the two vertices to all warehouses. is the normalized cost of the edge, is the normalized demand of the edge.

5. A path planning method for processing a multi-warehouse limited arc path problem according to claim 1, characterized in that: The adjacency matrix and eigenvector of the line graph are input into the graph convolutional network, and the initial embedding vector of each edge is generated by the graph convolutional network, including: The adjacency matrix and eigenvector of the line graph are input into the graph convolutional network, and a three-layer graph convolutional network is used for information aggregation, which can be expressed as: Where X represents the feature matrix composed of the feature vectors expressed by each vertex; W (0) 、W (1) and W (2) are the learnable weight matrices of each layer; It is a normalized adjacency matrix with self-loops; the first two layers use the ReLU activation function, and the third layer uses the tanh function to limit the embedded value to a bounded range; the final output H (0) as the initial embedding vector.

6. A path planning method for processing a multi-warehouse limited arc path problem according to claim 1, characterized in that: The obtained initial embedding vector is input into the edge-filtered attention-based encoder to obtain the encoded embedding vector, including: The initial embedding vector is input into an encoder of an N-layer encoding layer; the encoding layer consists of a multi-head edge filter attention layer (MHEFA), a feedforward layer (FFN), and an instance normalization layer (IN). The sub-layers are connected by residual connections; the encoding layer of the lth layer is expressed as: Where, represents the intermediate embedding vector of layer l, h (l) Represents the final embedding vector of layer l, IN (l) Represents the lth layer instance normalization layer, FFN (l) Represents the lth feed-forward layer.

7. A path planning method for processing a multi-warehouse limited arc path problem according to claim 6, characterized in that: The edge filter attention in the l-th encoding layer is expressed as: Q=W Q h (l-1) ,K=W K h (l-1) ,V=W V h (l-1) MHEFA(Q,K,V)=Concat(EFA (1) ,YES (2) ,…,YES (H) )W O Where, is a learnable parameter, α H It is a hyperparameter to prevent α from being initialized to 0. Through two independent multi-head attention mechanisms, different information is learned. For MHA1, information about edges with a large relationship to the current edge is learned. For MHA2, information about edges with a small relationship to the current edge is learned and excluded. W Q The query projection matrix, W K is the projection matrix of the key, W V It is the projection matrix of the value (Value); the embedded vector is divided into different subspaces, that is, different heads. The superscript (i) represents the i-th head, and the total number of heads is H.

8. A path planning method for processing a multi-warehouse limited arc path problem according to claim 1, characterized in that: The nearest neighbor strategy is used to construct multiple solution sequences. Starting from each warehouse, the n edges closest to the warehouse are selected as the first moment through the shortest distance matrix, and an m×n solution sequence is constructed, including: Construct multiple solution sequences based on the nearest neighbor strategy. At time t=0, a warehouse is selected as the starting point. At time t=1, the n edges with the closest distance are sorted by distance and selected as the choice at time t=1, thereby constructing n different solution sequences. Starting from time t = 2, the warehouse where the current vehicle departs, the encoded embedding vector corresponding to the road selected at the previous moment, and the remaining capacity of the current vehicle are concatenated into a context vector as the input of the decoder; Through the random sampling strategy, the edge of the current moment is selected, and the state is updated to make the selection at the next moment, so that the requirements of all edges are met.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images