Neural solution optimization method for large-scale vehicle routing problem based on distance-enhanced attention

Abstract

This invention belongs to the field of neural combinatorial optimization technology, and relates to a neural optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention. This invention constructs a joint distance-coordinate encoder and introduces a nested grouping strategy to dynamically adjust the receptive field of the distance matrix, guiding the model to gradually focus from the global spatial context to the local core path. Simultaneously, a multi-head gating mechanism is designed, assigning an independent and learnable gating scalar to each attention head to achieve adaptive fusion of distance priors and original attention scores. Furthermore, a two-stage progressive training strategy is adopted, enabling the model to stably migrate from small-scale tasks to large-scale application scenarios. Experimental results show that this invention achieves significantly better solution accuracy and generalization performance than existing methods in benchmarks such as CVRPLib, maintaining high computational efficiency when solving large-scale vehicle path planning problems, and effectively improving the spatial reasoning ability and cross-scale generalization ability of the neural solver.

Description

Technical Field

[0001] This invention belongs to the field of neural combinatorial optimization technology, and relates to a neural optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention. Background Technology

[0002] Vehicle routing problems are a classic combinatorial optimization problem that requires efficiently scheduling multiple vehicles to serve multiple customers while satisfying various constraints, in order to minimize the total travel distance or cost. Due to the nondeterministic, polynomial-hard nature of these problems, their difficulty increases exponentially with the problem size. While traditional optimization algorithms perform well on small to medium-sized instances, their computation time increases dramatically with the problem size, making them unsuitable for real-time, large-scale applications.

[0003] Traditional methods for solving vehicle routing problems are mainly divided into two categories: exact algorithms and heuristic algorithms. Exact algorithms, such as branch and bound and cutting plane methods, theoretically guarantee finding the global optimum, but their computational complexity is extremely high, limiting their application to small-scale problems. Heuristic algorithms, such as adaptive large neighborhood search and hybrid genetic search, have become mainstream in practical applications due to their high computational efficiency and have achieved optimal performance on various vehicle routing problems. Although these traditional methods perform well on small- to medium-scale instances and can provide high-quality solutions, they have the following inherent limitations: First, algorithm design heavily relies on expert knowledge, requiring the manual design of complex operators tailored to the problem characteristics; second, as the problem size increases, the computation time grows exponentially, reaching several hours or even longer for large-scale instances with hundreds of nodes, making it difficult to meet real-time decision-making requirements; third, when problem constraints change, the algorithm needs to be redesigned and its parameters tuned, lacking generalization ability.

[0004] To overcome the computational bottlenecks of traditional methods, neural combinatorial optimization methods based on neural networks have attracted widespread attention in recent years. These methods train neural networks to directly output approximate optimal solutions by learning the distribution patterns of a large number of problem instances. The inference phase can be completed in seconds or even milliseconds, and they have less reliance on domain knowledge.

[0005] The Attention Model (AM) was the first to introduce the Transformer architecture into the vehicle path planning problem, modeling it as a sequence generation task. It captures global dependencies between nodes through a self-attention mechanism and gradually generates path sequences during the decoding phase through the interaction of context vectors and node embeddings. This method achieves significantly faster solution speeds than traditional heuristics for small- to medium-scale problems, opening a new paradigm in neural solver research. Building on this, the Policy Optimization Method with Multiple Optimizations (POMO) further enhances the performance of neural solvers. This method employs a multi-starting-point strategy, initiating the decoding process from each node to generate multiple path trajectories. It combines instance augmentation techniques to leverage spatial symmetry and generate more candidate solutions, and uses the REINFORCE algorithm with a shared baseline to stabilize policy gradient estimation, effectively reducing training variance. Currently, POMO has become one of the most widely used baseline models in the field of neural combinatorial optimization.

[0006] While the aforementioned neural solvers offer significant computational efficiency advantages over traditional heuristic methods and can achieve near-optimal solutions, they primarily rely on node coordinates for decision-making, failing to fully utilize the distance information inherent in vehicle path planning. In vehicle path planning, distance information directly reflects the spatial proximity relationships between nodes; the next node is highly likely to be one of the nearest neighbors of the current node. Ignoring distance information leads to a lack of effective spatial guidance during model decision-making, especially as the problem scales up, easily resulting in "attention distraction"—the model struggles to distinguish key nodes from irrelevant nodes from a large number of nodes, leading to a decline in solution quality. To alleviate this problem, researchers have attempted to inject geometric and spatial prior information into neural architectures, such as the local and global policy ensemble method ELG and the distance-aware attention reshaping method DAR. ELG guides the model to make more robust decisions in unknown scenarios through the synergistic complementarity of global construction policies and local auxiliary policies. DAR manually adjusts the attention score using distance information between nodes, mitigating attention distraction by directly applying the distance penalty term to the probability calculation in the decoding stage, improving the model's generalization ability, and achieving state-of-the-art results. However, this method has the following shortcomings: First, distance information is only introduced in the decoding stage and does not participate in the feature extraction process of the encoder, which limits the full utilization of spatial priors; Second, the distance penalty adopts a fixed manual setting, which lacks learnability and cannot be adaptively adjusted according to the data distribution; Third, the fusion method of distance information in attention calculation is relatively mechanical and fails to give full play to the nonlinear modeling ability of neural networks. Summary of the Invention

[0007] To address the shortcomings of existing neural solvers in fully utilizing distance information, exhibiting distracted attention, and insufficient generalization ability in vehicle routing problems, this invention proposes a distance-enhanced attention-based neural solution optimization method for large-scale vehicle routing problems. This method constructs a joint distance-coordinate encoder, heuristically integrating distance information between nodes into the encoder's attention mechanism. By dynamically integrating geometric prior information, it maintains the solver's high focus on key nodes, enhancing its spatial reasoning ability and cross-scale, cross-distribution generalization performance, thereby improving the accuracy and generalization performance of solving vehicle routing problems.

[0008] Specifically, this invention proposes a heuristic nested grouping strategy for dynamically adjusting the distance matrix. As the encoder network depth increases, the receptive field gradually shrinks, guiding the model to progressively focus on local core paths from the global spatial context. In the shallow layers of the encoder, the receptive field is large, and the model can perceive a wide range of global spatial relationships. As the number of layers increases, the receptive field gradually shrinks, forcing the model to focus on the most relevant nearest neighbor nodes, thereby achieving a progressive spatial cognition from macro to micro during feature extraction.

[0009] Furthermore, this invention assigns an independent, learnable gating scalar to each attention head for adaptive fusion of distance prior information and the original attention score. Each attention head can autonomously adjust its dependence on distance information: some heads can focus on spatial vehicle path planning (gating value approaching 1), while others can ignore distance priors and focus on node-specific attributes (gating value approaching 0), thereby achieving multi-angle, multi-granular feature extraction.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A neural optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention is proposed, with the following specific steps: Step 1: Construct the initial embedding representation of the node The coordinate features and demand features of each node in a large-scale vehicle routing example are extracted. The nodes include warehouse nodes and customer nodes. Coordinate embedding and distance embedding are generated for each node. The coordinate embedding and distance embedding are concatenated along the feature dimension. Then, they are projected onto a unified original embedding dimension through a learnable linear fusion matrix to obtain the initial embedding representation of each node. This initial embedding representation is then input into the distance coordinate joint encoder along with the original distance matrix. Step 2: Construct a joint encoder for distance and coordinates In the layer-by-layer feature extraction process of the distance coordinate joint encoder, the node embedding representation of the previous layer (the initial embedding representation of the node in the first layer) is transformed linearly to generate queries, keys, and values, and the original self-attention score is calculated through a self-attention mechanism. The original distance matrix is ​​dynamically calculated according to the layer index of the current encoding layer and the local receptive field size is shrunk. A subset of the nearest neighbor nodes of the current node is selected, and the Euclidean distance values ​​of the selected nearest neighbor nodes are logarithmically transformed to amplify the discriminative differences between the nearest neighbor nodes, resulting in a distance matrix adjusted for the current encoding layer. The adjusted distance matrix is ​​mapped to a distance score matrix in a high-dimensional feature space through a multilayer perceptron. On this basis, an independent and learnable gating scalar is introduced for each attention head, and the weights of the distance score matrix are adaptively adjusted using the gating scalar. The original self-attention score is enhanced through a distance penalty mechanism with gating, thereby generating the node embedding representation of the current layer containing geometric space priors. After passing through all encoding layers, the global node embedding is output. Step 3: Decoder Autoregressive Construction Path Sequence During the decoding phase, based on the global node embedding output by the distance coordinate joint encoder, a query vector is generated and a basic alignment score is calculated by combining the vehicle dynamic features of the current decoding step. At the same time, a distance-guided alignment score adjustment mechanism is introduced in the decoding phase to finally generate the selection probability distribution of the next node and gradually autoregressively construct a complete path sequence. Step 4: Two-stage progressive training A strategy combining small-scale pre-training and incremental fine-tuning is adopted. In the first stage, pre-training is performed on a fixed small-scale set of instances to learn the basic vehicle path planning pattern. In the second stage, a course learning strategy is adopted to increase the problem size node by node, and a dynamic batch processing scheduling mechanism is used to adapt to the GPU memory. The network parameters are updated using a reinforcement learning strategy gradient algorithm, and finally a well-trained neural solver is obtained.

[0011] Furthermore, step 1 is detailed as follows: For including The problem involves vehicle routing across nodes, including one warehouse node and... For each delivery customer node, coordinate features and demand features are extracted. For warehouse and customer nodes, coordinate embeddings are generated through learnable linear transformations. in, Embedded for repository nodes, Embedded for client nodes, For warehouse node characteristics, For customer nodes Features and It is a learnable linear transformation matrix.

[0012] at the same time, For nodes To the node The distance from the original distance matrix Extract the warehouse (i.e., the node) ) to all nodes distance scalar And projected as distance embedding : in Time represents the warehouse node, Time represents the client node. It is a learnable linear transformation matrix.

[0013] After concatenating the coordinate embedding and the distance embedding, an initial embedding representation for all nodes is generated through linear fusion. : in, This represents a vector concatenation operation. It is a learnable fusion matrix.

[0014] Furthermore, step 2 is detailed below: For the Layer encoder, Embedded matrix from nodes in the previous layer Departure (when) At that time, the initial embedding representation of the node generated in step 1 is... ), for each attention head Calculate the query matrix Key matrix and value matrix ,in , and For the first The learnable projection matrix of the head. Then, the learnable projection matrix of the head is calculated. The original self-attention score matrix of each attention head : in, is the dimension of the key vector.

[0015] At the same time, the original distance matrix As input to the distance coordinate joint encoder, the receptive field size is dynamically calculated according to the inverse layer index. The reverse layer index refers to the encoder layer number starting from the deepest layer. Starting with the first layer, decrease gradually. The formula for calculating the receptive field size is: For each node Select the one with the smallest distance. For each neighboring node, replace its corresponding distance value with a logarithmic transformation value. For nodes that are not selected, the original distance values ​​are kept unchanged, resulting in an adjusted distance matrix. .

[0016] The purpose of the logarithmic transformation is twofold: firstly, it amplifies the discriminative differences between nearest neighbor nodes, providing stronger heuristic guidance for path decision-making; secondly, it compresses the distance value to a range similar to the attention score, preventing distance information from numerically dominating attention calculation.

[0017] Next, the adjusted distance matrix Mapped through a multilayer perceptron Different distance score matrices The multilayer perceptron comprises an input layer, a hidden layer, and an output layer. It employs the Tanh activation function to map scalar distance values ​​to a high-dimensional feature space, thereby enhancing its ability to model nonlinear spatial relationships.

[0018] Building upon this, an independent and learnable gating scalar is introduced for each attention head. The self-attention score is enhanced by adaptively adjusting the weights of the distance score matrix using the gated scalar, and by applying a gated distance penalty. : in, The Sigmoid function limits the gate value to 1 / 2. Within the range. The subtraction operation treats distance as a spatial penalty term; the smaller the distance, the smaller the penalty, thus assigning it a higher weight in the final attention score.

[0019] The enhanced attention scores of each head are normalized using Softmax and then applied to the value vector. Get output : Finally, the outputs of each layer are concatenated and linearly projected. After residual connection and layer normalization, the result is input into the feedforward network to generate the node embedding matrix of the current layer. : in, This represents the intermediate hidden state matrix output by the multi-head attention layer. To output the projection matrix, Indicates a feedforward network. This indicates a splicing operation. Presentation layer normalization operation. After... After layer-by-layer feature extraction using layer encoding, the final output is the global node embedding matrix. .

[0020] Furthermore, step 3 is detailed below: 3.1) Calculate the basic alignment score The global node embedding matrix The elements are In the decoding step ,from Extract the node embedding selected in the previous time step .Will With the vehicle's current remaining capacity (Indicates how much cargo remains in the vehicle to meet customer demand) Concatenate the data and perform a linear transformation to generate a query vector. : in, This is a learnable query projection matrix.

[0021] The context vector is obtained by interacting with the embeddings of all nodes through a multi-head attention mechanism (MHA). : in, and These are the projection matrices for the keys and values, respectively.

[0022] context vector With global node embedding matrix Perform dot product scaling to obtain the basic aligned fractional vector. : in, is the dimension of the embedded vector.

[0023] 3.2) Alignment score adjustment guided by distance Get the current node Distance scalar to all other nodes The vector formed Select the one with the smallest distance For each neighboring node, replace its distance value with a logarithmic transformation value. The adjusted distance vector is obtained. : in, yes The distance scalar in For the current node A set of indexes of the nearest neighbor nodes.

[0024] Introducing learnable gating scalars Calculate the alignment score for distance enhancement. : The enhanced alignment scores are activated using the Tanh function and then pruned to stabilize training and prevent premature convergence. in, For nodes Choose the logarithm. For pruning hyperparameters; A dynamic mask vector used to satisfy the physical constraints of the vehicle routing problem: visited client nodes and demand exceeding the vehicle's current remaining capacity. The corresponding node Unvisited feasible nodes that satisfy capacity constraints correspond to .

[0025] After normalization using the Softmax function, the probability distribution for selecting the next node is generated. : 3.3) State Update and Autoregressive Path Generation Based on the generated selection probability distribution Determine the next node to visit in the current step During the training phase, a probability distribution-based sampling strategy is used to explore diverse solution spaces; during the inference phase, a greedy strategy is used to directly select the node with the highest probability.

[0026] Determine the node Then, update the vehicle's current remaining capacity. (like If it is a warehouse node, then the vehicle capacity is restored to full capacity, and the mask vector is updated synchronously. Select the node Node embedding As the next decoding step Given the input, repeat steps 3.1) to 3.3 above to generate vehicle driving path sequences stepwise in an autoregressive manner. This continues until all customer nodes have been accessed and the vehicles have finally returned to the warehouse.

[0027] Furthermore, step 4 is detailed below: The neural solver of this invention employs a two-stage progressive training strategy, drawing on the "pre-training-fine-tuning" paradigm widely used in large-scale modeling to systematically improve the model's generalization ability across tasks of varying scales. Specifically, the first stage is the basic capability acquisition stage, where the model is fully trained on a fixed small-scale set of instances to quickly learn the basic decision-making patterns and spatial structure rules in vehicle routing tasks. The second stage is the progressive expansion and fine-tuning stage, where the model progressively expands the task scale "from easy to difficult, gradually increasing the scale," continuously improving its cross-scale generalization ability. This invention can be used to solve vehicle routing problems with capacity constraints, traveling salesman problems, and their variations.

[0028] 4.1) Small-scale pre-training The model is trained to convergence on a fixed small number of instances. Specifically, the problem size is set to... The training run consists of 256 rounds (approximately 400,000 steps) with a batch size of 64. This stage enables the model to quickly learn basic vehicle path planning patterns and stabilize parameters on a controllable problem scale, similar to the pre-training process of large models. By converging on large-scale, high-density simple samples, it lays a stable parameter foundation for subsequent cross-scale generalization.

[0029] 4.2) Progressive fine-tuning training This stage employs a course-based learning strategy, gradually increasing the complexity of the problems. The problem size starts from... Gradually increase to the target size (e.g.) Each training round adds one node, with 900 training rounds, each containing 64 steps. The model fully adapts and updates parameters at each intermediate scale. This process is similar to incremental fine-tuning in large models, improving the ability to model larger-scale instances by progressively expanding the context length and task complexity.

[0030] 4.3) Dynamic Batch Processing Furthermore, to adapt to memory constraints at different scales, this invention employs a dynamic batch processing scheduling mechanism, based on the current problem scale. Dynamically adjust batch size : This secondary scaling strategy ensures that the batch size decreases accordingly as the problem size increases, adapting to the GPU memory capacity. This achieves stable training on large-scale instances while maintaining training efficiency.

[0031] Both stages are driven by policy optimization algorithms, combining multi-starting-point trajectory sampling and a shared baseline strategy to further reduce gradient estimation variance and improve training stability. Through the aforementioned pre-training-progressive fine-tuning training paradigm, this invention effectively enhances the generalization performance and transfer capability of the neural solver in large-scale vehicle path planning tasks.

[0032] The beneficial effects of this invention are: This invention addresses the large-scale vehicle path planning problem by proposing a neural solution optimization method based on distance-enhanced attention. By constructing a joint distance-coordinate encoder, it significantly improves solution accuracy and generalization performance. First, this invention employs a nested grouping strategy to dynamically adjust the distance receptive field, enabling the model to establish a global spatial context in the shallow encoder and focus on local core paths in the deep encoder. This progressive spatial cognition mechanism effectively enhances the model's ability to handle large-scale vehicle path planning problems. Second, the multi-head gating mechanism designed in this invention assigns an independent gating scalar to each attention head, allowing different heads to autonomously adjust their dependence on distance priors. This multi-angle feature extraction mechanism endows the model with adaptive adjustment capabilities when facing problems of different scales and distributions, demonstrating excellent generalization performance. Finally, this invention employs a two-stage cross-scale training strategy, effectively improving the model's generalization ability in scenarios with varying scales, achieving stable transfer from fixed-scale training to large-scale generalization. Attached Figure Description

[0033] Figure 1 This is the overall flowchart of the present invention.

[0034] Figure 2 This is a diagram of the backbone network structure of the attention model of the basic architecture of this invention.

[0035] Figure 3 This is a structural diagram of the distance coordinate joint encoder of the present invention.

[0036] Figure 4 The figure shows the experimental results of this invention on 1000 node instances. Detailed Implementation

[0037] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0038] like Figure 1 As shown, this invention proposes a neural solver for the vehicle path planning problem based on distance-enhanced attention, suitable for solving large-scale vehicle path planning problems. This invention uses... Figure 2 The attention model shown is an improvement on the infrastructure. For example... Figure 3As shown, this invention employs a heuristic nested grouping strategy to dynamically adjust the distance receptive field and combines it with a multi-head gating attention mechanism to construct a joint encoder for distance coordinates, achieving a refined fusion of geometric spatial priors and high-dimensional feature weights. The following uses a vehicle routing problem with capacity constraints as an example to illustrate the specific implementation steps of this invention in detail.

[0039] (1) Initialization phase The network parameters of the encoder and decoder are initialized, including the multi-head attention layer, feedforward network, distance multilayer perceptron, and various gating scalars. The encoder consists of six identical stacked coding layers, each containing a multi-head self-attention sublayer and a feedforward network sublayer, all employing residual connections and layer normalization structures. The decoder is a single-layer structure containing a multi-head attention sublayer and a feedforward network sublayer. The distance multilayer perceptron's input layer contains one neuron to receive the scalar distance value, the hidden layer contains eight neurons using the Tanh activation function, and the output layer contains eight neurons corresponding to the distance scores of the eight attention heads. All gating scalars are initialized to 0.5 to ensure a moderate dependence on distance information in the initial stage of the model.

[0040] (2) Coding stage For each node in the input instance, coordinate embeddings and distance embeddings are generated. Specifically, for warehouse nodes and customer nodes, learnable linear transformation matrices are used respectively. and The original coordinate features are projected into 128-dimensional coordinate embedding vectors and then concatenated. Simultaneously, the distance scalar from the warehouse to each node is transformed using a linear transformation matrix. The projection is a 128-dimensional distance embedding vector. The coordinate embedding and distance embedding are concatenated along the feature dimensions to obtain a 256-dimensional concatenated vector, which is then processed by a linear fusion matrix. Project back to 128 dimensions to obtain the initial node embedding.

[0041] For the encoder's first layer( (incrementing from 1 to 6), the receptive field size of the current layer is calculated based on the inverse layer index. For the distance matrix Each node in Find the one with the smallest distance. For each of the three neighboring nodes (excluding itself), replace the distance values ​​of these neighboring nodes with logarithmic transformation values. The remaining positions retain their original distance values. The adjusted distance matrix... Input distance to a multilayer perceptron. For each node pair... Its distance scalar The data passes through an input layer, a hidden layer, and an output layer sequentially, generating an 8-dimensional distance feature vector. The distance feature vectors of all node pairs are then split along the attention head dimension, resulting in 8... Distance score matrix ,in .

[0042] Embedded matrix from the nodes of the previous layer Start, for each attention head Calculate the query matrix Key matrix Sum matrix ,in All are learnable projection matrices, with an output dimension of 16. Calculate the original self-attention score matrix. For each attention head Obtain its corresponding gated scalar Mapped to via the Sigmoid function After the interval is calculated, it is multiplied by the distance score matrix and subtracted from the original self-attention score to obtain the enhanced attention score matrix. The attention score matrix is ​​subjected to row-wise Softmax normalization to obtain the attention weight matrix, which is then multiplied by the value matrix to obtain the output of the head. The outputs of the eight attention heads are concatenated along the feature dimension, and the result is projected through the output matrix. The linear transformation back to 128 dimensions, followed by residual connections and layer normalization, is then input into the feedforward network. The feedforward network contains two linear layers, with the middle hidden layer having a dimension of 512 and employing the ReLU activation function. The output is then further processed through residual connections and layer normalization to obtain the node embeddings for the current layer. .

[0043] Repeat the above steps to complete the calculations for layers 1 through 6, and output the final node embedding matrix. .

[0044] (3) Decoding stage In the decoding step Get the index of the node selected in the previous time step. and its node embedding At the same time, obtain the vehicle's current remaining capacity. .Will With scalar splicing, through linear transformation matrix Projected as a 128-dimensional query vector Query vector As a query, the encoder outputs a node embedding matrix. The bond matrix is ​​obtained after linear transformation. Sum matrix Context vectors are calculated using a multi-head attention mechanism. . Context vector Embedded matrix with all nodes Perform dot product operation, divide by Scale the vector to obtain the base alignment score vector. .

[0045] Get the current node Euclidean distance vector to all other nodes Select the one with the smallest distance. neighboring nodes ( (The value is set to 100, consistent with existing technology), replacing the distance values ​​corresponding to these neighbor nodes with logarithmic transformation values. The remaining nodes retain their original distance values, resulting in the adjusted distance vector. Introducing a learnable decoding gate scalar. After being mapped by the Sigmoid function, the result is multiplied by the adjusted distance vector and subtracted from the base alignment score to obtain the enhanced alignment score. Crop the enhanced alignment score by setting the cropping hyperparameter. Limit the score to Within the range. Generate a dynamic mask. The number of visited customer nodes and the demand exceed the vehicle's current remaining capacity. The corresponding node Unvisited feasible nodes that satisfy capacity constraints correspond to Add the mask to the alignment score: Perform a Softmax operation on the alignment score after adding the mask to obtain the selection probability distribution of the next node. .according to Determine the current time The next node actually visited The system performs autoregressive updates to its dynamic state, repeating the decoding process to generate subsequent nodes step by step, until all client nodes have been accessed and the vehicle finally returns to the warehouse, ultimately outputting a complete vehicle travel path sequence. .

[0046] (4) Training phase In the first phase, a fixed scale was adopted. Training instances are generated according to existing technical standards, with client node coordinates within a unit square. Uniform random sampling within the area, the requirement is... Uniform random sampling is performed within the specified range, with a uniform vehicle capacity of 50. The REINFORCE algorithm combined with a multi-starting point strategy is used to update network parameters. Specifically, for each training instance, parameters are updated from... Each node initiates the decoding process, generating... For each complete path trajectory, calculate the total distance of each trajectory (taking the negative value as the reward), use the average reward of all trajectories as the baseline, calculate the policy gradient, and update the network parameters. The first phase consists of 256 training rounds, each containing approximately 1562 training steps, with a fixed batch size of 64. The Adam optimizer is used, with a learning rate set to... The weight decay is set to .

[0047] In the second phase, a course-based learning strategy will be employed to gradually increase the size of the problems. The initial size will be set to... The problem size increases by 1 after each round of training until the target size is reached. The training consists of 900 rounds, with each round containing 64 training steps. To accommodate memory limitations, the number of rounds is adjusted based on the current problem size. Dynamically adjust batch size The optimizer learning rate was adjusted to... The remaining hyperparameters remain the same as in the first stage.

[0048] (5) Experimental verification When the preset number of training rounds is reached, training is stopped, and the current network parameters are saved as the final model. During the testing phase, a greedy decoding strategy is adopted, without introducing an additional search mechanism, and the trained distance-enhanced attention neural solver is used directly to solve new instances.

[0049] Experimental results on the authoritative CVRPLib standard benchmark library in the field of operations research and optimization demonstrate that the proposed method achieves optimal performance across all scales. For problem sizes of 100-200, 200-500, and 500-1000, the proposed method achieves optimal gaps of 0.98%, 1.99%, and 3.06%, respectively, significantly outperforming the comparative methods. In particular, on the 500-1000 problem size, it improves upon the best baseline DAR (3.71%) by 0.65 percentage points, and the performance gap widens with increasing problem size, demonstrating excellent cross-scale generalization ability. In a 1000-node instance, the proposed method achieves a path length of 36.4381, superior to the 36.5755 path length achieved by DAR. Figure 4 The inference results of this method are demonstrated. In terms of inference time, this method is comparable to POMO, with only a small increase in overhead on large-scale instances, verifying the computational efficiency of this invention.

Claims

1. A neural optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention, characterized in that, The specific steps are as follows: Step 1: Construct the initial embedding representation of the node The coordinate features and demand features of each node in a large-scale vehicle routing example are extracted. The nodes include warehouse nodes and customer nodes. Coordinate embedding and distance embedding are generated for each node. The coordinate embedding and distance embedding are concatenated along the feature dimension. Then, they are projected onto a unified original embedding dimension through a learnable linear fusion matrix to obtain the initial embedding representation of each node. This initial embedding representation is then input into the distance coordinate joint encoder along with the original distance matrix. Step 2: Construct a joint encoder for distance and coordinates In the layer-by-layer feature extraction process of the distance coordinate joint encoder, the node embedding representation of the previous layer is transformed linearly to generate queries, keys, and values, and the original self-attention score is calculated through a self-attention mechanism. The original distance matrix is ​​dynamically calculated according to the layer index of the current encoding layer and the local receptive field size is shrunk. A subset of the nearest neighbor nodes of the current node is selected, and the Euclidean distance values ​​of the selected nearest neighbor nodes are logarithmically transformed to amplify the discriminative differences between the nearest neighbor nodes, resulting in a distance matrix adjusted for the current encoding layer. The adjusted distance matrix is ​​mapped to a distance score matrix in a high-dimensional feature space through a multilayer perceptron. On this basis, an independent and learnable gating scalar is introduced for each attention head. The weights of the distance score matrix are adaptively adjusted using the gating scalar, and the original self-attention score is enhanced through a distance penalty mechanism with gating, thereby generating the node embedding representation of the current layer containing geometric space priors. After passing through all encoding layers, the global node embedding is output. Step 3: Decoder Autoregressive Construction of Path Sequence During the decoding phase, based on the global node embedding output by the distance coordinate joint encoder, a query vector is generated and a basic alignment score is calculated by combining the vehicle dynamic features of the current decoding step. At the same time, a distance-guided alignment score adjustment mechanism is introduced in the decoding phase to finally generate the selection probability distribution of the next node and gradually autoregressively construct a complete path sequence. Step 4: Two-stage progressive training A strategy combining small-scale pre-training and incremental fine-tuning is adopted. In the first stage, pre-training is performed on a fixed small-scale set of instances to learn the basic vehicle path planning pattern. In the second stage, a course learning strategy is adopted to increase the problem size node by node, and a dynamic batch processing scheduling mechanism is used to adapt to the GPU memory. The network parameters are updated using a reinforcement learning strategy gradient algorithm, and finally a well-trained neural solver is obtained.

2. The neural network optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention as described in claim 1, characterized in that, Step 1 is as follows: For including The problem involves vehicle routing across nodes, including one warehouse node and... For each delivery customer node, extract the coordinate features and demand features of all nodes. For warehouse nodes and customer nodes, coordinate embeddings are generated through learnable linear transformations: in, Embedded for repository nodes, Embedded for client nodes, For warehouse node characteristics, For customer nodes Features and It is a learnable linear transformation matrix; at the same time, For nodes To the node The distance from the original distance matrix Extract the warehouse to all nodes distance scalar And projected as distance embedding : in Time represents the warehouse node, Time represents the client node. It is a learnable linear transformation matrix; After concatenating the coordinate embedding and the distance embedding, an initial embedding representation for all nodes is generated through linear fusion. : in, This represents a vector concatenation operation. It is a learnable fusion matrix.

3. The neural network optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention as described in claim 1, characterized in that, Step 2 is as follows: For the Layer encoder, Embedded matrix from nodes in the previous layer Start, for each attention head Calculate the query matrix Key matrix and value matrix ,in , and For the first The learnable projection matrix of the first head; then, the learnable projection matrix of the second head is calculated. The original self-attention score matrix of each attention head : in, The dimension of the key vector; At the same time, the original distance matrix As input to the distance coordinate joint encoder, the receptive field size is dynamically calculated according to the inverse layer index. The reverse layer index refers to the encoder layer number starting from the deepest layer. Starting with the first layer, decrease gradually; the formula for calculating the receptive field size is: For each node Select the one with the smallest distance. For each neighboring node, replace its corresponding distance value with a logarithmic transformation value. For nodes that are not selected, the original distance values ​​are kept unchanged, resulting in an adjusted distance matrix. ; Next, the adjusted distance matrix Mapped through a multilayer perceptron Different distance score matrices The multilayer perceptron comprises an input layer, a hidden layer, and an output layer. It employs the Tanh activation function to map scalar distance values ​​to a high-dimensional feature space, thereby enhancing its ability to model nonlinear spatial relationships. Introduce an independent, learnable gating scalar for each attention head. The self-attention score is enhanced by adaptively adjusting the weights of the distance score matrix using the gated scalar, and by applying a gated distance penalty. : in, The Sigmoid function limits the gate value to 1 / 2. Within the range; The enhanced attention scores of each head are normalized using Softmax and then applied to the value vector. Get output : Finally, the outputs of each layer are concatenated and linearly projected. After residual connection and layer normalization, the result is input into the feedforward network to generate the node embedding matrix of the current layer. : in, This represents the intermediate hidden state matrix output by the multi-head attention layer. To output the projection matrix, Indicates a feedforward network. This indicates a splicing operation. Presentation layer normalization operation; after After layer-by-layer feature extraction using layer encoding, the final output is the global node embedding matrix. .

4. The neural network optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention as described in claim 1, characterized in that, Step 3 is as follows: 3.1) Calculate the basic alignment score The global node embedding matrix The elements are In the decoding step ,from Extract the node embedding selected in the previous time step ;Will With the vehicle's current remaining capacity The query vector is generated by concatenating and linearly transforming the data. : in, For learnable query projection matrix; The context vector is obtained by interacting with the embeddings of all nodes through the multi-head attention mechanism (MHA). : in, and These are the projection matrices for the keys and values, respectively; context vector With global node embedding matrix Perform dot product scaling to obtain the basic aligned fractional vector. : in, The dimension of the embedded vector; 3.2) Alignment score adjustment guided by distance Get the current node Distance scalar to all other nodes The vector formed Select the one with the smallest distance For each neighboring node, replace its distance value with a logarithmic transformation value. The adjusted distance vector is obtained. : in, yes The distance scalar in For the current node The set of indexes of the nearest neighbor nodes; Introducing learnable gating scalars Calculate the alignment score for distance enhancement. : The enhanced alignment scores are activated using the Tanh function and then pruned to stabilize training and prevent premature convergence. in, For nodes Choose the logarithm. For clipping hyperparameters; A dynamic mask vector used to satisfy the physical constraints of the vehicle routing problem: visited client nodes and demand exceeding the vehicle's current remaining capacity. The corresponding node Unvisited feasible nodes that satisfy capacity constraints correspond to ; After normalization using the Softmax function, the probability distribution for selecting the next node is generated. : 3.3) State Update and Autoregressive Path Generation Based on the generated selection probability distribution Determine the next node to visit in the current step During the training phase, a probability distribution-based sampling strategy is used to explore diverse solution spaces; during the inference phase, a greedy strategy is used to directly select the node with the highest probability. Determine the node Then, update the vehicle's current remaining capacity. And update the mask vector synchronously. Select the node Node embedding As the next decoding step Given the input, repeat steps 3.1) to 3.3 above to generate vehicle driving path sequences stepwise in an autoregressive manner. This continues until all customer nodes have been accessed and the vehicles have finally returned to the warehouse.

5. The neural optimization method for solving large-scale vehicle path planning problems based on distance-enhanced attention as described in claim 1, characterized in that, Step 4 is as follows: 4.1) Small-scale pre-training The model is trained to convergence on a fixed small number of instances; specifically, the problem size is set to be... The training rounds consist of 256 rounds, and the batch size is 64. 4.2) Progressive fine-tuning training A course-based learning strategy is adopted to gradually increase the complexity of the problems; the problem size starts from... The training scale is gradually increased to the target size, with one node added in each training round. There are 900 training rounds, with each round containing 64 steps. The model is fully adapted and the parameters are updated at each intermediate scale. 4.3) Dynamic Batch Processing A dynamic batch processing scheduling mechanism is adopted, based on the current problem size. Dynamically adjust batch size : 。

Images