Method for solving traveling salesman problem based on quantum self-attention neural network

Abstract

The invention discloses a traveling salesman problem solving method based on a quantum self-attention neural network, and belongs to the field of quantum computing. A classic encoder-decoder structure is adopted, a quantum self-attention layer is designed in the encoder and comprises a quantum multi-head attention layer and a quantum full-connection feed-forward layer, and a variable component sub-circuit is used in a sub-layer to replace a full-connection layer of a classic neural network. Firstly, an encoder receives an input data set, node features are embedded into a quantum self-attention layer, and a complex relation between nodes is extracted through a quantum multi-head attention mechanism. The decoder then stepwise outputs the sequence of nodes to form an optimal path according to the embedding generated by the encoder. In the model training stage, a REINFORCE deep reinforcement learning algorithm is utilized, the total path length serves as an optimization target, and model parameters are updated through strategy gradients. Compared with the existing algorithm, the method has the advantage that the performance is well improved.

Description

Technical Field

[0001] The present invention relates to the field of quantum computing, and in particular to a solution to the traveling salesman problem based on a quantum self-attention neural network. Background Art

[0002] The Traveling Salesman Problem (TSP) is a classic problem in the field of combinatorial optimization. Its goal is to design a shortest path for a traveler to start from a starting point, visit all target cities only once, and then return to the starting point. Due to its simple description and wide range of practical applications, such as logistics path planning and communication network design, TSP has long been a research focus. However, TSP is an NP-hard problem. As the size of the problem increases, the complexity of solving it increases exponentially. Therefore, efficiently solving TSP has always been a challenge for academia and industry.

[0003] In recent years, deep learning, especially the Transformer model based on the self-attention mechanism, has made significant progress in solving TSP. For example, the Attention Model (AM) based on the self-attention mechanism can automatically learn optimization strategies from data by learning the dependencies between nodes, reducing the reliance on manually designed features. However, most of these methods are based on classic neural network models and have the following shortcomings:

[0004] Large parameter scale: Classic deep learning models require a large number of network parameters, which increases computational and storage overhead;

[0005] High data requirements: Due to the large scale of model parameters, a large amount of training data is often required, which is difficult to meet in some practical scenarios;

[0006] To solve the above problems, quantum computing introduces new possibilities. Quantum computing significantly improves parallel computing capabilities through the characteristics of quantum superposition and quantum entanglement, and provides a new way of information processing. Quantum Machine Learning (QML) combines quantum computing with classical machine learning and has shown great potential in optimization problems. For example, variational quantum circuits (VQC) can achieve the expression of complex functions through a small number of quantum bits, thereby reducing model parameters.

[0007] Based on this, the present invention proposes a TSP solution method based on quantum self-attention neural network, which combines quantum computing with self-attention mechanism, captures the complex relationship between nodes through quantum multi-head attention mechanism and quantum fully connected feedforward layer, and optimizes model parameters using deep reinforcement learning algorithm. This method not only significantly reduces model parameters and data requirements, but also outperforms existing classical methods in terms of path optimization quality and convergence speed, providing new ideas for solving TSP and other combinatorial optimization problems. Summary of the invention

[0008] The technical problem to be solved by the present invention is: how to effectively solve the problems of large parameter scale and high training data demand of classical neural network model, improve the performance of the algorithm for the traveling salesman problem, and provide a solution to the traveling salesman problem based on quantum self-attention neural network. First, the encoder processes the input node features through a quantum self-attention layer, wherein the quantum multi-head attention layer generates node embedding using a variational quantum circuit, captures the complex relationship between nodes, and improves the network stability through residual connection and batch normalization, providing high-quality feature representation for the decoder. The decoder gradually generates a path sequence, dynamically pays attention to unvisited nodes through an attention mechanism, and selects the current optimal node in combination with path context information. Finally, the decoder generates a complete path and returns to the starting node, calculates the total length of the path as the objective function value, optimizes the model parameters using a REINFORCE deep reinforcement learning algorithm, and continuously adjusts the model through a policy gradient method, so that the performance of the entire network and the generalization ability of the model are further improved and optimized, and a better performance structure is found.

[0009] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0010] S1: Define the traveling salesman problem instance s as a graph with n nodes, where nodes i∈{1,2,...,n} are represented by features X i Indicates that, X i are the coordinates of node i, and the graph is fully connected.

[0011] S2: The encoder processes the node features in the data set through the quantum self-attention layer. The quantum self-attention layer includes a quantum multi-head attention layer and a quantum fully connected feedforward layer. In the quantum multi-head attention layer, node embeddings are generated through variational quantum circuits to capture the complex relationships between nodes. Residual connections and batch normalization techniques are used to further improve the stability of the network and provide high-quality node embeddings for subsequent decoder operations.

[0012] S3: Based on the node embedding information generated by the encoder, the decoder outputs the node sequence in a step-by-step manner, selecting the optimal next node at each step. During the decoding process, the decoder dynamically pays attention to unvisited nodes through the attention mechanism, and combines the path context information to ensure that the path generation meets the constraints of the traveling salesman problem. In order to improve the decoding efficiency, a special context node is introduced to update the current path status in real time and guide the next node selection.

[0013] S4: Based on the node sequence generated by the decoder, a complete path is constructed and returned to the starting node to form a solution to the traveling salesman problem. The total length of the path is calculated and used as the objective function value of the traveling salesman problem to evaluate the quality of the current solution. The parameters of the encoder and decoder are optimized using the REINFORCE deep reinforcement learning algorithm. The model is continuously adjusted through the policy gradient method so that the path it generates gradually approaches the global optimal solution, thereby completing the solution to the traveling salesman problem.

[0014] Furthermore, in step S1, the traveling salesman problem instance s is represented as a fully connected graph containing n nodes, and each node i is represented by its feature X i Indicates that X i Usually the two-dimensional coordinates of the nodes; the input data set consists of a set of these node features {X1,X2,...,X n}, a certain number of nodes are uniformly selected from the unit square [0,1] to generate a random TSP data set.

[0015] Furthermore, the step S2 includes the following steps:

[0016] S21: In order to generate the initial node embedding, the encoder first performs a linear transformation on the input features and maps them to a high-dimensional embedding space. The specific operations are as follows:

[0017] H i =W input X i +b input ,i∈{1,2,...,n}

[0018] Among them, W input is the weight matrix, b input is the bias vector;

[0019] Initial node embedding H i is mapped into a quantum state and then input into a variational quantum circuit (VQC). A variational quantum circuit is a quantum circuit composed of multiple parameterized quantum gates. The input state is optimized by adjusting these parameters. The design of the variational quantum circuit is as follows: First, a parameterized rotation gate R is applied to the quantum bit. X, initialize the quantum state, then apply the CNOT gate to generate an entangled state between qubits, and finally obtain the new node embedding by measuring the quantum state where the information in the quantum state is converted into classical numerical values for further processing.

[0020] S22: For each node embedding First, calculate the query vector Q i , the key vector K i and the value vector V i :

[0021]

[0022] where W Q , W K , W V are the weight matrices for query, key, and value respectively;

[0023] The attention weight A ij is defined as:

[0024]

[0025] Use the attention weight to perform a weighted sum on the value vector V j to obtain the attention output of the node

[0026]

[0027] S23: After entering the quantum fully connected feed-forward layer, it is first input to the first layer of the quantum fully connected feed-forward layer, which performs a linear transformation on the input and processes it through the ReLU activation function to extract more complex features in the high-dimensional hidden space; the output of the hidden layer is fed into the second linear transformation, which maps the features back to the space with the same dimension as the original node embedding to maintain consistency with the original input; to enhance the gradient propagation ability and avoid the problem of gradient vanishing in the deep network, the quantum fully connected feed-forward layer adds a residual connection at the output stage, and the residual connection will be directly added to the output of the quantum fully connected feed-forward layer and then normalized to obtain the features of each node The final output H e and are expressed as:

[0028]

[0029] Furthermore, in step S3, the decoder proceeds sequentially during the decoding process. At time step t ∈ {1, 2,..., n}, the decoder generates the output π based on the node embedding generated by the encoder and the output generated at time t′ < tt′ To output node π t ; First, a special context node is added to the graph to represent the decoding context, which includes the embedding of the graph Previous node π t-1 and the embedding of the first node π1; we only need to calculate a query vector Q from the context node and use the encoder output to generate the key vector K and value vector V, thereby calculating the query vector Q and the attention scores of all nodes, and masking the nodes that have been visited within time step t, and finally using softmax to calculate the final output probability vector p, that is, the probability of selecting the node.

[0030] Furthermore, step S4 includes the following steps: first, the total path length L(π|s) of the traveling salesman problem is taken as the optimization target, and the objective function J(θ|s) is constructed. In the case of a given problem instance s, J(θ|s) is defined as:

[0031] J(θ|s)=E π～pθ(.|s) [L(π|s)]

[0032] We use policy gradient and stochastic gradient descent to optimize the quantum neural network parameters θ, expressed by the REINFORCE algorithm:

[0033]

[0034] in, is the gradient of the strategy network for generating the path, which is used to update the model parameters; b(s) is the benchmark function, which is used to reduce the variance of the gradient and improve the stability of training; L(π|s) is the actual path length, which measures the quality of the generated path;

[0035] The Adam optimizer is used to update the model parameters θ in the direction of minimizing the expected path length.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] (1) The present invention replaces the fully connected layer in the classical neural network model by introducing a variational quantum circuit. Only a small number of quantum bits are needed to capture the complex relationship between nodes, which greatly reduces the number of parameters required for the model, significantly reduces the computational complexity and memory consumption, and enables the model to run efficiently even under conditions of limited hardware resources.

[0038] (2) The present invention uses the quantum multi-head attention mechanism to accurately capture the global dependencies between nodes, reducing the model's dependence on large-scale training data sets. In addition, the quantum self-attention layer enhances the feature expression capability, giving the model stronger generalization performance.

[0039] (3) The present invention combines quantum computing and deep reinforcement learning. The decoder dynamically updates the path state to ensure that the path generation meets the constraints of the traveling salesman problem. The introduction of special context nodes further improves the decoding efficiency and can quickly generate a path sequence that meets the constraints. During the training process, the parameters of the encoder and decoder are optimized by the REINFORCE reinforcement learning algorithm, gradually approaching the global optimal solution, and significantly improving the quality of path optimization. Experimental results show that the present invention is superior to traditional methods in key indicators such as path length and convergence speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structure diagram of a quantum self-attention neural network in an embodiment of the present invention;

[0041] Figure 2 is a diagram of a variational quantum circuit structure in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the comparison between the experimental results of the embodiment of the present invention and the classical neural network model; DETAILED DESCRIPTION

[0043] The embodiments of the present invention are described in detail. The embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0044] The present invention uses randomly generated traveling salesman problem data (TSP5) for experiments, and provides a solution to the traveling salesman problem based on a quantum self-attention neural network. The following steps are included: step S1: defining the traveling salesman problem instance s as a graph with 5 nodes, randomly generating five nodes in each experiment, and constructing a corresponding data set; step S2: the encoder processes the node features in the data set through a quantum self-attention layer; step S3: the decoder outputs a node sequence in a step-by-step manner based on the node embedding information generated by the encoder, and selects the current optimal next node at each step; step S4: using the REINFORCE deep reinforcement learning algorithm to optimize the parameters of the encoder and decoder, and continuously adjusting the model through the policy gradient method so that the path generated by it gradually approaches the global optimal solution.

[0045] Dataset Generation

[0046] In the TSP5 experiment, we randomly generate a dataset of five nodes, each node is represented in the form of two-dimensional coordinates. A fully connected weighted graph is constructed by calculating the Euclidean distance between all nodes to form the input dataset of the traveling salesman problem.

[0047] Configuration of quantum self-attention neural network

[0048] refer to Figure 1 and Figure 2 In the structure of the quantum self-attention neural network model of the present invention, the encoder consists of N=3 layers of quantum self-attention layers. Each module contains a quantum multi-head attention layer and a quantum fully connected feedforward layer. The quantum self-attention layer uses a variational quantum circuit to generate node embeddings to capture the complex relationship between nodes. The number of quantum bits in the variational quantum circuit is set to 4 to strike a balance between result quality and computational complexity.

[0049] Model Training

[0050] The operating environment of the embodiment of the present invention adopts PyTorch and the quantum machine learning framework PennyLane. PennyLane is an open source library based on PyTorch that focuses on accelerating quantum computing. The library provides additional libraries and modules that enable users to easily build, optimize, and deploy quantum classical applications. In order to build a quantum self-attention neural network model, we implemented the classical module using PyTorch and the quantum module using PennyLane. Through the REINFORCE reinforcement learning algorithm, we optimized the parameters of the quantum self-attention neural network. We used the Adam optimizer to train the network parameters, and the learning rate was set to η=10 -4 , the decay factor in the REINFORCE algorithm is set to 0.8. Each training cycle processes 100 batches, each batch contains 512 samples, a total of 51200 sample data are trained, and the number of training cycles is set to 100.

[0051] Experimental comparison and evaluation

[0052] The experiment selected Pointer Network (Ptr-Net), Attention Model (AM), AM-critic (AM model based on actor-critic algorithm optimization) and AM-PPO (AM model combined with Proximal Policy Optimization algorithm) as comparison models. Ptr-Net uses the attention mechanism to select nodes on the path, the AM model improves Ptr-Net based on the Transformer architecture, and AM-PPO further enhances the model stability through the policy optimization algorithm.

[0053] The experimental results are as follows Figure 3As shown in the figure, it can be seen that the experimental results of the ptr-net model are the worst. After 10 epochs, the training traveling salesman path length target begins to converge. The AM model and AM-PPO model can converge soon after the start of training. Since the PPO algorithm can further improve the stability and convergence of the model, it performs better in solving the traveling salesman problem. Our model can optimize the training target soon after the start of training, and begins to converge and further optimize after 20 epochs. The final training result is also better than the previous classical neural network model. The advantage of quantum computing lies in its parallel computing capability. Quantum neural networks use quantum superposition and quantum entanglement to process multiple possible solutions simultaneously in one calculation. This enables quantum neural networks to explore the solution space of the TSP problem faster, especially in the case of a smaller number of cities, this parallel processing capability may bring significant efficiency improvements. In smaller-scale TSP problems, the number of quantum bits required is smaller and the calculation depth is shallower, so the impact of noise and error may be smaller, making quantum neural networks more stable in this case.

[0054] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A solution to the traveling salesman problem based on quantum self-attention neural network, characterized in that: The following steps are involved: S1: Define the traveling salesman problem instance s as a graph with n nodes, where nodes i∈{1,2,...,n} are represented by features X i Indicates that, X i are the coordinates of node i, and the graph is fully connected; S2: The encoder processes the node features in the data set through the quantum self-attention layer. The quantum self-attention layer includes the quantum multi-head attention layer and the quantum fully connected feedforward layer. In the quantum multi-head attention layer, node embedding is generated through variational quantum circuits to capture the complex relationship between nodes. Residual connection and batch normalization technology are used to further improve the stability of the network and provide high-quality node embedding for subsequent decoder operations. S3: Based on the node embedding information generated by the encoder, the decoder outputs the node sequence in a step-by-step manner, selecting the optimal next node at each step. During the decoding process, the decoder dynamically pays attention to unvisited nodes through the attention mechanism, and combines the path context information to ensure that the path generation meets the constraints of the traveling salesman problem. In order to improve the decoding efficiency, a special context node is introduced to update the current path status in real time and guide the next node selection; S4: Based on the node sequence generated by the decoder, a complete path is constructed and returned to the starting node to form a solution to the traveling salesman problem. The total length of the path is calculated and used as the objective function value of the traveling salesman problem to evaluate the quality of the current solution. The parameters of the encoder and decoder are optimized using the REINFORCE deep reinforcement learning algorithm. The model is continuously adjusted through the policy gradient method so that the path it generates gradually approaches the global optimal solution, thereby completing the solution to the traveling salesman problem.

2. A solution to the traveling salesman problem based on a quantum self-attention neural network according to claim 1, characterized in that: In step S1, the traveling salesman problem instance s is represented as a fully connected graph containing n nodes, and each node i is represented by its feature X i Indicates that X i Usually the two-dimensional coordinates of the nodes; the input data set consists of a set of these node features {X1,X2,...,X n }, a certain number of nodes are uniformly selected from the unit square [0,1] to generate a random TSP data set.

3. A solution to the traveling salesman problem based on a quantum self-attention neural network according to claim 1, characterized in that: In step S2, in order to generate the initial node embedding, the encoder first performs a linear transformation on the input features and maps them to a high-dimensional embedding space. The specific operation is: H i =W input X i +b input ,i∈{1,2,...,n} Among them, W input is the weight matrix, b input is the bias vector; Initial node embedding H i is mapped into a quantum state and then input into a variational quantum circuit (VQC). A variational quantum circuit is a quantum circuit composed of multiple parameterized quantum gates. The input state is optimized by adjusting these parameters. The design of the variational quantum circuit is as follows: First, a parameterized rotation gate R is applied to the quantum bit. X , to initialize the quantum state, then apply the CNOT gate to generate the entangled state between the quantum bits, and finally get the new node embedding by measuring the quantum state The information in the quantum state is converted into classical values ​​for further processing.

4. A solution to the traveling salesman problem based on a quantum self-attention neural network according to claim 3, characterized in that: For each node embedding First, calculate the query vector Q i , key vector K i Sum value vector V i : Among them, W Q , W K , W V are the weight matrices for query, key, and value, respectively; Attention weight A ij Defined as: Use attention weights to represent the value vector V j Weighted summation to get the node's attention output 5. A solution to the traveling salesman problem based on a quantum self-attention neural network according to claim 4, characterized in that: After entering the quantum fully connected feedforward layer, it is first input into the first layer of the quantum fully connected feedforward layer, which performs a linear transformation on the input and processes it through the ReLU activation function, thereby extracting more complex features in the high-dimensional hidden space; The output of the hidden layer is fed into the second layer of linear transformation, which maps the features back to a space consistent with the original node embedding dimension to maintain consistency with the original input. In order to enhance the gradient propagation capability and avoid the gradient vanishing problem in deep networks, the quantum fully connected feedforward layer adds a residual connection at the output stage. The residual connection The characteristics of each node are obtained by directly adding it to the output of the quantum fully connected feedforward layer and normalizing it. The final output of the encoder is H e and It is expressed as:

6. The method of claim 1 for solving the traveling salesman problem based on a quantum self-attention neural network, characterized in that: In the step S3, the decoder performs the decoding process sequentially. At time step t ∈ {1, 2,..., n}, the decoder outputs the node π based on the node embedding generated by the encoder and the output π generated at time t' < t. t′ to output the node π t ; First, a special context node is added to the graph to represent the decoding context, and the context includes the embedding of the graph the previous node π t-1 and the embedding of the first node π1; We only need to calculate a query vector Q from the context node and use the encoder output to generate the key vector K and value vector V, thereby calculating the query vector Q and the attention scores of all nodes, masking the nodes that have been visited within time step t, and finally using softmax to calculate the final output probability vector p, that is, the probability of selecting the node.

7. The method of claim 1 for solving the traveling salesman problem based on a quantum self-attention neural network, characterized in that: In step S4, the quantum self-attention neural network is trained using the REINFORCE algorithm, and the training process includes the following steps: First, the total path length L(π|s) of the traveling salesman problem is taken as the optimization target, and the objective function J(θ|s) is constructed. Given the problem instance s, J(θ|s) is defined as: J(θ|s)=E π～pθ (.|s)[L(π|s)] We use policy gradient and stochastic gradient descent to optimize the quantum neural network parameters θ, expressed by the REINFORCE algorithm: in, is the gradient of the strategy network for generating the path, which is used to update the model parameters; b(s) is the benchmark function, which is used to reduce the variance of the gradient and improve the stability of training; L(π|s) is the actual path length, which measures the quality of the generated path; The Adam optimizer is used to update the model parameters θ in the direction of minimizing the expected path length.

Images