A path planning method of a bidirectional gated recurrent network and a quantum transformer network

Abstract

The application provides a path planning method of a bidirectional gated recurrent network and a quantum Transformer network, and is used for solving a path planning problem of multiple mobile robots. The method uses a bidirectional gated recurrent network Hungarian method to solve a task allocation problem. A quantum computing method is combined into a Transformer neural network to form a quantum Transformer neural network. The quantum Transformer neural network is combined with a decision process of reinforcement learning to form a multi-agent deep reinforcement learning method based on the quantum Transformer neural network. The multi-agent deep reinforcement learning method based on the quantum Transformer neural network plans a path for a mobile robot which has been allocated a task. The path planning method of the bidirectional gated recurrent network and the quantum Transformer network can improve the accuracy of a task allocation scheme of a multiple mobile robot system, shorten a training time, give a path planning action with high accuracy, optimize a path of a mobile robot, and reduce collision between different mobile robots.

Description

Technical Field

[0001] This invention belongs to the field of path planning for mobile robots, and involves bidirectional gated recurrent network methods, Hungarian methods, quantum computing methods, Transformer neural network methods, and reinforcement learning decision-making methods, which are applicable to path planning for multiple mobile robots. Background Technology

[0002] Accurate distance information is crucial for accurate task allocation methods to produce accurate task allocation schemes. An accurate task allocation scheme ensures the shortest total path length for the mobile robot. After the initial task allocation, if the target point's location changes, the initial allocation scheme may be inaccurate. To correct this, the task allocation method needs to continuously recalculate the scheme as the target point's location changes. However, recalculating the allocation only after the target point's location changes may result in inaccurate recalculations. This inaccuracy stems from insufficient real-time performance, causing the target point's location to lag behind the actual location. Due to environmental obstacles, distance calculation methods cannot simply assume a straight-line distance between two points. They must provide a reasonable estimate of the distance between two points, assuming the path doesn't conflict with obstacles. The A* method can quickly obtain the path between two points in a static environment, leading to a reasonable distance estimate. However, the presence of moving obstacles can cause inaccurate calculations by the A* method, resulting in inaccurate task allocation schemes.

[0003] Traditional neural network methods require significant time to train on large datasets. When a complex scenario is modeled using data from a dataset, the resulting dataset is large, further impacting the training time. The self-attention structure is a crucial part of the Transformer neural network and also consumes considerable computational power. Furthermore, the accuracy of the attention coefficients output by the self-attention structure significantly affects the prediction accuracy of the Transformer neural network method. The prediction accuracy of the Transformer neural network method has a crucial impact on the accuracy of value estimation for path planning actions.

[0004] Furthermore, path planning for multiple mobile robots is not simply an aggregation of single-robot path planning. It requires considering path conflicts between the robots. Some path planning methods do not leverage the advantages of multi-agent collaborative planning but instead employ a hierarchical control approach. This approach first plans the path for each robot at the bottom layer and then coordinates the paths between different robots at the top layer to avoid collisions. However, the resulting robot paths from this hierarchical planning method are often not optimal.

[0005] Therefore, a bidirectional gated recurrent network (GRN) Hungarian method is proposed to improve the accuracy of task allocation schemes in multi-robot systems. The first GRN predicts the position of the moving target point. The second GRN predicts the position of moving obstacles. The A* method calculates distance information based on the predicted positions of the moving target point and moving obstacles. The Hungarian method uses the distance information obtained from the A* method to calculate the task allocation scheme. The Hungarian method, by using the distance information calculated from the predicted positions of the moving target point and moving obstacles to calculate the task allocation scheme, can react in advance to the dynamic changes of the multi-robot system, thus improving the accuracy of the task allocation scheme.

[0006] This paper proposes a method combining quantum computing with Transformer neural networks to improve the prediction accuracy and the ability of Transformer neural networks to handle large datasets. The variable quantum eigenvalue solver in quantum computing can be combined with classical optimization methods to optimize parameters. Therefore, combining the variable quantum eigenvalue solver with a self-attention structure can optimize the attention coefficients, thereby improving the prediction accuracy and the accuracy of value estimation for path planning actions in Transformer neural networks. Quantum computing methods can leverage the superposition principle of quantum states to achieve parallelism, making them suitable for handling large datasets. However, self-attention structures face excessive computational complexity when handling large datasets. Therefore, this paper combines the control rotation gate and quantum amplitude amplification methods from quantum computing into the self-attention structure of Transformer neural networks to shorten the training time.

[0007] This paper proposes a multi-agent deep reinforcement learning method based on quantum Transformer neural networks to collaboratively plan the paths of multiple mobile robots. During interaction with the environment, each agent calculates its individual Q-value using deep reinforcement learning based on quantum Transformer neural networks. At this point, the individual Q-value represents only the value of the agent's action to that individual agent, not its value to the entire multi-agent system. Therefore, the multi-agent deep reinforcement learning method based on quantum Transformer neural networks inputs the individual Q-values ​​of all agents in a given state into a hybrid network to obtain the overall Q-value of the system. The overall Q-value is then decomposed into the Q-values ​​of individual agents according to a predefined method. The multi-agent deep reinforcement learning method based on quantum Transformer neural networks is trained using the aforementioned method. After training, the multi-agent deep reinforcement learning method based on quantum Transformer neural networks plans the paths of the multiple mobile robots. This method leverages the advantages of collaborative planning among multiple mobile robots to obtain the Q-value of the multi-robot system in a given state, reducing the total path length of the mobile robots at the overall level. Moreover, the multi-agent deep reinforcement learning method based on the quantum Transformer neural network can reasonably decompose the overall Q value into individual Q values, thereby reducing the probability of collisions between mobile robots. Summary of the Invention

[0008] This invention proposes a path planning method using bidirectional gated recurrent networks and quantum Transformer networks. It combines bidirectional gated recurrent networks, the Hungarian method, quantum computing, Transformer neural networks, and reinforcement learning decision-making methods for path planning of multiple mobile robots. This method improves the accuracy of task allocation schemes, shortens training time, provides highly accurate path planning actions, optimizes the paths of mobile robots, and reduces collisions between different mobile robots. The steps in its application are as follows:

[0009] Step (1): In a multi-mobile robot system, there is at least one mobile robot and one task; the initial positions of the mobile robot and the task are random; the position of the task can change, and the task whose position can change is the target point for movement; the multi-mobile robot path planning method requires assigning tasks to the mobile robots before planning paths for them; the task assignment method requires assigning a mobile robot to each task to execute; without considering the conflict between paths of different mobile robots, the task assignment method reasonably assigns tasks to the mobile robots so that the total path length of the mobile robots is the shortest; the path planning method needs to plan the path from the position of the mobile robot to the position of the task for the mobile robot that has been assigned a task, and needs to fully consider the conflict between paths of different mobile robots.

[0010] Step (2): The bidirectional gated loop network consists of an input layer, a hidden layer, and an output layer; the hidden layer consists of a forward hidden layer and a backward hidden layer; the forward hidden layer and the backward hidden layer are two gated loop networks in opposite directions; the input layer passes information to the forward hidden layer and the backward hidden layer respectively; the forward hidden layer passes information within the forward hidden layer and passes information to the output layer; the backward hidden layer passes information within the backward hidden layer and passes information to the output layer;

[0011] Step (3): The bidirectional gated recurrent network in time Hidden layer state From the forward hidden layer state and the backward hidden layer state get:

[0012] (1)

[0013] (2)

[0014] (3)

[0015] in, For time The following forward hidden layer state, For time The following forward hidden layer state, For time The following backward hidden layer state, For time The following backward hidden layer state, As a way to input vector Encode into the corresponding nonlinear transformation function of the hidden layer state. for The function takes inputs other than the hidden state. These are the weight coefficients of the forward hidden layer. These are the weight coefficients of the backward hidden layer. For time Hidden layer state Corresponding to the forward hidden layer state and the backward hidden layer state The offset;

[0016] Step (4): Two bidirectional gated recurrent networks are used to predict the positions of the moving target point and the moving obstacles; the first bidirectional gated recurrent network is trained using the moving target point position dataset; the moving target point position dataset includes the past positions of all moving target points; the trained first bidirectional gated recurrent network outputs the predicted position of the moving target point; the second bidirectional gated recurrent network is trained using the moving obstacle position dataset; the moving obstacle position dataset includes the past positions of all moving obstacles; the trained second bidirectional gated recurrent network outputs the predicted position of the moving obstacles.

[0017] Step (5): A bidirectional gated recurrent network is combined with the Hungarian method to form a bidirectional gated recurrent network. The Hungarian method is used for task allocation in a multi-mobile robot system. The two bidirectional gated recurrent networks obtain the previous positions of the moving target points and the moving obstacles of the multi-mobile robot system, respectively. The two bidirectional gated recurrent networks output the predicted positions of the moving target points and the moving obstacles, respectively. The A* method calculates the distance information of the multi-mobile robot system based on the predicted positions of the moving target points and the moving obstacles. The distance information includes the distance information from the mobile robot to the moving target point and the distance information between the moving target points.

[0018] The Hungarian method uses distance information to calculate a task allocation scheme; it creates a weight matrix based on the distance information; the weight matrix represents the weight between each pair of vertices in the bipartite graph; the Hungarian method subtracts the minimum value of the weight matrix from each row of the matrix, and then subtracts the minimum value of the weight matrix from each column; the Hungarian method draws as many zero elements as possible with as few horizontal and vertical lines as possible; if the sum of the number of horizontal and vertical lines equals the order of the weight matrix, the Hungarian method finds the optimal solution.

[0019] If the sum of the number of horizontal and vertical lines is not equal to the order of the weight matrix, the Hungarian method needs to find the minimum value among the elements in the weight matrix that are not covered by lines, and update the weight matrix based on the minimum value. The Hungarian method's update operation adds the minimum value to the elements in the rows covered by horizontal lines, and subtracts the minimum value from the elements in the columns covered by vertical lines. The Hungarian method repeats the above process until the sum of the number of horizontal and vertical lines equals the order of the weight matrix.

[0020] Step (6): The self-attention structure is applied to the encoder and decoder of the Transformer neural network; the multi-head self-attention structure is the superposition of multiple self-attention structures; the multi-head self-attention structure that performs the masking operation in the self-attention structure is a masked multi-head self-attention structure; the input of the self-attention structure of the Transformer neural network is the query matrix. Key-value matrix and value matrix Output for:

[0021] (4)

[0022] in, For query matrix The number of columns, This is the matrix transpose operation. As an activation function, it can convert a matrix The sum of each row becomes 1; The function outputs the attention coefficient;

[0023] Step (7): The quantum computing method is combined with the self-attention structure to form a variable quantum eigenvalue solver self-attention structure; the variable quantum eigenvalue solver self-attention structure uses a control rotation gate operation to replace the dot product operation in the classical self-attention structure; the variable quantum eigenvalue solver self-attention structure uses a quantum amplitude amplification module to replace the operation in the classical self-attention structure. The attention coefficients output by the function module and the quantum amplitude amplification module are input into the variable quantum eigenvalue solver for optimization.

[0024] Step (8): Controlled rotation gate is a method used in quantum computing to control the interaction and rotation between bits; controlled rotation gate can be used to control single-bit rotation gate and can be extended to multi-bit rotation operations; controlled rotation gate can realize various types of controlled rotation gate by combining controlled NOT gate and single-bit rotation gate, thereby realizing complex quantum computing methods and logic circuits;

[0025] Step (9): In the quantum amplitude amplification module, a quantum state Decomposed into standard orthogonal ground state and Linear combination; amplitude amplification formula is:

[0026] (5)

[0027] (6)

[0028] in, The number of rotations in the quantum state. For amplitude amplification operator, For amplitude amplification operators Power of 1 The rotation angle in the quantum state. It is a sine function. It is a cosine function; the core of the quantum amplitude amplification module is to select the appropriate number of rotations. Make Closest to 1;

[0029] Step (10): In the variable quantum eigenvalue solver, the attention coefficients are the parameters to be optimized in the quantum circuit input to the variable quantum eigenvalue solver. The quantum circuit's pavement decomposes the Hamiltonian of the quantum system into a sum of measurable Pauli terms; the measurement module of the variable quantum eigenvalue solver measures the expected value of the decomposed Hamiltonian on the quantum circuit; the variable quantum eigenvalue solver adjusts the parameters to be optimized. To minimize the energy expectation; the variational quantum eigenfunction solver selects a trial wavefunction. and measured Energy Expectation :

[0030] (7)

[0031] (8)

[0032] in, As a set reference state, For the specified exponential activation operator, For the electron Hamiltonian; for The left-handed representation obtained by conjugate transpose;

[0033] Step (11): In the self-attention structure of the variational quantum eigenvalue solver, query the matrix. and key value matrix After being encoded into a quantum state by the quantum encoding module, the result is input into the control rotation gate; the output of the control rotation gate is input into the quantum amplitude amplification module; the masked quantum multi-head self-attention structure, after a masking operation, inputs the output of the control rotation gate into the quantum amplitude amplification module; the output of the quantum amplitude amplification module is the attention coefficient; the variable quantum eigenvalue solver receives the attention coefficient output from the quantum amplitude amplification module and optimizes the attention coefficient; the optimized attention coefficient is returned to the quantum amplitude amplification module; the attention coefficient is continuously passed between the quantum amplitude amplification module and the variable quantum eigenvalue solver until the final optimized result is obtained; the output of the optimized attention coefficient is decoded into a classical state by the quantum decoding module; the optimized attention coefficient and the value matrix are then processed. The final output of the variable quantum eigenvalue solver self-attention structure is obtained by performing a matrix dot product operation.

[0034] Step (12): Both the quantum multi-head self-attention structure and the masked quantum multi-head self-attention structure are obtained by superimposing multiple variable quantum eigenfunction solver self-attention structures; the quantum multi-head self-attention structure and the masked quantum multi-head self-attention structure integrate the output results of different variable quantum eigenfunction solver self-attention structures into an integration matrix; the output of the integration matrix after matrix transformation is the output of the quantum multi-head self-attention structure;

[0035] Step (13): The encoding block and decoding block based on the quantum multi-head self-attention structure are the quantum encoding block and the quantum decoding block, respectively; the quantum encoding block consists of a quantum multi-head self-attention structure, two residual connection and normalization layers, and a fully connected layer; the input matrix of the quantum encoding block is input into the quantum multi-head self-attention structure and the residual connection and normalization layer, respectively; the output of the quantum multi-head self-attention structure is the input of the residual connection and normalization layer; the output of the residual connection and normalization layer is the input of the next residual connection and normalization layer and the fully connected layer; the output of the fully connected layer is the input of the next residual connection and normalization layer; the output of the next residual connection and normalization layer is the output of the quantum encoding block;

[0036] The quantum decoding block consists of a quantum multi-head self-attention structure, a masked quantum multi-head self-attention structure, three residual connection and normalization layers, and a fully connected layer. The structure of the quantum decoding block is based on the quantum encoding block structure, with the addition of a masked quantum multi-head self-attention structure and residual connection and normalization layers. The input of the quantum multi-head self-attention structure of the quantum decoding block comes from the output of the quantum encoding block and the output of the residual connection and normalization layers.

[0037] Step (14): The Transformer neural network based on quantum encoding blocks and quantum decoding blocks is a quantum Transformer neural network; the quantum Transformer neural network consists of two parts: an encoder and a decoder; the encoder contains 6 quantum encoding blocks connected in sequence; the decoder contains 6 quantum decoding blocks connected in sequence; in the quantum Transformer neural network, environmental information is encoded into an input matrix and then input into the first quantum encoding block; except for the first quantum encoding block, the input of the other quantum encoding blocks is the output of the previous quantum encoding block; the output of the sixth quantum encoding block is the final output of the encoder, i.e., the encoded information matrix; the encoded information matrix is ​​the input of all 6 quantum decoding blocks; except for the first quantum decoding block, the input of the other quantum decoding blocks also includes the output of the previous quantum decoding block; the output of the sixth quantum decoding block is the final output of the decoder; the output of the decoder is the final output of the quantum Transformer neural network;

[0038] Step (15): The quantum Transformer neural network is combined with the decision-making process of reinforcement learning to form a multi-agent deep reinforcement learning method based on the quantum Transformer neural network; the mobile robot is a type of intelligent agent;

[0039] A multi-agent deep reinforcement learning method based on quantum Transformer neural networks was used to plan collision-free paths for mobile robots with assigned tasks. In a given state, a single mobile robot uses a quantum Transformer neural network to evaluate the Q-value of its actions in that state. The different Q-values ​​of different mobile robots in a given state are integrated into a hybrid network to obtain the joint Q-value of the multi-mobile robot system in that state. The hybrid network evaluates the impact of each mobile robot's action on the entire system and combines the Q-values. Expanded into the Q-value of a single mobile robot; joint Q-value Expanded to:

[0040] (9)

[0041] (10)

[0042] (11)

[0043] (12)

[0044] in, It is a constant that depends on the current state. For the first The first-order gain coefficient of each agent For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first and the Joint gain coefficient of each agent For the first To the Joint gain coefficient of each agent For k factorial, for The first-order differential operator, for The second-order differential operator, for of Differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator;

[0045] During the training process of the agent, at each time step, the probability of the agent choosing a random action is ε, and the probability of choosing the action with the maximum Q value is 1-ε.

[0046] Step (16): The trained multi-agent deep reinforcement learning method based on quantum Transformer neural network is used for path planning of multiple mobile robots to plan a collision-free path between the mobile robots.

[0047] The present invention has the following advantages and effects compared with the prior art:

[0048] (1) Common task allocation methods for multi-robot systems include linear programming and swarm intelligence methods. However, linear programming and swarm intelligence methods suffer from insufficient real-time performance in dynamic environments, leading to inaccurate task allocation schemes. This invention utilizes a bidirectional gated cyclic network to predict the positions of moving target points and moving obstacles. The A* method can calculate distance information using the predicted positions of moving target points and moving obstacles. The Hungarian method then calculates the task allocation scheme based on the distance information calculated by the A* method. Therefore, the Hungarian method can react in advance to the dynamic changes of the multi-robot system, improving the accuracy of the allocation scheme in dynamic environments.

[0049] (2) Conventional Transformer neural network methods take a lot of time to train with large datasets. This invention utilizes the superposition principle of quantum states in quantum computing to obtain the advantage of parallelism. By combining quantum computing methods with Transformer neural networks, the training time of Transformer neural networks can be effectively shortened.

[0050] (3) The attention coefficients output by the self-attention structure of the Transformer neural network have a significant impact on the accuracy of the prediction results. Traditional Transformer neural network methods do not optimize the attention coefficients. This invention combines a variable quantum eigenvalue solver with the self-attention structure of the Transformer neural network to optimize the attention coefficients, thereby improving the accuracy of the Transformer neural network predictions and the accuracy of value estimation for path planning actions.

[0051] (4) Existing multi-robot path planning methods often employ hierarchical planning. While hierarchical planning can mitigate collisions between robots to some extent, it fails to leverage the advantages of collaborative planning among multiple robots, and the planned paths are not sufficiently rational. The multi-agent deep reinforcement learning method based on the quantum Transformer neural network approach of this invention can utilize the advantages of collaborative planning among multiple robots to obtain the overall Q-value of the multi-robot system in a given state, thereby reducing the total length of the robot paths at the overall level.

[0052] (5) Existing multi-agent deep reinforcement learning methods lack a complete basis for decomposing the overall Q-value into individual Q-values. If the overall Q-value decomposition is inaccurate, collisions may occur between mobile robots. The multi-agent deep reinforcement learning method based on the quantum Transformer neural network method of this invention can reasonably decompose the overall Q-value into individual Q-values, thereby reducing the probability of collisions between mobile robots. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the execution process of the method of this invention.

[0054] Figure 2 This is a structural diagram of the bidirectional gated loop network of the method of the present invention.

[0055] Figure 3 This is a structural diagram of the bidirectional gated cyclic network Hungarian method of the present invention.

[0056] Figure 4 This is a structural diagram of the variable quantum eigenvalue solver of the method of the present invention.

[0057] Figure 5 This is a structural diagram of the self-attention structure of the variable quantum eigenvalue solver of the present invention.

[0058] Figure 6 This is a structural diagram of the quantum multi-head self-attention structure of the method of this invention.

[0059] Figure 7 This is a structural diagram of the quantum encoding block and quantum decoding block structure of the method of the present invention.

[0060] Figure 8 This is a diagram showing the overall structure of the quantum Transformer neural network of the method of this invention.

[0061] Figure 9 This is a structural diagram of the multi-agent deep reinforcement learning method based on the quantum Transformer neural network of the present invention. Detailed Implementation

[0062] The path planning method for bidirectional gated recurrent networks and quantum Transformer networks proposed in this invention is described in detail below with reference to the accompanying drawings:

[0063] Figure 1This is a flowchart illustrating the execution of the method of this invention. First, two bidirectional gated recurrent networks are trained for task allocation. The two trained bidirectional gated recurrent networks output the predicted positions of the moving target point and the predicted positions of the moving obstacles, respectively. The Hungarian method uses the predicted positions of the moving target point and the predicted positions of the moving obstacles to calculate the task allocation scheme. Then, quantum computing methods are combined with Transformer neural networks to form a quantum Transformer neural network. The quantum Transformer neural network is combined with reinforcement learning decision-making methods to form a multi-agent deep reinforcement learning method based on the quantum Transformer neural network. The multi-agent deep reinforcement learning method based on the quantum Transformer neural network is used to collaboratively solve the path planning problem of multiple mobile robots, that is, to plan a collision-free path for each mobile robot that has been assigned a task, while satisfying the condition of minimizing the sum of paths as much as possible. If the position of the moving target point changes, the Hungarian method recalculates the task allocation scheme. When all mobile robots reach the target point, the path planning process is completed.

[0064] Figure 2 This is a structural diagram of the bidirectional gated recurrent network of the method of this invention. For a given input sequence, the bidirectional gated recurrent network first passes the input sequence forward to one gated recurrent network, obtaining a forward state sequence. Then, the bidirectional gated recurrent network passes the input sequence backward to another gated recurrent network, obtaining a backward state sequence. The forward and backward state sequences are combined to form the final output. In the structural diagram, the input layer passes information to the forward hidden layer through W1 and to the backward hidden layer through W3. Then, the forward hidden layer passes information within itself through W2 and to the output layer through W4. The backward hidden layer passes information within itself through W5 and to the output layer through W6.

[0065] Figure 3 This is a structural diagram of the bidirectional gated loop network Hungarian method of the present invention. The bidirectional gated loop network Hungarian method employs two bidirectional gated loop networks. The two bidirectional gated loop networks respectively acquire the past positions of the moving target point and the moving obstacles in the multi-mobile robot system. The first bidirectional gated loop network is used to predict the position of the moving target point. The second bidirectional gated loop network is used to predict the position of the moving obstacles. The predicted positions of the moving target point and the moving obstacles are input into the A* method to obtain distance information. The Hungarian method then uses the distance information to obtain a task allocation scheme.

[0066] Figure 4This is a structural diagram of the variable quantum eigenvalue solver of the present invention. The variable quantum eigenvalue solver receives the parameters to be optimized from the classical solver. The eight plies in the quantum circuit decompose the Hamiltonian of the quantum system into measurable Pauli terms for measurement on the quantum circuit. Some plies exist on only one quantum circuit, while others span two. The measurement section measures the expected values ​​of the Hamiltonians on each of the decomposed quantum circuits. Then, the expected values ​​of the Hamiltonians from each part are combined to obtain the expected value of the entire Hamiltonian. The measured expected value of the entire Hamiltonian serves as the basis for parameter optimization. The parameters to be optimized are passed back and forth between the classical solver and the variable quantum eigenvalue solver until the final optimization of the parameters is completed.

[0067] Figure 5 This is a structural diagram of the self-attention structure of the variational quantum eigenvalue solver of the present invention. Query matrix. and key value matrix The input to the control rotation gate needs to be encoded into a quantum state by the quantum encoding module. For the variable quantum eigenvalue solver self-attention structure in a masked quantum multi-head self-attention structure, the output of the control rotation gate needs to undergo a masking operation before being output to the quantum amplitude amplification module. The output of the quantum amplitude amplification module is the attention coefficient. The attention coefficient is input to the variable quantum eigenvalue solver module to obtain the optimized attention coefficient. The attention coefficient result is continuously passed between the quantum amplitude amplification module and the variable quantum eigenvalue solver module until the final optimized result is obtained. The output of the optimized attention coefficient is decoded into a classical state by the quantum decoding module and compared with the value matrix. The final output is obtained after a series of matrix dot product operations.

[0068] Figure 6 This is a structural diagram of the quantum multi-head self-attention structure of the method of this invention. The quantum multi-head self-attention structure is a superposition of multiple variable quantum eigenfunction solver self-attention structures. Among them, the variable quantum eigenfunction solver self-attention structure of the masked quantum multi-head self-attention structure needs to perform a masking operation. The input of different variable quantum eigenfunction solver self-attention structures is a query matrix. Key-value matrix and value matrix The outputs of different variable quantum eigenvalue solvers with self-attention structures are integrated into a single output matrix. This output matrix is ​​then processed by appropriate matrix transformations to obtain the output of the quantum multi-head self-attention structure.

[0069] Figure 7This is a structural diagram of the quantum encoding block and quantum decoding block of the method of this invention. The quantum encoding block consists of a quantum multi-head self-attention structure, two residual connection and normalization layers, and a fully connected layer. The encoding block input matrix of the quantum encoding block is input into the quantum multi-head self-attention structure and the first residual connection and normalization layer. Simultaneously, the output of the quantum multi-head self-attention structure is also used as the input of the first residual connection and normalization layer. The output of the first residual connection and normalization layer is used as the input of the next residual connection and normalization layer and the fully connected layer. Simultaneously, the output of the fully connected layer is also used as the input of the next residual connection and normalization layer. The output of the next residual connection and normalization layer is the output of the quantum encoding block.

[0070] The quantum decoding block consists of two quantum multi-head self-attention structures, three residual connection and normalization layers, and one fully connected layer. One of the two quantum multi-head self-attention structures is a quantum multi-head self-attention structure that includes a masking operation. The connection relationships of the residual connection and normalization layer and the fully connected layer in the quantum decoding block are consistent with those in the quantum encoding block. The decoding block input matrix is ​​first input into the masked quantum multi-head self-attention structure and the first residual connection and normalization layer. The connection relationship between the masked quantum multi-head self-attention structure and the first residual connection and normalization layer is consistent with that in the quantum encoding block. The input to the quantum multi-head self-attention structure comes from the output of the quantum encoding block and the output of the first residual connection and normalization layer. The output of the first residual connection and normalization layer serves as the input to the second residual connection and normalization layer. The input to the second residual connection and normalization layer also includes the output of the quantum multi-head self-attention structure. The connection relationships of the fully connected layer and the third residual connection and normalization layer are consistent with those in the quantum encoding block.

[0071] Figure 8 This is a diagram illustrating the overall structure of the quantum Transformer neural network of the present invention. The quantum Transformer neural network consists of an encoder and a decoder, each containing six sequentially connected quantum encoding blocks and quantum decoding blocks. The input to the first quantum encoding block is the environment information matrix, and the inputs to the remaining quantum encoding blocks are the outputs of the previous quantum encoding block. The output of the sixth quantum encoding block is the final output of the encoder, i.e., the encoded information matrix. The encoded information matrix is ​​then sequentially input into the six quantum decoding blocks. The inputs to all six quantum decoding blocks, except for the first quantum decoding block, include the output of the previous quantum decoding block. The output of the last quantum decoding block is the final output of the decoder.

[0072] Figure 9This is a structural diagram of the multi-agent deep reinforcement learning method based on quantum Transformer neural networks, as described in this invention. In the interaction with the environment, a single agent calculates its individual Q-value using the quantum Transformer neural network method. However, this Q-value only represents the behavioral value of the agent's action for that single agent, not its behavioral value for the entire multi-agent system. The individual Q-values ​​of all agents at a given moment are input into a hybrid network. The output of the hybrid network is the overall Q-value representing the behavioral value of the entire multi-agent system. The overall Q-value is decomposed into the Q-values ​​of individual agents in their current state. The decomposed Q-values ​​are then fed back to the individual mobile robots. Through this process, cooperative path planning for multiple mobile robots is achieved.

[0073] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A path planning method for bidirectional gated recurrent networks and quantum Transformer networks, characterized in that, This paper combines bidirectional gated recurrent networks, the Hungarian method, quantum computing methods, Transformer neural networks, and reinforcement learning decision-making methods for path planning of multiple mobile robots. This improves the accuracy of task allocation schemes, shortens training time, provides highly accurate path planning actions, optimizes the paths of mobile robots, and reduces collisions between different mobile robots. The steps in its application are as follows: Step (1): In a multi-mobile robot system, there is at least one mobile robot and one task; the initial positions of the mobile robot and the task are random; the position of the task can change, and the task whose position can change is the target point for movement; the multi-mobile robot path planning method requires assigning tasks to the mobile robots before planning paths for them; the task assignment method requires assigning a mobile robot to each task to execute; without considering the conflict between paths of different mobile robots, the task assignment method reasonably assigns tasks to the mobile robots so that the total path length of the mobile robots is the shortest; the path planning method needs to plan the path from the position of the mobile robot to the position of the task for the mobile robot that has been assigned a task, and needs to fully consider the conflict between paths of different mobile robots. Step (2): The bidirectional gated recurrent network consists of an input layer, a hidden layer, and an output layer; the hidden layer consists of a forward hidden layer and a backward hidden layer; the forward hidden layer and the backward hidden layer are two gated recurrent networks in opposite directions; The input layer passes information to the forward hidden layer and the backward hidden layer respectively; The forward hidden layer passes information within itself and then passes the information to the output layer; The backward hidden layer passes information within itself and then passes the information to the output layer; Step (3): The bidirectional gated recurrent network in time Hidden layer state From the forward hidden layer state and the backward hidden layer state get: (1) (2) (3) in, For time The following forward hidden layer state, For time The following forward hidden layer state, For time The following backward hidden layer state, For time The following backward hidden layer state, As a way to input vector Encode into the corresponding nonlinear transformation function of the hidden layer state. for The function takes inputs other than the hidden state. These are the weight coefficients of the forward hidden layer. These are the weight coefficients of the backward hidden layer. For time Hidden layer state Corresponding to the forward hidden layer state and the backward hidden layer state The offset; Step (4): Two bidirectional gated recurrent networks are used to predict the positions of the moving target point and the moving obstacles; the first bidirectional gated recurrent network is trained using the moving target point position dataset; the moving target point position dataset includes the past positions of all moving target points; the trained first bidirectional gated recurrent network outputs the predicted position of the moving target point; the second bidirectional gated recurrent network is trained using the moving obstacle position dataset; the moving obstacle position dataset includes the past positions of all moving obstacles; the trained second bidirectional gated recurrent network outputs the predicted position of the moving obstacles. Step (5): A bidirectional gated recurrent network is combined with the Hungarian method to form a bidirectional gated recurrent network. The Hungarian method is used for task allocation in a multi-mobile robot system. The two bidirectional gated recurrent networks obtain the previous positions of the moving target points and the moving obstacles of the multi-mobile robot system, respectively. The two bidirectional gated recurrent networks output the predicted positions of the moving target points and the moving obstacles, respectively. The A* method calculates the distance information of the multi-mobile robot system based on the predicted positions of the moving target points and the moving obstacles. The distance information includes the distance information from the mobile robot to the moving target point and the distance information between the moving target points. The Hungarian method uses distance information to calculate a task allocation scheme; it creates a weight matrix based on the distance information; the weight matrix represents the weight between each pair of vertices in the bipartite graph; the Hungarian method subtracts the minimum value of the weight matrix from each row of the matrix, and then subtracts the minimum value of the weight matrix from each column; the Hungarian method draws as many zero elements as possible with as few horizontal and vertical lines as possible; if the sum of the number of horizontal and vertical lines equals the order of the weight matrix, the Hungarian method finds the optimal solution. If the sum of the number of horizontal and vertical lines is not equal to the order of the weight matrix, the Hungarian method needs to find the minimum value among the elements in the weight matrix that are not covered by lines, and update the weight matrix based on the minimum value. The Hungarian method's update operation adds the minimum value to the elements in the rows covered by horizontal lines, and subtracts the minimum value from the elements in the columns covered by vertical lines. The Hungarian method repeats the above process until the sum of the number of horizontal and vertical lines equals the order of the weight matrix. Step (6): The self-attention structure is applied to the encoder and decoder of the Transformer neural network; the multi-head self-attention structure is the superposition of multiple self-attention structures; the multi-head self-attention structure that performs the masking operation in the self-attention structure is a masked multi-head self-attention structure; the input of the self-attention structure of the Transformer neural network is the query matrix. Key-value matrix and value matrix Output for: (4) in, For query matrix The number of columns, This is the matrix transpose operation. As an activation function, it can convert a matrix The sum of each row becomes 1; The function outputs the attention coefficient; Step (7): The quantum computing method is combined with the self-attention structure to form a variable quantum eigenvalue solver self-attention structure; the variable quantum eigenvalue solver self-attention structure uses a control rotation gate operation to replace the dot product operation in the classical self-attention structure; the variable quantum eigenvalue solver self-attention structure uses a quantum amplitude amplification module to replace the operation in the classical self-attention structure. The attention coefficients output by the function module and the quantum amplitude amplification module are input into the variable quantum eigenvalue solver for optimization. Step (8): Controlled rotation gate is a method used in quantum computing to control the interaction and rotation between bits; controlled rotation gate can be used to control single-bit rotation gate and can be extended to multi-bit rotation operations; controlled rotation gate can realize various types of controlled rotation gate by combining controlled NOT gate and single-bit rotation gate, thereby realizing complex quantum computing methods and logic circuits; Step (9): In the quantum amplitude amplification module, a quantum state Decomposed into standard orthogonal ground state and Linear combination; amplitude amplification formula is: (5) (6) in, The number of rotations in the quantum state. For amplitude amplification operator, For amplitude amplification operators Power of 1 The rotation angle in the quantum state. It is a sine function. It is a cosine function; the core of the quantum amplitude amplification module is to select the appropriate number of rotations. Make Closest to 1; Step (10): In the variable quantum eigenvalue solver, the attention coefficients are the parameters to be optimized in the quantum circuit input to the variable quantum eigenvalue solver. The quantum circuit's pavement decomposes the Hamiltonian of the quantum system into a sum of measurable Pauli terms; the measurement module of the variable quantum eigenvalue solver measures the expected value of the decomposed Hamiltonian on the quantum circuit; the variable quantum eigenvalue solver adjusts the parameters to be optimized. To minimize the energy expectation; the variational quantum eigenfunction solver selects a trial wavefunction. and measured Energy Expectation : (7) (8) in, As a set reference state, For the specified exponential activation operator, For the electron Hamiltonian; for The left-handed representation obtained by conjugate transpose; Step (11): In the self-attention structure of the variational quantum eigenvalue solver, query the matrix. and key value matrix After being encoded into a quantum state by the quantum encoding module, the result is input into the control rotation gate; the output of the control rotation gate is input into the quantum amplitude amplification module; the masked quantum multi-head self-attention structure, after a masking operation, inputs the output of the control rotation gate into the quantum amplitude amplification module; the output of the quantum amplitude amplification module is the attention coefficient; the variable quantum eigenvalue solver receives the attention coefficient output from the quantum amplitude amplification module and optimizes the attention coefficient; the optimized attention coefficient is returned to the quantum amplitude amplification module; the attention coefficient is continuously passed between the quantum amplitude amplification module and the variable quantum eigenvalue solver until the final optimized result is obtained; the output of the optimized attention coefficient is decoded into a classical state by the quantum decoding module; the optimized attention coefficient and the value matrix are then processed. The final output of the variable quantum eigenvalue solver self-attention structure is obtained by performing a matrix dot product operation. Step (12): Both the quantum multi-head self-attention structure and the masked quantum multi-head self-attention structure are obtained by superimposing multiple variable quantum eigenfunction solver self-attention structures; the quantum multi-head self-attention structure and the masked quantum multi-head self-attention structure integrate the output results of different variable quantum eigenfunction solver self-attention structures into an integration matrix; the output of the integration matrix after matrix transformation is the output of the quantum multi-head self-attention structure; Step (13): The encoding block and decoding block based on the quantum multi-head self-attention structure are the quantum encoding block and the quantum decoding block, respectively; the quantum encoding block consists of a quantum multi-head self-attention structure, two residual connection and normalization layers, and a fully connected layer; the input matrix of the quantum encoding block is input into the quantum multi-head self-attention structure and the residual connection and normalization layer, respectively; the output of the quantum multi-head self-attention structure is the input of the residual connection and normalization layer; the output of the residual connection and normalization layer is the input of the next residual connection and normalization layer and the fully connected layer; the output of the fully connected layer is the input of the next residual connection and normalization layer; the output of the next residual connection and normalization layer is the output of the quantum encoding block; The quantum decoding block consists of a quantum multi-head self-attention structure, a masked quantum multi-head self-attention structure, three residual connection and normalization layers, and a fully connected layer. The structure of the quantum decoding block is based on the quantum encoding block structure, with the addition of a masked quantum multi-head self-attention structure and residual connection and normalization layers. The input of the quantum multi-head self-attention structure of the quantum decoding block comes from the output of the quantum encoding block and the output of the residual connection and normalization layers. Step (14): The Transformer neural network based on quantum encoding blocks and quantum decoding blocks is a quantum Transformer neural network; the quantum Transformer neural network consists of two parts: an encoder and a decoder; the encoder contains 6 quantum encoding blocks connected in sequence; the decoder contains 6 quantum decoding blocks connected in sequence; in the quantum Transformer neural network, environmental information is encoded into an input matrix and then input into the first quantum encoding block; except for the first quantum encoding block, the input of the other quantum encoding blocks is the output of the previous quantum encoding block; the output of the sixth quantum encoding block is the final output of the encoder, i.e., the encoded information matrix; the encoded information matrix is ​​the input of all 6 quantum decoding blocks; except for the first quantum decoding block, the input of the other quantum decoding blocks also includes the output of the previous quantum decoding block; the output of the sixth quantum decoding block is the final output of the decoder; the output of the decoder is the final output of the quantum Transformer neural network; Step (15): The quantum Transformer neural network is combined with the decision-making process of reinforcement learning to form a multi-agent deep reinforcement learning method based on the quantum Transformer neural network; the mobile robot is a type of intelligent agent; A multi-agent deep reinforcement learning method based on quantum Transformer neural networks was used to plan collision-free paths for mobile robots with assigned tasks. In a given state, a single mobile robot uses a quantum Transformer neural network to evaluate the Q-value of its actions in that state. The different Q-values ​​of different mobile robots in a given state are integrated into a hybrid network to obtain the joint Q-value of the multi-mobile robot system in that state. The hybrid network evaluates the impact of each mobile robot's action on the entire system and combines the Q-values. Expanded into the Q-value of a single mobile robot; joint Q-value Expanded to: (9) (10) (11) (12) in, It is a constant that depends on the current state. For the first The first-order gain coefficient of each agent For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first The Q-value of an agent, For the first and the Joint gain coefficient of each agent For the first To the Joint gain coefficient of each agent For k factorial, for The first-order differential operator, for The second-order differential operator, for of Differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator, for The first-order differential operator; During the training process of the agent, at each time step, the probability of the agent choosing a random action is ε, and the probability of choosing the action with the maximum Q value is 1-ε. Step (16): The trained multi-agent deep reinforcement learning method based on quantum Transformer neural network is used for path planning of multiple mobile robots to plan a collision-free path between the mobile robots.

Images