Intelligent robot autonomous navigation trajectory planning method combined with reinforcement learning

Abstract

This invention discloses a method for autonomous navigation trajectory planning of intelligent robots that combines reinforcement learning. This invention relates to the fields of artificial intelligence and robot control. The method constructs a three-layer coupled architecture: a macroscopic intention game layer, a microscopic physical adaptation layer, and an absolute safety hard constraint layer. The macroscopic layer extracts obstacle interaction intentions based on a dynamic social topology graph and a graph convolutional network, outputting topological sub-objectives. The microscopic layer obtains dynamic context vectors based on a temporal coding network, reconstructs policy network weights through a hypernetwork, and outputs candidate actions. The safety layer estimates safety boundaries online based on a neural network, constructs a safe action manifold, projects and corrects candidate actions, and outputs the executed action. This invention can prevent robot freezing while ensuring zero-collision safety hard constraints and achieve millisecond-level dynamic adaptation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and robot control, specifically to a method for planning autonomous navigation trajectories for intelligent robots that combines reinforcement learning. Background Technology

[0002] Reinforcement learning is a machine learning method in which an intelligent agent continuously interacts with the environment and optimizes its own strategy based on reward signals to maximize cumulative rewards. In the field of autonomous navigation of intelligent robots, the robot's own pose, speed and surrounding environment perception information are usually used as state inputs, and linear velocity and angular velocity commands are used as action outputs. A navigation strategy to guide the robot to the target point is obtained through training a policy network.

[0003] The closest existing technology to this invention is a robot navigation method that combines safety reinforcement learning with a safety filter based on a control obstacle function: the policy network is trained based on a constrained Markov decision process and outputs nominal control commands, and then a safety filter based on a fixed analytical expression is used to solve a quadratic programming problem, which modifies the nominal control commands into execution commands that satisfy safety constraints.

[0004] The existing technologies mentioned above have the following shortcomings: the fixed-form control obstacle function cannot adjust the safety boundary in real time according to the dense dynamic obstacle environment; the set of safety actions is prone to shrinking into an empty set when obstacles are densely interspersed, causing the robot to freeze in place; the policy network parameters remain fixed after deployment, making it difficult to achieve instantaneous adaptation when encountering physical and dynamic mutations not covered in the training phase; the existing solutions lack a reasoning mechanism for the interaction intentions of surrounding dynamic obstacles, which can easily lead to deadlock in intention game in human-robot mixed scenarios, affecting passage efficiency and trajectory smoothness. Therefore, how to avoid robot freezing under the premise of ensuring zero-collision safety hard constraints for robot movement, while taking into account dynamic adaptive capabilities and social interaction compliance, has become an urgent technical problem to be solved in the field of autonomous navigation of intelligent robots. Summary of the Invention

[0005] The technical problem this invention aims to solve is that in existing intelligent robot autonomous navigation schemes, the fixed-form control obstacle function relied upon by safety reinforcement learning cannot adjust the safety boundary in real time according to the dense dynamic obstacle environment. This causes the robot to easily freeze in place and frequently stop abruptly in dense human-robot mixed environments. Existing reinforcement learning navigation strategies keep the policy network parameters fixed after deployment, making it difficult to achieve instantaneous adaptation when encountering physical and dynamic mutations not covered in the training phase. This can easily lead to catastrophic failures during the virtual-real migration process. Existing solutions lack explicit reasoning mechanisms for interaction intentions with surrounding dynamic obstacles, which can easily lead to intention game deadlock in complex interaction scenarios such as narrow passages where humans and robots are mixed, affecting passage efficiency and trajectory smoothness.

[0006] To address the aforementioned technical problems, this invention provides a method for autonomous navigation trajectory planning of intelligent robots that incorporates reinforcement learning, comprising:

[0007] Obtain the robot's own pose and velocity information, as well as the position, velocity, and category information of N surrounding dynamic obstacles, and construct a dynamic social topology graph containing robot nodes and obstacle nodes;

[0008] The dynamic social topology graph is input into a pre-trained spatiotemporal graph convolutional network to update node features and extract latent variables of interaction intent corresponding to each node.

[0009] The latent variables of the interaction intentions of all nodes and the coordinates of the robot's navigation target point are concatenated and then input into the pre-trained macro-policy network for calculation to obtain the topology-level sub-target points.

[0010] Acquire the timing data output by the robot's inertial measurement unit and the timing current data of the drive motor, and construct the sensing sequence within a preset time window;

[0011] The sensing sequence is input into a time-series coding network to obtain a dynamic context vector representing the robot's current physical dynamic state;

[0012] The dynamic context vector is input into the hypernetwork module to generate the dynamic weight parameters of the reinforcement learning policy backbone network;

[0013] Configure the policy backbone network with the dynamic weight parameters, input the topology-level sub-target points and robot local state observations into the configured policy backbone network for forward calculation, and output the original candidate action instructions;

[0014] The system acquires the set of dynamic obstacles and the robot's local state perceived at the current moment, inputs them into a pre-trained neural network safety boundary estimation module, and obtains the safety boundary function value that changes with the real-time perception data of the environment.

[0015] For each obstacle in the set of dynamic obstacles, a safety constraint inequality is constructed, and the intersection of all safety constraint inequalities is taken to construct a safety motion manifold.

[0016] Using the original candidate action instruction as a reference point, a projection optimization problem is solved within the safe action manifold to obtain the execution action instruction corrected by safe projection.

[0017] The execution command is sent to the robot chassis actuator to drive the robot to move.

[0018] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows:

[0019] First, since the safety action manifold is constructed based on the safety boundary function that changes with real-time environmental perception data obtained through online learning of neural networks, rather than using a fixed analytical safety boundary, the safety action manifold can adaptively shrink or expand according to the real-time density of obstacles. In dense crowd scenarios, it can still retain narrow but non-empty passable gaps, fundamentally avoiding the problem of the robot getting stuck in place due to the shrinkage of the safety action set into an empty set caused by a fixed safety boundary. At the same time, by solving the projection optimization problem, it ensures that the robot's movement always meets the zero-collision safety hard constraint.

[0020] Second, since the extraction of the dynamic context vector and the generation of the weight parameters of the policy backbone network by the supernetwork only involve one forward propagation operation and do not involve online gradient updates, the weight reconstruction process can be completed in milliseconds. This allows the robot to achieve instantaneous adaptive adjustment of the policy without additional online trial and error training when it encounters physical dynamic mutations not covered in the training phase, thus avoiding catastrophic failures in the virtual-real migration process.

[0021] Third, because the macro-intention game layer compresses the interaction intentions of surrounding dynamic obstacles into low-dimensional topological sub-goals through dynamic social topology graph and spatiotemporal graph convolutional network, the underlying reinforcement learning policy network does not need to directly process the original high-dimensional state of all obstacles, but only needs to make local decisions based on the compressed low-dimensional sub-goals. Therefore, it effectively reduces the reinforcement learning state space, reduces the training difficulty of policy network in multi-agent interaction scenarios, alleviates the deadlock problem of intention game in human-machine mixed scenarios, and improves the passage efficiency and trajectory smoothness of robot navigation. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the three-layer coupled navigation architecture used in the intelligent robot autonomous navigation trajectory planning method of the present invention;

[0023] Figure 2 This is a flowchart illustrating the overall algorithm of the intelligent robot autonomous navigation trajectory planning method of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating the principle of dynamic social topology graph construction and spatiotemporal graph convolutional intention reasoning in the macroscopic intention game layer of this invention.

[0025] Figure 4 This is a block diagram illustrating the principle of adaptive weight reconstruction of the hypernetwork dynamics in the microphysical adaptive layer of this invention.

[0026] Figure 5 This is a schematic diagram of the adaptive safety boundary estimation and differentiable convex optimization action projection mechanism in the absolute safety hard constraint layer of this invention. Detailed Implementation

[0027] The following is in conjunction with the appendix Figure 1-5 The specific embodiments of the present invention will be further described below. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0028] The intelligent robot autonomous navigation trajectory planning method combined with reinforcement learning provided by this invention is based on the construction of a three-layer coupled hierarchical reinforcement learning navigation architecture, consisting of a macroscopic intention game layer, a microscopic physical adaptive layer, and an absolute safety hard constraint layer, as follows: Figure 1 As shown, the three layers are tightly coupled and work together through data interfaces to complete a complete closed-loop process from environmental perception to the final output of safety control commands. The overall algorithm execution flow is as follows: Figure 2 As shown.

[0029] I. Macro-level Intent Game Theory

[0030] The macro-intention game layer is used to address the problems of convergence difficulties in reinforcement learning and deadlock in narrow channels caused by the increasing number of obstacles in the state space of multi-agent games. Its internal processing principle is as follows: Figure 3 As shown.

[0031] In one specific embodiment of the present invention, the robot acquires its own pose and velocity information in real time through perception modules such as LiDAR and depth cameras, as well as the position, velocity, and category information of N surrounding dynamic obstacles detected and tracked, and constructs a dynamic social topology map accordingly. , where the set of nodes Including the robot's own nodes and obstacle nodes to edge set Each edge in the graph represents a potential interaction relationship between two corresponding nodes whose Euclidean distance is less than a preset interaction sensing radius. In this embodiment, the interaction sensing radius is set to 5 meters.

[0032] For each node in the graph Construct an initial feature vector, which is composed of the node's two-dimensional position component, two-dimensional velocity component, and category code component at the current moment. The category code is used to distinguish between pedestrians, vehicles, and other obstacle types.

[0033] The initial features of each node are input into the pre-trained spatiotemporal graph convolutional network, and the node features are updated layer by layer according to the following relationship:

[0034] ;

[0035] in, Represents a node In the Hidden feature vectors after layer graph convolution. The value is taken as the initial feature vector of this node. Represents nodes There is a set of neighboring nodes with connecting edges between them. Represents a node With nodes The normalized distance weighting coefficients are used to weight the contributions of different neighboring nodes. Indicates the first The learnable weight matrix of layer graph convolution. This represents a nonlinear activation function. The above calculation is repeated along the time dimension, enabling the network to simultaneously capture the spatial topological relationships and temporal evolution relationships between obstacles. In this embodiment, the total number of layers in the graph convolutional network... Set to 3 layers.

[0036] Node The hidden feature sequences output from each graph's convolutional layer are input into a recurrent neural network unit to extract the latent variables of the node's interaction intent:

[0037] ;

[0038] in, This represents a gated cyclic unit network. This represents the total number of layers in the graph convolutional network. Nodes were encoded Information regarding the potential game intentions of the robot and other nodes, such as whether they have a tendency to yield or to maintain their original speed and go straight.

[0039] Implicit variables of the interaction intent of all nodes And the coordinates of the navigation target point that the robot ultimately needs to reach. After concatenation, the data is input into the macro-policy network to calculate the topological sub-target points at the current time step:

[0040] ;

[0041] in, Represents a macro-strategy network. These are the learnable parameters of the network. Indicates in The topological sub-target points output at each step, which are used by the underlying reinforcement learning policy network as local navigation targets.

[0042] Therefore, the underlying reinforcement learning policy network does not need to directly process the original high-dimensional state of all N obstacles. It only needs to make local obstacle avoidance and trajectory tracking decisions based on the low-dimensional sub-objectives compressed by intention reasoning, thus achieving effective dimensionality reduction of the underlying reinforcement learning state space.

[0043] II. Microphysical Adaptive Layer

[0044] The micro-physical adaptive layer is used to solve the problem of catastrophic failure in virtual-real migration caused by abrupt changes in physical dynamics. Its internal processing principle is as follows: Figure 4 As shown.

[0045] The robot continuously collects triaxial acceleration and triaxial angular velocity data output from its inertial measurement unit, as well as real-time current data from each drive motor. This data is then buffered at a fixed sampling frequency to construct a sensor sequence within the most recent time window.

[0046] ;

[0047] in, Indicates the first The three-axis acceleration vector at each sampling time point, Indicates the first The three-axis angular velocity vector at each sampling time. Indicates the first The current vector of each drive motor at each sampling time, This represents the total number of sampling points within a preset time window. In this embodiment, the duration of this time window is 0.5 seconds, and the number of sampling points... The value is 50.

[0048] Sensing sequence Input the pre-trained temporal coding network to calculate the dynamic context vector:

[0049] ;

[0050] In this embodiment, the temporal coding network is implemented using a lightweight temporal Transformer encoder, which consists of a multi-head self-attention layer and a feedforward network layer. This vector implicitly represents the physical dynamic context information such as the friction characteristics of the ground where the robot is currently located and the distribution state of its own load center of gravity. Moreover, this representation can be automatically learned without the need for manual explicit labeling of specific physical parameter values.

[0051] dynamic context vector Input the hypernetwork module to calculate the dynamic weight parameters of the backbone network that generates the reinforcement learning policy:

[0052] ;

[0053] in, , These represent the weight matrix and bias vector of the first fully connected layer in the hypernetic network module, respectively. , These represent the weight matrix and bias vector of the second fully connected layer in the hypernetic network module, respectively. Represents a non-linear activation function. This indicates the calculated value of the strategy backbone network in... The dynamic weight parameter tensor used at all times.

[0054] Dynamic weight parameters Inject the policy backbone network with it as the weight, and use the sub-target points output from the macroscopic layer. and robot local state observation As input, forward computation yields the original candidate action instructions for the current moment:

[0055] ;

[0056] in, Indicates For the policy backbone network with current weight parameters, This represents the robot's local state observations, including local cost map information and its own velocity information. This represents the topology-level sub-target points output by the macroscopic layer. This indicates the original candidate action instructions output by the policy backbone network, which have not yet been corrected by security projection.

[0057] Since the extraction of the dynamic context vector and the generation of the supernetwork weight parameters involve only one forward propagation operation and do not involve any backpropagation gradient updates, the weight reconstruction process can be completed within a millisecond timescale, achieving an instantaneous adaptive response to physical dynamic changes without the need to collect a large amount of real trial and error data for online gradient updates in new scenarios.

[0058] III. Absolutely Safe Hard Constraint Layer

[0059] The absolute safety hard constraint layer is used to overcome the limitation that safety reinforcement learning can only satisfy probabilistic expectation safety, and at the same time solves the robot freezing problem caused by fixed safety boundaries. Its internal processing principle is as follows: Figure 5 As shown.

[0060] Obtain the set of all dynamic obstacles detected by the perception module at the current moment. and robot local state The value of the current safety boundary function is calculated by inputting it into a pre-trained neural network safety boundary estimation module (i.e., a Neural-CBF network). ,in The network's learnable parameters differ from those constructed using fixed analytical methods. Instead, this network estimates the safety boundary online based on real-time sensing data and can dynamically adjust according to the real-time distribution of obstacles in the environment.

[0061] Construct safety constraint inequalities that require the time derivative of the safety boundary function along the system dynamics to satisfy:

[0062] ;

[0063] in, Represents the safety boundary function The time derivative of the robot system's dynamic model is related to the current state. and the action to be requested Related, This represents a monotonically increasing K-like function used to adjust the tightness of safety constraints; a larger value indicates a looser constraint.

[0064] For each obstacle currently sensed, construct the aforementioned safety constraint inequalities, and take the intersection of all constraints to construct the safety action manifold at the current moment:

[0065] ;

[0066] in, This represents the robot's overall motion space. Indicates the current state The set of possible actions that simultaneously satisfy safety constraints for all obstacles.

[0067] The original candidate actions output by the micro-layer Using the reference point, solve the differentiable convex optimization problem with the shortest projection distance within the safety action manifold to obtain the execution action corrected by the safety projection:

[0068] ;

[0069] in, This indicates the safety action to be issued and executed after safety projection correction. The optimization problem is a standard quadratic programming problem. The differentiable convex optimization solution layer can be implemented using a differentiable quadratic programming solver based on the KKT conditional implicit function theorem. This solver solves the quadratic programming problem to obtain the optimal solution in the forward computation and derives the gradient of the optimal solution with respect to the input parameters according to the KKT conditions in the backward computation, thereby supporting the back propagation of the gradient along this layer and thus supporting end-to-end joint training of the policy network and the safety layer.

[0070] When the feasible region area is less than a preset threshold due to the dense distribution of obstacles in the safety manifold, slack variables are introduced to soften the constraints, and the above optimization problem is reconstructed as follows:

[0071] ;

[0072] ;

[0073] in, This indicates the introduction of non-negative slack variables, which allow safety constraints to be temporarily softened to a necessary extent in extreme scenarios, thereby ensuring that a feasible solution always exists for the optimization problem. This represents the relaxation penalty coefficient, used to balance the relationship between action safety margin and the solvability of the optimization problem. The larger the value, the stricter the penalty for constraint softening.

[0074] The final solution will yield the safe execution action. The command is sent to the robot chassis actuator, which drives the robot to move according to the action instruction. The new state after execution is fed back to the perception module, and the cycle processing of the next control cycle begins. Since the safety boundary is updated online with real-time perception data, the safety area can be dynamically contracted or expanded according to the actual passable gaps in dense crowds. Therefore, the safety action manifold can retain non-empty feasible solutions in most scenarios, avoiding the problem of the robot being forced to freeze due to the action manifold being empty because of the fixed safety boundary.

[0075] IV. Three-tier collaborative training mechanism

[0076] The spatiotemporal graph convolutional network, macroscopic policy network, temporal coding network, supernetwork module, policy backbone network, and neural network safety boundary estimation module in this invention are based on a physical simulation platform during the offline training phase. They generate diverse training data covering different friction coefficients, different load center of gravity distributions, and different crowd density scenarios through domain randomization technology. An end-to-end joint training method is adopted, and a reward function composed of multiple indicators such as navigation efficiency, safety constraint satisfaction, and social adaptability is used for joint optimization. Since the safety projection layer and the supernetwork weight reconstruction are differentiable calculation processes, the gradient can be backpropagated along the complete forward calculation path from the executed action to the original parameters of the macroscopic policy network, temporal coding network, and policy backbone network, realizing the collaborative and integrated training of multiple network modules. After training, the model can be directly deployed to a real physical robot. During the deployment phase, each network module only performs forward inference calculations and does not involve online parameter updates.

[0077] V. Application Examples

[0078] The following describes the implementation of this invention in a specific application scenario. Suppose an indoor delivery robot is performing a delivery task in a hospital corridor environment. There are multiple pedestrians in the corridor whose walking directions are uncertain. The robot first obtains its own position and speed information and that of all surrounding pedestrians through its perception module, constructs a dynamic social topology graph, and inputs it into a spatiotemporal graph convolutional network. This yields latent variables reflecting whether each pedestrian intends to yield, and then generates topological sub-target points that bypass densely populated pedestrian areas and point towards the delivery target point. Subsequently, the robot uses inertial measurement unit data from the most recent 0.5 seconds and electrical... The machine current data is used to generate policy network weights in real time through a time-series coding network and a super network, which are adapted to the current ground material and load state. The output is a raw candidate action that points to the sub-target point and meets the local obstacle avoidance requirements. Finally, the absolute safety hard constraint layer estimates the safety boundary online and constructs a safety action manifold based on the real-time position and speed of all pedestrians. The raw candidate actions are projected and corrected into execution actions that meet the zero-collision constraints and then sent to the robot chassis. This drives the robot to pass smoothly through the passable gaps between pedestrians while maintaining safety, avoiding the stationary phenomenon caused by traditional fixed safety boundary schemes.

[0079] VI. Alternative Implementation Methods

[0080] In the absolute safety hard constraint layer, in addition to the above-mentioned implementation method of differentiable convex optimization projection layer, the safety constraint inequality can also be used as the constraint condition of the model predictive control rolling optimization problem. Within the preset finite prediction time domain, the rolling solution tracks the control sequence that satisfies the original candidate action instructions and the safety constraints to obtain the execution action instructions. This alternative implementation method can also dynamically adjust the safety boundary according to the real-time environment and avoid the action manifold being empty.

[0081] In the microphysical adaptive layer, in addition to using a temporal Transformer encoder to extract the dynamic context vector, a lightweight long short-time memory network or a temporal convolutional network can also be used to encode the timing data of the inertial measurement unit and the motor current. This can also extract the context vector that implicitly represents the current physical dynamic state and input it into the subsequent super network module to complete the weight reconstruction. This alternative implementation can further reduce the model inference latency on edge computing platforms with limited computing resources.

[0082] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent substitutions or improvements made by those skilled in the art based on the technical solutions disclosed in the present invention without departing from the concept of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for autonomous navigation trajectory planning of intelligent robots combining reinforcement learning, characterized in that, include: The robot acquires its own pose and velocity information, as well as the position, velocity, and category information of N surrounding dynamic obstacles, and constructs a dynamic social topology graph. The dynamic social topology graph includes robot nodes representing the robot itself and obstacle nodes representing each dynamic obstacle. The dynamic social topology graph is input into a pre-trained spatiotemporal graph convolutional network for node feature updates. The latent interaction intent variables corresponding to the robot nodes and each obstacle node are extracted. The latent interaction intent variables of all nodes and the coordinates of the robot's navigation target point are concatenated and input into a pre-trained macro-policy network for calculation to obtain topology-level sub-target points. The system acquires time-series acceleration data, time-series angular velocity data, and time-series current data of the robot's drive motors from the robot's inertial measurement unit. It constructs a sensing sequence within a preset time window and inputs the sensing sequence into a time-series coding network to obtain a dynamic context vector. The dynamic context vector represents the robot's current physical dynamic state. The dynamic context vector is then input into a super network module to generate dynamic weight parameters for the reinforcement learning policy backbone network. The policy backbone network is configured with the dynamic weight parameters. The topological sub-target points and robot local state observations are input into the configured policy backbone network for forward computation, and the original candidate action commands are output. The system acquires the set of dynamic obstacles and the robot's local state perceived at the current moment. It inputs the set of dynamic obstacles and the robot's local state into a pre-trained neural network safety boundary estimation module to obtain the safety boundary function value. The safety boundary function value changes with the real-time environmental perception data. Based on the safety boundary function value, a safety constraint inequality is constructed for each obstacle in the set of dynamic obstacles. The intersection of all the safety constraint inequalities is taken to construct a safe action manifold. Using the original candidate action command as a reference point, the system solves the projection optimization problem within the safe action manifold to obtain the execution action command corrected by the safe projection. The execution action command is sent to the robot chassis actuator to drive the robot to move according to the execution action command.

2. The method according to claim 1, characterized in that, The spatiotemporal graph convolutional network updates node features according to the following relationship: ； in, Represents a node In the Hidden feature vectors after layer graph convolution. For nodes The initial feature vector, Represents nodes There exists a set of neighboring nodes connected by an edge. Represents a node With nodes Normalized distance weighting coefficients between them Indicates the first The learnable weight matrix of layer graph convolution. Represents a non-linear activation function; Node The hidden feature sequences output from each graph's convolutional layer are input into a recurrent neural network unit to obtain nodes. The implicit variables of the interaction intent.

3. The method according to claim 1, characterized in that, The dynamic weight parameters of the policy backbone network generated by the hypernetwork module satisfy the following relationship: ； in, Represents the dynamic context vector, , These represent the weight matrix and bias vector of the first fully connected layer in the hypernetic network module, respectively. , These represent the weight matrix and bias vector of the second fully connected layer in the hypernetic network module, respectively. Represents a non-linear activation function. This represents the generated dynamic weight parameters.

4. The method according to claim 1, characterized in that, The security constraint inequalities satisfy the following relationship: ； in, This indicates the value of the safety boundary function. This represents the learnable parameters of the neural network safety boundary estimation module. This represents the time derivative of the safety boundary function along the robot system dynamics model. Indicates the local state of the robot. Indicates the action to be requested. This represents a monotonically increasing K-like function used to adjust the tightness of safety constraints.

5. The method according to claim 4, characterized in that, The projection optimization problem is a quadratic programming problem in which the feasible action with the shortest Euclidean distance to the original candidate action instruction is determined within the safe action manifold as the execution action instruction. The quadratic programming problem is solved through a differentiable convex optimization solution layer, and the solution process supports gradient backpropagation.

6. The method according to claim 5, characterized in that, When the feasible region area of ​​the safety action manifold is less than a preset threshold, non-negative relaxation variables are introduced into the quadratic programming problem to soften the safety constraint inequality. The objective function of the quadratic programming problem also includes the product of the squared term of the non-negative relaxation variable and the relaxation penalty coefficient to ensure that the projection optimization problem has a feasible solution.

7. The method according to claim 4, characterized in that, The projection optimization problem includes: using the safety constraint inequality as the constraint condition for the model predictive control rolling optimization problem, solving the control sequence that tracks the trajectory error of the original candidate action command and satisfies the preset convergence condition within a preset finite prediction time domain, and obtaining the execution action command.

8. The method according to claim 1, characterized in that, The spatiotemporal graph convolutional network, the supernetwork module, the policy backbone network, and the neural network safety boundary estimation module are jointly trained on a physical simulation platform during the offline training phase. During the training process, the gradients are propagated backward from the execution action command to the parameters of the spatiotemporal graph convolutional network, the temporal coding network, and the policy backbone network along the complete forward computation path. When deployed to the robot after training, each network only performs forward inference computation.

9. An intelligent robot autonomous navigation trajectory planning system combining reinforcement learning, characterized in that, include: The macro-intention game module is used to acquire the robot's own pose and velocity information, as well as the position, velocity, and category information of N surrounding dynamic obstacles, to construct a dynamic social topology graph. It extracts latent variables of interaction intentions through a spatiotemporal graph convolutional network, and calculates the latent variables of interaction intentions and the coordinates of the robot's navigation target point through a pre-trained macro-policy network to obtain topology-level sub-target points. The microphysics adaptive module is used to acquire the timing data output by the robot's inertial measurement unit and the timing current data of the drive motor. It obtains the dynamic context vector through the timing coding network, generates the dynamic weight parameters of the policy backbone network through the super network module, and outputs the original candidate action commands. The absolute safety hard constraint module is used to acquire the set of dynamic obstacles perceived at the current moment and the local state of the robot. The safety boundary function value is obtained through the neural network safety boundary estimation module, a safety action manifold is constructed, and the original candidate action instructions are projected and corrected to obtain the action instructions to be executed. The topological sub-target points output by the macroscopic intention game module are used as inputs to the microscopic physical adaptive module, and the original candidate action instructions output by the microscopic physical adaptive module are used as inputs to the absolute safety hard constraint module.

10. The system according to claim 9, characterized in that, It also includes a robot chassis actuator, which is connected to the absolute safety hard constraint module and is used to receive the execution action command and drive the robot to move; The system is also connected to an inertial measurement unit and a drive motor, which are used to provide the timing data and the timing current data to the microphysical adaptive module, respectively.

Images