A four-legged robot path planning method and system based on brain-like decision-making and a medium

Abstract

The present application relates to a kind of four-legged robot path planning method, system and medium based on brain-like decision-making, constructs environment scene, obtains the multi-dimensional observation state vector for four-legged robot and environment;Improved hybrid pulse reinforcement learning neural network is constructed, and multi-dimensional observation state vector is hybrid coded, is converted into time sequence pulse sequence after it, is handled by Actor network, outputs action, multi-dimensional observation state vector is input into Critic network, obtains value function estimate value;Actor network and Critic network are trained;Deploy the Actor network after training to four-legged robot, generates control instruction according to real-time multi-dimensional observation state vector, drives four-legged robot to execute task, until reaches target point;System includes four-legged robot platform and control terminal, realizes control terminal and medium.The present application improves space-time information representation capability, effectively realizes the best balance of four-legged robot computing resource and endurance.

Description

Technical Field

[0001] This invention relates to the technical field of computer systems based on specific computational models, and in particular to a method, system, and medium for quadruped robot path planning based on brain-like decision-making in the fields of brain-like computing, artificial intelligence, and deep reinforcement learning. Background Technology

[0002] With the rapid development of robotics technology, quadruped robots, as a typical type of legged robot, are widely used in complex scenarios such as disaster relief, field surveying, and industrial inspection due to their excellent mobility and terrain adaptability. Existing technologies such as A * Algorithms such as Dijkstra's algorithm or RRT not only rely on the accuracy of the environment model, but their computational complexity also increases exponentially with the complexity of the environment, making it difficult to guarantee performance in dynamic, unknown, or highly disturbed complex environments. With the rapid development of artificial intelligence in recent years, deep reinforcement learning (DRL) algorithms based on artificial neural networks have been widely used in path planning and have achieved significant results, but they also face many challenges, such as high energy consumption, high latency, and poor robustness. To address these issues, brain-like decision-making methods based on spiking neural networks (SNNs), which possess advantages such as event-driven, low-power consumption, and time-series awareness, have been used in this field. However, existing SNN methods have slight shortcomings in decision-making accuracy, which also poses a significant challenge to the algorithm training process. Therefore, to balance the constraints of decision-making accuracy and energy consumption, research often employs a hybrid training framework combining SNNs and traditional neural networks, applying it to tasks such as robot path planning.

[0003] Chinese patent CN120742904A discloses a path planning-based obstacle avoidance control method and system for quadruped robots. It constructs a semantic grid map by fusing environmental point clouds, image semantic segmentation results, and robot motion information, and then performs global path planning based on this map. As the robot travels along the path, it generates a local reference path based on semantics and occupancy status, and executes multi-level obstacle avoidance strategies to obtain foot trajectories, achieving autonomous obstacle avoidance in complex terrain. However, this method requires continuous calculation of the entire map information, resulting in significant energy consumption for the robot's decision-making process.

[0004] Chinese patent CN111811532A proposes a path planning method and apparatus based on spiking neural networks. The method divides the map into target, normal, and obstacle regions and models them as a neural network. It uses the self-charging current of the target neurons to trigger the propagation of pulse waves and updates the weights in real time by combining the plasticity mechanism of synapses. Finally, it backtracks to generate a path from the current position to the target region based on the magnitude of the synaptic weights. However, considering that the plasticity mechanism of synapses is difficult to apply in deep networks, it is difficult to use in scenarios with complex external environments.

[0005] Chinese patent CN120439279A proposes an indoor autonomous navigation method for a robotic dog based on a spiking neural network. It utilizes a fisheye top-down view image to drive a SNN model for path planning and combines RGB-D data to identify six types of obstacles for obstacle avoidance. This method leverages the sparse computational characteristics of spiking neural networks to effectively solve the problems of high computational load and high energy consumption in traditional visual navigation, significantly improving the robot's endurance. However, relying on the SNN network framework, it cannot achieve high control accuracy. Summary of the Invention

[0006] This invention addresses the problems of poor environmental adaptability, high energy consumption and large motion jitter in deep reinforcement learning, and insufficient decision-making accuracy of traditional spiking neural networks in the path planning of quadruped robots in the prior art. It provides a method, system and medium for quadruped robot path planning based on brain-like decision-making.

[0007] The technical solution adopted in this invention is a path planning method for quadruped robots based on brain-like decision-making. The method constructs an environmental scene and obtains a multi-dimensional observation state vector for the quadruped robot and the environment.

[0008] An improved hybrid pulse reinforcement learning neural network is constructed, which performs hybrid encoding on the multidimensional observation state vector, converts it into a time-series pulse sequence, processes the time-series pulse sequence through an Actor network, outputs an action, and inputs the multidimensional observation state vector into a Critic network to obtain the corresponding value function estimate.

[0009] Train the Actor network and Critic network;

[0010] The trained Actor network is deployed to the quadruped robot, and control commands are generated based on the real-time multidimensional observation state vector to drive the quadruped robot to perform tasks until the quadruped robot reaches the target point.

[0011] Preferably, the multidimensional observation state vector Including the robot's own status and environmental information, ,in, For the normalized position of the robot itself, This represents the unit vector pointing from the quadruped robot to the target. The target distance is normalized to the target point. This is the virtual lidar sensing vector.

[0012] Preferably, the improved hybrid pulse reinforcement learning neural network includes an Actor network and a Critic network configured in combination;

[0013] The Actor network consists of a sequentially connected input encoding layer, at least one second-order dynamic hidden layer, and an output decoding layer.

[0014] Preferably, the multidimensional observation state vector is encoded by an input coding layer, including population coding and deterministic coding performed sequentially;

[0015] Population coding utilizes a set of neurons with learnable Gaussian receptive fields to cover each dimension of the state vector. For the i-th state dimension, the calculation of the simulated stimulus intensity of the j-th coding neuron is related to the multidimensional observed state vector, the mean and standard deviation of the Gaussian receptive field.

[0016] The deterministic encoding uses the simulated stimulus intensity as the presynaptic input, updates the pseudomembrane voltage through a deterministic accumulation process, and generates a deterministic pulse sequence when the pseudomembrane voltage exceeds a threshold, while performing a soft reset operation.

[0017] Preferably, the second-order dynamic hidden layer is composed of fully connected second-order dynamic neurons, and its dynamic behavior is described by coupled membrane potential variables and recovery variables through discretized difference equations;

[0018] When the membrane potential exceeds the threshold and a pulse is emitted, a dynamic reset mechanism is executed.

[0019] Preferably, the Actor network and Critic network are trained using a policy optimization method based on the MDC-TD3 algorithm;

[0020] Construct two identical but parameter-independent deep Critic networks; sample data from the experience replay buffer and add truncated noise when calculating the target action; evaluate the target action using the two target Critic networks and calculate the target Q value by taking the minimum of the two values; adopt a delayed update strategy to make the update frequency of the pulse Actor network lower than that of the deep Critic network.

[0021] During backpropagation, a rectangular window function is used as a substitute derivative of the pulse firing function to achieve backpropagation of the error gradient along time and end-to-end update.

[0022] Preferably, a composite reward function is used to evaluate the behavior of the quadruped robot. The composite reward function includes a goal achievement reward, a collision or boundary crossing penalty, an asymmetric distance guidance reward based on distance improvement, an obstacle avoidance safety reward based on the nearest obstacle distance, and a discrete grid exploration reward.

[0023] Preferably, the control quantity corresponding to the motion space of the quadruped robot is a relative speed command.

[0024] A brain-like decision-making-based path planning system for quadruped robots includes:

[0025] The quadruped robot platform is equipped with a virtual LiDAR perception module, a positioning module, and an underlying motion controller.

[0026] The control terminal is equipped with a model trained by the brain-like decision-making-based quadruped robot path planning method. It is used to receive environmental state information, calculate path planning strategies in real time, generate speed control commands, and send them to the underlying motion controller to drive the quadruped robot to achieve autonomous navigation and obstacle avoidance.

[0027] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the aforementioned brain-like decision-based quadruped robot path planning method.

[0028] This invention relates to a method, system, and medium for path planning of quadruped robots based on brain-like decision-making. The method involves constructing an environmental scene and obtaining multi-dimensional observation state vectors for the quadruped robot and its environment. An improved hybrid pulse reinforcement learning neural network is constructed to hybrid-encode the multi-dimensional observation state vectors, converting them into a time-series pulse sequence. This time-series pulse sequence is then processed by an Actor network to output actions. The multi-dimensional observation state vectors are then input into a Critic network to obtain corresponding value function estimates. The Actor and Critic networks are trained. The trained Actor network is deployed to the quadruped robot, generating control commands based on the real-time multi-dimensional observation state vectors to drive the quadruped robot to perform tasks until it reaches the target point. The system includes a quadruped robot platform and a control terminal, with the control terminal and medium implemented based on the method.

[0029] The beneficial effects of this invention are as follows:

[0030] (1) A dual encoding mechanism is used to encode the state information in the environment into pulse signals, so as to achieve a more refined and robust representation of the environmental information and complete the efficient conversion from environmental state to neural pulse;

[0031] (2) In the spiking neural network, second-order dynamic neurons are used to enhance the nonlinear dynamic characteristics, capture more complex time patterns in the input signal, improve the spatiotemporal information representation ability, and realize high-dimensional nonlinear mapping of the state space.

[0032] (3) The Actor network with sparse firing characteristics of spiking neurons is used as the motion strategy generator. The event-driven mechanism is effectively utilized to significantly reduce redundant floating-point operations in the network and reduce system energy consumption. The network is then deployed on the quadruped robot to effectively achieve the best balance between computing resources and endurance of the quadruped robot. Attached Figure Description

[0033] Figure 1This is a flowchart of the method of the present invention;

[0034] Figure 2 This is a two-dimensional scene diagram constructed according to the present invention;

[0035] Figure 3 The Actor network diagram of the MDC-TD3 algorithm based on hybrid coding of the present invention is shown below;

[0036] Figure 4 This is a schematic diagram of the system structure of the present invention;

[0037] Figure 5 This is a comparison diagram of the simulation reward effect of the present invention;

[0038] Figure 6 This is a simulation trajectory diagram of the present invention;

[0039] Figure 7 This is an experimental result diagram of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0041] This invention relates to a path planning method for quadruped robots based on brain-like decision-making. The method improves the path planning ability of quadruped robots in complex environments. It uses a spiking neural network to map states to actions, which greatly reduces energy consumption during training and improves the endurance of quadruped robots. In the hybrid encoding mechanism, population encoding maps the continuous state space to the receptive field of high-dimensional neurons, improving the system's spatial resolution and nonlinear representation ability of environmental features. Then, it combines deterministic encoding to convert stimulus intensity into a pulse sequence with deterministic temporal characteristics. While preserving environmental dynamic information, it eliminates training noise introduced by traditional random encoding, introduces randomness, and significantly improves the convergence speed and decision accuracy of the path planning strategy.

[0042] like Figure 1 As shown, the method includes the following steps:

[0043] (1) Construct an environmental scene and obtain a multi-dimensional observation state vector for the quadruped robot and the environment;

[0044] (2) Construct an improved hybrid pulse reinforcement learning neural network, which performs hybrid encoding on the multidimensional observation state vector, converts it into a time-series pulse sequence, processes the time-series pulse sequence through an Actor network, outputs an action, and inputs the multidimensional observation state vector into a Critic network to obtain the corresponding value function estimate.

[0045] (3) Train the Actor network and Critic network;

[0046] (4) Deploy the trained Actor network to the quadruped robot, generate control commands based on the real-time multidimensional observation state vector, drive the quadruped robot to perform tasks until the quadruped robot reaches the target point.

[0047] The method will be described in detail below with reference to the embodiments.

[0048] (1) Construct an environmental scene and obtain a multi-dimensional observation state vector for the quadruped robot and the environment;

[0049] In this embodiment, as Figure 2 As shown, a two-dimensional simulation environment with multiple obstacles in a continuous state space is constructed. It is defined as a two-dimensional bounded planar region with a preset size, and the geometric dimension vector is set to That is, the dimension along the x-axis. The dimension along the y-axis is 32 units. The area is divided into 20 units, each representing 1 meter; a challenging "S"-shaped obstacle course is pre-set within this area; the quadruped robot moves in time steps... The position is Starting point position The target location is set to To simulate collision risks in a real physical environment and prevent the robot from passing through narrow gaps, a radius limit for the quadruped robot was also set within the environment. Safety margin and gap penetration allowance This is used for subsequent physical collision detection calculations and defines the effective collision determination threshold. , .

[0050] The multidimensional observation state vector Including the robot's own status and environmental information, ,in,

[0051] For the normalized position of the robot itself, and , For world size vectors, This represents element-wise division;

[0052] This represents the unit vector pointing from the quadruped robot to the target. In this embodiment ;

[0053] The target distance is normalized to the target point. , Let be the diagonal length from the initial position to the target point, and its Euclidean distance be . ;

[0054] The system simulates the virtual lidar sensing vector. Linear virtual lidar, in this embodiment , No. Angle of the rays , its reading This represents the normalized distance along the ray direction to the nearest obstacle boundary or environmental boundary.

[0055]

[0056] in, A collection of obstacles. For any obstacle, dist() is the distance function. In this embodiment, the maximum detection radius of the radar is... It covers approximately 78% of the map's diagonal length, ensuring the quadruped robot can perceive the layout of distant obstacles; to prevent missing thin-walled obstacles, the ray casting algorithm employs a high-density sampling strategy with a sampling interval of... Set to 0.05.

[0057] Furthermore, the quadruped robot's action space The corresponding control quantity is the relative speed command. , and They are respectively direction and The velocity in the direction, allowing for continuously variable transmission, and its actual displacement update follows the following dynamic equations:

[0058]

[0059] Where clip is the clipping function, and the maximum motion speed is... meters per step, position at the next moment From current location The integral is obtained, but it needs to pass through the collision detection function before updating:

[0060]

[0061] in, The sampling time interval;

[0062] When a collision risk is detected, the quadruped robot keeps its coordinates unchanged and resets its velocity vector to simulate the hindrance effect of an inelastic collision.

[0063] (2) Construct an improved hybrid pulse reinforcement learning neural network, which performs hybrid encoding on the multidimensional observation state vector, converts it into a time-series pulse sequence, processes the time-series pulse sequence through an Actor network, outputs an action, and inputs the multidimensional observation state vector into a Critic network to obtain the corresponding value function estimate.

[0064] In this invention, a hybrid impulsive reinforcement learning framework (MDC-TD3) combining a multi-scale dynamic coding improved impulsive policy network with a deep deterministic policy gradient is proposed for continuous control in quadrupedal robot path planning tasks.

[0065] The improved hybrid pulse reinforcement learning neural network includes an Actor network and a Critic network configured in combination;

[0066] like Figure 3 As shown, the Actor network comprises a sequentially connected input encoding layer, at least one second-order dynamic hidden layer, and an output decoding layer. During the process, a hybrid encoding method combining population coding and deterministic coding is used to extract environmental state information. Second-order dynamic neurons are used to transmit impulses, and finally, the average firing rate is used as the output, which is further decoded into action decisions. The specific hierarchical structure of the policy network is as follows. It contains a four-layer structure:

[0067] (2-1) Input coding layer, receive state information from environmental information, and output the time-series pulse sequence after population coding;

[0068] (2-2) Hidden layer, containing a fully connected layer of 256 second-order dynamic neurons, receives pulse signals from the coding layer and performs spatiotemporal feature extraction;

[0069] (2-3) Hidden layer, also composed of a fully connected layer containing 256 second-order dynamic neurons, (2-2) output pulse of hidden layer, further abstracting higher-order features;

[0070] (2-4) Output the decoding layer and calculate the average firing rate of the second hidden layer within the simulation time window.

[0071] The following is a detailed explanation of Actor networks.

[0072] (2-1) Input coding layer

[0073] The multidimensional observation state vector is encoded using an input encoding layer, including sequential population encoding and deterministic encoding. In this invention, to address the problem of training instability caused by random noise introduced by traditional Poisson encoding, a deterministic population encoding mechanism based on Gaussian receptive fields is adopted, where the environmental state vector is... For each dimension in the state vector ( The system allocates a group of sizes. The population of encoding neurons in this embodiment , ;

[0074] Population coding utilizes a set of neurons with learnable Gaussian receptive fields to cover each dimension of the state vector. For the i-th state dimension, the calculation of the simulated stimulus intensity of the j-th coding neuron is related to the multidimensional observed state vector, the mean and standard deviation of the Gaussian receptive field.

[0075] For the The state dimension, the first Simulated stimulus intensity of each encoding neuron satisfy,

[0076]

[0077] in, and These represent the mean and standard deviation of the Gaussian receptive field, respectively. For environmental input.

[0078] Deterministic encoding uses the simulated stimulus intensity as the presynaptic input, updates the pseudomembrane voltage through a deterministic accumulation process, and generates a deterministic pulse sequence when the pseudomembrane voltage exceeds a threshold, while simultaneously performing a soft reset operation; specifically, the environmental state vector is sequentially expanded as follows: 3D feature matrix The eigenvectors are flattened into vectors, and then deterministic pulse generation is used to transform the system into eigenvectors. As a presynaptic input, a pulse sequence is generated through a deterministic accumulation process, defining the pseudomembrane voltage. At the simulation time step The Each neuron is updated to,

[0079]

[0080] Determine whether a deterministic pulse sequence has been generated.

[0081]

[0082] That is, when the pseudo-film voltage At that time, a deterministic pulse is generated. ;

[0083] When a pulse is emitted ( After that, the pseudo-membrane voltage is soft reset: The generated deterministic pulse sequence As a direct input to subsequent networks.

[0084] (2-2), (2-3) Hidden Layers

[0085] The second-order dynamic hidden layer consists of fully connected second-order dynamic neurons, used for time-step feature extraction of pulse sequences. Its dynamic behavior is described by coupled membrane potential variables and recovery variables through discretized difference equations.

[0086] An SNN network is used to extract feature information, so that at time step , No. The first layer Each neuron receives pulse input from the previous layer. Calculate synaptic current :

[0087]

[0088] in, This is the synaptic weight matrix. For bias vectors, For the first The number of neurons in a layer.

[0089] The second-order dynamic neuron introduces a recovery variable. With membrane potential The coupled dynamic equations are updated using second-order dynamic discretization difference:

[0090]

[0091]

[0092]

[0093]

[0094] in, , , These represent the network layer index, the intra-layer neuron index, and the current time step, respectively. , They represent the first Layer One neuron in Membrane potential and recovery variable at time t; This indicates that the neuron is in The synaptic input current received at all times; These are learnable kinetic parameters, controlling the coupling strength between membrane potential and recovery variable, respectively.

[0095] Its second pulse delivery and dynamic reset mechanism are as follows:

[0096]

[0097] The pulse firing threshold is set to 1 in this embodiment; if a pulse firing occurs... Then the second-order dynamic neuron performs a dynamic reset:

[0098]

[0099] in, To reset the potential, This is the jump increment of the recovery variable after the pulse.

[0100] (2-4) Output Decoding Layer

[0101] Finally, block decoding and action mapping are used to statistically analyze the output of hidden layer (2-3). Average pulse firing rate vector within time step ,

[0102]

[0103] And through the output layer weights and bias Calculate the original action ,

[0104]

[0105] Its value range is limited by the Tanh function. In order to adapt to the asymmetric constraints of the actual implementing mechanism In this embodiment, , The system Perform a linear transformation to obtain the final action. :

[0106]

[0107]

[0108] (3) Train the Actor network and Critic network;

[0109] (3-1) Constructing a composite reward function

[0110] A composite reward function is used to evaluate the behavior of the quadruped robot, which includes a reward for achieving the goal. Collision or boundary crossing penalties Asymmetric distance-guided rewards based on distance improvement Obstacle avoidance safety reward based on the nearest obstacle distance and Discrete Grid Exploration Rewards .

[0111] To address the sparse reward and local minima problem in complex obstacle environments, this embodiment designs a composite reward function consisting of five parts:

[0112]

[0113] Goal Achievement Rewards In the middle, when the quadruped robot is at a distance from the target When the unit is reached, the task is considered successful and a high reward of +100.0 is given;

[0114] This indicates a penalty for collisions or going out of bounds, calculated as -2.0 from the reward each time.

[0115] Asymmetric distance-guided reward:

[0116]

[0117] in, For distance improvement, and In this embodiment, the forward and backward coefficients are represented. , To simulate the quadruped robot repeatedly scoring points, the backward coefficient was set to 1.2 times the forward coefficient.

[0118] Represents obstacle avoidance safety rewards, based on the distance to the nearest obstacle. Within the safety threshold range Provide internal incentives:

[0119]

[0120] in, In this embodiment, the linear interpolation reward with the highest weight is applied. , , ;

[0121] To reward exploration in discrete grids, the environment is discretized into grids, and a +0.3 exploration incentive is given for the first grid visited.

[0122] The design condition for terminating a single round is as follows: the round ends when the distance between the quadruped robot and the target point is less than 0.8 units or the training time exceeds 400 steps.

[0123] (3-2) Training an improved hybrid spiking reinforcement learning neural network

[0124] The Actor network and Critic network were trained using a policy optimization method based on the MDC-TD3 algorithm.

[0125] Construct two deep Critic networks with identical structures and independent parameters. and ;

[0126] Its network architecture consists of three fully connected layers, with the input being a state vector. With action vectors The layers are assembled as follows: the first layer is a fully connected layer with 256 neurons and the ReLU activation function; the second layer is also a fully connected layer with 256 neurons and the ReLU activation function; the third layer is the output layer with no activation function, directly outputting the data. Value scalar;

[0127] Data is sampled from the experience replay buffer, and truncated noise is added when calculating the target action; the target action is evaluated using two target Critic networks, and the target Q value is calculated by taking the minimum of the two; a delayed update strategy is adopted so that the update frequency of the pulse Actor network is lower than that of the deep Critic network.

[0128] During backpropagation, a rectangular window function is used as a substitute derivative of the pulse firing function to achieve backpropagation of the error gradient along time and end-to-end update.

[0129] Specifically, in the hybrid training process based on the TD3 algorithm, the system maintains six neural networks: a policy network. Target-Policy Network Two value networks Two target value networks The training cycle is as follows:

[0130] First, sample from the experience playback buffer. tuples In this embodiment, ; Calculate the next state target action At that time, truncation noise is added to the smoothing strategy:

[0131]

[0132]

[0133] Among them, the standard deviation of noise Cut off the boundary ;

[0134] Next, two target value networks are used for evaluation. And take the minimum value to calculate the target. value,

[0135]

[0136] When the single round ends. ,otherwise Eliminate by using the minimum of two target value networks. The value exceeds the estimated deviation. Recalculate the predicted value and the target value. The mean square error is calculated and updated via backpropagation. ,

[0137]

[0138] Gradient descent is performed using the Adam optimizer, with a learning rate set to... A delayed update strategy is adopted, where the policy network updates only once for every two updates to the value network, aiming to maximize the loss function of the first policy network.

[0139]

[0140] Also taking into account the pulse firing function Non-differentiable, define a rectangular window function As an alternative derivative:

[0141]

[0142] In calculating gradient Gradient flow First, the Critic network feeds back to the output layer of the Actor network, and then along the time axis. and network layer Backpropagation. For second-order kinetic neurons, the gradient propagates not only through the membrane potential. Propagation also requires restoring variables. Propagation. The specific chain rule unfolds as follows:

[0143]

[0144] Through this mechanism, the error signal travels through each pulse moment, updating all learnable parameters, including the weights. Bias Encoding parameters and dynamic parameters .

[0145] Finally, perform a soft update on all target network parameters:

[0146]

[0147]

[0148] In this embodiment, Each time, only 0.5% of the actual network parameters are updated to the target network to ensure the smoothness of the training objective.

[0149] (4) Deploy the trained Actor network to the quadruped robot, generate control commands based on the real-time multidimensional observation state vector, drive the quadruped robot to perform tasks until the quadruped robot reaches the target point.

[0150] like Figure 4 As shown, the present invention also relates to a path planning system for a quadruped robot based on brain-like decision-making, comprising:

[0151] The quadruped robot platform is equipped with a virtual LiDAR perception module, a positioning module, and an underlying motion controller.

[0152] The control terminal is equipped with a model trained by the brain-like decision-making-based quadruped robot path planning method. It is used to receive environmental state information, calculate path planning strategies in real time, generate speed control commands, and send them to the underlying motion controller to drive the quadruped robot to achieve autonomous navigation and obstacle avoidance.

[0153] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned brain-like decision-making-based quadruped robot path planning method.

[0154] In practical implementation, the autonomous navigation and obstacle avoidance scenario of the quadruped robot is first built in a Gym-based two-dimensional simulation platform, such as... Figure 2 As shown, the scenario is set as an unstructured terrain area containing an "S"-shaped obstacle layout and random obstacle blocks. In this scenario, the quadruped robot agent perceives the environment through LiDAR. Its observation state is composed of a normalized self-position, a unit vector pointing to the target, the target distance, and a 48-dimensional virtual LiDAR perception vector; the action space is defined as the robot's relative velocity control commands. The system sets a composite reward function to guide the robot to plan a path quickly and safely.

[0155] In the policy construction and network building phases, this invention employs a hybrid encoding mechanism: firstly, population encoding assigns state information for each dimension to a group of neurons with learnable Gaussian receptive fields, mapping low-dimensional continuous states to high-dimensional features; subsequently, deterministic encoding transforms activation intensity into deterministic pulse sequences, eliminating random noise. The policy network uses a four-layer fully connected structure, with hidden layers composed of second-order dynamic neurons, utilizing the coupled dynamic equations of recovery variables and membrane potentials to capture complex spatiotemporal features. The network ultimately decodes the velocity action command by statistically analyzing the average firing rate of the output layer within the simulation window. Simultaneously, two identical deep value networks are constructed to evaluate policy value.

[0156] During the model training and parameter optimization phases, a hybrid TD3 algorithm is used for training. During training, data is sampled from the experience replay buffer, truncated noise is added to smooth the strategy when calculating the target action, and the minimum value of the two target value networks is used to calculate the target action. To eliminate overestimation bias, the value network is updated by minimizing the mean squared error, while the policy network employs a delayed update mechanism. The optimization objective is to maximize the value score. To address the impulsive non-differentiability problem, a rectangular window function is defined as an alternative derivative, allowing the error gradient to propagate backward along the time axis, updating the second-order dynamic parameters and weights. Finally, soft updates are performed on the target network parameters to ensure training stability.

[0157] During the physical deployment and execution phase, the trained strategy is deployed to the Unitree GO1 quadruped robot. The system uses a Horizon Robotics X3 board as its computing core, acquires high-precision positioning information through a Wittek WTRTK-4G module, and inputs it into the SNN network along with airborne radar data. The speed commands calculated by the network in real time are sent to the robot's underlying controller via the network port, driving the quadruped robot to complete obstacle avoidance and path planning tasks in simulated environments such as tennis courts.

[0158] During the simulation, a quadruped robot (simplified as a point mass) was trained five times in a two-dimensional simulation environment with multiple obstacles in a continuous state space, with 80,000 time steps each, until the reward curve reached stability. Simultaneously, ablation experiments were conducted, performing multiple simulations of hybrid coding mechanisms and second-order dynamic neurons. The simulation reward comparison is shown in the figure below. Figure 5As shown, the lines and shaded areas represent the mean and variance of the reward value, respectively. The black line represents the algorithm proposed in this paper, the red line represents the simulation effect of population deterministic encoding under ordinary LIF neurons, and the blue line represents the simulation effect of population encoding under ordinary LIF neurons. Through simulation result comparison, it can be demonstrated that the second-order dynamic neurons in the proposed algorithm can control the quadruped robot to obtain the maximum reward and reach the target position faster than LIF neurons. It also highlights that population deterministic encoding has a larger reward than ordinary encoding, demonstrating that the hybrid encoding mechanism has a better information representation effect and can overcome the local optimum problem in path planning for quadruped robots.

[0159] The simulations conducted can be rendered as follows: Figure 6 As shown, the trained algorithm can reach the target point by passing through gaps in the shortest time, achieving the effect of path planning. The quadruped robot can reach the target position in 75 time steps.

[0160] To further verify the strong practical application effect of the trained algorithm, an experiment was conducted in a real-world scenario. The experiment used the Unitree GO1 quadruped robot, equipped with rich sensing interfaces, as the main motion subject to verify path planning in complex terrain. The positioning system employed the Wittek WTRTK-4G module, based on a radar system, achieving centimeter-level high-precision positioning with an error of less than 10cm. Data transmission relied on the Mercury M9G wireless networking module, constructing a low-latency (less than 20ms) MESH network with automatic reassembly capability to ensure stable communication. The computing core was equipped with the Horizon Solar X3 board (5TopsAI computing power), performing the conversion from latitude and longitude to UTM and local coordinate systems in the ROS environment. Closed-loop communication was established with the robot's underlying control system via the network port to ensure real-time calculation and execution of planning commands. The experimental site was a 32x20m tennis court. The quadruped robot's movement direction and speed were set consistent with the simulation experiment, and an obstacle environment identical to the simulation environment was set up to ensure that the quadruped robot achieved the same effect and performance as in the simulation environment in the real-world experiment.

[0161] The experimental results can be obtained through repeated training. Figure 7 As shown, the quadruped robot can reach the target position from the initial position according to the proposed algorithm without encountering obstacles in the environment. Furthermore, the experimental trajectory is almost identical to the simulated trajectory, verifying that the proposed method has good application performance.

[0162] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0167] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A path planning method for a quadruped robot based on brain-like decision-making, characterized in that: The method constructs an environmental scene and obtains a multi-dimensional observation state vector for the quadruped robot and the environment. An improved hybrid pulse reinforcement learning neural network is constructed, which performs hybrid encoding on the multidimensional observation state vector, converts it into a time-series pulse sequence, processes the time-series pulse sequence through an Actor network, outputs an action, and inputs the multidimensional observation state vector into a Critic network to obtain the corresponding value function estimate. Train the Actor network and Critic network; The trained Actor network is deployed to the quadruped robot, and control commands are generated based on the real-time multidimensional observation state vector to drive the quadruped robot to perform tasks until the quadruped robot reaches the target point.

2. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 1, characterized in that: The multidimensional observation state vector Including the robot's own status and environmental information, ,in, For the normalized position of the robot itself, This represents the unit vector pointing from the quadruped robot to the target. The target distance is normalized to the target point. This is the virtual lidar sensing vector.

3. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 1, characterized in that: The improved hybrid pulse reinforcement learning neural network includes an Actor network and a Critic network configured in combination; The Actor network consists of a sequentially connected input encoding layer, at least one second-order dynamic hidden layer, and an output decoding layer.

4. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 3, characterized in that: The multidimensional observation state vector is encoded by an input encoding layer, including population encoding and deterministic encoding performed sequentially; Population coding utilizes a set of neurons with learnable Gaussian receptive fields to cover each dimension of the state vector. For the i-th state dimension, the calculation of the simulated stimulus intensity of the j-th coding neuron is related to the multidimensional observed state vector, the mean and standard deviation of the Gaussian receptive field. The deterministic encoding uses the simulated stimulus intensity as the presynaptic input, updates the pseudomembrane voltage through a deterministic accumulation process, and generates a deterministic pulse sequence when the pseudomembrane voltage exceeds a threshold, while performing a soft reset operation.

5. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 3, characterized in that: The second-order dynamic hidden layer is composed of fully connected second-order dynamic neurons, and its dynamic behavior is described by coupled membrane potential variables and recovery variables through discretized difference equations. When the membrane potential exceeds the threshold and a pulse is emitted, a dynamic reset mechanism is executed.

6. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 1, characterized in that: The Actor network and Critic network were trained using a policy optimization method based on the MDC-TD3 algorithm. Construct two identical but parameter-independent deep Critic networks; sample data from the experience replay buffer and add truncated noise when calculating the target action; The target action is evaluated using two target Critic networks, and the target Q-value is calculated by taking the minimum of the two values. A delayed update strategy is adopted to make the update frequency of the pulse Actor network lower than that of the deep Critic network; During backpropagation, a rectangular window function is used as a substitute derivative of the pulse firing function to achieve backpropagation of the error gradient along time and end-to-end update.

7. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 6, characterized in that: A composite reward function is used to evaluate the behavior of the quadruped robot. The composite reward function includes goal achievement reward, collision or boundary crossing penalty, asymmetric distance guidance reward based on distance improvement, obstacle avoidance safety reward based on the nearest obstacle distance, and discrete grid exploration reward.

8. The method for path planning of a quadruped robot based on brain-like decision-making according to claim 1, characterized in that: The control quantity corresponding to the motion space of the quadruped robot is the relative speed command.

9. A path planning system for a quadruped robot based on brain-like decision-making, characterized in that: include: The quadruped robot platform is equipped with a virtual LiDAR perception module, a positioning module, and an underlying motion controller. The control terminal is equipped with a model trained by the brain-like decision-making-based quadruped robot path planning method as described in any one of claims 1 to 8. It is used to receive environmental state information, calculate path planning strategies in real time, generate speed control commands, and send them to the underlying motion controller to drive the quadruped robot to achieve autonomous navigation and obstacle avoidance.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by the processor, the program implements the brain-like decision-making-based quadruped robot path planning method as described in any one of claims 1 to 8.

Images