A Mapless Dynamic Obstacle Avoidance Navigation Method for Mobile Robots
By introducing temporal modeling and policy constraint mechanisms into the navigation model, the problems of unsmooth obstacle avoidance and safety of robots in dynamic obstacle environments in existing technologies are solved, and the autonomous navigation and obstacle avoidance capabilities of mobile robots are realized.
Patent Information
- Application Number
- CN202611159855.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-02
- Publication Date
- 2026-11-17
AI Technical Summary
Existing deep reinforcement learning navigation algorithms struggle to effectively perceive the movement trends of dynamic obstacles in scenarios without maps and with frequently changing obstacles. This leads to robots exhibiting local head-shaking, frequent and drastic motion corrections during obstacle avoidance, and insufficient safety distances, affecting the smoothness and safety of navigation.
A navigation model based on deep reinforcement learning is adopted, and a temporal modeling module such as a long short-term memory network is introduced to extract the temporal features of continuous observation states. Smoothness constraints and safety constraints are added to the loss function of the policy network to constrain and control the smoothness and safety of the instructions.
It improves the navigation success rate, safety margin, and trajectory smoothness of mobile robots in complex dynamic environments, and realizes autonomous obstacle avoidance navigation in mapless dynamic environments.
Smart Images

Figure CN123111273A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot autonomous navigation and artificial intelligence control, and in particular to a mapless dynamic obstacle avoidance navigation method for mobile robots. Background Technology
[0002] Achieving safe and efficient autonomous navigation in complex environments is a core capability for mobile robots. However, traditional navigation methods that rely on pre-built environmental maps suffer from poor environmental adaptability and weak generalization ability in scenarios without maps or with frequently changing dynamic obstacles. Deep reinforcement learning technology can directly learn control strategies from sensor observations, enabling end-to-end decision-making from environmental perception to action output, providing a novel solution to the problem of mapless navigation.
[0003] However, existing deep reinforcement learning navigation algorithms typically rely solely on single observations at the current moment for decision-making, failing to adequately utilize continuous historical observations. This makes it difficult for robots to effectively perceive the movement trends of dynamic obstacles. Furthermore, existing models lack effective constraint mechanisms in the action output stage, leading to problems such as localized head-swaying, frequent and drastic action corrections, and insufficient safety distances during actual obstacle avoidance. Consequently, it is difficult to guarantee the smoothness of the mobile robot's trajectory and obstacle avoidance safety. Summary of the Invention
[0004] The purpose of this invention is to solve the technical problems mentioned above and to propose a map-free dynamic obstacle avoidance navigation method for mobile robots, comprising the following steps: Obtain continuous observation status information of the mobile robot; The continuous observation state information is input into a pre-trained navigation model, and the navigation model outputs control commands to control the mobile robot to perform autonomous obstacle avoidance navigation in a dynamic environment. The navigation model is a model built based on deep reinforcement learning. During the training process, a temporal modeling module is introduced to extract the temporal features of continuously observed state information. At the same time, a policy constraint mechanism is introduced into the policy network loss function of the navigation model to constrain the smoothness and safety of the output control commands.
[0005] In a preferred embodiment, the continuous observation state information is composed of state vectors from multiple adjacent moments; the state vector at a single moment is composed of the following information spliced together: the ranging information sequence of the lidar in different angular sectors, the relative distance between the robot and the target point, the azimuth difference between the robot's current orientation and the target direction, the robot's current linear velocity, and the robot's current angular velocity.
[0006] In the preferred embodiment, the navigation model is specifically a soft actor-critic algorithm model that incorporates a long short-term memory network module; the temporal modeling refers to using the long short-term memory network module to extract temporal features from continuous observation states to characterize the movement trend of dynamic obstacles and the process of environmental changes.
[0007] In a preferred embodiment, the long short-term memory network module is introduced into the policy network and value network of the navigation model, respectively. In the policy network, the sequence of continuously observed state information passes through a fully connected layer and a long short-term memory network layer in sequence, and then outputs action distribution parameters. In the value network, the sequence of continuously observed state information is processed by a long short-term memory network module to extract temporal features. These temporal features are then fused with action features to output a state-action value estimate.
[0008] In the preferred embodiment, the policy constraint mechanism is reflected in the improved policy network loss function; The improved policy network loss function is specifically calculated as the sum of the original policy loss function and the additional policy constraint terms; The policy constraint terms, together with the original policy loss, participate in the backpropagation and parameter update of the policy network.
[0009] In a preferred embodiment, the strategy constraint term is formed by linearly weighting and summing the smoothing constraint term and the safety constraint term after multiplying them by their respective weight coefficients. The smoothing constraint term is used to suppress drastic changes in action output between adjacent time steps; The safety constraints are used to enhance the strategy's sensitivity to the risk of nearby obstacles.
[0010] In a preferred embodiment, the smoothing constraint term is specifically calculated as follows: the square of the difference between the linear velocity at the current moment and the linear velocity at the previous moment is calculated, the square of the difference between the angular velocity at the current moment and the angular velocity at the previous moment is calculated, the two squared values are added together, and the mean of the sum is taken as the value of the smoothing constraint term.
[0011] In a preferred embodiment, the safety constraint is specifically calculated by determining whether the distance between the robot and the nearest obstacle at the current moment is less than a set safety distance threshold. When the distance is greater than or equal to the safe distance threshold, the safety constraint term takes the value of zero; When the distance is less than the safe distance threshold, the difference between the safe distance threshold and the distance is calculated. This difference is multiplied by the set risk penalty growth coefficient to obtain a product. The power value is calculated with the natural constant as the base and the product as the exponent. Finally, the power value is subtracted by one to obtain the value of the safety constraint term, so that the safety constraint term increases as the robot approaches the obstacle.
[0012] In a preferred embodiment, the navigation model employs a combined reward function during training in a simulation environment. The combined reward function is calculated by adding the target guidance reward, safety perception reward, motion constraint reward, and termination feedback reward together, and subtracting the fixed step cost to obtain the total reward. Among them, the target-guided reward is used to guide the robot to move toward the target point; Safety perception rewards are used to constrain the distance between the robot and obstacles; Motion-constrained rewards are used to suppress low-speed stagnation and abrupt turning behaviors in robots; Termination feedback rewards are used to give positive rewards when the robot reaches the target point and negative rewards when the robot collides.
[0013] In the preferred embodiment, in practical applications, the real-time continuous state information collected by the mobile robot and its sensors in the actual environment is directly input into the trained navigation model; the model directly generates continuous linear velocity and angular velocity control commands to achieve end-to-end autonomous navigation from sensor observation to motion control commands.
[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: By introducing a Long Short-Term Memory (LSTM) network module into the SAC algorithm, this invention enables the navigation model to extract dynamic obstacle movement trends and environmental temporal change characteristics using continuous historical observation information, thus overcoming the problem of insufficient dynamic perception capability when relying solely on current observations for decision-making. Simultaneously, this invention incorporates a policy constraint mechanism consisting of smoothing and safety constraints into the policy network loss function, effectively suppressing unstable behaviors such as local head swings, sudden velocity changes, and abrupt turns, and enhancing the robot's sensitivity to near-field obstacle risks. Therefore, this invention can improve the navigation success rate, safety margin, and trajectory smoothness of mobile robots in complex dynamic environments, while eliminating the need for pre-constructing an environmental map, providing an effective solution for autonomous obstacle avoidance navigation of mobile robots in map-less dynamic environments. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the overall structure of the TC-SAC navigation model according to an embodiment of the present invention.
[0016] Figure 2 This is a schematic diagram of the temporal modeling structure of the policy network and value network in an embodiment of the present invention.
[0017] Figure 3 This is a schematic diagram of a mapless dynamic obstacle avoidance simulation training environment for a mobile robot according to an embodiment of the present invention. Detailed Implementation
[0018] Example 1 This embodiment provides a map-free dynamic obstacle avoidance navigation method for mobile robots, including the following steps: Obtain continuous observation status information of the mobile robot; The continuous observation state information is input into a pre-trained navigation model, and the navigation model outputs control commands to control the mobile robot to perform autonomous obstacle avoidance navigation in a dynamic environment. The navigation model is a model built based on deep reinforcement learning. During the training process, a temporal modeling module is introduced to extract the temporal features of continuously observed state information. At the same time, a policy constraint mechanism is introduced into the policy network loss function of the navigation model to constrain the smoothness and safety of the output control commands.
[0019] Preferably, the continuous observation state information is composed of state vectors at multiple adjacent moments; the state vector at a single moment is composed of the following information spliced together: the ranging information sequence of the lidar in different angular sectors, the relative distance between the robot and the target point, the azimuth difference between the robot's current orientation and the target direction, the robot's current linear velocity, and the robot's current angular velocity.
[0020] Preferably, the navigation model is a soft actor-critic algorithm model that incorporates a long short-term memory network module; the temporal modeling refers to using the long short-term memory network module to extract temporal features from continuous observation states to characterize the movement trend of dynamic obstacles and the process of environmental change.
[0021] Preferably, the long short-term memory network module is introduced into the policy network and value network of the navigation model, respectively; In the policy network, the sequence of continuously observed state information passes through a fully connected layer and a long short-term memory network layer in sequence, and then outputs action distribution parameters. In the value network, the sequence of continuously observed state information is processed by a long short-term memory network module to extract temporal features. These temporal features are then fused with action features to output a state-action value estimate.
[0022] Preferably, the policy constraint mechanism is embodied in the improved policy network loss function; The improved policy network loss function is specifically calculated as the sum of the original policy loss function and the additional policy constraint terms; The policy constraint terms, together with the original policy loss, participate in the backpropagation and parameter update of the policy network.
[0023] Preferably, the strategy constraint term is formed by linearly weighting and summing the smoothing constraint term and the safety constraint term after multiplying them by their respective weight coefficients; The smoothing constraint term is used to suppress drastic changes in action output between adjacent time steps; The safety constraints are used to enhance the strategy's sensitivity to the risk of nearby obstacles.
[0024] Preferably, the specific calculation method of the smoothing constraint term is as follows: calculate the square of the difference between the linear velocity at the current moment and the linear velocity at the previous moment, calculate the square of the difference between the angular velocity at the current moment and the angular velocity at the previous moment, add the above two squared values together, and take the mean of the sum as the value of the smoothing constraint term.
[0025] Preferably, the specific calculation method of the safety constraint is as follows: determine whether the distance between the robot and the nearest obstacle at the current moment is less than the set safety distance threshold; When the distance is greater than or equal to the safe distance threshold, the safety constraint term takes the value of zero; When the distance is less than the safe distance threshold, the difference between the safe distance threshold and the distance is calculated. This difference is multiplied by the set risk penalty growth coefficient to obtain a product. The power value is calculated with the natural constant as the base and the product as the exponent. Finally, the power value is subtracted by one to obtain the value of the safety constraint term, so that the safety constraint term increases as the robot approaches the obstacle.
[0026] Preferably, the navigation model employs a combined reward function during training in a simulation environment; the combined reward function is calculated by adding the target guidance reward, safety perception reward, motion constraint reward, and termination feedback reward, and subtracting the fixed step cost to obtain the total reward; Among them, the target-guided reward is used to guide the robot to move toward the target point; Safety perception rewards are used to constrain the distance between the robot and obstacles; Motion-constrained rewards are used to suppress low-speed stagnation and abrupt turning behaviors in robots; Termination feedback rewards are used to give positive rewards when the robot reaches the target point and negative rewards when the robot collides.
[0027] Preferably, in practical applications, the real-time continuous state information collected by the mobile robot and its sensors in the actual environment is directly input into the trained navigation model; the model directly generates continuous linear velocity and angular velocity control commands to achieve end-to-end autonomous navigation from sensor observation to motion control commands.
[0028] Example 2 like Figure 1As shown, this invention proposes a mapless dynamic obstacle avoidance navigation method for mobile robots that integrates temporal modeling and policy constraints. Its core lies in constructing and training a deep reinforcement learning agent called TC-SAC. This method first acquires environmental perception information and the robot's own motion state information through sensors such as LiDAR and odometry mounted on the mobile robot; then, it forms a state sequence from the state information at multiple consecutive time points and inputs it into the SAC model, which incorporates a long short-term memory network module, to extract the dynamic obstacle movement trend and environmental temporal change characteristics; further, it adds a policy constraint term to the policy network loss function to constrain the smoothness and safety of the action output; finally, the trained model outputs linear velocity and angular velocity control commands to control the mobile robot to complete autonomous navigation and obstacle avoidance in a mapless dynamic environment. The entire scheme does not rely on a pre-constructed environmental map, achieving end-to-end learning from sensor observations to motion control commands.
[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0030] This embodiment uses the TurtleBot3 mobile robot as the platform and is conducted in the Gazebo simulation environment.
[0031] S1. Build a mapless dynamic obstacle avoidance simulation training environment for mobile robots, configure the mobile robot and its sensors, and set dynamic obstacles, static obstacles and random target points.
[0032] S1.1. A mapless dynamic obstacle avoidance training environment for a mobile robot is built using the robot operating system ROS2 and the Gazebo simulation platform. In this embodiment, the simulation training environment is set to 6m×6m in size, and several walls and bricks are set in the environment as static obstacles to form the basic scene structure.
[0033] S1.2 Configure the TurtleBot3 Waffle mobile robot as the intelligent agent. The mobile robot is equipped with a two-dimensional LiDAR and an odometry module. The two-dimensional LiDAR is used to obtain distance information of obstacles around the robot, and the odometry module is used to obtain the robot's own pose and motion state information.
[0034] S1.3. Set multiple cylinders as dynamic obstacles in the simulation training environment, such as... Figure 3 As shown, dynamic obstacles move along preset or random paths to simulate uncertain dynamic disturbances such as pedestrians and moving objects in the environment; at the same time, target points are randomly generated in the environment, enabling the robot to learn navigation strategies from different initial states to different target positions during training.
[0035] S1.4. Set task judgment conditions. In this embodiment, when the distance between the robot and the target point is less than the preset arrival threshold of 0.20m, the robot is judged to have successfully reached the target; when the distance between the robot and the obstacle is less than the preset collision threshold of 0.13m, the robot is judged to have collided and the current training round ends.
[0036] S2, design the state space, action space, and combinatorial reward function.
[0037] S2.1 The state space consists of the mobile robot's environmental perception information, target relative position information, and its own motion state information. Specifically, the 360-degree scan data from the LiDAR is downsampled into a 40-dimensional distance vector, with each dimension representing the distance information of obstacles within the corresponding angular sector; the target relative position information includes the distance between the robot and the target point, as well as the azimuth difference between the robot's current orientation and the target direction; the robot's own motion state information includes the robot's current linear velocity and angular velocity.
[0038] As shown in formula (1), the state vector at a single moment can be expressed as: (1) in, to This indicates the ranging information of the lidar in different angular sectors. This represents the distance between the robot and the target point. This indicates the angle between the robot's current orientation and the target direction. This indicates the robot's current linear velocity. This represents the robot's current angular velocity. Therefore, in this embodiment, the state vector at a single moment is 44-dimensional.
[0039] S2.2 To enhance the model's ability to perceive the movement trend of dynamic obstacles, the state vectors of multiple consecutive adjacent time points are constructed into a state sequence, as shown in formula (2), which is expressed as: (2) in, This represents the length of the state sequence. In this embodiment, the state sequence length is 8, meaning that the state vectors at 8 consecutive time points are input into the Long Short-Term Memory (LSTM) network module for temporal feature extraction.
[0040] S2.3 The action space is the two-dimensional continuous control command of the mobile robot, as shown in formula (3): (3) in, To control the linear velocity, To control angular velocity. In this embodiment, the control linear velocity range is set. m / s, control angular velocity range set to rad / s.
[0041] S2.4 The combined reward function is used to guide the robot to simultaneously consider target arrival, obstacle avoidance, and motion stability. The robot at any given time... The instant reward is shown in formula (4): (4) in, Indicates a goal-oriented reward system. This indicates a reward for perceived safety. This indicates a reward based on exercise constraints. This indicates the termination of feedback rewards. The fixed step cost is 1. The fixed step cost is used to impose a small penalty on each step of movement, so as to encourage the robot to reduce ineffective movements and long periods of wandering, thereby improving navigation efficiency.
[0042] The target guidance reward is used to guide the robot toward the target point and continuously shorten the distance between the robot and the target point. This reward consists of a direction deviation term and a target distance term, as shown in formula (5): (5) The direction deviation term is defined as follows: (6) In the formula, This represents the angle between the robot's current orientation and the target direction. The target distance term is shown in formula (7): (7) in, This indicates the distance between the robot and the target location. This value encourages the robot to continuously approach the target and adjusts accordingly during the approach phase based on directional deviations to improve end-point navigation accuracy.
[0043] The safety perception reward is used to enhance the robot's ability to perceive local collision risks, as shown in formula (8): (8) in, Used to characterize the spatial risk between the robot and the nearest obstacle at the current moment. This is used to reflect the recent changing trend of local risk. The current spatial risk term is shown in formula (9): (9) in, This represents the minimum gap between the robot and the nearest obstacle at the current moment. This represents the early obstacle avoidance threshold, which is taken in this paper. 0.30m. This parameter applies a stronger penalty within close range and retains a weaker penalty that decreases with distance outside the safe range, in order to prompt the robot to adjust its movement strategy in advance.
[0044] To further reflect the changing trends of local risks, let the minimum obstacle gaps at the current time and the previous time be respectively... and Its change is defined as: (10) when This indicates that the distance between the robot and the nearest obstacle is decreasing, meaning the risk of local collision is increasing. Based on this, the local risk change term is shown in formula (11): (11) in, The activation threshold for risk change terms. 0.45m; The penalty intensity coefficient is set to 30; The upper limit for distance variation trimming is set to 0.05m. When the minimum obstacle gap continues to decrease within the close range, it indicates that the risk of local collision is increasing. At this point, additional penalties are applied to guide the robot to take avoidance actions earlier.
[0045] The motion constraint reward is to reduce unstable behaviors such as excessive forward rushing, stationary stagnation, and violent head shaking, as shown in formula (12): (12) The linear velocity constraint term is written as: (13) In the formula This indicates the robot's current linear velocity. This indicates that the reference linear velocity is 0.30 m / s. This parameter encourages the robot to maintain a reasonable forward speed and avoids excessive deviation. When the robot's linear velocity is too low, the following condition must be met: When the speed is 0.05 m / s, it is considered that the robot may stop or rotate in place, and a fixed penalty of -5 is added on the original basis.
[0046] The angular velocity constraint term is defined as follows: (14) in, This indicates the robot's current angular velocity. This value is used to suppress excessive turning movements, reduce control oscillations and frequent head swings, thereby improving motion stability during navigation.
[0047] The termination feedback reward is used to perform a global evaluation of the navigation results at the round level, as shown in formula (15): (15) in , A significant positive reward is given when the robot successfully reaches the target location, and a significant negative reward is given when the robot collides with an obstacle or wall. No further termination feedback is given in other cases. This feature constrains the policy outcome at the round level, reinforcing the preferences for successfully reaching the target and avoiding collisions.
[0048] S3. Construct a deep reinforcement learning network model based on the SAC algorithm, and introduce long short-term memory network modules into its policy network and value network to form an SAC+LSTM navigation model.
[0049] S3.1 Construct the SAC deep reinforcement learning network model. The SAC model includes a policy network, two value networks, a target value network, and an experience replay buffer.
[0050] S3.2, Long Short-Term Memory (LSTM) network modules are introduced into the policy network and value network of the deep reinforcement learning network model, respectively, such as... Figure 2 As shown in the diagram, in the policy network, the continuous state sequence passes through a fully connected layer, a long short-term memory network layer, and another fully connected layer in sequence, outputting the mean and logarithmic standard deviation of the action distribution. Continuous actions are then generated through reparameterized sampling and tanh mapping. In the value network, the continuous state sequence is input to the long short-term memory network layer via a state branch to extract temporal features, while action information is extracted via an action branch to extract action features. The temporal and action features are concatenated and then output as a state-action value estimate via a fusion layer. This enables the model to utilize historical observation information to characterize the movement trends of dynamic obstacles and the temporal changes in the environment, improving the obstacle avoidance decision-making ability of mobile robots in map-free dynamic environments.
[0051] S4. Introduce a policy constraint term into the policy network loss function of the SAC+LSTM navigation model to form the TC-SAC navigation model, such as... Figure 1 As shown.
[0052] S4.1. Based on the SAC+LSTM navigation model, a policy constraint mechanism consisting of smoothing constraints and safety constraints is introduced to ensure that the policy network can learn high-value actions while maintaining the continuity of action output and obstacle avoidance safety. The improved policy network loss function is shown in formula (16): (16) in, This represents the original policy loss function of the SAC algorithm. This represents an additional policy constraint term. This policy constraint term, along with the original policy loss, participates in the backpropagation and parameter updates of the policy network.
[0053] S4.2 The policy constraint term is composed of a linear weighted sum of a smoothing constraint term and a safety constraint term, as shown in formula (17): (17) in, Represents the smoothing constraint term. Indicates safety constraints. and These represent the weight coefficients of the smoothing constraint and the safety constraint, respectively.
[0054] S4.3 The smoothing constraint term is used to suppress drastic changes in action output between adjacent time steps, and to reduce frequent acceleration / deceleration and sharp turning phenomena during local obstacle avoidance, as shown in formula (18): (18) in, and These represent the linear velocity and angular velocity at the current moment, respectively. and Let these represent the linear velocity and angular velocity at the previous instant, respectively. This indicates that the mean of the corresponding terms is calculated.
[0055] S4.4 The safety constraint term is used to enhance the sensitivity of the strategy to the risk of nearby obstacles and guide the robot to maintain a reasonable safe distance during local obstacle avoidance, as shown in formula (19): (19) in, This indicates the distance between the robot and the nearest obstacle at the current moment. Indicates the safe distance threshold. This represents the risk penalty growth coefficient. When... Greater than or equal to When, the safety constraint term is 0; when Less than As the robot approaches an obstacle, the safety constraints increase, thus providing stronger inhibition against high-risk actions.
[0056] S5. In the simulation training environment, the robot is trained using the TC-SAC algorithm until the navigation strategy converges, and the trained navigation model is obtained.
[0057] S5.1 Initialize the policy network, two value networks, and the target value network in the TC-SAC navigation model, and initialize the experience replay buffer. Set the training parameters, including the learning rate, discount factor, soft update coefficient, batch size, and state sequence length. In this embodiment, the learning rate can be set to 0.0005, the discount factor to 0.97, the soft update coefficient to 0.005, the batch size to 64, the state sequence length to 8, and the optimizer to AdamW.
[0058] S5.2 At the beginning of each training round, initialize the position of the mobile robot and randomly generate target points in the simulation training environment. The robot interacts with the environment based on the actions output by the current policy network to obtain the next state, immediate reward, and round termination flag.
[0059] S5.3. The single-step state, action, reward, next state, and termination flag obtained from the robot's interaction with the simulation environment are stored in the experience replay buffer in chronological order, while retaining boundary information between different training rounds. Unlike the ordinary single-step random sampling method, during training, a continuous length of [value missing] is randomly sampled from the experience replay buffer. The current state sequence and the next state sequence are constructed based on the time segment as shown in formulas (2) and (20): (20) in, This indicates the length of the state sequence. In this embodiment, The mini-batch samples obtained from sampling include state sequences, actions, rewards, next state sequences, and termination flags, which are used for subsequent network parameter updates.
[0060] S5.4. The sampled state sequence is input into the Long Short-Term Memory (LSTM) network modules in the policy network and value network. The LSM network modules extract temporal features from continuous historical observation information. The target action and target value are calculated based on the next state sequence, and the parameters of the two value networks are updated. Subsequently, the improved policy network loss function is applied. Update the strategy network parameters, and finally update the target value network parameters using a soft update method.
[0061] S5.5 Repeat steps S5.2 to S5.4 to continuously optimize the navigation strategy as the robot interacts with the simulation environment. When the number of training rounds reaches a preset value, or when indicators such as average reward and navigation success rate tend to stabilize, the navigation strategy is considered to have converged, and the parameters of the trained TC-SAC navigation model are saved.
[0062] S6. Input the real-time status information collected by the mobile robot and its sensors in the simulation environment into the trained TC-SAC navigation model. The model outputs linear velocity and angular velocity control commands to control the mobile robot to complete autonomous navigation and obstacle avoidance in a mapless dynamic environment.
[0063] S6.1 Load the trained TC-SAC navigation model parameters in the simulation environment and initialize the mobile robot, dynamic obstacles, static obstacles, and target point.
[0064] S6.2 The mobile robot acquires lidar ranging information and odometer information in real time through simulated sensors, and constructs the current state vector by combining the target point location information.
[0065] S6.3. The state vectors from multiple consecutive time points are combined into a state sequence and input into the trained TC-SAC navigation model. After forward inference, the model outputs continuous actions, namely linear velocity control commands and angular velocity control commands.
[0066] S6.4 Apply the linear velocity control command and angular velocity control command to the simulated mobile robot, so that the robot adjusts its movement direction and speed in real time according to the current environmental state, avoids dynamic and static obstacles, and moves towards the target point.
[0067] S6.5. Repeat steps S6.2 to S6.4 to form a closed-loop process from perception to decision-making to control, until the robot successfully reaches the target point, collides with another robot, or reaches the maximum number of task steps. This enables autonomous navigation and obstacle avoidance for mobile robots in map-less dynamic environments.
[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A map-free dynamic obstacle avoidance navigation method for a mobile robot, characterized in that, Includes the following steps: Obtain continuous observation status information of the mobile robot; The continuous observation state information is input into a pre-trained navigation model, and the navigation model outputs control commands to control the mobile robot to perform autonomous obstacle avoidance navigation in a dynamic environment. The navigation model is a model built based on deep reinforcement learning. During the training process, a temporal modeling module is introduced to extract the temporal features of continuously observed state information. At the same time, a policy constraint mechanism is introduced into the policy network loss function of the navigation model to constrain the smoothness and safety of the output control commands.
2. The map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 1, characterized in that, The continuous observation state information consists of state vectors from multiple adjacent moments; the state vector at a single moment is composed of the following information: the ranging information sequence of the lidar in different angular sectors, the relative distance between the robot and the target point, the azimuth difference between the robot's current orientation and the target direction, the robot's current linear velocity, and the robot's current angular velocity.
3. The map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 1, characterized in that, The navigation model is specifically a soft actor-critic algorithm model that incorporates a long short-term memory network module; the temporal modeling refers to using the long short-term memory network module to extract temporal features from continuous observation states to characterize the movement trend of dynamic obstacles and the process of environmental changes.
4. The map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 3, characterized in that, The long short-term memory network module is respectively introduced into the policy network and value network of the navigation model; In the policy network, the sequence of continuously observed state information passes through a fully connected layer and a long short-term memory network layer in sequence, and then outputs action distribution parameters. In the value network, the sequence of continuously observed state information is processed by a long short-term memory network module to extract temporal features. These temporal features are then fused with action features to output a state-action value estimate.
5. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 1, characterized in that, The policy constraint mechanism is embodied in the improved policy network loss function; The improved policy network loss function is specifically calculated as the sum of the original policy loss function and the additional policy constraint terms; The policy constraint terms, together with the original policy loss, participate in the backpropagation and parameter update of the policy network.
6. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 5, characterized in that, The policy constraint term is composed of a linear weighted sum of smoothing constraint term and security constraint term multiplied by their respective weight coefficients. The smoothing constraint term is used to suppress drastic changes in action output between adjacent time steps; The safety constraints are used to enhance the strategy's sensitivity to the risk of nearby obstacles.
7. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 6, characterized in that, The specific calculation method for the smoothing constraint term is as follows: calculate the square of the difference between the linear velocity at the current moment and the linear velocity at the previous moment, calculate the square of the difference between the angular velocity at the current moment and the angular velocity at the previous moment, add the two squared values together, and take the mean of the sum as the value of the smoothing constraint term.
8. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 6, characterized in that, The specific calculation method for the safety constraint is as follows: determine whether the distance between the robot and the nearest obstacle at the current moment is less than the set safety distance threshold; When the distance is greater than or equal to the safe distance threshold, the safety constraint term takes the value of zero; When the distance is less than the safe distance threshold, the difference between the safe distance threshold and the distance is calculated. This difference is multiplied by the set risk penalty growth coefficient to obtain a product. The power value is calculated with the natural constant as the base and the product as the exponent. Finally, the power value is subtracted by one to obtain the value of the safety constraint term, so that the safety constraint term increases as the robot approaches the obstacle.
9. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to claim 1, characterized in that, The navigation model employs a combined reward function during training in a simulation environment. The combined reward function is calculated by adding the target guidance reward, safety perception reward, motion constraint reward, and termination feedback reward, and subtracting the fixed step cost to obtain the total reward. Among them, the target-guided reward is used to guide the robot to move toward the target point; Safety perception rewards are used to constrain the distance between the robot and obstacles; Motion-constrained rewards are used to suppress low-speed stagnation and abrupt turning behaviors in robots; Termination feedback rewards are used to give positive rewards when the robot reaches the target point and negative rewards when the robot collides.
10. A map-free dynamic obstacle avoidance navigation method for a mobile robot according to any one of claims 1 to 9, characterized in that, In practical applications, the real-time continuous state information collected by the mobile robot and its sensors in the actual environment is directly input into the trained navigation model; the model directly generates continuous linear velocity and angular velocity control commands, realizing end-to-end autonomous navigation from sensor observation to motion control commands.
