Mobile robot deep reinforcement learning motion planning method based on collision probability

Abstract

The invention provides a mobile robot deep reinforcement learning motion planning method based on collision probability, and belongs to the field of mobile robot navigation planning. According to the method, firstly, in order to evaluate the risk of collision between the robot and an obstacle in a complex dynamic environment, a collision probability estimation function is designed based on the relative distance and speed between the robot and the obstacle and is used for marking key obstacles around the robot; secondly, considering a key obstacle and a global motion planning target, and designing a combination form state observation space; and finally, designing a reward function for guiding the robot to avoid the key obstacle aiming at the key obstacle, bringing the information of the key obstacle into a state space and designing a reward function in a corresponding form, thereby realizing effective obstacle avoidance of the high-risk obstacle and reducing the time cost of motion planning. Meanwhile, it is verified that the algorithm improves the safe and rapid motion planning performance of the mobile robot in a complex dynamic environment in a test environment.

Description

Technical Field

[0001] The present invention relates to the field of mobile robot navigation planning, and in particular to a deep reinforcement learning motion planning method for mobile robots based on collision probability. Background Art

[0002] Autonomous mobile robots, with their strong motion planning capabilities and high degree of autonomy, are increasingly being used in various aspects of social life, particularly as service robots providing guidance in places like banks, hospitals, restaurants, and train stations. Mobile robots operating in these environments often face a common challenge: rapidly changing, dynamic environments, particularly those with people moving in all directions. This environment can be considered a complex dynamic environment. When performing motion planning tasks in complex dynamic environments, robots must not only plan for crowds and other dynamic obstacles in real time, but also for static obstacles in the environment. This significantly increases the demands placed on autonomous mobile robots' motion planning.

[0003] Traditional model-based motion planning methods often require a known map for global planning, followed by local planning for dynamic obstacle avoidance. However, in complex, ever-changing dynamic environments, they can easily fall into endless local planning, leading to the Frozen Robot Problem (FRP), where the robot becomes stuck and unable to move. The key to addressing these issues is how to utilize limited resources, such as confined spaces, limited perception, and computational load, to improve motion planning efficiency while maintaining safety. Recent work on obstacle avoidance navigation for intelligent sweeping robots and autonomous vehicles has demonstrated the effectiveness of deep reinforcement learning-based methods for motion planning. Compared to model-based methods, deep reinforcement learning-based methods do not require prior map information and are more applicable to unknown and dynamically changing environments. Furthermore, deep reinforcement learning-based methods can handle high-dimensional input states and model complex nonlinear relationships, thereby learning more accurate motion planning strategies. However, due to the limitations of the training environment, this approach suffers from generalization and scalability issues.

[0004] When mobile robots plan motion in complex dynamic environments, various hardware resources are limited. As the number of dynamic obstacles continues to increase, it becomes difficult to simultaneously process multiple obstacles observed in the surrounding environment, making it difficult to achieve more effective and safe obstacle avoidance using limited resources. To this end, this paper proposes a deep reinforcement learning motion planning method for mobile robots based on collision probability. Summary of the Invention

[0005] The present invention aims to solve the problems of low motion planning efficiency and low obstacle avoidance success rate caused by limited computing resources when a mobile robot performs motion planning in a complex dynamic environment. A deep reinforcement learning motion planning method for a mobile robot based on collision probability is proposed. First, a probability function for calculating the collision between a mobile robot and a dynamic obstacle is designed to evaluate the criticality of the dynamic obstacles around the robot. Then, a specific observation space and reward function are designed for such critical obstacles. Finally, the TD3 (Twin Delayed Deep Deterministic PolicyGradient) algorithm is used to train the model in a constructed simulation environment. The trained model effectively reduces the motion planning time, improves the success rate of obstacle avoidance, and realizes safe and fast motion planning of the mobile robot. In order to achieve the above, the technical solution proposed by the present invention is as follows:

[0006] A deep reinforcement learning motion planning method for a mobile robot based on collision probability includes the following steps:

[0007] Step 1: Calculate the probability of collision between the mobile robot and the surrounding dynamic obstacles, and mark the top K obstacles with the highest collision probability as key obstacles.

[0008] The relative information between the mobile robot and the dynamic obstacle mainly includes the relative position and relative velocity. In order to calculate the collision probability of the dynamic obstacle around the mobile robot at a certain time t, a probability function in the form of weighted average is designed. The calculation formula is as follows:

[0009] P=α1P d +α2P v (1)

[0010] Where: P represents the probability of collision between the mobile robot and a dynamic obstacle; α1, α2∈[0,1] represent the weight coefficients of different probability components and α1+α2=1; P d 、P v They are collision probability calculation functions based on distance and speed respectively.

[0011] The collision probability based on distance can be expressed as the function P d express:

[0012]

[0013] in: represents the distance between the robot and the obstacle at time t, l max and l min Respectively represent the maximum and minimum values of the lidar scanning range.

[0014] Collision probability P based on velocityv The calculation formula is as follows:

[0015]

[0016] Where: P v represents the collision probability; υ and υ o respectively represent the translational velocities of the mobile robot and the dynamic obstacle; similarly, |υ max | and |υ omax | respectively represent the maximum values of the translational velocities of the robot and the dynamic obstacle; θ is the direction angle of the relative velocity, and α, β are angular constants.

[0017] Step 2: In order to utilize the information of the key obstacles, an observation space combining multiple information is designed. This observation space mainly consists of four parts. The measurement data returned by the lidar of the mobile robot at time t is directly used as The set target point position and the calculated path point position are The position (r x , r y ) and velocity r υ of the robot at the current moment are The velocities (v xi , v yi ) and positions (p xi , p yi ) information of the K key obstacles with the highest calculated collision probability are where i ∈ [1, 2,..., K]. The observation space combining multiple information can be represented in the following form:

[0018]

[0019] Where: o t represents the observation space of the mobile robot at time t; is the measurement data returned by the lidar; is the information related to the target point; is the position and velocity information of the mobile robot; is the observation information of the key obstacles.

[0020] Step 3: Define the action space of the mobile robot.

[0021] The action space consists of two parts: the translational velocity and the rotational velocity of the mobile robot. Both are defined in the local coordinate system of the robot. At the same time, a continuous action space is used to obtain a more accurate control strategy. The size of the action space is determined by the actual hardware platform. Therefore, the action space of the robot can be defined as:

[0022]

[0023] Among them: a t Actions generated by deep reinforcement learning models, represent the translational velocity and rotational velocity respectively.

[0024] Step 4: Design a new composite reward function to guide model training.

[0025] The calculation formula of the new composite reward function is:

[0026] r=r g +r p +r c +r a (6)

[0027] Where: r is the total reward function of the deep reinforcement learning motion planning method, and the reward function form of each other item is described as follows.

[0028] r g It is the reward used to guide the robot to reach its goal and can be defined as follows:

[0029]

[0030] where r goal =200,g error =0.2m,nstep max =500, r dg =32. Where: It represents the distance from the robot to the target point at the end of the current step at time t; represents the distance from the robot to the target point at the end of the previous step at time t-1; g error Indicates the error range allowed for reaching the target; nstep represents the total number of steps the robot moves in this round at time t, max is the maximum number of steps set for each round.

[0031] r p The reward for reaching a waypoint is the intersection of the straight line connecting the robot's current position and the target position and the circle with the maximum radius of the LiDAR return data. It is used to guide the robot to move to the waypoint. The reward is as follows:

[0032]

[0033] where p error =0.2m indicates the allowable error range for reaching the path point; Represents the distance from the robot to the path point at time t.

[0034] r cIt is a reward used to punish the robot for approaching obstacles or colliding with them, and can be defined as follows:

[0035]

[0036] where r collision =-200;d robot =0.12m; r do =-2. Where: It represents the distance from the robot to an obstacle at time t, d robot It is the set limit collision distance.

[0037] r a It is a new reward item designed using velocity obstacle theory (VO) to avoid key obstacles. It is expressed as follows:

[0038] r a =r cpvo (θ m -θ cpvo ) (10)

[0039] Where: r cpvo =5 is the obstacle avoidance reward coefficient; is the set maximum steering angle, that is, it is expected that the steering angle of the robot each time does not exceed the set maximum value; θ cpvo Indicates the actual steering angle of the robot when considering key obstacles.

[0040] Step 5: Selection of deep reinforcement learning algorithm.

[0041] A reinforcement learning algorithm is selected to train the DRL model of the method of the present invention. Here, the Twin Delayed Deep Policy Gradient (TD3) algorithm is used, and the Adam optimizer is used to find the optimal policy parameters.

[0042] Step 6: Build the model training and testing environments.

[0043] Use the Gazebo robotics simulation platform to build a model training environment, as well as a simulation environment for model testing and performance comparison. This includes setting the mobile robot's starting and target positions, and setting the size, speed, and number of dynamic obstacles.

[0044] Step 7: Training process and testing results.

[0045] First, the hyperparameters required for model training, such as the learning rate and maximum step size, were set. Secondly, performance indicators such as success rate, time cost, average path length, and average speed were set to evaluate the model's adaptability in different environments.

[0046] The beneficial effects of the present invention are:

[0047] The collision probability-based mobile robot motion planning method designed in this invention has two advantages: First, the collision probability can be used to identify critical obstacles with higher risks. The speed and position information of these critical obstacles is used as part of the DRL state space, and a new reward function form based on VO theory is introduced into the reward function to guide the intelligent agent to avoid these critical obstacles, effectively improving the success rate of motion planning and reducing the time cost of motion planning. Second, in complex environments with a higher number of dynamic obstacles, this method can achieve safer and faster motion planning performance using limited hardware resources. Compared with other DRL-based methods, it has better performance in terms of success rate and average speed, and has strong adaptability and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is the overall flow chart of the deep reinforcement learning motion planning method for mobile robots based on collision probability proposed by the present invention;

[0049] Figure 2 A schematic diagram showing the principles of the design components of the collision probability function of the present invention;

[0050] Figure 3 Schematic diagram of the VO theory used for the key obstacle analysis of the present invention;

[0051] Figure 4 A schematic diagram of the principle of the present invention for calculating and updating path points;

[0052] Figure 5 1 is the overall system structure diagram of the method of the present invention;

[0053] Figure 6 The Gazebo environment map used by the present invention to train the algorithm model;

[0054] Figure 7 An environmental map for model testing in the present invention;

[0055] Figure 8 Path planning and speed distribution diagrams of different methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0057] The embodiment of the present invention provides a deep reinforcement learning motion planning method for a mobile robot based on collision probability. The overall process is as follows: Figure 1 As shown, the specific steps of implementing the present invention are as follows:

[0058] Step 1: Calculate the probability of collision between the mobile robot and the surrounding dynamic obstacles, and mark the eight obstacles with the highest collision probability as key obstacles.

[0059] The relative information between the mobile robot and the dynamic obstacle mainly includes relative position and relative speed, such as Figure 2 As shown in the figure, A represents the robot, B1 and B2 represent obstacles, d1 and d2 are the distances from the robot to the two obstacles, and v A 、 and Represent the speeds of the three respectively, the solid line represents the current time t, and the dotted line represents the time t+1. Figure 2 It can be seen that at time t, B1 is closer, that is, d1<d2, while B2 is faster, that is, This causes the robot to collide with B1 and B2 at the same time at time t+1. In order to calculate the collision probability of the dynamic obstacles around the mobile robot at time t, a probability function in the form of weighted average is designed. The calculation formula is as follows:

[0060] P=α1P d +α2P v (1)

[0061] Where: P represents the probability of collision between the mobile robot and a dynamic obstacle; α1, α2∈[0,1] represent the weight coefficients of different probability components and α1+α2=1; P d 、P v are the collision probability calculation functions based on distance and speed, respectively. Here, α1 = α2 = 0.5 indicates that distance and speed have the same impact on the collision probability. At time t, a close, slow obstacle and a far, fast obstacle have the same probability of colliding with the robot.

[0062] The collision probability based on distance can be expressed as the function P d express:

[0063]

[0064] in: represents the distance between the robot and the obstacle at time t, l max and l min Respectively represent the maximum and minimum values of the lidar scanning range.

[0065] Collision probability P based on velocity v The calculation formula is as follows:

[0066]

[0067] Where: P v represents the collision probability. Figure 3 You can get v A and v B Represent the translation speeds of the mobile robot and the obstacle respectively; similarly and denote the maximum translational speed of the robot and obstacle respectively; θ is the relative speed v A,B ; α and β are angle constants calculated by VO theory through simple geometric relationships. Figure 3 A is used to represent the position p A , radius r A , speed v A The robot, B represents the position p B , radius r B , speed v B Obstacles; v A,B is the speed of the robot relative to the obstacle, λ A,B The straight line representing the relative velocity; represents the circular area obtained by mapping the obstacle into the robot's configuration space, λ r and λ f yes Two tangent lines of A,B Represents the collision cone, VO A,B A speed barrier.

[0068] To define the speed barrier VO A,B , first you need to define the circular area of the mapping It can be expressed as follows:

[0069]

[0070] Where: d(p S ,p B ) indicates p S and p B Then, the collision cone CC A,B It can be defined as:

[0071]

[0072] Finally, the speed barrier VO A,B Defined as:

[0073]

[0074] in: represents the Minkowski vector and operator.

[0075] CC A,B The physical meaning is that for any relative speed v A,B ∈CC A,B will cause the robot to collide with the obstacle at some time in the future. A,B It can also be defined as:

[0076] CC A,B ∈[α-β,α+β](7)

[0077] The calculation formulas for α and β can be expressed as:

[0078]

[0079] Use Collision Cone CC A,B , the robot can know which heading angles will cause a collision with an obstacle. Then, the robot uses these collision cones and the direction angle θ of the path-point p , find the required heading angle θ through a sampling-based search algorithm cpvo .

[0080] Step 2: In order to utilize the information of key obstacles, an observation space with multiple information combinations is designed.

[0081] The observation space is mainly composed of four parts. The measurement data returned by the mobile robot laser radar at time t is directly used as The set target point position and the calculated path point position are The robot's current position (r x ,r y ) and speed r υ for Calculate the speed of the K key obstacles with the highest collision probability (v xi ,v yi ) and position (p xi ,p yi ) Information is where i∈[1,2,…,K]. The observation space of multiple information combinations can be expressed as follows:

[0082]

[0083] Among them: t represents the observation space of the mobile robot at time t; The measurement data returned by the lidar; Information related to the target point; is the position and speed information of the mobile robot; Observation information of key obstacles.

[0084] Step 3: Define the action space of the mobile robot.

[0085] The action space consists of the translational velocity and rotational velocity of the mobile robot, both of which are defined in the robot's local coordinate system. The robot's forward direction is defined as the positive direction of the local coordinate system's X-axis. Therefore, the robot's linear velocity is always in the positive direction of the axis, while the angular velocity is in the positive direction when rotating counterclockwise. In addition, a continuous action space is used to obtain more accurate control strategies. The size of the action space is determined by the actual hardware platform. The Turtlebot3-Burger two-wheel differential mobile robot is used here, so the robot's action space can be defined as:

[0086]

[0087] Among them: a t Actions generated by deep reinforcement learning models, They represent the translational velocity and rotational velocity in the local coordinate system of the mobile robot, respectively.

[0088] Step 4: Design a new composite reward function to guide model training.

[0089] The calculation formula of the new composite reward function is:

[0090] r=r g +r p +r c +r a (11)

[0091] Where: r is the total reward function of the deep reinforcement learning motion planning method, and the reward function form of each other item is described as follows.

[0092] r g It is the reward used to guide the robot to reach its goal and can be defined as follows:

[0093]

[0094] where r goal =200,g error =0.2m,nstep max =500, r dg =32. Where: It represents the distance from the robot to the target point at the end of the current step at time t; represents the distance from the robot to the target point at the end of the previous step at time t-1; g errorIndicates the error range allowed for reaching the target; nstep represents the total number of steps the robot moves in this round at time t, max is the maximum number of steps per round.

[0095] r p is the reward for reaching the path-point, such as Figure 4 The figure shows the calculation and update diagram of the path point. The path point is the intersection of the robot's current position, the target position and the maximum measurement distance of the laser radar. t 、B t Represent the positions of the robot and the path point at time t, A t+1 、B t+1 is the position of the robot and the path point at time t+1, and C represents the position of the target point. Figure 4 It can be seen that if the robot cannot reach the waypoint directly, that is, the robot avoids obstacles, the waypoint must be updated. If the waypoint is not updated, it must first go to the original waypoint and then move to the target point. Assuming that the robot moves in an ideal straight line, the sum of the two sides of the triangle is greater than the third side, which shows that the total path length increases, so the waypoint needs to be updated. In order to make the robot move to the target point faster without the need for obstacle avoidance and at the same time have a smoother path, a waypoint reward function is set. The reward is calculated as follows:

[0096]

[0097] where p error =0.2m indicates the allowable error range for reaching the path point; Represents the distance from the robot to the path point at time t.

[0098] r c It is a reward used to punish the robot for approaching obstacles or colliding with them, and can be defined as follows:

[0099]

[0100] where r collision =-200;d robot =0.12m; r do =-2. Where: It represents the distance from the robot to a dynamic obstacle at time t, d robot It is the set limit distance. When the distance between the robot and the obstacle reaches this distance, it means a collision has occurred.

[0101] r a It is a new reward item designed using VO theory to avoid key obstacles, expressed as follows:

[0102] r a =r cpvo (θ m -θ cpvo ) (15)

[0103] Where: r cpvo =5 is the key obstacle avoidance reward coefficient; is the set maximum steering angle, that is, it is expected that the steering angle of the robot each time does not exceed the set value; θ cpvo Indicates the actual steering angle of the robot when considering key obstacles.

[0104] Step 5: Selection of deep reinforcement learning algorithm.

[0105] A reinforcement learning algorithm is selected to train the DRL model of the method of the present invention. Here, the Twin Delayed Deep Policy Gradient (TD3) algorithm is used, and the Adam optimizer is used to find the optimal policy parameters.

[0106] like Figure 5 The following diagram shows the overall system architecture. The hardware section includes the Turtlebot3-Burger mobile robot model and sensor types. Next, the state space observation section includes information such as lidar measurement data, key obstacle data, target point-related data, and robot state data. Finally, the state observation space is input into the network to generate action commands, which are then sent to the robot via ROS for execution.

[0107] Step 6: Build the model training and testing environments.

[0108] The Gazebo robot simulation platform was used to build Figure 6 The model training environment shown has a map size of 4x4m, and the center is located at the origin of the global coordinate system (0,0). The robot's starting position is located in the lower left corner with coordinates (-1.5,1.5), and the target position is (1.5,1.5), which is located in the upper right corner of the map. The blue cylinder is used to represent dynamic obstacles, with a size of 0.05x0.2m, a speed of [0,0.22]m / s, and a random direction. 20 moving obstacles are set according to the size of the training environment. In order to more conveniently describe the test environment and reflect the superiority of the solution, the concept of environmental dynamic density is defined. For an area size of S / m 2 , in an environment where the number of dynamic obstacles is N, the dynamic density ρ can be expressed as follows:

[0109]

[0110] ρ represents the number of dynamic obstacles per unit area in the environment, which can directly evaluate the dynamic intensity in different test environments and ensure that the performance of the solution is not affected by the area size. 2 The environmental area is set up, and six test environments with different dynamic densities are set up, namely ρ = 0.5, ρ = 1.0, ρ = 1.5, ρ = 2.0, ρ = 2.5, and ρ = 3.0. Figure 7 As shown in (a), (b), (c), (d), (e) and (f).

[0111] Step 7: Training process and testing results.

[0112] First, we set hyperparameters such as the learning rate and maximum step size required for model training. Next, we set performance metrics such as success rate, time cost, average path length, and average speed to evaluate the model's adaptability in different environments. We conducted 100 rounds of testing in each test environment with different values of ρ. The percentage of robots successfully reaching their target was the test success rate in that environment. The average time spent in all successful rounds was the model's time cost in that environment. The average cumulative path length of the robot successfully reaching its target was the average path length of the model in that environment. The average speed was the ratio of the average path length to the time cost.

[0113] To verify the effectiveness of the method of the present invention, the traditional DRL, the DRL-VO motion planning method that does not consider the collision probability, and the DRL-CPVO motion planning method proposed in the solution of the present invention were trained and tested respectively. All three methods were carried out in the same training and testing environment. The test results are shown in Table 1. It can be seen that as ρ increases, the motion planning success rate of the three methods decreases, but the DRL-CPVO method proposed in the present invention always has a higher success rate, and the gap between the success rates increases with the continuous increase of ρ, indicating that the method of the present invention is more applicable in environments with larger ρ, and has the shortest average path length and the fastest average speed.

[0114] Table I Comparison of the results of three planning methods under different ρ environments

[0115]

[0116]

[0117] Figure 8The following are the motion planning simulation results of the three methods that are closest to the average case in the test environment with ρ = 3.0. Figure (a) is a path planning comparison diagram, Figure (b) is a linear velocity distribution diagram, and Figure (c) is an angular velocity distribution diagram. As can be seen from the path planning diagrams of the three methods, the path generated by DRL-CPVO is shorter and smoother. The linear velocity distribution histogram shows a higher proportion of the robot's actions running at maximum speed, which ensures the efficiency of the motion planning task. In the angular velocity distribution histogram, the proportion of the robot's angular velocity near 0 is relatively high for the method of the present invention. This is the result of setting the VO reward item to guide the robot to avoid key obstacles.

[0118] It should be pointed out that the above is a preferred embodiment of the present invention. Any changes made based on the technical solution of the present invention and the resulting functional effects do not exceed the scope of the technical solution of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A deep reinforcement learning motion planning method for mobile robots based on collision probability, characterized by: The following steps are involved: Step 1: Calculate the probability of collision between the mobile robot and surrounding dynamic obstacles, and mark the top K obstacles with the highest collision probability as key obstacles; Step 2: Design an observation space that combines multiple information using the position and velocity information of key obstacles, robot sensor data, and target points. Step 3: Input the observation space into the motion planning strategy model to obtain the robot action and design the action space of the mobile robot; Step 4: Design a composite reward function to guide model training; Step 5: Select a deep reinforcement learning algorithm to train the mobile robot motion planning model; Step 6: Build the training and testing environments required for the model; Step 7: Train and test the model.

2. The method for deep reinforcement learning motion planning for mobile robots based on collision probability according to claim 1, characterized in that: The function formula used to calculate the collision probability in step 1 is as follows: P=α1P d +α2P v (1) Where: P represents the probability of collision between the mobile robot and a dynamic obstacle; α1, α2∈[0,1] represent the weight coefficients of different probability components and α1+α2=1; P d 、P v They are collision probability calculation functions based on distance and speed, respectively. The calculation formulas are as follows: Where: υ and υ o respectively represent the translational velocities of the mobile robot and the dynamic obstacle; |υ max | and |υ omax | respectively represent the maximum values of the translational velocities of the robot and the obstacle; θ is the direction angle of the relative velocity, and α and β are angle values calculated from geometric relationships; in: represents the distance between the mobile robot and the dynamic obstacle at time t; l max and l min Respectively represent the maximum and minimum values of the lidar measurement range.

3. The method for deep reinforcement learning motion planning for mobile robots based on collision probability according to claim 2, characterized in that: In step 2, the observation space of the multiple information combinations is as follows: Among them: t represents the observation space of the mobile robot at time t; The measurement data returned by the lidar; Information related to the target point; is the position and speed information of the mobile robot; Observation information of key obstacles.

4. The method for deep reinforcement learning motion planning for mobile robots based on collision probability according to claim 3, characterized in that: In step 3, the action space of the mobile robot is as follows: Among them: a t Actions generated by the motion planning policy model, are the translational velocity and rotational velocity in the local coordinate system of the mobile robot, respectively.

5. The method for deep reinforcement learning motion planning of a mobile robot based on collision probability according to claim 4, characterized in that: In step 4, the formula for calculating the compound reward function is: r＝r g +r p +r c +r a (6) Where: r is the total reward function of the deep reinforcement learning motion planning method, and the reward function form of each other item is described as follows; r g It is the reward used to guide the robot to reach its goal and is defined as follows: where r goal =200,g error =0.2m,nstep max =500, r dg =32; where: It represents the distance from the robot to the target point at the end of the current step at time t. represents the distance from the robot to the target point at the end of the previous step at time t-1, g error Indicates the error range allowed for reaching the target, nstep indicates the total number of steps the robot moves in this round at time t, and nstep max is the maximum number of steps set for each round; r p is the reward for reaching a waypoint, which is the intersection of the line connecting the robot's current position and the target position and the maximum scanning distance of the lidar. The corresponding reward is calculated as follows: where p error =0.2m represents the allowable error range for reaching the sub-target point; Indicates the distance from the robot to the sub-target point at time t; r c It is a reward used to punish the robot for approaching obstacles or colliding with them. It has the following form: where r collision =-200;d robot =0.12m; r do =-2; where: It represents the distance from the robot to a dynamic obstacle at time t, d robot It is the set limit collision distance.

6. The method for deep reinforcement learning motion planning of a mobile robot based on collision probability according to claim 5, characterized in that: In step 5, the TD3 algorithm based on the Actor-Critic framework is selected to train the mobile robot motion planning model.

7. The method for deep reinforcement learning motion planning of a mobile robot based on collision probability according to claim 6, characterized in that: The process for step 7 is as follows: Define performance indicators for evaluating mobile robot motion planning methods, test the trained models in different environments, and compare the model test results in different environments.

Images