Robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning

Abstract

The invention discloses a robot path planning and obstacle avoidance method combining an artificial potential field and reinforcement learning. The method comprises the following steps: carrying out sensor data acquisition and fusion; environment complexity evaluation is carried out, a multi-scale planning mechanism is established, and global planning and local planning are determined based on switching conditions; carrying out artificial potential field path planning based on global planning; carrying out reinforcement learning path planning based on local planning; and judging whether the target point is reached, and returning to the determined plan until the target point is reached. According to the invention, the data of the infrared sensor and the laser radar are fused, the complementary characteristics of the two sensors are fully utilized, the comprehensive perception of static and dynamic obstacles is realized, and the adaptability of the robot in different environments is improved. According to the method, through reinforcement learning, strategy optimization and uncertainty estimation, the obstacle avoidance capability of the robot in a dynamic environment is enhanced, and the local optimum problem of a traditional artificial potential field method is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot autonomous navigation and path planning, and specifically relates to a robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning. Background Art

[0002] With the rapid development of robotics, the ability of robots to autonomously navigate and avoid obstacles in complex environments has become a research focus. Existing path planning and obstacle avoidance methods mainly rely on the combination of sensor technology and algorithms to ensure that robots can safely complete tasks in unknown or dynamic environments. However, many limitations in existing technologies limit the performance of robots in complex environments.

[0003] First, the limitations of traditional sensor technology

[0004] Currently, the mainstream methods of robot path planning and obstacle avoidance mainly rely on single sensors, such as LiDAR, depth cameras, ultrasonic sensors or infrared sensors. These sensors have their own advantages, but they also expose limitations in complex environments.

[0005] LiDAR: LiDAR can provide high-precision three-dimensional space detection capabilities, especially in distance measurement and spatial layout. However, LiDAR is prone to errors in extremely poor light or complex reflective environments (such as glass or foggy environments). In addition, LiDAR reacts relatively slowly when detecting dynamic obstacles (such as pedestrians and animals), making it difficult to cope with rapidly changing scenes.

[0006] Depth camera: Depth camera can capture the three-dimensional depth information of the environment and is usually used for navigation of indoor robots or drones. However, its performance is strongly dependent on the ambient lighting conditions and is prone to misdetection in strong or low light conditions. In particular, the effectiveness of depth cameras will be reduced in nighttime or low light scenes.

[0007] Infrared sensor: Infrared sensors can effectively detect temperature differences in low light or light-varying environments, and are particularly suitable for detecting heat sources (such as humans, animals, etc.), and have unique advantages in certain application scenarios. However, their disadvantage is that they cannot accurately provide distance and spatial layout information, making it difficult to independently complete robot navigation tasks.

[0008] Ultrasonic sensor: Ultrasonic sensors are not affected by environmental factors such as dust, haze, and light, and can maintain good working performance even in harsh environments. However, its disadvantage is that there may be a "blind spot" very close to the sensor, and the sensor cannot detect the object. If there are multiple objects in the environment, the reflected ultrasonic echoes may interfere with each other, resulting in inaccurate measurement data, especially in complex environments.

[0009] Second, the limitations of single sensor path planning algorithms

[0010] Path planning algorithms such as Dijkstra algorithm, A* algorithm, Artificial Potential Field (APF) and Rapid-exploring Random Tree (RRT) are widely used in robot navigation. Although these algorithms perform well in ideal environments, in complex dynamic environments, the perception limitations of a single sensor make it difficult for these algorithms to cope with changing scenarios, showing the following problems:

[0011] Local optimal problem: The traditional artificial potential field method is prone to fall into the local optimal point in the local environment, that is, the robot is trapped by the repulsive force of obstacles and it is difficult to continue moving towards the target point.

[0012] Poor real-time performance: Most traditional path planning algorithms require large amounts of computation in complex environments and are difficult to meet real-time requirements, especially in scenarios where dynamic obstacles frequently appear and the robot is unable to update the path in a timely manner.

[0013] Insufficient exploration capabilities: These algorithms usually perform path planning based on a global known map and perform poorly when faced with unknown environments or areas with uncertainty, resulting in inefficiency when the robot explores new areas.

[0014] In response to the above problems, China CN202311032731.X combines deep reinforcement learning and artificial potential field method to achieve precise motion control of the robot arm, especially in unstructured dynamic environments. China CN202311361250.3 combines global artificial potential field with local reinforcement learning, and introduces dynamic updates of global and local maps to solve the problems of low path planning success rate and slow algorithm convergence in complex local environments. China CN201911020333.X adds a domain potential field with strong attraction in a small range, and at the same time uses reinforcement learning to learn to avoid multi-target situations and reduce local optimal problems. China CN202411156553.6 combines artificial potential field with reinforcement learning, uses a combination of centralized training and decentralized execution in multi-agent scenarios, and handles multi-target problems through an allocation mechanism with the least combined gravity, which improves the decision-making efficiency of the agent in a dynamic environment and the optimization of path allocation.

[0015] The above patents all use a combination of reinforcement learning and artificial potential fields, but mainly improve the efficiency and success rate of path planning through algorithm optimization (such as deep learning, distributed course learning, etc.). However, the above patented technologies cannot meet the needs of how to improve the use of path planning when facing changing situations. Summary of the invention

[0016] The present invention is proposed to solve the problems existing in the prior art, and its purpose is to provide a robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning.

[0017] The technical solution of the present invention is: a robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning, comprising the following steps:

[0018] A. Perform sensor data collection and fusion;

[0019] B. Conduct environmental complexity assessment, establish a multi-scale planning mechanism, and determine global planning and local planning based on switching conditions;

[0020] C. Based on global planning, artificial potential field path planning is performed;

[0021] D. Perform reinforcement learning path planning based on local planning;

[0022] E. Determine whether the target point has been reached and return to step B until the target point has been reached.

[0023] Furthermore, step A performs sensor data collection and fusion, and the specific process is as follows:

[0024] First, the infrared sensor collects the thermal radiation information of the surrounding obstacles, especially the temperature distribution of the obstacles;

[0025] Then, the lidar provides distance information of obstacles and three-dimensional spatial data of the environment;

[0026] Then, the reliability of the infrared sensor and lidar is determined based on the current ambient light and obstacle type conditions.

[0027] Finally, the temperature information of the infrared sensor is combined with the distance data of the lidar through weighted fusion to form a unified environmental perception model.

[0028] Furthermore, in the fusion output process, the details are as follows:

[0029] First, when the infrared sensor detects a high-temperature obstacle, it gives priority to avoiding such obstacles; the lidar plays a role in mainly detecting static objects and providing path planning data;

[0030] Then, when the robot is operating in a low-light environment, the infrared sensor is given more weight, ensuring that the robot can still accurately identify obstacles and avoid danger even when it cannot rely on the lidar;

[0031] Later, in an environment with better lighting, the weight of lidar data increases, and the data from the infrared sensor will be used as an auxiliary input to detect dynamic obstacles or obstacles with large temperature changes, thereby improving the comprehensiveness of perception and obstacle avoidance capabilities in complex environments.

[0032] Furthermore, step B evaluates the complexity of the environment, establishes a multi-scale planning mechanism, and determines the global plan and local plan based on the switching conditions. The specific process is as follows:

[0033] First, obtain the overall information of the current environment based on fusion perception;

[0034] Then, the environment complexity assessment model is used to determine the distribution and density of obstacles in the environment and whether there are dynamic obstacles.

[0035] Then, the global plan / local plan is determined based on the switching conditions.

[0036] Furthermore, the environmental complexity assessment model is as follows:

[0037] Overall rating of environmental complexity: C complexity It is the weighted sum of the following key environmental parameters, specifically defined as:

[0038]

[0039] in:

[0040] w1, w2, w3, and w4 are the weights of each key parameter, which are adjusted according to the specific application scenario;

[0041] d density The obstacle density is calculated by the number of obstacles detected by the LiDAR and their distribution range. The more obstacles there are, the higher the density is and the more complex the environment is.

[0042] d obs It is the shortest distance between the robot and the obstacle. The closer to the obstacle, the higher the complexity. It is directly obtained from the laser radar, and its reciprocal is used as a parameter for complexity evaluation. The shorter the distance, the larger the reciprocal, and the higher the complexity.

[0043] v obs is the moving speed of the obstacle, which is collected by the laser radar;

[0044] T obs is the temperature of the obstacle, collected by the infrared sensor;

[0045] Based on fusion perception, the overall information of the current environment is obtained and the overall score of the complexity of the above environment is determined.

[0046] Furthermore, the switching condition in step B determines the switching between global planning and local planning by setting a threshold. When the indicator exceeds the threshold, the global planning is switched to the local planning, and vice versa.

[0047] Furthermore, step C performs artificial potential field path planning based on global planning. The specific process is as follows:

[0048] First, global planning is suitable for paths with fewer obstacles and longer distances;

[0049] The robot then uses the gravitational field model to guide it to move quickly toward the target point, while using the repulsive field model to avoid static obstacles.

[0050] Furthermore, step D performs reinforcement learning path planning based on local planning. The specific process is as follows:

[0051] First, when the robot approaches an obstacle-dense area or a dynamic obstacle, it switches to local planning mode;

[0052] Then, the robot's obstacle avoidance path is updated in real time through a high-precision local potential field model.

[0053] Furthermore, the problem that the robot is trapped by the combined force of local obstacles and cannot continue to move to the target point in local planning is solved as follows:

[0054] In order to deal with the above-mentioned local optimal problem, the robot determines which areas have insufficient information through uncertainty estimation, and then guides it to actively explore these areas, thereby enhancing the comprehensiveness of path planning and preventing the robot from falling into local optimality. This is solved by combining reinforcement learning algorithms.

[0055] Furthermore, the reinforcement learning algorithm uses the state function, action function, and reward function to make decisions and select the optimal action that the robot should take to avoid collision and reach the target point as quickly as possible.

[0056] The beneficial effects of the present invention are as follows:

[0057] The present invention integrates infrared sensor and lidar data, fully utilizes the complementary characteristics of the two sensors, realizes comprehensive perception of static and dynamic obstacles, and improves the adaptability of the robot in different environments.

[0058] By introducing an adaptive multi-scale planning mechanism, the present invention enables the robot to seamlessly switch between global and local operations, ensuring efficient path planning and obstacle avoidance.

[0059] The present invention enhances the robot's obstacle avoidance capability in a dynamic environment through strategy optimization and uncertainty estimation by reinforcement learning, and solves the local optimal problem of the traditional artificial potential field method. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a control flow chart of the present invention. DETAILED DESCRIPTION

[0061] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings and embodiments:

[0062] like Figure 1 As shown, the robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning includes the following steps:

[0063] A. Perform sensor data collection and fusion;

[0064] B. Conduct environmental complexity assessment, establish a multi-scale planning mechanism, and determine global planning and local planning based on switching conditions;

[0065] C. Based on global planning, artificial potential field path planning is performed;

[0066] D. Perform reinforcement learning path planning based on local planning;

[0067] E. Determine whether the target point has been reached and return to step B until the target point has been reached.

[0068] Step A performs sensor data collection and fusion. The specific process is as follows:

[0069] First, the infrared sensor collects the thermal radiation information of the surrounding obstacles, especially the temperature distribution of the obstacles;

[0070] Then, the lidar provides distance information of obstacles and three-dimensional spatial data of the environment;

[0071] Then, the reliability of the infrared sensor and lidar is determined based on the current ambient light and obstacle type conditions.

[0072] Finally, the temperature information of the infrared sensor is combined with the distance data of the lidar through weighted fusion to form a unified environmental perception model.

[0073] In the fusion output process, the details are as follows:

[0074] First, when the infrared sensor detects a high-temperature obstacle, it gives priority to avoiding such obstacles; the lidar plays a role in mainly detecting static objects and providing path planning data;

[0075] Then, when the robot is operating in a low-light environment, the infrared sensor is given more weight, ensuring that the robot can still accurately identify obstacles and avoid danger even when it cannot rely on the lidar;

[0076] Later, in an environment with better lighting, the weight of lidar data increases, and the data from the infrared sensor will be used as an auxiliary input to detect dynamic obstacles or obstacles with large temperature changes, thereby improving the comprehensiveness of perception and obstacle avoidance capabilities in complex environments.

[0077] Step B evaluates the complexity of the environment, establishes a multi-scale planning mechanism, and determines the global plan and local plan based on the switching conditions. The specific process is as follows:

[0078] First, obtain the overall information of the current environment based on fusion perception;

[0079] Then, the environment complexity assessment model is used to determine the distribution and density of obstacles in the environment and whether there are dynamic obstacles.

[0080] Then, the global plan / local plan is determined based on the switching conditions.

[0081] The environmental complexity assessment model is as follows:

[0082] Overall rating of environmental complexity: C complexity It is the weighted sum of the following key environmental parameters, specifically defined as:

[0083]

[0084] in:

[0085] w1, w2, w3, and w4 are the weights of each key parameter, which are adjusted according to the specific application scenario;

[0086] d density The obstacle density is calculated by the number of obstacles detected by the LiDAR and their distribution range. The more obstacles there are, the higher the density is and the more complex the environment is.

[0087] d obs It is the shortest distance between the robot and the obstacle. The closer to the obstacle, the higher the complexity. It is directly obtained from the laser radar, and its reciprocal is used as a parameter for complexity evaluation. The shorter the distance, the larger the reciprocal, and the higher the complexity.

[0088] v obs is the moving speed of the obstacle, which is collected by the laser radar;

[0089] T obs is the temperature of the obstacle, collected by the infrared sensor;

[0090] Based on fusion perception, the overall information of the current environment is obtained and the overall score of the complexity of the above environment is determined.

[0091] The switching condition in step B determines the switching between global planning and local planning by setting a threshold. When the indicator exceeds the threshold, the global planning is switched to the local planning, and vice versa.

[0092] Step C performs artificial potential field path planning based on global planning. The specific process is as follows:

[0093] First, global planning is suitable for paths with fewer obstacles and longer distances;

[0094] The robot then uses the gravitational field model to guide it to move quickly toward the target point, while using the repulsive field model to avoid static obstacles.

[0095] Step D performs reinforcement learning path planning based on local planning. The specific process is as follows:

[0096] First, when the robot approaches an obstacle-dense area or a dynamic obstacle, it switches to local planning mode;

[0097] Then, the robot's obstacle avoidance path is updated in real time through a high-precision local potential field model.

[0098] Solve the problem that the robot is trapped by the combined force of local obstacles in local planning and cannot continue to move to the target point. The details are as follows:

[0099] In order to deal with the above-mentioned local optimal problem, the robot determines which areas have insufficient information through uncertainty estimation, and then guides it to actively explore these areas, thereby enhancing the comprehensiveness of path planning and preventing the robot from falling into local optimality. This is solved by combining reinforcement learning algorithms.

[0100] The reinforcement learning algorithm uses the state function, action function, and reward function to make decisions and select the optimal action that the robot should take to avoid collision and reach the target point as quickly as possible.

[0101] Specifically, in local planning, the repulsive field of the obstacle on the robot is calculated and dynamically adjusted according to the relative distance, speed and temperature difference between the obstacle and the robot. In particular, the higher the temperature of the obstacle, the stronger the repulsive force, indicating that it is a biological or dynamic obstacle. Within the robot's perception range, a potential field model based on the superposition of the gravitational field and the repulsive field of the fused data is generated to guide the robot to avoid obstacles and move towards the target.

[0102] Specifically, the method for constructing the artificial potential field model is as follows:

[0103] The establishment of gravitational field

[0104] U att(q) represents the gravitational field of the robot under path planning, which is used to guide the robot to move towards the target point. Assume that the current position of the robot is q and the target point is q goal , then the gravitational field can be expressed as:

[0105]

[0106] in:

[0107] k att is the gravitational coefficient, which is used to control the robot's attraction toward the target.

[0108] ||qq goal || is the robot's current position q and the target point q goal The Euclidean distance between .

[0109] The establishment of repulsive field

[0110] U rep (q) is used to process the local planning of the robot in the obstacle-dense area. Assume that there are N obstacles in the environment, and the coordinates of the obstacles are q i Considering the moving speed and temperature of the obstacle at the same time, the repulsive field can be expressed as:

[0111]

[0112] in:

[0113] k rep is the repulsion coefficient, which is used to control the robot's avoidance of obstacles.

[0114] d(q,q i ) is the robot’s current position q and the obstacle’s position q i The distance between.

[0115] d safe is the safe distance, when d(q,q i ) is lower than the preset safety distance d safe , the robot makes obstacle avoidance actions according to the repulsive field.

[0116] f(T obs ) is the temperature influence function, which adjusts the strength of the repulsive field according to the temperature of the obstacle. The higher the temperature of the obstacle, the stronger the repulsive force generated.

[0117] The temperature influence function can be expressed as:

[0118] f(T obs )=1+χ·T obs

[0119] Among them, χ is the temperature coefficient, which is used to adjust the effect of temperature on the repulsive field.

[0120] f(v obs ) is the speed influence function, which adjusts the strength of the repulsive field according to the moving speed of the obstacle. The higher the moving speed of the obstacle, the stronger the repulsive force generated.

[0121] The speed influence function can be expressed as:

[0122]

[0123] in, is the speed coefficient, which is used to adjust the effect of speed on the repulsive field.

[0124] The total potential field U formed by the environment total (q) is the resultant force field of the robot at position q, which represents the robot's behavior under the combined effects of the gravitational force of the target point and the repulsive force of the obstacle. It can be expressed as the superposition of the gravitational field and the repulsive field:

[0125] U total (q) = U att (q)+U rep (q)

[0126] Then the total force on the robot in a complex environment can be calculated as:

[0127] F(q)=-▽U total (q)

[0128] Among them, ▽U total (q) is the gradient of the total potential field with respect to the robot's position q. The negative sign indicates that the robot will move toward the position with the minimum potential energy (the target point) while staying away from obstacles.

[0129] Specifically, in a complex dynamic environment, robots usually encounter a variety of uncertain situations, such as dense obstacles and frequent changes in dynamic obstacles. Although the traditional artificial potential field method can handle static obstacles well, it is easy to fall into the local optimum, that is, the robot is trapped by the combined force of local obstacles and cannot continue to move to the target point. Therefore, relying solely on artificial potential fields may not be able to completely solve these problems. In order to deal with this local optimal problem, through uncertainty estimation, the robot can determine which areas have insufficient information, and then guide it to actively explore these areas, thereby enhancing the comprehensiveness of path planning and avoiding the robot from falling into the local optimum. This can be solved by combining reinforcement learning algorithms.

[0130] Specifically, the Proximal Policy Optimization Agents (PPO) is a deep reinforcement learning algorithm that can improve the success rate of obstacle avoidance and path planning by learning and optimizing the robot's behavior strategy in a complex and dynamic environment. The Proximal Policy Optimization Agents optimizes the policy model by adjusting the relative position between the robot and obstacles and target points, improving the strength of the repulsive field, and the environmental data perceived by the robot in real time.

[0131] Specifically, the specific process of the reinforcement learning algorithm is as follows:

[0132] d1. Determine the state function s t

[0133] First, the state function is used to describe the robot's current environment state.

[0134] Then, the state function s t Represents the robot’s environmental perception information at time step t. The state space of reinforcement learning not only contains the position information of the robot and obstacles, but also incorporates the gravitational field and repulsive field information in the artificial potential field.

[0135] Finally, by adding the gravitational field (attraction of the target point) and the repulsive field (repulsion of obstacles) to the state description, the robot can consider the distribution of the global field in the state function. The above expansion of the state space helps the reinforcement learning algorithm to perceive the current environment state more comprehensively, providing richer information for the optimization of obstacle avoidance strategies.

[0136] s t ={robot position, obstacle position, target point position, gravitational field strength, repulsive field strength, sensor data}

[0137] The dynamic change information of the gravitational field and the repulsive field is calculated by the artificial potential field method, updated in real time at each moment, and input into the state space of reinforcement learning. Sensor data can include obstacle distance information perceived by the laser radar, temperature information of the infrared sensor, etc.

[0138] d2. Determine the action function a t

[0139] First, the action function a t represents the action taken by the robot at time step t.

[0140] Then, the robot adjusts the current state s t Decide how to move at the next moment (such as turning, backing up, accelerating or decelerating). The action space can be continuous or discrete.

[0141] a t ~πθ (a t ∣s t )

[0142] Among them, π θ (a t ∣s t ) represents the policy function, and the parameter θ is the weight of the deep neural network, which is used to describe the state s t Next select action a t The probability distribution of .

[0143] d3. Conduct uncertainty estimation

[0144] First, uncertainty can be measured by information entropy.

[0145] Then, in a certain state s t Under this condition, the probability of strategy π choosing each action a is π θ (a t ∣s t ). Information entropy is used to describe the uncertainty of this state:

[0146]

[0147] The higher the information entropy, the greater the uncertainty of the state, and the robot needs to explore more in the area.

[0148] d4. Determine the reward function r t

[0149] First, the reward function r t It is the robot that is taking action a t The reward value is then set based on the relative distance between the robot and the obstacle, the target point, and the obstacle avoidance situation.

[0150] Specifically, approaching the target point will receive positive rewards, avoiding obstacles will also increase rewards, and collision or moving away from the target point will result in negative rewards. In order to encourage the robot to explore uncertain areas, the uncertainty measure is introduced into the reward function of the proximal policy optimization algorithm. When the robot enters certain areas with high uncertainty in the dynamic environment, the proximal policy optimization algorithm can automatically enhance the exploration behavior. The exploration is improved by increasing the entropy of the policy, and the reward feedback is promptly transmitted back to the policy network to adjust future action selection.

[0151] r t = -α·d goal +β·d obs +δ·H(s t )+κ·F att -γ·F rep

[0152] in,

[0153] α is the distance penalty coefficient from the target point, d goal is the current distance between the robot and the target point;

[0154] β is the reward coefficient for obstacle distance, d obs is the shortest distance between the robot and the obstacle;

[0155] δ is a weight factor that controls the exploration intensity and determines the robot's exploration intensity in the uncertain area. t ) is the exploration reward function;

[0156] κ is the reward coefficient of the gravitational field, F att Represents the strength of the gravitational field.

[0157] γ is the penalty coefficient of the repulsive field, F rep Represents the strength of the repulsive field.

[0158] d5. Determine the advantage function A πθ

[0159] First, the advantage function It reflects the advantage of choosing a certain action over other actions in a given state. It measures the advantage of the current action over other possible actions and is the basis for updating the strategy.

[0160] Then, in a dynamic environment, by averaging the rewards of the first several time steps k, the jitter caused by the dynamic changes of the environment can be reduced.

[0161]

[0162] Among them, ξ is the discount factor, which indicates the discount degree of future rewards; is the state value function, which means that in state s t The expected reward value under .

[0163] d6. Determine the objective function of the proximal strategy optimization algorithm

[0164] Through active exploration and uncertainty estimation, the policy update of reinforcement learning is based on the feedback obtained from exploration, thereby continuously optimizing the path planning. The objective function of the proximal policy optimization algorithm limits the amplitude of the policy update to prevent the policy from being updated too quickly, causing the policy to become unstable. The objective function usually uses a clipped probability ratio to maintain stability:

[0165]

[0166] in,

[0167] is the ratio of the new strategy to the old strategy.

[0168] ∈ is a clipping parameter that controls the update amplitude of the strategy to prevent the ratio from deviating too much.

[0169] clip(r t (θ), 1-∈, 1+∈) limits the update step size so that the policy does not change excessively.

[0170] d7. Update the strategy

[0171] The proximal policy optimization algorithm maximizes its objective function L PPO (θ) to update the strategy:

[0172]

[0173] Where η is the learning rate, It is the gradient of the policy network parameters, and optimizing the network makes the policy continuously improved.

[0174] Through the proximal policy optimization algorithm, at each time step, the robot perceives the state of the environment (state function s t ), select an action according to the current strategy (action function a t ), and according to the feedback reward signal (reward function r t ) update strategy. Through active exploration and uncertainty estimation, the robot can not only plan paths in known environments, but also dynamically respond to changes in unknown environments, thereby optimizing the overall path planning process. This mechanism effectively prevents the robot from falling into local optimal points and improves exploration efficiency.

[0175] Compared with other algorithms, the proximal policy optimization algorithm maintains the stability of learning by limiting the amplitude of policy updates, avoiding excessive changes in the policy during each update.

[0176] As an implementation method, during the application process, the collaboration between the artificial potential field and reinforcement learning can be carried out through the following steps:

[0177] Global Guidance: In long-distance path planning, the robot uses an artificial potential field method to determine the global direction of motion. The gravitational field guides the robot to the target point, while the repulsive field ensures that the robot avoids static obstacles.

[0178] Local optimization: When the robot approaches a dynamic obstacle, the reinforcement learning algorithm makes local obstacle avoidance decisions based on the current state. Reinforcement learning analyzes and improves the repulsive strength of the obstacle in the repulsive field and selects the optimal action to avoid the obstacle. This allows the robot to flexibly respond to complex local environments.

[0179] Real-time updates: During the local obstacle avoidance process, the reinforcement learning algorithm continuously optimizes its own strategy and dynamically adjusts the robot’s actions based on the learned experience, ensuring that the robot can both avoid obstacles efficiently and stay on track toward the global goal.

[0180] This collaborative mechanism effectively solves the defect of traditional artificial potential field methods that are prone to falling into local optimality. At the same time, it improves the robot's ability to cope with dynamic and complex environments and ensures the coordination and consistency between global path planning and local obstacle avoidance decisions.

[0181] The present invention can be applied to, but is not limited to, unmanned vehicles, ground robots, drones, intelligent storage robots, and other fields that require autonomous navigation and obstacle avoidance, and is particularly suitable for scenes with complex lighting, dynamic obstacles, or unknown environments.

[0182] Based on the infrared sensor and laser radar integrated on the robot, the present invention organically combines the artificial potential field method with reinforcement learning, so that the robot can achieve efficient collaboration in global path planning and local obstacle avoidance decision-making. The artificial potential field method is used for global path planning to guide the robot to move toward the target point while avoiding static obstacles; reinforcement learning is used to handle local complex dynamic obstacle avoidance tasks. By combining the artificial potential field method, continuous learning and strategy optimization are carried out to improve the robot's performance in dynamic environments, and the robot can achieve accurate perception of obstacles and real-time path planning in complex environments.

[0183] The present invention integrates the temperature data of the infrared sensor integrated on the robot with the distance data of the laser radar, and forms a full range of environmental perception capabilities through complementary cooperation. The dynamic changes of obstacles are taken into account. The weights of the laser radar and infrared sensing are dynamically adjusted according to factors such as light and temperature in the environment to ensure that the robot can efficiently perceive and avoid obstacles in different environments.

[0184] The adaptive multi-scale planning mechanism of the present invention: an adaptive multi-scale regional planning mechanism is designed to seamlessly switch between global and local planning according to the different complexities of the environment, ensuring the efficiency and safety of path planning. An improved artificial potential field model is constructed based on infrared and laser radar data to enhance the dynamic adaptability of the potential field model.

[0185] The local decision optimization of reinforcement learning in the present invention: reinforcement learning is used to handle complex obstacle avoidance tasks in local environments. When the robot approaches an obstacle, the environment state input is updated in real time through fused sensor data. The reward mechanism of reinforcement learning is dynamically adjusted in combination with the artificial potential field model. Through the reinforcement learning method, the path planning strategy is continuously iterated and optimized to ensure that the robot maintains efficient obstacle avoidance capabilities in complex dynamic environments.

Claims

1. A robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning, characterized by: The following steps are involved: A. Perform sensor data collection and fusion; B. Conduct environmental complexity assessment, establish a multi-scale planning mechanism, and determine global and local planning based on switching conditions; C. Based on global planning, artificial potential field path planning is performed; D. Perform reinforcement learning path planning based on local planning; E. Determine whether the target point has been reached and return to step B until the target point has been reached.

2. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 1, characterized in that: Step A performs sensor data collection and fusion. The specific process is as follows: First, the infrared sensor collects the thermal radiation information of the surrounding obstacles, especially the temperature distribution of the obstacles; Then, the lidar provides distance information of obstacles and three-dimensional spatial data of the environment; Then, the reliability of the infrared sensor and lidar is determined based on the current ambient light and obstacle type conditions. Finally, the temperature information of the infrared sensor is combined with the distance data of the lidar through weighted fusion to form a unified environmental perception model.

3. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 2, characterized in that: In the fusion output process, the details are as follows: First, when the infrared sensor detects a high-temperature obstacle, it gives priority to avoiding such obstacles; the lidar plays a role in mainly detecting static objects and providing path planning data; Then, when the robot is operating in a low-light environment, the infrared sensor is given more weight, ensuring that the robot can still accurately identify obstacles and avoid danger even when it cannot rely on the lidar; Later, in an environment with better lighting, the weight of lidar data increases, and the data from the infrared sensor will be used as an auxiliary input to detect dynamic obstacles or obstacles with large temperature changes, thereby improving the comprehensiveness of perception and obstacle avoidance capabilities in complex environments.

4. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 1, characterized in that: Step B evaluates the complexity of the environment, establishes a multi-scale planning mechanism, and determines the global plan and local plan based on the switching conditions. The specific process is as follows: First, obtain the overall information of the current environment based on fusion perception; Then, the environment complexity assessment model is used to determine the distribution and density of obstacles in the environment and whether there are dynamic obstacles. Then, the global plan / local plan is determined based on the switching conditions.

5. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 4, characterized in that; The environmental complexity assessment model is as follows: Overall rating of environmental complexity: C complexity It is the weighted sum of the following key environmental parameters, specifically defined as: in: w1, w2, w3, and w4 are the weights of each key parameter, which are adjusted according to the specific application scenario; d density The obstacle density is calculated by the number of obstacles detected by the LiDAR and their distribution range. The more obstacles there are, the higher the density is and the more complex the environment is. d obs It is the shortest distance between the robot and the obstacle. The closer to the obstacle, the higher the complexity. It is directly obtained from the laser radar, and its reciprocal is used as a parameter for complexity evaluation. The shorter the distance, the larger the reciprocal, and the higher the complexity. v obs is the moving speed of the obstacle, which is collected by the laser radar; T obs is the temperature of the obstacle, collected by the infrared sensor; Based on fusion perception, the overall information of the current environment is obtained and the overall score of the complexity of the above environment is determined.

6. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 5, characterized in that: The switching condition in step B determines the switching between global planning and local planning by setting a threshold. When the indicator exceeds the threshold, the global planning is switched to the local planning, and vice versa.

7. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 1, characterized in that: Step C performs artificial potential field path planning based on global planning. The specific process is as follows: First, global planning is suitable for paths with fewer obstacles and longer distances; The robot then uses the gravitational field model to guide it to move quickly toward the target point, while using the repulsive field model to avoid static obstacles.

8. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 1, characterized in that: Step D performs reinforcement learning path planning based on local planning. The specific process is as follows: First, when the robot approaches an obstacle-dense area or a dynamic obstacle, it switches to local planning mode; Then, the robot's obstacle avoidance path is updated in real time through a high-precision local potential field model.

9. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 8, characterized in that: Solve the problem that the robot is trapped by the combined force of local obstacles in local planning and cannot continue to move to the target point. The details are as follows: In order to deal with the above-mentioned local optimal problem, the robot determines which areas have insufficient information through uncertainty estimation, and then guides it to actively explore these areas, thereby enhancing the comprehensiveness of path planning and preventing the robot from falling into local optimality. This is solved by combining reinforcement learning algorithms.

10. The robot path planning and obstacle avoidance method combining artificial potential field and reinforcement learning according to claim 1, characterized in that: The reinforcement learning algorithm uses the state function, action function, and reward function to make decisions and select the optimal action that the robot should take to avoid collision and reach the target point as quickly as possible.

Images