A machine learning-based method and apparatus for robot motion path planning

Abstract

This invention provides a machine learning-based robot motion path planning method and apparatus, relating to the field of robot path planning technology. The method includes: acquiring and storing the position information of obstacles and target points in the environment; constructing an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state; optimizing the artificial potential field using a variable neighborhood search algorithm, incorporating a reinforcement learning strategy; using nodes on the basic path generated by the artificial potential field as the initial starting points for ants in an ant colony algorithm, and obtaining the final path through multiple rounds of iterative optimization using ant pheromone release, transition probability selection, and pheromone evaporation update mechanisms; evaluating the optimized path and dynamically adjusting the path. This robot motion path planning method can cope with complex environmental changes and effectively improve the safety and efficiency of path planning.

Description

Technical Field

[0001] This invention relates to the field of robot path planning technology, and in particular to a robot motion path planning method and apparatus based on machine learning. Background Technology

[0002] Path planning for mobile robots (MR) is a crucial branch of mobile robot research and forms the foundation for their control. Depending on the degree of environmental information known, path planning is divided into global path planning based on known environmental information and local path planning based on unknown or partially known environmental information. With the rapid development of technology and the widespread application of robots, the demands on robots are increasing, especially in terms of intelligence. Autonomous path planning is a vital step in achieving robot intelligence; it refers to planning the optimal path for a robot to safely reach a designated target location from its starting point without collisions.

[0003] Currently, there are many commonly used global path planning methods for mobile robots, such as the grid method and the artificial potential field method. For the grid method, the required storage space increases dramatically as the space increases, and the decision-making speed decreases. The artificial potential field method (APF) calculates by assuming the existence of a virtual force field in space and simulating the motion of a particle in the potential field. The APF method has low computational cost and high real-time performance, but the robot may get stuck in local minima during its movement. These points are where the attraction and repulsion forces are in equilibrium, but they are not the target positions, causing the robot to get stuck in a local equilibrium point, like being trapped in a "trap," unable to continue moving towards the target. Other algorithms, such as the A* algorithm, particle swarm optimization, and genetic algorithms, can also be used for robot path planning, but they also have corresponding problems. Although there are many methods to solve the path planning problem of mobile robots, due to the limitations of map scale and the number of obstacles, a single algorithm is difficult to cope with complex environmental changes. Summary of the Invention

[0004] To address the limitations of existing robot path planning technologies, such as map scale and the number of obstacles, which make it difficult for a single algorithm to handle complex environmental changes, this invention provides a robot motion path planning method and apparatus based on machine learning.

[0005] On the one hand, a machine learning-based robot motion path planning method is provided, which is implemented by a machine learning-based robot motion path planning device. The method includes:

[0006] Step S1, Environment Modeling and Initialization: The robot uses its onboard sensors to perceive the working environment, obtains the location information of obstacles and target points, converts the location information of obstacles and target points into a digital map for storage, and initializes the relevant parameters of the artificial potential field algorithm and ant colony algorithm.

[0007] Step S2: Use the robot motion model and MPC to predict the robot's motion state over a future period of time, and construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state.

[0008] Step S3: Optimize the artificial potential field using a variable neighborhood search algorithm, and introduce a reinforcement learning strategy into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field.

[0009] Step S4: Using the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization via the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone evaporation update mechanism.

[0010] Step S5: Evaluate the optimized final path regarding path length, path smoothness, safety, convergence speed, and adaptability. Monitor environmental changes in real time during robot movement and dynamically adjust the path according to the changes.

[0011] Optionally, the step of converting the location information of obstacles and the location information of target points into a digital map for storage includes:

[0012] The grid method is used to divide the robot's movement space into grids of a certain size. Obstacles are assigned a value of 1, and passable grids are assigned a value of 0. The grids are encoded using sequential encoding, so that the position information of obstacles and the position information of target points are transformed into a 0-1 matrix model.

[0013] Optionally, the step of constructing an artificial potential based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state specifically involves:

[0014] Define the gravitational field function that introduces relative velocity as follows: The repulsive field function is The total potential field in which the robot is located is Then we have:

[0015] ;

[0016] ;

[0017] Where q is the robot's current position and v is the robot's speed. It is the target speed. The gravitational coefficient, The target point location, It is a weighting coefficient used to balance the effects of velocity and position. The speed of the obstacle's movement. The relative speed between the robot and the obstacle. Let d(q) be the repulsive force coefficient, and d(q) be the distance between the robot and the obstacle. For threshold distance, It is a coefficient used to adjust the intensity of repulsion based on relative velocity.

[0018] Optionally, the optimization objective function of the MPC takes into account factors such as the surrounding environment and the movement distance, and incorporates kinematic constraints, dynamic constraints, and obstacle constraints.

[0019] Optionally, optimizing the artificial potential field using a variable neighborhood search algorithm includes:

[0020] The neighborhood structure of the variable neighborhood search algorithm is constructed, and six neighborhoods are defined as: up, down, left, right, far from the obstacle, and outside the exclusion range of the obstacle; among them, the six neighborhood actions represent the generation of sub-target points in six directions, with the size being the step size;

[0021] Construct an evaluation function for the variable neighborhood search algorithm. The evaluation function is as follows: ,in, , , For the weighting coefficients, satisfying > > , The size of the total potential field for the current sub-target. The newly generated path length for the sub-target. The shortest distance from the obstacle to the two sub-targets is the distance between them. The sub-targets are local key points in the artificial potential field where the gravitational gradient changes. , For gravitational potential field, It is a repulsive potential field;

[0022] The evaluation function compares the scores of the six neighborhoods in the neighborhood structure, selects the one with the best score as the local optimum, and then perturbs it. The perturbation object is the coordinates of the sub-target point of the current local optimum. The perturbation method is to randomly shift the position of the sub-target point within the neighborhood step size. After the perturbation, the score is recalculated and a new local optimum is selected again. This process is repeated iteratively to optimize the sub-target point until the robot jumps out of the local trap area driven by the attraction of the sub-target point.

[0023] Optionally, the introduction of a reinforcement learning strategy into the variable neighborhood search algorithm includes:

[0024] The six neighborhoods are simultaneously used as the reward matrix Q for both state s and action a to update the feedback from the environment.

[0025] The improvement is introduced by changing the environment. Adaptive rewards and penalties are given based on the generated solution. The following four cases are defined: the reward for a local optimum is R1, the reward for improving the environment is R2, the reward for both local optimum and improving the environment is R3, and the reward for neither of the above three cases is R4.

[0026] Optionally, the transition probability is calculated using a roulette wheel method, specifically:

[0027] ;

[0028] in, It is a heuristic function, representing the expected degree of an ant moving from grid i to grid j at time t. This represents the expected degree of an ant moving from grid i to grid s at time t. It represents the pheromone concentration, indicating the pheromone concentration between grids i and j at time t. This represents the pheromone concentration between two grid cells, i and s, at time t. , represents the weights of pheromones and heuristic information in the transition probability, respectively; allowed represents the grid that the robot can move to in the next step.

[0029] Optionally, the pheromone evaporation and renewal mechanism specifically includes: ,in, This represents the pheromone concentration between grids i and j. Let represent the pheromone concentration left by the k-th ant along its path between grids i and j. Let M be the volatile factor, and M be the total number of ants in the robot's abstraction.

[0030] Optionally, the multi-round iterative optimization to obtain the final path includes:

[0031] The robot is placed at the starting point, and it selects the next candidate point to move to based on the pheromone concentration and heuristic information from the pheromone release mechanism of ants.

[0032] Based on the path traversed by the ants and its length, a pheromone evaporation and update mechanism is implemented for the pheromones along the path.

[0033] In the process of ant colony search simulated by the robot, a potential field force is introduced to guide the ants' search direction. Based on the potential field information of the current ant's location, the direction of the potential field force on the ant is calculated, so that the ant tends to move towards the target point during the search process, while avoiding approaching obstacles.

[0034] After each iteration, it is determined whether the convergence condition is met. If the convergence condition is not met, the next iteration continues. If the convergence condition is met, the iteration stops and the next step is performed. The convergence condition is that the maximum number of iterations is reached or the optimal path length changes less than a set threshold in multiple consecutive iterations.

[0035] On the other hand, a machine learning-based robot motion path planning device is provided, which is applied to the machine learning-based robot motion path planning method. The device includes:

[0036] The initialization module uses the sensors on the robot to perceive the working environment, obtain the location information of obstacles and target points in the environment, convert the location information of obstacles and target points into digital map format for storage, and initialize the relevant parameters of artificial potential field algorithm and ant colony algorithm.

[0037] The module constructs an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state, using the robot motion model and MPC to predict the robot's motion state over a future period.

[0038] The optimization module optimizes the artificial potential field using a variable neighborhood search algorithm, and introduces a reinforcement learning strategy into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field.

[0039] The iterative module uses nodes on the basic path as the initial starting point for ants in the ant colony algorithm. Through multiple rounds of iterative optimization, the final path is obtained by using the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone evaporation update mechanism.

[0040] The adjustment module evaluates the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. It also monitors environmental changes in real time during robot movement and dynamically adjusts the path accordingly.

[0041] On the other hand, a machine learning-based robot motion path planning device is provided, the machine learning-based robot motion path planning device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the above-described machine learning-based robot motion path planning methods is implemented.

[0042] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described machine learning-based robot motion path planning methods.

[0043] The beneficial technical effects of this invention are as follows:

[0044] This invention effectively combines the improved artificial potential field method with model prediction planning and control, and applies it to the construction of artificial potential fields in robot path planning. It features high control speed, high safety, and high stability. At the same time, introducing relative velocity during the construction of artificial potential fields can better adapt to dynamic environments, predict possible collisions in advance, and guide the movement path of the agent more rationally, thereby effectively improving the safety and efficiency of path planning.

[0045] This invention combines the ant colony algorithm with the artificial potential field method. By utilizing the strong global search capability of the ant colony algorithm, the problem of the artificial potential field method being prone to getting trapped in local minima is optimized. When the artificial potential field method causes the robot to get trapped in a local minimum between obstacles, the ant colony algorithm can use the random movement characteristics of ants to find a new path direction and guide the robot to jump out of the local optimum.

[0046] In dynamic environments, the artificial potential field method may not update the potential field in a timely manner. Ant colony algorithm can readjust the ants' search path according to environmental changes. By updating the pheromone concentration and changing the path selection strategy, the robot can better avoid dynamic obstacles. At the same time, in the process of finding a path, the ant colony algorithm can comprehensively consider factors such as path length and path smoothness. When combined with the artificial potential field method, it can generate a smoother path while avoiding collisions (an advantage of the artificial potential field method).

[0047] Ant colony optimization (ACO) has strong global search capabilities, but it also suffers from slow convergence, susceptibility to local optima, and premature convergence. Reinforcement learning, on the other hand, allows agents (ant colonies) to escape local optima through continuous trial and error and learning, adjusting their strategies based on environmental feedback. Combining ACO and reinforcement learning enhances search capabilities, enabling agents to adjust their behavior strategies in a timely manner according to dynamic environmental changes and find better global solutions. Furthermore, information such as pheromone concentration from ACO can serve as state features in reinforcement learning, helping it converge to the optimal strategy more quickly. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a machine learning-based robot motion path planning method provided in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the neighborhood layout structure of the VNS algorithm used in a machine learning-based robot motion path planning method provided in an embodiment of the present invention, wherein (a) is searching for sub-targets in the upward direction, (b) is searching for sub-targets in the downward direction, (c) is searching for sub-targets in the left direction, (d) is searching for sub-targets in the right direction, (e) is searching for sub-target points in the direction away from obstacles, and (f) is searching for sub-target points outside the exclusion range of obstacles;

[0051] Figure 3 A flowchart illustrating the learning feedback strategy of a machine learning-based robot motion path planning method provided in an embodiment of the present invention;

[0052] Figure 4 This is a structural block diagram of a robot motion path planning device based on machine learning provided in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of a robot motion path planning device based on machine learning provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0056] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0057] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0058] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0059] This invention provides a machine learning-based robot motion path planning method and apparatus. The method can be implemented by a machine learning-based robot motion path planning device, which can be a terminal or a server. The flow of a machine learning-based robot motion path planning method is as follows: Figure 1 As shown, the processing flow of the method includes the following steps:

[0060] Step S1: The robot uses its onboard sensors to perceive the working environment, acquiring the location information of obstacles and target points. This information is then converted into a digital map and stored. Simultaneously, the parameters for the artificial potential field algorithm and the ant colony algorithm are initialized. Here, "target point" refers to the desired location for the robot's movement task. In the context of robot path planning, this is the specific coordinates the robot needs to reach. For example, in a cleaning robot scenario, it might be the cleaning endpoint of a room; in a delivery robot scenario, it might be the designated delivery location for goods. This is also the destination that the path planning aims to guide the robot to.

[0061] In one feasible implementation, in step S1, the sensors mounted on the robot perceive the working environment, including:

[0062] LiDAR: By emitting a laser beam and measuring the time of the reflected light, it determines the distance to surrounding objects. It can quickly and accurately scan the contour information of obstacles in all directions around the robot, forming point cloud data. Each point in the point cloud contains information such as its three-dimensional coordinate position in space.

[0063] Visual sensors (cameras): These sensors capture images of the environment and, based on computer vision algorithms, can identify the outlines and features of objects in the images. By analyzing images from different perspectives and combining them with relevant visual ranging techniques (such as using the principle of binocular vision to estimate distance through parallax), the position of objects in space can be indirectly obtained.

[0064] Ultrasonic sensor: It emits ultrasonic pulses and receives reflected waves, and calculates the distance to obstacles based on the time difference.

[0065] The data collected by the sensors is then preprocessed. Sensor data often contains noise interference; for example, LiDAR point cloud data may contain some outliers. Filtering algorithms, such as mean filtering and median filtering, can be used to remove these noise points, improving data quality and allowing subsequent analysis to be based on more accurate data. Furthermore, data from different sensors may be based on their own coordinate systems; for example, camera image coordinates and LiDAR 3D coordinates differ. Appropriate coordinate transformation matrices are needed to unify them to the robot's global coordinate system, facilitating fusion and subsequent map building operations.

[0066] The robot acquires the location information of obstacles and target points in the environment through sensors and converts it into a digital map for storage. Specifically, a grid method is used to transform the robot's motion environment into a 0-1 matrix model. The robot's movement space is divided into grids of a certain size. Obstacles are assigned a value of 1, and passable grids are assigned a value of 0. The grids are encoded using sequential encoding. As the robot moves and continuously collects new data, the map information needs to be updated in real time. For example, if new obstacles are discovered or previously marked areas no longer have obstacles, the state information of the corresponding grids or topological node edges in the map needs to be modified accordingly. The size of the grid needs to be determined based on the actual application scenario and accuracy requirements of the robot.

[0067] After establishing the digital map, the relevant parameters of the artificial potential field algorithm and the ant colony algorithm are initialized simultaneously. The relevant parameters of the artificial potential field algorithm mainly include the attraction coefficient, repulsion coefficient, obstacle influence radius, and step size. The relevant parameters of the ant colony algorithm include the number of ants, pheromone evaporation coefficient, pheromone heuristic factor, and maximum number of iterations.

[0068] Step S2: Predict the robot's motion state over a future period using the robot motion model and MPC (Model Predictive Control). Construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state.

[0069] In step S2, the robot's motion state is predicted over a future period using the robot motion model and MPC. The predicted robot motion state specifically includes position (such as coordinate information), velocity, acceleration, and motion-related parameters such as robot attitude (such as heading angle) and angular velocity. The specific implementation method is as follows:

[0070] A robot's motion model can generally be represented by the following equations:

[0071]

[0072] in( , () represents the robot's planar position at time k. It is the robot's heading angle. It is the robot's linear velocity. It is the robot's angular velocity. It is the time step.

[0073] Model-based optimization control (MPC) is a model-based optimization control strategy that determines the optimal control input at the current time by solving a finite-time optimization problem in each control cycle. Its main steps include:

[0074] Predictive Model: Utilizing the robot's motion model, based on the current state and the hypothesized future control input sequence, predict the robot's state sequence over a future period. Specifically, using the robot's motion model, predict the robot's state sequence from the current moment, at fixed time step intervals Δt (e.g., 0.1 seconds), for N consecutive steps (e.g., 10 steps, corresponding to the time range from the current moment to the current moment + N×Δt). This sequence is a "pre-simulation of actual motion" derived from the model, and needs to be compared and optimized with the "desired state sequence (human-defined ideal trajectory)" through an objective function.

[0075] Objective function setting: Define an objective function to measure the difference between the predicted state sequence and the desired state sequence. For example, the objective function may include factors such as the distance between the robot and the target position, and the smoothness of the robot's motion.

[0076] Constraints to consider: Consider the robot's kinematic constraints (such as speed and acceleration limits), dynamic constraints (such as motor torque limits), and obstacle constraints (such as collision avoidance).

[0077] Optimization solution: Solve the control input sequence that minimizes the objective function under the constraints by using optimization algorithms (such as the quadratic programming algorithm QP).

[0078] Control implementation: Apply the first control input from the optimized control input sequence to the robot, and then repeat the above process in the next control cycle.

[0079] Meanwhile, the optimization objective function of MPC takes into account various factors such as the surrounding environment and movement distance, and incorporates kinematic constraints, dynamic constraints and obstacle constraints.

[0080] The objective function is typically a cost function that needs to be minimized. It takes into account the various factors mentioned above, and a general form can be expressed as: ,in, This represents the cost associated with the distance traveled. This indicates the cost associated with the surrounding environment. This represents the cost associated with motion smoothness.

[0081] Cost of travel distance Represented as: ,in, It is the distance cost weight. It is the robot's first Step position, It is the first The expected reference position is N, where N is the number of prediction steps.

[0082] Represented as: ,in, Here, M represents the environmental cost weighting coefficient, and M represents the number of obstacles. For the robot in the At each step, the distance to the i-th obstacle; P is the number of special regions in the environment. Let J be the penalty / reward coefficient for the j-th special region. For the indicator function, when the robot is in the... When the step is in the j-th special region, The value is 1; when not within this region, the value is 0.

[0083] The expression is: ,in, It is a smoothness weight (e.g., 0.3); It is the first Stepping speed, It's angular velocity. It is the weight of angular velocity change (e.g., 0.5), used to balance the smoothness of linear velocity and angular velocity.

[0084] Environmental factors are crucial for the robot's safe and efficient movement. For example, the robot needs to consider its distance from surrounding obstacles to avoid collisions. It may also need to consider special areas in the environment, such as hazardous areas and priority passage zones.

[0085] The motion distance factor is as follows: Robots typically want to reach the target location via the shortest or most economical path. The motion distance can be measured by the Euclidean distance between the robot's current position and the target position or other path length metrics.

[0086] Kinematic constraints include position and attitude constraints, velocity and acceleration constraints; dynamic constraints include force and torque constraints, capability constraints; obstacle constraints include static obstacle constraints and dynamic obstacle constraints. By setting these constraints, the safety, stability and reliability of the robot in the path planning process are ensured.

[0087] Optionally, in step S2, an artificial potential field is constructed based on the robot's current position and speed, the target point's position and speed, obstacle information, and predicted state, specifically as follows:

[0088] Define the gravitational field function that introduces relative velocity as follows: The repulsive field function is The total potential field in which the robot is located is Then we have:

[0089]

[0090]

[0091] Where q is the robot's current position and v is the robot's speed. It is the target speed. The gravitational coefficient, The target point location, It is a weighting coefficient used to balance the effects of velocity and position. The speed of the obstacle's movement. The relative speed of the robot to the obstacle. Let d(q) be the repulsive force coefficient, and d(q) be the distance between the robot and the obstacle. For threshold distance, It is a coefficient used to adjust the intensity of repulsion based on relative velocity.

[0092] Traditional artificial potential field algorithms primarily consider the distances between the robot and the target point, as well as the distances to obstacles, to construct gravitational and repulsive potential fields. This invention, based on a relative velocity artificial potential field, further introduces the factor of relative velocity into the traditional algorithm. This aims to allow the robot to not only consider positional relationships during movement but also dynamically adjust the forces acting on it based on the velocity differences between itself, the target, and obstacles. This results in more rational trajectory planning, avoiding collisions and efficiently approaching the target. However, one of the main problems with artificial potential field methods in local path planning is the local optimum problem. When obstacles exist near the target point, the robot may become trapped in a local optimum under the combined force of repulsive and gravitational forces, preventing it from moving forward. For example, when the repulsive force from the obstacle and the gravitational force from the target reach equilibrium, the robot may stall at a non-globally optimal position, failing to reach the target point. By introducing the factor of relative velocity, the forces acting on a robot no longer depend solely on its position. Taking the robot's position between an obstacle and a target as an example, even if the attractive and repulsive forces are balanced at a certain position, an additional force will be generated because the relative velocity between the robot and the obstacle (or target) is not zero. For example, the above... This will significantly increase the repulsive force, disrupt the original balance, and prompt the robot to change its path to avoid getting stuck in a local optimum.

[0093] Step S3: The artificial potential field is optimized by introducing a variable neighborhood search algorithm with reinforcement learning strategy, and a basic path is generated based on the optimized artificial potential field.

[0094] Optionally, in step S3, the artificial potential field is optimized using a variable neighborhood search algorithm, specifically as follows:

[0095] The neighborhood structure of the variable neighborhood search algorithm is constructed, and six neighborhoods are defined as: up, down, left, right, far from the obstacle, and outside the exclusion range of the obstacle; among them, the six neighborhood actions represent the generation of sub-target points in six directions, with the size being the step size;

[0096] The evaluation function for the variable neighborhood search algorithm is constructed as follows: ,in, , , Let be the weighting coefficient, satisfying > > , The size of the total potential field for the current sub-target. The newly generated path length for the sub-target. The shortest distance from the obstacle to the two sub-targets is the distance between them. The sub-targets are local key points in the artificial potential field where the gravitational gradient changes. , For gravitational potential field, It is a repulsive potential field;

[0097] The scores of six neighborhoods in the neighborhood structure are compared using an evaluation function. The algorithm calculates and selects the solution with the best score as the local optimum. Then, it perturbs the sub-target point coordinates of the current local optimum. The perturbation method is to randomly shift the position of the sub-target point within the neighborhood step size (such as adjusting the coordinates ± step size). After the perturbation, the score is recalculated, and a new local optimum is selected again. This process is repeated iteratively to optimize the sub-target point until the robot jumps out of the local trap area under the attraction of the sub-target point and safely completes the path planning task.

[0098] In one feasible implementation, such as Figure 2 As shown, the path generated by the small circle in the middle is the original path when the robot encounters a local minimum. The path generated by the four-pointed star is the path that guides the robot to travel using the sub-target points generated by the Variable Neighborhood Search (VNS) algorithm with reinforcement learning strategy, which is the optimized path. The two circular objects at the top and bottom and the triangular area in the upper right corner are obstacles. The circle formed by the dashed lines around the circular objects is the repulsive force range of the obstacles. Figure 2 In the diagram, (a), (b), (c), and (d) represent searching for sub-targets in the upward, downward, leftward, and rightward directions, respectively. Figure 2 In the diagram, (e) represents searching for sub-target points in a direction away from the obstacle; Figure 2 In the equation (f), the search for sub-target points is conducted outside the repulsion range of the obstacle. At this time, the area where the four-pointed star is located within the repulsion range is within the repulsion range of the obstacle.

[0099] Meanwhile, this invention introduces a reinforcement learning strategy into the variable neighborhood search algorithm, specifically:

[0100] The feedback from the environment is updated by using six neighborhoods as the reward matrix Q for both state s and action a. The action selection strategy is as follows:

[0101] ;

[0102] in, It is a random probability value. This is the critical value used to determine the action random(a) and the maximum reward action max(Q(s,a)). The smaller the variable neighborhood, the faster the convergence speed of the algorithm.

[0103] The improvement is introduced by the change of environment. Adaptive rewards and penalties are given according to the generated solution. The following four cases are defined: the reward for a local optimum is R1, the reward for improving the environment is R2, the reward for both local optimum and improving the environment is R3, and the reward for neither of the above three cases is R4.

[0104] like Figure 3 As shown, a robot path planning process based on the fusion of Artificial Potential Field (APF) and reinforcement learning is presented. The upper layer uses reinforcement learning to learn actions, set rewards, and update them, driving decision optimization. The lower layer generates a basic path based on APF. If it gets stuck in a local optimum or other situations, the feedback is sent to the upper layer, which then adjusts the actions through reinforcement learning. This process is iterated until a globally optimal solution that balances path length, smoothness, and safety is finally output. This achieves adaptive path planning in dynamic environments, overcoming the shortcomings of single algorithms and improving the intelligence and environmental adaptability of the planning. The learning and environmental feedback strategy is adaptive and reactive because updating the reward value in a single step can improve the selection of the local search neighborhood. If there is no improvement, a penalty is imposed. The reward for environmental change improvement is defined as follows:

[0105]

[0106] In the formula, R1 and R4 are constants, and R4≤0≤R1≤R2≤R3. C1 and C2 represent the local optimal solution and the environmental improvement, respectively, obtained by comparing the neighborhood solution evaluation function and the path safety index.

[0107] ;

[0108] definition In the formula, (x1, y1) and (x2, y2) are the position coordinates of the first and last keypoints segmented when the robot first encounters a local minimum, respectively. The equation of the straight line formed by the two key points mentioned above. It is a constant and R1 < <R3, , These are the last sub-objective of the previous solution (i.e., the locally optimal sub-objective path solution of the previous iteration) and the last sub-objective of the current solution, respectively. The algorithm systematically expands the search range by improving the environment, avoiding repeated searches in already searched areas, and increasing the probability of escaping local optima.

[0109] Step S4: Using the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization via the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone evaporation update mechanism.

[0110] In one feasible implementation, in step S4, the transition probability is calculated using a roulette wheel method, and the calculation method is as follows:

[0111] ;

[0112] in, It is a heuristic function, representing the expected degree of an ant moving from grid i to grid j at time t. This represents the expected degree of an ant moving from grid i to grid s at time t. It represents the pheromone concentration, indicating the pheromone concentration between grids i and j at time t. This represents the pheromone concentration between two grid cells, i and s, at time t. , represents the weights of pheromones and heuristic information in the transition probability, respectively; allowed represents the grid that the robot can move to in the next step.

[0113] In step S4, the final path is obtained through multiple rounds of iterative optimization, specifically as follows:

[0114] The robot is placed at the starting point. Based on the pheromone concentration from the ants' pheromone release mechanism and heuristic information, the robot selects the next candidate point to move to. This heuristic information can comprehensively consider distance and potential field information, among others. During the ants' movement, the taboo list is continuously updated to record the candidate nodes already visited, preventing repeated visits. When an ant reaches the destination or the maximum number of steps, a foraging cycle is completed, and the path traversed by each ant and its length are recorded.

[0115] Based on the paths traversed by ants and their lengths, a pheromone evaporation and renewal mechanism is implemented along these paths. A common method involves adding a certain amount of pheromone to the paths traversed by ants, while simultaneously evaporating a portion of the pheromone according to a evaporation coefficient, mimicking the natural dissipation process of pheromones. This gradually increases the pheromone concentration on the better paths, guiding subsequent ants to choose these paths more readily.

[0116] In the robot-simulated ant colony search process, a potential field force is introduced to guide the ants' search direction. Based on the potential field information of the current ant's location, the direction of the potential field force on the ant is calculated, so that the ant tends to move towards the target point during the search process, while avoiding approaching obstacles.

[0117] After each iteration, it is determined whether the convergence condition is met. If the convergence condition is not met, the next iteration continues. If the convergence condition is met, the final path is obtained and the next step is taken. The convergence condition is to reach the maximum number of iterations or the optimal path length changes less than a set threshold in multiple consecutive iterations.

[0118] The pheromone evaporation and renewal mechanism mentioned above is as follows: ,in, This represents the pheromone concentration between grids i and j. For the first The concentration of pheromones left by an ant along its flight path between grids i and j. Let M be the volatile factor, and M be the total number of ants (abstracted by the robot). After the robot moves between grids, the pheromone concentration left along the path each time is: ,in, is the fitness value of the robot path, and Q is the weight coefficient;

[0119] The heuristic message expression is:

[0120] ;

[0121] in, The potential force coefficient, For potential field gravity, For the robot's transfer direction and the angle of the potential force, This is the potential force attenuation coefficient, which causes the artificial potential field to gradually decrease during the iteration process; To construct heuristics related to the path at time t, This represents the Euclidean distance from the robot's current position to the j-th target position (or feasible grid cell). It is the weighting coefficient of heuristic information.

[0122] Potential field gravity The calculation method is as follows: ,in, It is the gradient of the potential function with respect to the position. It is the partial derivative of the potential field function with respect to the velocity;

[0123] The potential force attenuation coefficient is: ,in, N represents the current iteration number of the algorithm, and N represents the total number of iterations of the algorithm.

[0124] Step S5: Evaluate the optimized final path regarding path length, path smoothness, safety, convergence speed, and adaptability. Monitor environmental changes in real time during robot movement and dynamically adjust the path according to the changes.

[0125] In one feasible implementation, step S5 evaluates the optimized path regarding path length, path smoothness, safety, convergence speed, and adaptability, specifically as follows:

[0126] Path length evaluation includes quantitative and qualitative analysis:

[0127] Quantitative analysis: Compare the path length values ​​before and after optimization. Using robot path planning software or tools for recording motion trajectories, obtain the length data of the original path and the optimized path, and calculate the proportion of length reduction. For complex 3D paths or paths with multiple sub-paths, decompose the path into multiple line segments, calculate the length of each segment separately, and sum them for comparison.

[0128] Qualitative analysis: Determine the effectiveness of path length optimization based on the robot's task type. For example, for material handling robots, a shorter path means more materials can be moved per unit time, improving work efficiency; for inspection robots, a shorter path allows inspection tasks to be completed faster, reducing delays in fault detection.

[0129] Considering the relationship between a robot's energy consumption and path length, generally speaking, the shorter the path, the less energy the robot consumes. For battery-powered robots, this directly affects their endurance and work cycle.

[0130] Path smoothness evaluation, including curvature reduction analysis and steering angle analysis:

[0131] Curvature analysis: Calculate the curvature at each point on the path. Using mathematical methods, such as calculating the second derivative based on the path's parametric equations, curvature information is obtained. The smaller the curvature, the smoother the path at that point. By plotting a curve showing the change in curvature as a function of path length, the change in path smoothness can be visually observed. The maximum, minimum, and average values ​​of curvature are statistically analyzed, and these values ​​are compared before and after optimization. If the maximum curvature decreases and the average curvature approaches zero after optimization, it indicates that the path is smoother.

[0132] Steering Angle Analysis: Record the robot's steering angle at each turning point on the path. Paths with frequent large-angle turns exhibit poor smoothness. Compare the distribution of steering angles before and after optimization. If the number of large-angle turns decreases and the variance of the steering angles decreases after optimization, the path is smoother. Observe the continuity of steering angles. A smooth path should change gradually during turns, rather than abruptly. This can be evaluated by analyzing the rate of change of the steering angles.

[0133] Actual exercise effect evaluation:

[0134] Observe the robot's movement in the experimental environment. If the robot does not exhibit obvious shaking, swaying, or sudden stops and starts during movement, it indicates good path smoothness. For robots with loads, such as handling robots, observe the stability of the load during movement. A smooth path can reduce load swaying and lower the risk of damage to goods.

[0135] Safety assessment includes obstacle avoidance capability, hazardous area identification and avoidance, and emergency response capability.

[0136] Obstacle avoidance capability: Construct a test environment containing various obstacles to simulate the obstacles that the robot may encounter in real-world scenarios, such as static obstacles (walls, equipment, etc.) and dynamic obstacles (pedestrians, other mobile robots, etc.). Record the robot's reaction distance when encountering an obstacle, that is, the distance from detecting the obstacle to taking avoidance measures. The shorter the reaction distance, the higher the robot's safety, but it is also necessary to consider avoiding misjudgments that lead to frequent path changes. Statistically analyze the number of times the robot successfully avoids obstacles and the number of collisions. The higher the success rate, the more reliable the path planning is in terms of safety.

[0137] Hazard identification and avoidance: Define hazardous areas, such as high temperature, high pressure, and radiation areas. Use the robot's sensors to detect whether it can accurately identify these hazardous areas and plan a path in advance to avoid them. For hazardous areas with unclear safety boundaries, assess whether the robot can maintain a sufficient safe distance to avoid entering the hazardous area due to errors or environmental interference.

[0138] Emergency response capability: Simulate emergency situations, such as sudden obstacles, communication interruptions, sensor malfunctions, etc., and observe whether the robot has preset emergency strategies, such as emergency braking, waiting in place for recovery, or switching to an alternative path planning mode, to ensure the safety of itself and the surrounding environment.

[0139] Convergence speed evaluation includes algorithm iteration count, time measurement, and convergence stability assessment:

[0140] Algorithm Iteration Count Statistics: During the path optimization algorithm's execution, intermediate results for each iteration are recorded, and the number of iterations required from the initial path to convergence to the final optimized path is counted. The fewer the number of iterations, the faster the convergence speed. Tests are conducted for environments of different scales and complexities, and the changes in the number of iterations are observed. If the number of iterations remains within a reasonable range even in complex scenarios, it indicates that the algorithm's convergence speed has good robustness.

[0141] Time Measurement: A high-precision timer is used to record the time taken from starting the path optimization algorithm to obtaining the final optimized path. The shorter the time, the faster the convergence speed. Considering the impact of factors such as computer performance on time measurement, comparative tests are conducted on the same hardware platform to ensure the accuracy of the evaluation results.

[0142] Convergence stability assessment: Run the path optimization algorithm multiple times and observe whether the final path reached each time is consistent. If the convergence result is stable, it indicates that the algorithm not only converges quickly but also has high reliability; if the convergence result fluctuates greatly, further analysis is needed, which may be due to the algorithm itself having local optima or interference from environmental noise and other factors.

[0143] Adaptability evaluation includes testing in different environments, task diversity testing, and interference resistance testing.

[0144] Different Environment Testing: Place the robot in various environments, such as indoor office environments, warehouse environments, and complex outdoor terrain environments, and observe whether the robot can successfully optimize a path suitable for each environment based on its characteristics, such as spatial layout, obstacle distribution, and terrain features. For changing environments, such as changes in the position of indoor furniture or changes in terrain caused by outdoor weather, evaluate whether the robot can quickly adapt and re-optimize its path.

[0145] Task diversity testing: Set various different task objectives, such as patrolling, transportation, and search, and observe whether the robot can adjust its path planning strategy according to the task requirements when performing different tasks. For example, patrolling tasks may require covering a wider area, while transportation tasks may focus more on path length and load stability. For changes in task requirements, such as adjustments to task priorities or the addition of new tasks, evaluate whether the robot can promptly change its path planning to adapt to the new task requirements.

[0146] Anti-interference capability test: Different types of interference factors are introduced, such as electromagnetic interference, noise interference, and changes in lighting, to observe whether the robot can work normally under these interferences and whether path optimization is affected. If the robot can still maintain good path planning ability under a certain degree of interference, it indicates that its adaptability is strong.

[0147] Optionally, in step S5, the robot monitors environmental changes in real time and dynamically adjusts the path according to the changes during the robot's movement. Specifically, as the robot moves along the planned path, the lidar and camera continuously monitor the environment. When a new obstacle is detected, the grid map is immediately updated, the grid with the obstacle is re-marked, and then the above steps of constructing the basic path guidance using the artificial potential field method, optimizing the path using the ant colony algorithm, and evaluating the path are executed again to dynamically adjust the robot's movement path.

[0148] In summary, this invention effectively combines the improved artificial potential field method with model predictive planning and control for the construction of artificial potential fields in robot path planning, exhibiting characteristics such as high control speed, high safety, and high stability. Furthermore, introducing relative velocity during artificial potential field construction allows for a more accurate description of the dynamic relationship between the agent and the target or obstacle. By considering relative velocity, the artificial potential field can better adapt to dynamic environments, predict potential collisions in advance, and more rationally guide the agent's movement path, thereby effectively improving the safety and efficiency of path planning.

[0149] Secondly, the ant colony algorithm is combined with the artificial potential field method. The strong global search capability of the ant colony algorithm is leveraged to address the problem of the artificial potential field method easily getting trapped in local minima. The ant colony algorithm can explore a wider area through the random search of individual ants and the guidance of pheromones. In robot path planning, when the artificial potential field method causes the robot to get stuck in a local minimum between obstacles, the ant colony algorithm can utilize the random movement characteristics of the ants to find a new path direction, guiding the robot out of the local optimum. In dynamic environments, the artificial potential field method may not update the potential field in a timely manner. The ant colony algorithm can readjust the ant search path according to environmental changes, and by updating the pheromone concentration, it can change the path selection strategy, enabling the robot to better avoid dynamic obstacles. Simultaneously, during the path finding process, the ant colony algorithm can comprehensively consider factors such as path length and path smoothness. Combined with the artificial potential field method, it can generate smoother paths while avoiding collisions (an advantage of the artificial potential field method).

[0150] Finally, while the ant colony algorithm possesses strong global search capabilities and can guide the artificial potential field method to solve local optima, it also suffers from slow convergence, susceptibility to local optima, and premature convergence. Its performance is poor in unknown environments and it cannot cope with complex and changing conditions. This invention introduces reinforcement learning, which, through continuous trial and error and learning, adjusts its strategy based on environmental feedback, guiding the agent (ant colony) out of local optima. The combination of these two approaches enhances search capabilities, enabling the agent to adjust its behavior strategy in a timely manner according to dynamic environmental changes and find a better global solution. Simultaneously, information such as pheromone concentration from the ant colony algorithm can serve as state features for reinforcement learning, helping it converge to the optimal strategy more quickly.

[0151] Figure 4 This is a block diagram of a robot motion path planning device based on machine learning, provided in an embodiment of the present invention. This device is used in a machine learning-based robot motion path planning method. (Refer to...) Figure 4 The device includes an initialization module 410, a construction module 420, an optimization module 430, an iteration module 440, and an adjustment module 450. Wherein:

[0152] The initialization module 410 is used to perceive the working environment through the sensors on the robot, obtain the location information of obstacles and target points in the environment, convert the location information of obstacles and target points into digital map form for storage, and initialize the relevant parameters of artificial potential field algorithm and ant colony algorithm.

[0153] The construction module 420 is used to predict the robot's motion state over a future period of time using the robot motion model and MPC, and to construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and the robot's motion state.

[0154] The optimization module 430 is used to optimize the artificial potential field through a variable neighborhood search algorithm, and at the same time, a reinforcement learning strategy is introduced into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field.

[0155] The iteration module 440 is used to take the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm, and obtain the final path through multiple rounds of iterative optimization by the ant pheromone release mechanism, the transfer probability selection mechanism and the pheromone evaporation update mechanism.

[0156] The adjustment module 450 is used to evaluate the optimized path in terms of path length, path smoothness, safety, convergence speed, and adaptability. It monitors environmental changes in real time during robot movement and dynamically adjusts the path according to the changes.

[0157] Figure 5 This is a schematic diagram of the structure of a machine learning-based robot motion path planning device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, a machine learning-based robot motion path planning device may include the above-mentioned... Figure 4 The illustrated robot motion path planning device is based on machine learning. Optionally, the machine learning-based robot motion path planning device 510 may include a first processor 2001.

[0158] Optionally, the machine learning-based robot motion path planning device 510 may also include a memory 2002 and a transceiver 2003.

[0159] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0160] The following is combined Figure 5 A detailed introduction to each component of the machine learning-based robot motion path planning device 510 is provided below:

[0161] The first processor 2001 is the control center of the machine learning-based robot motion path planning device 510. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0162] Optionally, the first processor 2001 can perform various functions of the machine learning-based robot motion path planning device 510 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0163] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0164] In a specific implementation, as one example, the machine learning-based robot motion path planning device 510 may also include multiple processors, for example... Figure 5 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0165] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0166] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the machine learning-based robot motion path planning device 510. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0167] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0168] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0169] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the machine learning-based robot motion path planning device 510. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0170] It should be noted that, Figure 5 The structure of the machine learning-based robot motion path planning device 510 shown in the diagram does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0171] Furthermore, the technical effects of the machine learning-based robot motion path planning device 510 can be referenced from the technical effects of the machine learning-based robot motion path planning method described in the above method embodiments, and will not be repeated here.

[0172] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0173] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0174] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0175] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0176] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0177] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0179] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0180] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine learning-based robot motion path planning method, characterized in that, The method includes: Step S1, Environment Modeling and Initialization: The robot uses its onboard sensors to perceive the working environment, obtains the location information of obstacles and target points, converts the location information of obstacles and target points into a digital map for storage, and initializes the relevant parameters of the artificial potential field algorithm and ant colony algorithm. Step S2: Use the robot motion model and MPC to predict the robot's motion state over a future period of time, and construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state. Step S3: Optimize the artificial potential field using a variable neighborhood search algorithm, and introduce a reinforcement learning strategy into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field. Step S4: Using the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization via the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone evaporation update mechanism. Step S5: Evaluate the optimized final path regarding path length, path smoothness, safety, convergence speed, and adaptability. Monitor environmental changes in real time during robot movement and dynamically adjust the path according to the changes. Specifically, the construction of the artificial potential based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state includes: Define the gravitational field function that introduces relative velocity as follows: The repulsive field function is The total potential field in which the robot is located is Then we have: ； ； Where q is the robot's current position and v is the robot's speed. It is the target speed. The gravitational coefficient, The target point location, It is a weighting coefficient used to balance the effects of velocity and position. The speed of the obstacle's movement. The relative speed between the robot and the obstacle. Let d(q) be the repulsive force coefficient, and d(q) be the distance between the robot and the obstacle. For threshold distance, It is a coefficient used to adjust the intensity of repulsive force according to the relative velocity; The optimization of the artificial potential field using a variable neighborhood search algorithm includes: The neighborhood structure of the variable neighborhood search algorithm is constructed, and six neighborhoods are defined as: up, down, left, right, far from the obstacle, and outside the exclusion range of the obstacle; among them, the six neighborhood actions represent the generation of sub-target points in six directions, with the size being the step size; Construct an evaluation function for the variable neighborhood search algorithm. The evaluation function is as follows: ,in, , , Let be the weighting coefficient, satisfying > > , The size of the total potential field for the current sub-target. The newly generated path length for the sub-target. The shortest distance from the obstacle to the two sub-targets is the distance between them. The sub-targets are local key points in the artificial potential field where the gravitational gradient changes. , For gravitational potential field, It is a repulsive potential field; The evaluation function compares the scores of the six neighborhoods in the neighborhood structure, selects the one with the best score as the local optimum, and then perturbs it. The perturbation object is the coordinates of the sub-target point of the current local optimum. The perturbation method is to randomly shift the position of the sub-target point within the neighborhood step size. After the perturbation, the score is recalculated and a new local optimum is selected again. This process is repeated iteratively to optimize the sub-target point until the robot jumps out of the local trap area driven by the attraction of the sub-target point.

2. The robot motion path planning method based on machine learning according to claim 1, characterized in that, The process of converting the location information of obstacles and the location information of target points into digital map format for storage includes: The robot's motion space is divided into grids using the grid method. Obstacles are assigned a value of 1, and passable grids are assigned a value of 0. The grids are encoded using sequential encoding, so that the position information of obstacles and target points are transformed into a 0-1 matrix model.

3. The robot motion path planning method based on machine learning according to claim 1, characterized in that, The optimization objective function of the MPC takes into account factors such as the surrounding environment and the distance of movement, and incorporates kinematic constraints, dynamic constraints, and obstacle constraints.

4. The robot motion path planning method based on machine learning according to claim 1, characterized in that, Introducing reinforcement learning strategies into the variable neighborhood search algorithm, including: The six neighborhoods are simultaneously used as the reward matrix Q for both state s and action a to update the feedback from the environment. The improvement is introduced by changing the environment. Adaptive rewards and penalties are given based on the generated solution. The following four cases are defined: the reward for a local optimum is R1, the reward for improving the environment is R2, the reward for both local optimum and improving the environment is R3, and the reward for neither of the above three cases is R4.

5. The robot motion path planning method based on machine learning according to claim 1, characterized in that, The transition probability is calculated using a roulette wheel betting method, specifically: ； in, It is a heuristic function, representing the expected degree of an ant moving from grid i to grid j at time t. This represents the expected degree of an ant moving from grid i to grid s at time t. It represents the pheromone concentration, indicating the pheromone concentration between grids i and j at time t. This represents the pheromone concentration between two grid cells, i and s, at time t. , represents the weights of pheromones and heuristic information in the transition probability, respectively; allowed represents the grid that the robot can move to in the next step.

6. The robot motion path planning method based on machine learning according to claim 1, characterized in that, The pheromone evaporation and renewal mechanism is as follows: ,in, This represents the pheromone concentration between grids i and j. Let represent the pheromone concentration left by the k-th ant along its path between grids i and j. Let M be the volatile factor, and M be the total number of ants in the robot's abstraction.

7. The robot motion path planning method based on machine learning according to claim 6, characterized in that, The multi-round iterative optimization yields the final path, including: The robot is placed at the starting point, and it selects the next candidate point to move to based on the pheromone concentration and heuristic information from the pheromone release mechanism of ants. Based on the path traversed by the ants and its length, a pheromone evaporation and update mechanism is implemented for the pheromones along the path. In the process of ant colony search simulated by the robot, a potential field force is introduced to guide the ants' search direction. Based on the potential field information of the current ant's location, the direction of the potential field force on the ant is calculated, so that the ant tends to move towards the target point during the search process, while avoiding approaching obstacles. After each iteration, it is determined whether the convergence condition is met. If the convergence condition is not met, the next iteration continues. If the convergence condition is met, the iteration stops and the next step is performed. The convergence condition is that the maximum number of iterations is reached or the optimal path length changes less than a set threshold in multiple consecutive iterations.

8. A machine learning-based robot motion path planning device, wherein the machine learning-based robot motion path planning device is used to implement the machine learning-based robot motion path planning method as described in any one of claims 1-7, characterized in that, The device includes: The initialization module uses the sensors on the robot to perceive the working environment, obtain the location information of obstacles and target points in the environment, convert the location information of obstacles and target points into digital map format for storage, and initialize the relevant parameters of artificial potential field algorithm and ant colony algorithm. The module constructs an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state, using the robot motion model and MPC to predict the robot's motion state over a future period. The optimization module optimizes the artificial potential field using a variable neighborhood search algorithm, and introduces a reinforcement learning strategy into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field. The iterative module uses nodes on the basic path as the initial starting point for ants in the ant colony algorithm. Through multiple rounds of iterative optimization, the final path is obtained by using the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone evaporation update mechanism. The adjustment module evaluates the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. It also monitors environmental changes in real time during robot movement and dynamically adjusts the path accordingly.

Images