Robot motion path planning method and device based on machine learning

Abstract

The invention provides a robot motion path planning method and device based on machine learning, and relates to the technical field of robot path planning, and the method comprises the steps: obtaining and storing the position information of an obstacle and a target point in an environment; constructing an artificial potential field according to the current position of the robot, the speed of the robot, the target point position, the target speed, the obstacle information and the motion state of the robot; the artificial potential field is optimized through a variable neighborhood search algorithm, and a reinforcement learning strategy is introduced into the variable neighborhood search algorithm; nodes on the basic path generated by the artificial potential field method serve as initial starting points of ants in the ant colony algorithm, and a final path is obtained through multi-round iterative optimization through an ant release pheromone mechanism, a transition probability selection mechanism and a pheromone volatilization updating mechanism; and evaluating the optimized path and dynamically adjusting the path. The robot motion path planning method can cope with complex environment changes, and effectively improves the safety and efficiency of path planning.

Description

Technical Field

[0001] The present invention relates to the technical field of robot path planning, and in particular to a robot motion path planning method and device based on machine learning. Background Art

[0002] Path planning for mobile robots (MR) is a key branch of MR research and the foundation of their control. Depending on the level of known environmental information, path planning can be categorized as global path planning based on known environmental information or local path planning based on unknown or partially known environmental information. With the rapid advancement of technology and the widespread application of robots, expectations for robots are increasing, particularly regarding their intelligence. Autonomous robot path planning is a crucial step in achieving this intelligence. Path planning involves planning the optimal path for a robot to safely and collision-freely reach its target location from its starting point.

[0003] Currently, many global path planning methods for mobile robots are commonly used, such as grid methods and artificial potential field methods. However, grid methods require significantly more storage space and slow down decision-making as the space increases. Artificial potential field methods (APFs) simulate the motion of particles within a hypothetical force field, minimizing computational complexity and ensuring high real-time performance. However, during motion, the robot may become trapped in local minima—points where gravitational and repulsive forces reach equilibrium, but not the target location. This can cause the robot to become stuck in a local equilibrium point, like a "trap," unable to continue toward its target. Other algorithms, such as the A* algorithm, particle swarm optimization, and genetic algorithms, can also be used for robot path planning, but they also present corresponding challenges. While numerous approaches have been developed to solve the mobile robot path planning problem, they are limited by map size and the number of obstacles, making a single algorithm incapable of navigating complex environmental changes. Summary of the Invention

[0004] In order to solve the problem in the prior art that robot path planning is limited by map scale and the number of obstacles, and a single algorithm is difficult to cope with complex environmental changes, the present invention provides a robot motion path planning method and device based on machine learning.

[0005] In one aspect, a method for robot motion path planning based on machine learning is provided. The method is implemented by a robot motion path planning device based on machine learning, and the method includes:

[0006] Step S1: Environmental modeling and initialization: The robot senses the working environment through the sensors carried by the robot, obtains the location information of obstacles and target points in the environment, converts the location information of obstacles and target points into digital map form and stores them, and initializes the relevant parameters of the artificial potential field algorithm and the ant colony algorithm.

[0007] Step S2, using the robot motion model and MPC to predict the robot's motion state in the future, and constructing an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state;

[0008] Step S3, optimizing the artificial potential field by a variable neighborhood search algorithm, while introducing a reinforcement learning strategy into the variable neighborhood search algorithm, and generating a basic path based on the optimized artificial potential field;

[0009] Step S4: Using the nodes on the basic path as the initial starting points of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization through the ant pheromone release mechanism, the transition probability selection mechanism, and the pheromone volatilization update mechanism;

[0010] Step S5: Evaluate the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. During the robot's movement, monitor environmental changes in real time and dynamically adjust the path according to the changes.

[0011] Optionally, the step of converting the obstacle location information and the target location information into a digital map format for storage includes:

[0012] The grid method is used to divide the robot motion space into grids of a certain size, assign obstacles a value of 1, and assign passable grids a value of 0. The grids are encoded using sequential coding, so that the position information of obstacles and target points is converted into a 0-1 matrix model.

[0013] Optionally, the artificial potential is constructed according to the current position of the robot, the speed of the robot, the position of the target point, the target speed, the obstacle information and the motion state of the robot, specifically:

[0014] The gravitational field function that introduces relative velocity is defined as , the repulsive field function is , the total potential field where the robot is located is , then:

[0015] ;

[0016] ;

[0017] Among them, q is the current position of the robot, v is the speed of the robot, is the target speed, is the gravitational coefficient, is the target point position, is a weight coefficient used to balance the effects of speed and position. is the obstacle moving speed, is the relative moving speed between the robot and the obstacle, is the repulsion coefficient, d(q) is the distance between the robot and the obstacle, is the threshold distance, is a coefficient used to adjust the strength of the repulsive force according to the relative velocity.

[0018] Optionally, the optimization objective function of the MPC takes into account factors of the surrounding environment and movement distance, and adds kinematic constraints, dynamic constraints and obstacle constraints.

[0019] Optionally, the optimizing the artificial potential field by using a variable neighborhood search algorithm includes:

[0020] Construct the neighborhood structure of the variable neighborhood search algorithm and define six neighborhoods: up, down, left, right, away from obstacles, and outside the exclusion range of obstacles; the six neighborhood actions represent the six directions of sub-target point generation, and the size is the step size;

[0021] Construct the evaluation function of the variable neighborhood search algorithm. The evaluation function is: ,in, 、 、 is the weight coefficient, satisfying > > , is the total potential field size of the current sub-goal, Newly generated path length for the sub-goal, is the shortest distance from the obstacle to the line connecting the two sub-targets, and the sub-targets are the local key points where the gravitational gradient changes in the artificial potential field. , is the gravitational potential field, is the repulsive potential field;

[0022] The scores of the six neighborhoods in the neighborhood structure are compared through an evaluation function, and the one with the best score is selected as the local optimal solution. Then, the perturbation is performed. The perturbation object is the coordinates of the sub-target point of the current local optimal solution. The perturbation method is to randomly offset the position of the sub-target point within the neighborhood step size. After the perturbation, the score is recalculated and a new local optimal solution is selected again. The sub-target point is iteratively optimized repeatedly until the robot jumps out of the local trap area driven by the attraction of the sub-target point.

[0023] Optionally, introducing a reinforcement learning strategy into the variable neighborhood search algorithm includes:

[0024] The six neighborhoods are used as the reward matrix Q of state s and action a to update the feedback brought by the environment;

[0025] Improvements to environmental changes are introduced, and adaptive rewards and penalties are given based on the generated solutions. The following four cases are defined: the reward for the local optimal solution is R1, the reward for the improvement on the environment is R2, the reward for both the local optimal solution and the improvement on the environment is R3, and the reward for none of the above three cases is R4.

[0026] Optionally, the transfer probability is calculated by a roulette method, specifically:

[0027] ;

[0028] in, is the heuristic function, which represents the expected degree of the ant moving from grid i to grid j at time t. represents the expected degree of the ant moving from grid i to grid s at time t, is the pheromone concentration, which represents the pheromone concentration between grids i and j at time t. represents the pheromone concentration between the two grids i and s at time t, 、 Respectively represent the weights of pheromone and heuristic information in the transition probability; allowed represents the grid where the robot can walk next.

[0029] Optionally, the pheromone volatilization and updating mechanism is specifically as follows: ,in, represents the pheromone concentration between grids i and j, is the pheromone concentration left by the kth ant on the route between grids i and j, is the volatility factor, and M is the total number of ants abstracted by the robot.

[0030] Optionally, the multiple rounds of iterative optimization to obtain the final path include:

[0031] The robot is placed at the starting point and selects the next candidate point to move to based on the pheromone concentration and heuristic information of the ant pheromone release mechanism;

[0032] According to the path taken by the ants and the length of the path, the pheromone on the path is updated by the pheromone volatilization mechanism;

[0033] During the robot-simulated ant colony search process, the potential field force is introduced to guide the ants' search direction. Based on the potential field information of the ants' current location, the direction of the potential field force on the ants is calculated, so that the ants tend to move towards the target point during the search process while avoiding obstacles.

[0034] After each iteration is completed, it is determined whether the convergence condition is met. If not, the next iteration is continued; if the convergence condition is met, the iteration is stopped and the next step is entered; the convergence condition is that the maximum number of iterations is reached or the optimal path length changes by less than a set threshold in multiple consecutive iterations.

[0035] On the other hand, a robot motion path planning device based on machine learning is provided, which is applied to the robot motion path planning method based on machine learning, and the device includes:

[0036] The initialization module uses the robot's onboard sensors to perceive the working environment, obtain the location information of obstacles and target points in the environment, convert the location information of obstacles and target points into digital map form and store it, and initialize the relevant parameters of the artificial potential field algorithm and the ant colony algorithm;

[0037] The construction module uses the robot motion model and MPC to predict the robot's motion state in the future, and constructs an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information and robot motion state;

[0038] The optimization module optimizes the artificial potential field through a variable neighborhood search algorithm. At the same time, a reinforcement learning strategy is introduced into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field.

[0039] The iterative module uses the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm. Through the ant pheromone release mechanism, the transition probability selection mechanism, and the pheromone volatilization update mechanism, the final path is obtained through multiple rounds of iterative optimization;

[0040] The adjustment module evaluates the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. It monitors environmental changes in real time during robot movement and dynamically adjusts the path based on the changes.

[0041] On the other hand, a robot motion path planning device based on machine learning is provided, and the robot motion path planning device based on machine learning includes: a processor; a memory, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned robot motion path planning methods based on machine learning is implemented.

[0042] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned robot motion path planning methods based on machine learning.

[0043] The beneficial technical effects of the present invention are:

[0044] The present invention effectively combines the improved artificial potential field method with model prediction planning and control to apply to the construction of artificial potential fields in robot path planning. It has the characteristics of high control speed, high safety, and high stability. At the same time, when constructing the artificial potential field, the introduction of relative speed can better adapt to dynamic environments, predict possible collisions in advance, and more reasonably guide the motion path of the intelligent body, thereby effectively improving the safety and efficiency of path planning.

[0045] The present invention combines the ant colony algorithm with the artificial potential field method. Utilizing the ant colony algorithm's strong global search capabilities, the algorithm optimizes the problem of the artificial potential field method easily falling into local minima. When the artificial potential field method causes the robot to fall into a local minimum point between obstacles, the ant colony algorithm can use the random movement characteristics of ants to find a new path direction and guide the robot out of the local optimal solution.

[0046] In a dynamic environment, the artificial potential field method may not update the potential field in a timely manner. The ant colony algorithm can readjust the ants' search path according to environmental changes. By updating the pheromone concentration and changing the path selection strategy, the robot can better avoid dynamic obstacles. At the same time, the ant colony algorithm can comprehensively consider factors such as path length and path smoothness during the path search process. When combined with the artificial potential field method, it can generate a smoother path while avoiding collisions (an advantage of the artificial potential field method).

[0047] The ant colony algorithm has the characteristics of strong global search capabilities, but the ant colony algorithm itself has problems such as slow convergence, easy to fall into local optimality, and premature convergence; using reinforcement learning, through continuous trial and error and learning, the strategy can be adjusted according to environmental feedback to guide the intelligent agent (ant colony) to jump out of the local optimal solution. The combination of the two can enhance the search capability, enable the intelligent agent to adjust its behavior strategy in time according to the dynamic changes of the environment, and find a better global solution. At the same time, information such as pheromone concentration in the ant colony algorithm can be used as a state feature of reinforcement learning, helping reinforcement learning to converge to the optimal strategy faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A flowchart of a robot motion path planning method based on machine learning provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the neighborhood layout structure of the VNS algorithm used in a machine learning-based robot motion path planning method provided by an embodiment of the present invention, wherein (a) searches for sub-targets in an upward direction, (b) searches for sub-targets in a downward direction, (c) searches for sub-targets in a leftward direction, (d) searches for sub-targets in a rightward direction, (e) searches for sub-target points in a direction away from obstacles, and (f) searches for sub-target points outside the obstacle exclusion range.

[0051] Figure 3 A schematic diagram of a flow chart of a learning feedback strategy for a robot motion path planning method based on machine learning provided by an embodiment of the present invention;

[0052] Figure 4 This is a structural block diagram of a robot motion path planning device based on machine learning provided by an embodiment of the present invention;

[0053] Figure 5 It is a structural schematic diagram of a robot motion path planning device based on machine learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0056] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0057] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0058] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0059] The present invention provides a method and device for robot motion path planning based on machine learning. The method can be implemented by a robot motion path planning device based on machine learning, which can be a terminal or a server. The process of a robot motion path planning method based on machine learning is as follows: Figure 1 As shown, the processing flow of the method includes the following steps:

[0060] Step S1: The robot uses its onboard sensors to sense the working environment, obtain the location information of obstacles and the target point in the environment, convert the obstacle and target location information into a digital map, and initialize the relevant parameters of the artificial potential field algorithm and the ant colony algorithm. The "target point" refers to the desired location of the robot during the motion task. In the context of robot motion path planning, it is the specific coordinate location the robot needs to reach. For example, in the cleaning robot scenario, it is a certain cleaning end point in the room, in the delivery robot scenario, it is a designated delivery location, etc. It is also the destination that the path planning guides the robot to.

[0061] In a feasible implementation, in step S1, the sensor carried by the robot senses the working environment, including:

[0062] LiDAR: It determines the distance to surrounding objects by emitting laser beams and measuring the time it takes to reflect the light. It can quickly and accurately scan the outline information of obstacles in all directions around the robot, forming point cloud data. Each point in the point cloud contains information such as the three-dimensional coordinate position in space.

[0063] Visual sensor (camera): uses a camera to capture environmental images. Based on computer vision algorithms, it can identify the outlines and features of objects in the image. By analyzing images from different perspectives and combining relevant visual ranging technologies (such as binocular vision principles to estimate distance through parallax), it can indirectly obtain the position of objects in space.

[0064] Ultrasonic sensor: emits ultrasonic pulses and receives reflected waves, and calculates the distance to the obstacle based on the time difference.

[0065] The data collected by the sensors is then preprocessed. The collected sensor data often contains noise interference. For example, the point cloud data of the LiDAR may contain some erroneous outliers. Filtering algorithms, such as mean filtering and median filtering, can be used to remove these noise points, improve data quality, and enable subsequent analysis to be based on more accurate data. Furthermore, the data obtained by different sensors may be based on their own coordinate systems. For example, the camera image coordinates and the LiDAR three-dimensional coordinates are different. These need to be unified into the robot's global coordinate system through the corresponding coordinate transformation matrix to facilitate fusion and subsequent map construction operations.

[0066] Sensors are used to obtain the location information of obstacles and target points in the environment and convert it into a digital map for storage. Specifically, a grid method is used to convert the robot's motion environment into a 0-1 matrix model. The robot's motion space is divided into grids of a certain size. Obstacles are assigned a value of 1, and passable grids are assigned a value of 0. The grids are then encoded using a sequential code. As the robot moves and continuously collects new data, the map information needs to be updated in real time. For example, if a new obstacle is discovered or an area previously marked as an obstacle no longer exists, the status information of the corresponding grid or topological node edge in the map must be modified accordingly. The size of the grid is determined based on the actual application scenario and accuracy requirements of the robot.

[0067] After establishing the digital map, the relevant parameters of the artificial potential field algorithm and the ant colony algorithm are initialized at the same time. The relevant parameters of the artificial potential field algorithm mainly include the attraction coefficient, repulsion coefficient, obstacle influence radius, and step size. The relevant parameters of the ant colony algorithm include the number of ants, pheromone volatility coefficient, pheromone heuristic factor, and maximum number of iterations.

[0068] Step S2: Use the robot motion model and MPC (Model Predictive Control) to predict the robot's motion state in the future, and construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state.

[0069] In step S2, the robot motion model and MPC are used to predict the robot motion state in the future. The predicted robot motion state specifically includes position (such as coordinate information), speed, acceleration, as well as the robot's posture (such as heading angle, etc.), angular velocity and other motion-related parameters. The specific implementation method is as follows:

[0070] The robot motion model can generally be expressed by the following equation:

[0071]

[0072] in( , ) is the plane position of the robot at time k, is the heading angle of the robot, is the robot's linear velocity, is the angular velocity of the robot, is the time step.

[0073] MPC is a model-based optimization control strategy that solves a finite-time optimization problem within each control cycle to determine the optimal control input at the current moment. Its main steps include:

[0074] Prediction model: Utilizing the robot's motion model, the robot's state sequence over a period of time is predicted based on the current state and a hypothetical future control input sequence. The robot's state sequence is predicted starting from the current moment, with a fixed time step length Δt (e.g., 0.1 seconds) and for N consecutive steps (e.g., 10 steps, corresponding to the time range from the current moment to the current moment + N × Δt). This sequence is a "preview of the actual motion" derived from the model and is compared with the "desired state sequence (a manually defined ideal trajectory)" through an objective function optimization.

[0075] Objective function setting: Define an objective function to measure the difference between the predicted state sequence and the expected state sequence. For example, the objective function can include factors such as the distance between the robot and the target position and the smoothness of the robot's motion.

[0076] Constraint considerations: Consider the robot's kinematic constraints (such as velocity and acceleration limits), dynamic constraints (such as motor torque limits), and obstacle constraints (such as collision avoidance).

[0077] Optimization solution: Use an optimization algorithm (such as the quadratic programming algorithm QP) to solve the control input sequence that minimizes the objective function while satisfying the constraints.

[0078] Control implementation: The first control input in the optimized control input sequence is applied to the robot, and the above process is repeated in the next control cycle.

[0079] At the same time, the optimization objective function of MPC takes into account multiple factors such as the surrounding environment and movement distance, and adds kinematic constraints, dynamic constraints and obstacle constraints.

[0080] The objective function is usually a cost function that needs to be minimized. It takes into account the above factors. A general form can be expressed as: ,in, represents the cost associated with the movement distance, represents the cost associated with the surrounding environment, Represents the cost associated with motion smoothness.

[0081] Movement distance cost Expressed as: ,in, is the distance cost weight, It's the robot Step position, It is is the expected reference position, and N is the number of prediction steps.

[0082] Expressed as: ,in, is the environmental cost weight coefficient, M is the number of obstacles, For the robot step, the distance to the ith obstacle; P is the number of special areas in the environment, is the penalty / reward coefficient of the jth special area, is the indicator function. When the robot is in When the step is in the jth special area, When it is not in the area, the value is 0.

[0083] The expression is: ,in, is the smoothness weight (e.g. 0.3); It is Step line speed, is the angular velocity, is the weight of angular velocity change (e.g. 0.5), which is used to balance the smoothness of linear velocity and angular velocity.

[0084] Environmental factors: Information about the surrounding environment is crucial for the safe and efficient movement of a robot. For example, the robot needs to consider the distance to surrounding obstacles to avoid collisions. It may also need to consider special areas in the environment, such as danger zones and priority areas.

[0085] The motion distance factor is: the robot usually hopes to reach the target position by the shortest or most economical path. The motion distance can be measured by the Euclidean distance between the current position of the robot and the target position or other path length metrics.

[0086] The kinematic constraints added include position and posture constraints, velocity and acceleration constraints; the dynamic constraints include force and torque constraints, capacity constraints; the obstacle constraints include static obstacle constraints and dynamic obstacle constraints. By setting these constraints, the safety, stability and reliability of the robot during the path planning journey are guaranteed.

[0087] Optionally, in step S2, an artificial potential field is constructed based on the current position and speed of the robot, the target point position and speed, the obstacle information and the predicted state, specifically:

[0088] The gravitational field function that introduces relative velocity is defined as , the repulsive field function is , the total potential field where the robot is located is , then:

[0089]

[0090]

[0091] Among them, q is the current position of the robot, v is the speed of the robot, is the target speed, is the gravitational coefficient, is the target point position, is a weight coefficient used to balance the effects of speed and position. is the obstacle moving speed, is the relative moving speed between the robot and the obstacle, is the repulsion coefficient, d(q) is the distance between the robot and the obstacle, is the threshold distance, is a coefficient used to adjust the strength of the repulsive force according to the relative velocity.

[0092] In traditional artificial potential field algorithms, the main considerations are the distance between the robot and the target point and the distance from obstacles to construct the gravitational and repulsive potential fields. The present invention is based on an artificial potential field based on relative velocity. It further introduces the factor of relative velocity on the basis of the traditional artificial potential field algorithm. The goal is to allow the robot to not only consider the position relationship during movement, but also dynamically adjust the force situation based on the speed difference between itself and the target and obstacles, thereby more rationally planning the movement trajectory, avoiding collisions and efficiently moving towards the target. At the same time, one of the main problems with the artificial potential field method in local path planning is the local optimal solution. When there is an obstacle near the target point, the robot may be trapped in the local optimal solution under the combined force of repulsion and gravity, resulting in the robot being unable to move forward. For example, when the repulsive force generated by the robot approaching the obstacle and the gravitational force generated by the target reach equilibrium, it may stagnate at a position that is not globally optimal and unable to reach the target point. When the relative velocity factor is introduced, the force on the robot no longer depends solely on the position. For example, when the robot is between an obstacle and a target, even if the gravitational force and repulsive force are balanced at a certain position, an additional force will be generated because the relative velocity between the robot and the obstacle (or target) is not zero. The term will significantly increase the repulsive force, break the original balance, and prompt the robot to change its path to avoid falling into a stagnation state in the local optimum.

[0093] Step S3: Optimize the artificial potential field by introducing a variable neighborhood search algorithm with a reinforcement learning strategy, and generate a basic path based on the optimized artificial potential field.

[0094] Optionally, in step S3, the artificial potential field is optimized by a variable neighborhood search algorithm, specifically:

[0095] Construct the neighborhood structure of the variable neighborhood search algorithm and define six neighborhoods: up, down, left, right, away from obstacles, and outside the exclusion range of obstacles; the six neighborhood actions represent the six directions of sub-target point generation, and the size is the step size;

[0096] Construct the evaluation function of the variable neighborhood search algorithm, specifically: ,in, 、 、 is the weight coefficient, satisfying > > , is the total potential field size of the current sub-goal, Newly generated path length for the sub-goal, is the shortest distance from the obstacle to the line connecting the two sub-targets, and the sub-targets are the local key points where the gravitational gradient changes in the artificial potential field. , is the gravitational potential field, is the repulsive potential field;

[0097] The scores of the six neighborhoods in the neighborhood structure are compared by the evaluation function. Calculate and select the one with the best score as the local optimal solution, and then perform perturbation. The perturbation object is the coordinates of the sub-target point of the current local optimal solution. The perturbation method is to randomly offset the position of the sub-target point within the neighborhood step size (such as coordinate ± step size adjustment). After the perturbation, recalculate the score and select a new local optimal solution again. Repeat the iterative optimization of the sub-target point until the robot jumps out of the local trap area driven by the attraction of the sub-target point and safely completes the path planning task.

[0098] In one possible implementation, Figure 2 As shown in the figure, the path generated by the small dot in the middle is the original path of the robot when it encounters a local minimum. The path generated by the four-pointed star is the path guided by the sub-target points generated by the variable neighborhood search algorithm VNS that introduces the reinforcement learning strategy, that is, the optimized path. The upper and lower circular objects and the triangular area in the upper right corner of the figure are obstacles, and the circle formed by the dotted lines outside the circular objects is the obstacle repulsion range. Figure 2 (a), (b), (c), and (d) are searching for sub-targets in the upward, downward, left, and right directions, respectively; Figure 2 (e) in the figure is searching for the sub-target point in the direction away from the obstacle; Figure 2 (f) in the figure is searching for sub-target points outside the obstacle repulsion range. At this time, the area where the four-pointed star is located within the repulsion range is the area within the obstacle repulsion range.

[0099] At the same time, the present invention introduces a reinforcement learning strategy into the variable neighborhood search algorithm, specifically:

[0100] The six neighborhoods are used as the reward matrix Q of state s and action a to update the feedback from the environment. The action selection strategy is as follows:

[0101] ;

[0102] in, is the random probability value, is the critical value used to judge the action random(a) and the maximum reward action max(Q(s,a)), The smaller the variable, the faster the neighborhood search algorithm converges.

[0103] Improvements to environmental changes are introduced, and adaptive rewards and penalties are given based on the generated solution. The following four cases are defined: the reward for the local optimal solution is R1, the reward for environmental improvement is R2, the reward for both local optimality and environmental improvement is R3, and the reward for none of the above three cases is R4;

[0104] like Figure 3 As shown in the figure, a robot path planning process based on the fusion of artificial potential field (APF) and reinforcement learning is presented. The upper layer uses reinforcement learning to select actions, set rewards, and update them to drive decision optimization. The lower layer relies on APF to generate basic paths. If it falls into a local optimum or other situation, it will be fed back to the upper layer. The upper layer then adjusts its actions through reinforcement learning, iterates repeatedly, and finally outputs a global optimal solution that takes into account path length, smoothness, and safety. This achieves adaptive path planning in a dynamic environment, makes up for the shortcomings of a single algorithm, and improves planning intelligence and environmental adaptability. The learning and environmental feedback strategy is adaptive and reactive because a single-step update of the reward value can improve the selection of the local search neighborhood. If there is no improvement, a penalty is imposed. The reward for environmental change improvement is defined as:

[0105]

[0106] Where R1 and R4 are constants, and R4≤0≤R1≤R2≤R3. C1 and C2 are the local optimal solution and environmental improvement, respectively, which are obtained by comparing the neighborhood solution evaluation function and the path safety index.

[0107] ;

[0108] definition , where (x1, y1) and (x2, y2) are the position coordinates of the first and last key points of the segmentation when the robot first encounters the local minimum point. is the equation of the straight line formed by the two key points mentioned above, is a constant and R1< <R3, 、 are the last sub-goal of the previous solution (i.e. the local optimal sub-goal path solution of the previous iteration) and the last sub-goal of the current solution. By improving the environment, the algorithm systematically expands the search range, avoids repeated searches in the searched area, and increases the probability of escaping the local optimal solution.

[0109] Step S4: Using the nodes on the basic path as the initial starting points of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization using the ant pheromone release mechanism, the transfer probability selection mechanism, and the pheromone volatilization update mechanism.

[0110] In a feasible implementation, in step S4, the transition probability is calculated by a roulette method, which is calculated as follows:

[0111] ;

[0112] in, is the heuristic function, which represents the expected degree of the ant moving from grid i to grid j at time t. represents the expected degree of the ant moving from grid i to grid s at time t, is the pheromone concentration, which represents the pheromone concentration between grids i and j at time t. represents the pheromone concentration between the two grids i and s at time t, 、 Respectively represent the weights of pheromone and heuristic information in the transition probability; allowed represents the grid where the robot can walk next.

[0113] In step S4, multiple rounds of iterative optimization are performed to obtain the final path, which is specifically:

[0114] The robot is placed at the starting point and selects the next candidate point to move to based on the pheromone concentration and heuristic information from the ant's pheromone release mechanism. This heuristic information can take into account distance information, potential field information, and other factors. As the ant moves, the taboo table is continuously updated to record candidate nodes it has passed through to prevent duplicate visits. When the ant reaches the end point or reaches the maximum step limit, the foraging process is complete, and the path and length of each ant are recorded.

[0115] Based on the paths taken by the ants and their lengths, a pheromone volatilization mechanism is implemented to update the pheromones along these paths. A typical update method involves adding a certain amount of pheromone to the paths the ants have taken, while simultaneously volatilizing a portion of the pheromone according to the volatilization coefficient, simulating the natural dissipation process of the pheromone. This gradually increases the pheromone concentration along the preferred paths, guiding subsequent ants to prefer these paths.

[0116] During the robot-simulated ant colony search process, potential field forces are introduced to guide the ants' search direction. Based on the potential field information of the ants' current location, the direction of the potential field force on the ants is calculated, so that the ants tend to move towards the target point during the search process while avoiding approaching obstacles.

[0117] After each iteration is completed, it is determined whether the convergence condition is met. If not, the next iteration is continued; if the convergence condition is met, the final path is obtained and the next step is entered; the convergence condition is that the maximum number of iterations is reached or the optimal path length changes by less than a set threshold in multiple consecutive iterations.

[0118] The above-mentioned pheromone volatilization and renewal mechanism is specifically as follows: ,in, represents the pheromone concentration between grids i and j, For the The concentration of pheromone left by ants in the route between grids i and j, is the volatility factor, M is the total number of ants abstracted by the robot, and when the robot transfers between grids, the pheromone concentration left between paths each time is: ,in, is the fitness value of the robot path, Q is the weight coefficient;

[0119] The heuristic information expression is:

[0120] ;

[0121] in, is the potential field force coefficient, is the potential field attraction, is the robot's transfer direction and the angle of the potential field force, is the potential field force attenuation coefficient, which makes the artificial potential field gradually decrease with the iteration process; is the heuristic information related to path construction at time t, represents the Euclidean distance from the robot's current position to the jth target position (or feasible grid), is the weight coefficient of the heuristic information.

[0122] Potential field gravity The calculation method is: ,in, is the gradient of the potential function with respect to position, is the partial derivative of the potential field function with respect to velocity;

[0123] The potential field force attenuation coefficient is: ,in, is the current iteration number of the algorithm, and N is the total number of iterations of the algorithm.

[0124] Step S5: Evaluate the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. During the robot's movement, monitor environmental changes in real time and dynamically adjust the path according to the changes.

[0125] In a feasible implementation, in step S5, the optimized path is evaluated in terms of path length, path smoothness, safety, convergence speed, and adaptability, specifically:

[0126] Path length evaluation, including quantitative analysis and qualitative analysis:

[0127] Quantitative analysis: Compare the lengths of the paths before and after optimization. Use robot path planning software or a tool that records motion trajectories to obtain the lengths of the original and optimized paths and calculate the reduction ratio. For complex three-dimensional paths or those with multiple sub-paths, decompose the path into multiple line segments, calculate the lengths of each segment separately, and sum them for comparison.

[0128] Qualitative analysis: The effectiveness of path length optimization is determined based on the robot's mission type. For example, for a material handling robot, a shorter path means more material can be moved per unit time, improving work efficiency. For an inspection robot, a shorter path allows inspection tasks to be completed more quickly, reducing delays in fault detection.

[0129] Consider the relationship between the robot's energy consumption and the path length. Generally speaking, the shorter the path, the less energy the robot consumes. For battery-powered robots, this directly affects their endurance and working cycle.

[0130] Path smoothness evaluation, including curvature analysis and turning angle analysis:

[0131] Curvature Analysis: Calculate the curvature at each point on the path. This information is derived using mathematical methods, such as calculating the second-order derivative of the path's parametric equation. The smaller the curvature, the smoother the path at that point. By plotting the curvature versus path length, you can visually observe how the path's smoothness changes. Calculate the maximum, minimum, and average curvature values ​​and compare them before and after optimization. If the maximum curvature decreases after optimization and the average approaches zero, the path is smoother.

[0132] Steering Angle Analysis: Record the robot's steering angle at each turning point along the path. Paths with frequent large-angle turns are less smooth. Compare the distribution of steering angles before and after optimization. If the number of large-angle turns decreases after optimization, and the variance of steering angles decreases, the path is smoother. Observe the continuity of steering angles; a smooth path should show gradual, rather than sudden, changes during turns. This can be assessed by analyzing the rate of change of the steering angle.

[0133] Actual exercise effect evaluation:

[0134] Observe the robot's motion in the experimental environment. If the robot exhibits no noticeable jitter, shaking, or sudden stops and starts during movement, the path is smooth. For robots with loads, such as transport robots, observe the stability of the load during movement. A smooth path can reduce load shaking and the risk of damage to the cargo.

[0135] Safety evaluation, including obstacle avoidance capability, dangerous area identification and avoidance, and emergency response capability:

[0136] Obstacle avoidance capability: Build a test environment containing various obstacles to simulate obstacles that the robot may encounter in real-world scenarios, such as static obstacles (walls, equipment, etc.) and dynamic obstacles (pedestrians, other mobile robots, etc.). Record the robot's reaction distance when encountering an obstacle, that is, the distance from obstacle detection to the initiation of avoidance measures. The shorter the reaction distance, the safer the robot. However, it is also important to avoid frequent path changes caused by misjudgments. Count the number of successful obstacle avoidances and collisions by the robot. The higher the success rate, the more reliable the path planning in terms of safety.

[0137] Dangerous area identification and avoidance: Define dangerous areas, such as high temperature, high pressure, and radiation areas, and use the robot's sensors to detect whether it can accurately identify these dangerous areas and plan paths to avoid them in advance; for dangerous areas with blurred safety boundaries, evaluate whether the robot can maintain a sufficient safety distance to avoid entering the dangerous area due to errors or environmental interference.

[0138] Emergency handling capabilities: Simulate emergency situations such as sudden obstacles, communication interruptions, sensor failures, etc., and observe whether the robot has preset emergency strategies, such as emergency braking, waiting on the spot to recover, or switching to an alternative path planning mode, to ensure the safety of itself and the surrounding environment.

[0139] Convergence speed evaluation, including algorithm iteration statistics, time measurement, and convergence stability assessment:

[0140] Algorithm Iteration Statistics: During the execution of the path optimization algorithm, the intermediate results of each iteration are recorded. The number of iterations required to converge from the initial path to the final optimized path is counted. The fewer iterations, the faster the convergence rate. Tests are conducted in environments of varying scale and complexity, observing how the number of iterations changes. If the number of iterations remains within a reasonable range even in complex scenarios, the algorithm's convergence rate is robust.

[0141] Time measurement: Use a high-precision timer to record the time it takes from starting the path optimization algorithm to obtaining the final optimized path. The shorter the time, the faster the convergence. Consider the impact of factors such as computer performance on time measurement, and conduct comparative tests on the same hardware platform to ensure the accuracy of the evaluation results.

[0142] Convergence stability assessment: Run the path optimization algorithm multiple times to observe whether the final path converged to is consistent each time. If the convergence results are stable, the algorithm is not only fast but also highly reliable. If the convergence results fluctuate significantly, further analysis is required. This may indicate that the algorithm itself has a local optimal solution or is being interfered with by factors such as environmental noise.

[0143] Adaptability evaluation, including different environment tests, task diversity tests, and anti-interference tests:

[0144] Different Environment Testing: The robot is placed in a variety of environments, such as indoor offices, warehouses, and outdoor environments with complex terrain. The robot is observed to see whether it can successfully optimize a path suitable for each environment based on its characteristics, such as spatial layout, obstacle distribution, and topography. The robot is also evaluated for its ability to quickly adapt and re-optimize its path to changing environments, such as changes in indoor furniture placement and outdoor terrain caused by weather.

[0145] Task Diversity Testing: A variety of mission objectives are set, such as patrol, transport, and search, to observe whether the robot can adjust its path planning strategy based on the mission requirements. For example, a patrol mission may require covering a wider area, while a transport mission may prioritize path length and payload stability. In response to changes in mission requirements, such as adjustments to task priorities or the addition of new tasks, the robot is evaluated for its ability to adapt its path planning strategy to the new mission requirements.

[0146] Interference resistance test: Introducing various types of interference factors, such as electromagnetic interference, noise interference, and lighting changes, to observe whether the robot can function normally under these interferences and whether path optimization is affected. If the robot can still maintain good path planning capabilities under certain interference levels, it indicates strong adaptability.

[0147] Optionally, in step S5, the environmental changes are monitored in real time during the movement of the robot and the path is dynamically adjusted according to the changes. Specifically, while the robot is moving along the planned path, the lidar and camera continuously monitor the environment. When a new obstacle is detected, the grid map is immediately updated, the grid with the obstacle is re-marked, and then the above-mentioned artificial potential field method is executed again to construct the basic path guidance, the ant colony algorithm is used to optimize the path, and the path evaluation steps are performed to dynamically adjust the robot's movement path.

[0148] In summary, this invention effectively combines an improved artificial potential field method with model predictive planning and control to construct an artificial potential field for robot path planning, resulting in high control speed, high safety, and high stability. Furthermore, the inclusion of relative velocity during artificial potential field construction more accurately describes the dynamic relationship between the agent and the target or obstacle. By considering relative velocity, the artificial potential field can better adapt to dynamic environments, predict possible collisions in advance, and more rationally guide the agent's motion path, thereby effectively improving the safety and efficiency of path planning.

[0149] Secondly, the ant colony algorithm (ACA) is combined with the artificial potential field method (APF), leveraging the ACA's strong global search capabilities to address the ACA's tendency to get stuck in local minima. The ACA, through the random search of individual ants and the guidance of pheromones, can explore a wider area. In robot path planning, when the ACA causes the robot to be trapped in a local minimum between obstacles, the ACA can leverage the ants' random movement to find a new path and guide the robot out of the local optimum. In dynamic environments, the ACA may not update the potential field in a timely manner. The ACA can adjust the ants' search path based on environmental changes, modifying the path selection strategy by updating pheromone concentrations, and enabling the robot to better avoid dynamic obstacles. Furthermore, the ACA can comprehensively consider factors such as path length and smoothness during path finding. When combined with the ACA, it can generate smoother paths while avoiding collisions (an advantage of the ACA).

[0150] Finally, while the ant colony algorithm (ACA) possesses strong global search capabilities and can guide artificial potential field methods to local optimal solutions, it also suffers from inherent problems such as slow convergence, susceptibility to local optimality, and premature convergence. This makes it less effective in unknown environments and unable to cope with complex and changing circumstances. The present invention introduces reinforcement learning, which, through trial and error and learning, adjusts its strategy based on environmental feedback, guiding the intelligent agent (ant colony) out of local optimal solutions. This combination enhances search capabilities, enabling the agent to adjust its behavior strategy in response to dynamic environmental changes and find a more optimal global solution. Furthermore, information such as pheromone concentration in the ACA can be used as state features for reinforcement learning, helping it converge more quickly to the optimal strategy.

[0151] Figure 4 This is a block diagram of a robot motion path planning device based on machine learning provided by an embodiment of the present invention, which is used in a robot motion path planning method based on machine learning. Figure 4 The device includes an initialization module 410, a construction module 420, an optimization module 430, an iteration module 440, and an adjustment module 450.

[0152] Initialization module 410 is used to sense the working environment through the sensors carried by the robot, obtain the location information of obstacles and the location information of the target point in the environment, convert the location information of the obstacles and the location information of the target point into a digital map format for storage, and initialize the relevant parameters of the artificial potential field algorithm and the ant colony algorithm;

[0153] A construction module 420 is configured to use the robot motion model and MPC to predict the robot motion state within a certain period of time in the future, and to construct an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and the robot motion state;

[0154] An optimization module 430 is configured to optimize the artificial potential field using a variable neighborhood search algorithm, introduce a reinforcement learning strategy into the variable neighborhood search algorithm, and generate a basic path based on the optimized artificial potential field;

[0155] Iterative module 440 is configured to use the nodes on the basic path as the initial starting points of the ants in the ant colony algorithm, and obtain the final path through multiple rounds of iterative optimization through the ant pheromone release mechanism, the transition probability selection mechanism, and the pheromone volatilization update mechanism;

[0156] The adjustment module 450 is used to evaluate the optimized path in terms of path length, path smoothness, safety, convergence speed and adaptability, monitor environmental changes in real time during robot movement, and dynamically adjust the path according to the changes.

[0157] Figure 5 is a schematic diagram of the structure of a robot motion path planning device based on machine learning provided by an embodiment of the present invention, such as Figure 5 As shown, the robot motion path planning device based on machine learning may include the above Figure 4 Optionally, the robot motion path planning device 510 based on machine learning may include a first processor 2001 .

[0158] Optionally, the robot motion path planning device 510 based on machine learning may further include a memory 2002 and a transceiver 2003 .

[0159] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0160] The following combination Figure 5 The components of the machine learning-based robot motion path planning device 510 are described in detail:

[0161] The first processor 2001 is the control center of the machine learning-based robot motion path planning device 510 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs) or one or more field programmable gate arrays (FPGAs).

[0162] Optionally, the first processor 2001 can perform various functions of the robot motion path planning device 510 based on machine learning by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0163] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.

[0164] In a specific implementation, as an embodiment, the robot motion path planning device 510 based on machine learning may also include multiple processors, such as Figure 5 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0165] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0166] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be accessed through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0167] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0168] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0169] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0170] It should be noted that Figure 5 The structure of the machine learning-based robot motion path planning device 510 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0171] In addition, the technical effects of the robot motion path planning device 510 based on machine learning can refer to the technical effects of the robot motion path planning method based on machine learning described in the above method embodiment, and will not be repeated here.

[0172] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0173] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0174] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0175] It should be understood that the term "and / or" as used herein simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0176] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0177] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0178] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0179] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0180] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0181] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0183] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot motion path planning method based on machine learning, characterized in that: The method comprises: Step S1: Environmental modeling and initialization: The robot senses the working environment through the sensors carried by the robot, obtains the location information of obstacles and target points in the environment, converts the location information of obstacles and target points into digital map form and stores them, and initializes the relevant parameters of the artificial potential field algorithm and the ant colony algorithm. Step S2, using the robot motion model and MPC to predict the robot's motion state in the future, and constructing an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information, and robot motion state; Step S3, optimizing the artificial potential field by a variable neighborhood search algorithm, while introducing a reinforcement learning strategy into the variable neighborhood search algorithm, and generating a basic path based on the optimized artificial potential field; Step S4: Using the nodes on the basic path as the initial starting points of the ants in the ant colony algorithm, the final path is obtained through multiple rounds of iterative optimization through the ant pheromone release mechanism, the transition probability selection mechanism, and the pheromone volatilization update mechanism; Step S5: Evaluate the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. During the robot's movement, monitor environmental changes in real time and dynamically adjust the path according to the changes.

2. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The step of converting the obstacle location information and the target location information into a digital map format for storage includes: The grid method is used to divide the robot motion space into grids, assigning obstacles a value of 1 and passable grids a value of 0. The grids are encoded using sequential coding, so that the position information of obstacles and target points is converted into a 0-1 matrix model.

3. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The artificial potential is constructed according to the current position of the robot, the speed of the robot, the position of the target point, the target speed, the obstacle information and the motion state of the robot, specifically: The gravitational field function that introduces relative velocity is defined as , the repulsive field function is , the total potential field where the robot is located is , then: ； ； Among them, q is the current position of the robot, v is the speed of the robot, is the target speed, is the gravitational coefficient, is the target point position, is a weight coefficient used to balance the effects of speed and position. is the obstacle moving speed, is the relative moving speed between the robot and the obstacle, is the repulsion coefficient, d(q) is the distance between the robot and the obstacle, is the threshold distance, is a coefficient used to adjust the strength of the repulsive force according to the relative velocity.

4. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The optimization objective function of the MPC takes into account factors of the surrounding environment and motion distance, and adds kinematic constraints, dynamic constraints and obstacle constraints.

5. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The optimization of the artificial potential field by using a variable neighborhood search algorithm includes: Construct the neighborhood structure of the variable neighborhood search algorithm and define six neighborhoods: up, down, left, right, away from obstacles, and outside the exclusion range of obstacles; the six neighborhood actions represent the six directions of sub-target point generation, and the size is the step size; Construct the evaluation function of the variable neighborhood search algorithm. The evaluation function is: ,in, 、 、 is the weight coefficient, satisfying > > , is the total potential field size of the current sub-goal, Newly generated path length for the sub-goal, is the shortest distance from the obstacle to the line connecting the two sub-targets, and the sub-targets are the local key points where the gravitational gradient changes in the artificial potential field. , is the gravitational potential field, is the repulsive potential field; The scores of the six neighborhoods in the neighborhood structure are compared through an evaluation function, and the one with the best score is selected as the local optimal solution. Then, the perturbation is performed. The perturbation object is the coordinates of the sub-target point of the current local optimal solution. The perturbation method is to randomly offset the position of the sub-target point within the neighborhood step size. After the perturbation, the score is recalculated and a new local optimal solution is selected again. The sub-target point is iteratively optimized repeatedly until the robot jumps out of the local trap area driven by the attraction of the sub-target point.

6. The robot motion path planning method based on machine learning according to claim 5, characterized in that: Introducing reinforcement learning strategies into the variable neighborhood search algorithm, including: The six neighborhoods are used as the reward matrix Q of state s and action a to update the feedback brought by the environment; Improvements to environmental changes are introduced, and adaptive rewards and penalties are given based on the generated solutions. The following four cases are defined: the reward for the local optimal solution is R1, the reward for the improvement on the environment is R2, the reward for both the local optimal solution and the improvement on the environment is R3, and the reward for none of the above three cases is R4.

7. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The transfer probability is calculated by the roulette method, specifically: ； in, is the heuristic function, which represents the expected degree of the ant moving from grid i to grid j at time t. represents the expected degree of the ant moving from grid i to grid s at time t, is the pheromone concentration, which represents the pheromone concentration between grids i and j at time t. represents the pheromone concentration between the two grids i and s at time t, 、 Respectively represent the weights of pheromone and heuristic information in the transition probability; allowed represents the grid where the robot can walk next.

8. The robot motion path planning method based on machine learning according to claim 1, characterized in that: The pheromone volatilization and updating mechanism is specifically as follows: ,in, represents the pheromone concentration between grids i and j, is the pheromone concentration left by the kth ant on the route between grids i and j, is the volatility factor, and M is the total number of ants abstracted by the robot.

9. The robot motion path planning method based on machine learning according to claim 8, characterized in that: The multiple rounds of iterative optimization to obtain the final path include: The robot is placed at the starting point and selects the next candidate point to move to based on the pheromone concentration and heuristic information of the ant pheromone release mechanism; According to the path taken by the ants and the length of the path, the pheromone on the path is updated by the pheromone volatilization mechanism; During the robot-simulated ant colony search process, the potential field force is introduced to guide the ants' search direction. Based on the potential field information of the ants' current location, the direction of the potential field force on the ants is calculated, so that the ants tend to move towards the target point during the search process while avoiding obstacles. After each iteration is completed, it is determined whether the convergence condition is met. If not, the next iteration is continued; if the convergence condition is met, the iteration is stopped and the next step is entered; the convergence condition is that the maximum number of iterations is reached or the optimal path length changes by less than a set threshold in multiple consecutive iterations.

10. A robot motion path planning device based on machine learning, wherein the robot motion path planning device based on machine learning is used to implement the robot motion path planning method based on machine learning according to any one of claims 1 to 9, characterized in that: The device comprises: The initialization module uses the robot's onboard sensors to perceive the working environment, obtain the location information of obstacles and target points in the environment, convert the location information of obstacles and target points into digital map form and store it, and initialize the relevant parameters of the artificial potential field algorithm and the ant colony algorithm; The construction module uses the robot motion model and MPC to predict the robot's motion state in the future, and constructs an artificial potential field based on the robot's current position, robot speed, target point position, target speed, obstacle information and robot motion state; The optimization module optimizes the artificial potential field through a variable neighborhood search algorithm. At the same time, a reinforcement learning strategy is introduced into the variable neighborhood search algorithm to generate a basic path based on the optimized artificial potential field. The iterative module uses the nodes on the basic path as the initial starting point of the ants in the ant colony algorithm. Through the ant pheromone release mechanism, the transition probability selection mechanism, and the pheromone volatilization update mechanism, the final path is obtained through multiple rounds of iterative optimization; The adjustment module evaluates the optimized final path in terms of path length, path smoothness, safety, convergence speed, and adaptability. It monitors environmental changes in real time during robot movement and dynamically adjusts the path based on the changes.

Images