Dual planning and autonomous learning based robot path planning and control method

Abstract

The application discloses a kind of mechanical arm path planning and control method based on double programming and autonomous learning, the method is first using double programming idea to carry out initial planning and secondary planning, introduce the distance cost of mobile path to screen initial mobile path of initial planning by calculating Euclidean distance, then according to whether target position moves, introduce intelligent sampling factor to plan, determine the obstacle avoidance trajectory route of mechanical arm moving to target point, reduce the turning point of mechanical arm movement, improve moving efficiency;Then design a kind of recurrent neural network that can adaptively search and learn, effectively improve the convergence speed of reinforcement learning network in autonomous motion scene, enhance the PID control stability of mechanical arm model, solve the output action continuity problem of linear velocity and yaw angular velocity of mechanical arm in algorithm.In addition, the output action of mechanical arm is introduced into the reward and punishment evaluation function, and the moving path of the mechanical arm is reasonably analyzed, so as to facilitate the improvement and optimization of subsequent path planning.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot arm path planning, and in particular relates to a robot arm path planning and control method based on dual planning and autonomous learning. Background Art

[0002] With the rapid development of the e-commerce and logistics industries in recent years, the application of mobile warehouse robots has become increasingly widespread to meet the demand for more efficient goods turnover in and out of the supply chain. Mobile warehouse robots are intelligent automated equipment that can be used in large-scale e-commerce warehouses and smart logistics factories. They are a new type of intelligent product that combines high operational precision, high execution efficiency, and high automation for convenient and fast operation. Traditional logistics warehouses often require human personnel to move goods. However, with the continuous advancement of intelligent warehouses, mobile palletizing robots, with their advantages such as high operational precision and low error, short operation time and high efficiency, low labor loss, and low cost, are becoming an indispensable part of the construction of future smart factories.

[0003] As palletizing robots are increasingly used, optimizing the shortcomings of industrial applications of robots has become a problem that warehouse managers need to consider and solve. Regarding the current industrial applications of robotic arms, Science and Technology Daily has pointed out that the current industrial applications of palletizing robotic arms have the following shortcomings: low efficiency of the robotic arm's obstacle avoidance navigation algorithm, lack of high-efficiency path planning algorithms and high-precision servo systems, large algorithm limitations, low accuracy of robotic arm motion operations, and difficulty in applying to complex environments. Currently, many researchers have put forward their own suggestions for improvement. Huang Qiuyu of Zhejiang University proposed in his invention patent "A Factory Cargo Transfer and Palletizing Robot Arm" (CN202310199470.4) that the stability of the robot's operation can be improved by designing a stable mounting box for the robot arm. Huang Yalan of Shanghai Institute of Electric Power used a super-redundant robotic arm path planning method in his patent "A Super-Redundant Robot Arm Path Planning Method" (CN202310253774.4) to predict and optimize the robot's operating posture. However, the above method did not mention the redundancy problem of the current robot arm's obstacle avoidance route design, and the robot arm also has low PID stability during movement, which makes it often unable to perform autonomous learning in complex situations, resulting in low applicability. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a robot arm path planning and control method based on dual planning and autonomous learning, so as to realize the reasonable planning of the robot arm's operation trajectory, reduce the turning points in the path movement process, reduce the operating cost of the robot arm, and improve the accuracy of the robot arm's movement precision.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A robot arm path planning and control method based on dual planning and autonomous learning includes the following steps:

[0007] Step 1: Obtain the robot's location coordinates, movement speed, yaw angular velocity, and obstacle information to create a factory grid map.

[0008] Step 2: Simplify the factory grid map, remove irrelevant obstacle coordinate points, and obtain a simplified grid map;

[0009] Step 3: Use random sampling in the simplified grid map for initial planning and search to generate the initial movement path;

[0010] Step 4: Calculate the path distance cost of the initial moving path to select the optimal moving path; if the path distance cost of the optimal moving path is too high, re-search and generate the initial moving path;

[0011] Step 5: Determine whether the target position has moved. If so, change the target position and re-plan the initial movement path.

[0012] Step 6: If the target position has not moved, perform secondary planning and introduce the intelligent sampling factor to expand the guided path to obtain the intelligent sampling area. Combine the factory grid map with the intelligent sampling area to generate a new sampling area. Continuously iterate the search within the new sampling area to optimize the optimal movement path and obtain the final movement path.

[0013] Step 7: Use the recurrent neural network based on improved adaptive search learning to improve the PID control algorithm for the robot arm movement, perform network training on the PID control parameters separately, accelerate the convergence of PID control, and realize the control of the robot arm.

[0014] As a preferred technical solution, the base of the robotic arm is equipped with a position sensing sensor, which senses its position coordinates (x, y, z) with the help of a wireless sensor network composed of routers;

[0015] A plurality of speed sensing sensors are placed on the robotic arm to obtain the moving speed v and the yaw angular velocity ω of the robotic arm;

[0016] The robotic arm is also equipped with a laser radar as an obstacle sensing sensor of the robotic arm. The laser radar point cloud is clustered using the Euclidean distance clustering method based on the KD tree to obtain the coordinates and length, width and height of the obstacle relative to the robotic arm itself.

[0017] As a preferred technical solution, in step 3, the search generates an initial moving path, specifically:

[0018] Set the node set of the initial moving path and set it to an empty set;

[0019] The coordinates of the robot's position are used as the starting node and stored sequentially in the node set of the initial movement path;

[0020] Search for a random node q in the simplified grid map rand , the search formula is:

[0021]

[0022] Among them, q goal is the target node at the target location, P is a random number in [0,1], and a is a custom constant whose value is determined by the number of obstacles;

[0023] Get a random node q in a simplified grid map rand The nearest neighbor node q nearst , calculate the new node q nee , the formula is:

[0024]

[0025] Among them, d represents the step size or increment, which is used to determine the nearest neighbor node q nearest Expand to new node q new distance and direction;

[0026] After generating a new node, take the new node as the starting node and store it in the node set of the initial moving path in sequence. Continue to randomly sample and obtain random nodes for iterative search until P>1-a appears. Then, the random node q rand As the target node q goal And store them sequentially in the node set of the initial moving path, and end the search to generate the initial moving path.

[0027] As a preferred technical solution, in step 4, the path distance cost calculation formula of the initial moving path is:

[0028]

[0029] Where d1 is the Euclidean distance in the joint space of the manipulator, d2 is the Euclidean distance in the Cartesian space of the manipulator, and Δθ is the Euclidean distance of the manipulator posture; α, β, and γ are the weights of d1, d2, and Δθ respectively;

[0030] The Euclidean distance d between two adjacent nodes in the initial moving path in the manipulator joint space j The calculation formula is:

[0031]

[0032] Where k is the angle in the joint space of the robot arm, q 1i is the angle of the previous node i, q 2i is the i-th angle of the next node;

[0033] Sum the Euclidean distances between all adjacent nodes in the initial movement path in the manipulator joint space to obtain the Euclidean distance d1 of the initial movement path in the manipulator joint space;

[0034] The Euclidean distance d between two adjacent nodes in the initial moving path in the Cartesian space of the robot arm d The calculation formula is:

[0035]

[0036] Among them, (x1, y1, z1) is the vector representation of the previous node in Cartesian space, and (x2, y2, z2) is the vector representation of the next node in Cartesian space;

[0037] Sum the Euclidean distances between all adjacent nodes in the initial moving path in the robotic arm Cartesian space to obtain the Euclidean distance d2 of the initial moving path in the robotic arm Cartesian space;

[0038] The calculation formula of the Euclidean distance Δa between two adjacent nodes in the initial moving path in the robot arm posture is:

[0039]

[0040] Among them, r, p, are the rotation angles around the x, y, and z axes in Cartesian space, r1, p1, is the rotation angle of the previous node around the x, y, z axis in Cartesian space, r2, p2, is the rotation angle of the latter node around the x, y, z axis in Cartesian space;

[0041] Sum the Euclidean distances between all adjacent nodes in the initial moving path at the robot arm posture to obtain the Euclidean distance Δθ of the initial moving path at the robot arm posture;

[0042] After obtaining the path distance costs of each initial moving path, compare them and select the initial moving path with the minimum path distance cost as the optimal moving path;

[0043] Set a path distance cost threshold. If the calculated optimal moving path's path distance cost is greater than the path distance cost threshold, re-search and generate the initial moving path.

[0044] If no initial moving path that meets the requirements is found after multiple searches, the search is stopped, a null value is output, and the on-site personnel are prompted.

[0045] As a preferred technical solution, in step 6, the secondary planning is specifically as follows:

[0046] If the target position does not move, the turning node in the optimal moving path is set as the beacon node q brand , introducing the intelligent sampling factor τ to expand the path and obtain the intelligent sampling area;

[0047] Combine the factory grid map with the smart sampling area to generate new sampling areas;

[0048] Customize the intelligent sampling factor τ to determine whether P>1-τ occurs. If so, select the beacon node q brand Start searching for the best moving path from the starting node to the beacon node q brand The path of the points remains unchanged;

[0049] If this does not happen, the starting node coordinates of the optimal moving path are used as the starting node and the search is iterated in the new sampling area;

[0050] When searching, a random node q rand The search formula is:

[0051]

[0052] Where P is a random number in [0,1], τ is a custom intelligent sampling factor, and its value is [0,1].

[0053] During the random node search process, when P>1-0.5τ occurs, the node q finally searched will be final As the final node of this search;

[0054] Get the final node q of this search in the new sampling area final The nearest neighbor node q nearst , calculate the new node q new , the formula is:

[0055]

[0056] Among them, d represents the step size or increment, which is used to determine the nearest neighbor node q nearest Expand to new node q new distance and direction;

[0057] Generate a new node q new After that, calculate and compare the new node q new Or the nearest neighbor node q nearst The path distance cost as the end point;

[0058] If the nearest neighbor node q nearstThe path distance cost as the end point is less than the path distance starting from the new node q new As the path distance cost of the end point, the nearest neighbor node q nearst As the final node, generate the final moving path; otherwise, use the new node q new As the final node, the final movement path is generated.

[0059] As a preferred technical solution, in step 7, the training steps of the improved adaptive search learning recurrent neural network are:

[0060] Initialize the parameters of the recurrent neural network, including population size, maximum number of iterations, scaling factor, perturbation factor, and initial learning rate;

[0061] Obtain historical samples of the robot arm's PID control parameters and divide them into training samples and test samples in proportion;

[0062] The training samples are input into the recurrent neural network, and the improved adaptive search learning algorithm is used to optimize the number of hidden layer nodes, learning rate and number of iterations of the recurrent neural network until the stopping condition is reached;

[0063] The test samples are input into the recurrent neural network with the optimal parameters to obtain the optimal PID parameters of the robotic arm motion.

[0064] As a preferred technical solution, the optimization steps of the improved adaptive search learning algorithm are:

[0065] Initialize the adaptive search learning algorithm, calculate the fitness value of each individual in the population and sort it, record the current best and worst individual information, and divide the discoverers, followers, and guards;

[0066] Adaptively adjust the number of discoverers and followers. The adaptive adjustment formula is:

[0067]

[0068] Among them, F num and G num are the number of discoverers and followers respectively; N is the population size; b is the proportional coefficient, which is used to control the ratio of discoverers and followers, and its value is (0,0.5); iter max is the maximum number of iterations; ρ is the perturbation factor, which takes a value of [0, 0.1] and is used to slightly perturb the nonlinear decreasing r; t is the current number of iterations;

[0069] Mutate the discoverer and follower, and update the positions of the discoverer, follower, and sentinel;

[0070] Determine whether the maximum number of iterations has been reached or whether the individual fitness value has reached the expected limit threshold. If not, recalculate the fitness value of each individual in the population and adjust the number of discoverers and followers for iterative learning; otherwise, output the number of discoverers, followers and alerts.

[0071] As a preferred technical solution, let the population N be expressed as:

[0072] X=[X1,X2,…,X N ] T ,X i =[X i,1 ,X i,2 ,…,X i,m ]

[0073] Among them, X i represents the i-th population, m is the number of individuals in the population;

[0074] The position of an individual in the population is expressed as: Among them, h1,h2,…,h n are the number of neurons in the n hidden layers of the recurrent neural network; ψ are the learning rate and number of iterations of the recurrent neural network;

[0075] The fitness value calculation formula is:

[0076]

[0077] Among them, f() is the fitness function;

[0078] The discoverers are individuals ranked in the top 10%-20% of fitness values; the vigilants are randomly selected from 10%-20% of individuals except the discoverers; the followers are individuals except the discoverers and vigilants;

[0079] When mutating the discoverer and follower, the Cauchy mutation distribution function is used to perform perturbations to achieve positional variation of the discoverer and follower;

[0080] The Cauchy variation distribution function f(d) is expressed as:

[0081]

[0082] Where d is a random number that is updated randomly and its value is between positive and negative infinity, and inf represents infinity;

[0083] After the positions of the discoverer and follower are mutated, the new positions are expressed as:

[0084]

[0085] in, is the individual position of the i-th individual in the discoverer or follower after the position mutation in the t-th iteration, is the individual position of the i-th individual in the discoverer or follower before the mutation in the t-th iteration.

[0086] As a preferred technical solution, the position update formula of the discoverer is:

[0087]

[0088] in, Indicates the position of the i-th individual in the discoverer at the t+1th iteration, t and iter max Represent the current number of iterations and the maximum number of iterations respectively; represents the position of the i-th individual in the discoverer after the t-th iteration; δ is a random number in the range (0,1); R2 represents the warning value, which is in the range [0,1]; ST represents the safety value, which is in the range [0.5,1]; Q represents a random number that obeys the [0,1] normal distribution; L represents a matrix whose elements are all 1;

[0089] The follower's position update formula is:

[0090]

[0091] in, is the position of the i-th individual in the follower at the t+1th iteration; is the position of the individual with the worst fitness value in the population after the tth iteration; is the position of the i-th individual in the follower after the t-th iteration; is the position of the p-th individual in the discoverer at the t+1th iteration; A represents a matrix in which each element is randomly assigned a value of -1 or 1, and satisfies A * =A * (AA T ) -1 ;

[0092] The position update formula of the sentinel is:

[0093]

[0094] in, is the position of the i-th individual in the vigilance at the t+1th iteration, is the position of the individual with the best fitness value among the vigilants after the tth iteration; ∈ is the parameter for controlling the step length, which obeys the normal distribution with mean 0 and variance 1; k is a random number in the range of [-1,1], which determines the movement direction of the individual in the population and is also the step length control parameter; f irepresents the fitness value of individual i in the vigilance after the tth iteration; f g represents the fitness value of the best individual among the vigilants after the tth iteration; f w Represents the fitness value of the worst individual among the vigilants; μ is a small constant used to avoid the denominator being 0.

[0095] As a preferred technical solution, the method further includes: setting robot arm movement constraints, introducing sparse rewards to set the reward function, evaluating the final movement path and recording the practicality of the path to facilitate subsequent optimization and improvement of the robot arm path planning;

[0096] The robot arm movement constraint is expressed as:

[0097]

[0098] Among them, V max 、ω max are the maximum moving speed and maximum yaw angular velocity of the manipulator, respectively; V1 and ω1 are the set maximum moving speed and yaw angular velocity of the manipulator, respectively;

[0099] The reward function is expressed as:

[0100] K=κ1+κ2+κ3

[0101] Among them, κ1 is the sparse reward, κ2 is the distance reward, and κ3 is the motion angle reward;

[0102] The sparse reward κ1 formula is:

[0103]

[0104] The formula for the motion angle reward κ3 is:

[0105]

[0106] Among them, i is the i-th joint angle in the current motion of the robot arm, θ i is the i-th joint angle of the end of the robot arm at the target object, and k is the number of joint angles of the robot arm.

[0107] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0108] 1. The dual planning concept of initial planning and secondary planning is used to plan the path. First, the established factory grid map is simplified to determine the initial movement path. Then, it is determined whether the target node has moved. The intelligent sampling factor is introduced, and secondary planning is performed in the new sampling area to obtain the final movement path. Based on the dual planning concept, the deviation of the robot arm caused by sudden changes in the target node can be avoided. The obstacle avoidance path of the robot arm is simplified, the deviation of the robot arm's movement and turning is reduced, and the smoothness of the overall operation is improved.

[0109] 2. Introducing adaptive adjustment and Cauchy variation function optimization. Traditional adaptive algorithms are prone to falling into local optimality. Iterative adjustment and mutation of the discoverer and follower are performed to optimize the global iteration position of the discoverer and follower, so as to avoid the algorithm from falling into local constraints and thus ending the iteration, and ensure the global search capability of the algorithm.

[0110] 3. An improved adaptive search learning algorithm is used to optimize the traditional recurrent neural network to improve the adaptive deep learning ability of the neural network; and the optimized recurrent neural network with adaptive learning function is used to learn and strengthen the robot arm movement PID parameters to improve the stability and continuity of the overall movement of the robot arm.

[0111] 4. Design a reasonable reward and punishment function for the robotic arm, introduce a sparse reward mechanism, and consider whether the robotic arm's movement angle changes. Make a full and objective evaluation of the robotic arm's learning speed and operation quality to facilitate subsequent improvements to the robotic arm's movement.

[0112] 5. Establish a simplified 2D grid map space to effectively avoid node route redundancy, prune routes with excessively high costs by calculating path distance costs, improve the success rate of path planning, and shorten the running time and number of iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0114] Figure 1 Flowchart of a robot arm path planning and control method based on dual planning and autonomous learning in an embodiment of the present invention.

[0115] Figure 2 Schematic diagram of traditional obstacle avoidance path planning based on positive feedback mechanism.

[0116] Figure 3 Flowchart of a recurrent neural network based on improved adaptive search learning in an embodiment of the present invention.

[0117] Figure 4 This is a flowchart of an improved adaptive search learning algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0118] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0119] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0120] like Figure 1 As shown, the robot arm path planning and control method based on dual planning and autonomous learning in this embodiment includes the following steps:

[0121] Step 1: Obtain the robot's location coordinates, movement speed, yaw angular velocity, and obstacle information to create a factory grid map.

[0122] Step 2: Simplify the factory grid map, remove irrelevant obstacle coordinate points, and obtain a simplified grid map;

[0123] Step 3: Use random sampling in the simplified grid map for initial planning and search to generate the initial movement path;

[0124] Step 4: Calculate the path distance cost of the initial moving path to select the optimal moving path; if the path distance cost of the optimal moving path is too high, re-search and generate the initial moving path;

[0125] Step 5: Determine whether the target position has moved. If so, change the target position and re-plan the initial movement path.

[0126] Step 6: If the target position has not moved, perform secondary planning and introduce the intelligent sampling factor to expand the guided path to obtain the intelligent sampling area. Combine the factory grid map with the intelligent sampling area to generate a new sampling area. Continuously iterate the search within the new sampling area to optimize the optimal movement path and obtain the final movement path.

[0127] Step 7: Use the recurrent neural network based on improved adaptive search learning to improve the PID control algorithm for the robot arm movement, perform network training on the PID control parameters separately, accelerate the convergence of PID control, and realize the control of the robot arm.

[0128] Furthermore, in order to obtain real-time information about the robot arm's operation, it is necessary to install a variety of inductive sensors on the robot arm:

[0129] A position sensor is placed on the base of the robotic arm, and its location coordinates (x, y, z) are sensed using a wireless sensor network composed of routers. Multiple speed sensors are placed on the robotic arm to obtain the arm's movement speed v and yaw angular velocity ω. A lidar is also used as the robotic arm's obstacle sensing sensor. A KD-tree-based Euclidean distance clustering method is used to cluster the lidar point cloud to obtain the coordinates, length, width, and height of obstacles relative to the robotic arm itself. This information is then used to create a factory grid map.

[0130] Further, such as Figure 2 As shown in the figure, the principle of the traditional obstacle avoidance path planning algorithm is to simulate the group animals in nature that release a gas with an information factor odor during the group foraging process to transmit information between groups. The gas concentration is inversely proportional to the path length. The higher the concentration, the shorter the path and the greater the probability of being selected. In this way, path selection is achieved through a positive feedback mechanism.

[0131] In order to reduce the time loss caused by the inverse solution of the robot arm's kinematics, this application improves the traditional obstacle avoidance path planning algorithm, introduces a dual planning path algorithm based on the dual planning idea, and performs initial planning and secondary planning to reduce the path operation cost and improve operation efficiency.

[0132] On the one hand, the task of initial planning is to divide the motion grid and perform pre-planning to obtain the initial movement path, specifically:

[0133] Set the node set of the initial moving path and set it to an empty set;

[0134] The coordinates of the robot's position are used as the starting node and stored sequentially in the node set of the initial movement path;

[0135] Search for a random node q in the simplified grid map rand , the search formula is:

[0136]

[0137] In formula (1), q goalis the target node at the target location, P is a random number in [0,1], and a is a custom constant whose value is determined by the number of obstacles. When there are fewer obstacles, a is set to a larger value, and when there are more obstacles, a is set to a smaller value;

[0138] Get a random node q in a simplified grid map rand The nearest neighbor node q nearst , calculate the new node q new , the formula is:

[0139]

[0140] In formula (2), d represents the step size or increment, which is used to determine the nearest neighbor node q nearest Expand to new node q new distance and direction;

[0141] After generating a new node, take the new node as the starting node and store it in the node set of the initial moving path in sequence. Continue to randomly sample and obtain random nodes for iterative search until P>1-a appears. Then, the random node q rand As the target node q goal And store them sequentially in the node set of the initial moving path, and end the search to generate the initial moving path.

[0142] Furthermore, after obtaining the initial movement path in step 3, it needs to be screened. Therefore, step 4 defines the path distance cost of the initial movement path. That is, when two initial movement paths are close to the target node position, the path distance cost of the initial movement path is determined by determining which initial movement path has the lowest path distance cost. The path distance cost of the initial movement path is generally defined by Euclidean distance. For a robot arm, the end position corresponding to two sets of joint angles with a small difference may be very different. In addition, due to the multi-solution nature of the inverse solution, the end position corresponding to two completely different joint angles may be completely consistent. Therefore, the path distance cost is defined by the weighted value of the Euclidean distance in the robot arm joint space, the Euclidean distance in the robot arm Cartesian space, and the Euclidean distance of the robot arm posture, expressed as:

[0143]

[0144] In formula (3), d1 is the Euclidean distance of the manipulator joint space, d2 is the position of the manipulator in Cartesian space, Δθ is the Euclidean distance of the manipulator posture; α, β, and γ are the weight values ​​of d1, d2, and Δθ respectively;

[0145] Among them, the Euclidean distance d1 of the robot arm joint space is obtained by summing the Euclidean distances between all adjacent nodes in the initial moving path in the robot arm joint space; the Euclidean distance d between two adjacent nodes in the initial moving path in the robot arm joint space is obtained by summing the Euclidean distances between all adjacent nodes in the initial moving path in the robot arm joint space.j The calculation formula is:

[0146]

[0147] In formula (4), k is the angle in the joint space of the robot arm, q 1i is the angle of the previous node i, q 2i is the i-th angle of the next node;

[0148] The Euclidean distance d2 in the Cartesian space of the robot arm is obtained by summing the Euclidean distances between all adjacent nodes in the initial moving path in the Cartesian space of the robot arm; the Euclidean distance d2 between two adjacent nodes in the initial moving path in the Cartesian space of the robot arm is obtained by summing the Euclidean distances between all adjacent nodes in the Cartesian space of the robot arm d The calculation formula is:

[0149]

[0150] In formula (5), (x1, y1, z1) is the vector representation of the previous node in Cartesian space, and (x2, y2, z2) is the vector representation of the next node in Cartesian space;

[0151] The Euclidean distance Δθ of the manipulator posture is obtained by summing the Euclidean distances between all adjacent nodes in the initial movement path at the manipulator posture. The calculation formula of the Euclidean distance Δa between two adjacent nodes in the initial movement path at the manipulator posture is:

[0152]

[0153] In formula (6), r, p, are the rotation angles around the x, y, and z axes in Cartesian space, r1, p1, is the rotation angle of the previous node around the x, y, z axis in Cartesian space, r2, p2, is the rotation angle of the latter node around the x, y, z axis in Cartesian space;

[0154] After obtaining the path distance costs of each initial moving path, compare them and select the initial moving path with the minimum path distance cost as the optimal moving path;

[0155] Set a path distance cost threshold. If the calculated path distance cost of the initial moving path is greater than the path distance cost threshold, a new search is performed to generate the initial moving path. If no initial moving path that meets the requirements is found after multiple searches, the search is stopped, a null value is output, and a prompt is given to the on-site personnel.

[0156] It should be understood that the adjacent nodes mentioned here refer to the adjacent nodes constituting the initial moving path in sequence, that is, calculating the path distance cost is to calculate the sum of the distances between the constituent nodes in the initial moving path in sequence.

[0157] On the other hand, the task of secondary programming is to determine whether the target object has moved, introduce intelligent sampling factors, correct the turning nodes of the optimal movement path selected in the initial planning, optimize the optimal movement path, and obtain the final movement path. Since the change of the target position may make the optimal movement path planned in the initial planning no longer applicable, and the movement path should minimize the number of turns, the optimal movement path needs to be re-searched and optimized; therefore, secondary programming first needs to determine whether the target position has moved. If it has moved, the target node position is first changed and the initial planning search is re-performed to generate the initial movement path; if there has been no movement, secondary planning is performed, specifically as follows:

[0158] If the target position does not move, the turning node in the optimal moving path is set as the beacon node q brand , the intelligent sampling factor τ is introduced to expand the path and obtain the intelligent sampling area; since the selection of turning nodes needs to consider the overall structure of the robotic arm during the movement and grasping process of the robotic arm, the turning nodes are selected from the spatial turning points corresponding to the mechanical axes used for obvious spatial transformation in the moving path of the robotic arm.

[0159] Combine the factory grid map with the smart sampling area to generate new sampling areas;

[0160] Customize the intelligent sampling factor τ to determine whether P>1-τ occurs. If so, select the beacon node q brand Start searching for the best moving path from the starting node to the beacon node q brand The path of the points remains unchanged;

[0161] If this does not happen, the starting node coordinates of the optimal moving path are used as the starting node and the search is iterated in the new sampling area;

[0162] When searching, a random node q rand The search formula is:

[0163]

[0164] In formula (7), P is a random number in [0, 1], and τ is a customized intelligent sampling factor, which is generally in the range of [0, 1]. However, if obstacles increase after the intelligent sampling area is combined with the factory grid map, the value of the intelligent sampling factor can be appropriately increased.

[0165] During the random node search process, when P>1-0.5τ occurs, the node q finally searched will be final As the final node of this search;

[0166] Get the final node q of this search in the new sampling area finalThe nearest neighbor node q nearst , calculate the new node q new , the formula is:

[0167]

[0168] In formula (8), d represents the step size or increment, which is used to determine the nearest neighbor node q nearest Expand to new node q new distance and direction;

[0169] Generate a new node q new After that, calculate and compare the new node q new Or the nearest neighbor node q nearst The path distance cost as the end point;

[0170] If the nearest neighbor node q nearst The path distance cost as the end point is less than the path distance starting from the new node q new As the path distance cost of the end point, the nearest neighbor node q nearst As the final node, generate the final moving path; otherwise, use the new node q new As the final node, the final movement path is generated.

[0171] By introducing intelligent sampling factors into the quadratic planning, the probability of generating random factors according to the random node search formula is increased. When expanding the secondary search, the number of nodes explored in the forward direction can be increased, the search length can be increased, and the purpose of optimizing the initial planning can be achieved. In this way, a cost-effective, collision-free, obstacle-avoidance path with a small number of turning angles can be obtained from the starting point to the final target point.

[0172] The dual planning path algorithm (OURS) based on the dual planning idea in this application is run in an assumed simplified 2D grid map space, with the starting point set to (0,0) and the end point set to (95,95). Under the same conditions of the initial point, target point, obstacles and other variables, three other common planning algorithms (A * , RRT, RRT * ) conducted the same comparative experiment, and conducted 10 comparisons. The number of iterations, running time, and path length were averaged over the number of successful attempts. The comparison results are shown in Table 1:

[0173]

[0174] Table 1 Algorithm performance comparison

[0175] It can be seen that for the same robotic arm operation scenario, all four algorithms have a high success rate. However, this application proposes a dual planning path algorithm (OURS) based on the dual planning concept, which introduces Euclidean distance to calculate the path distance cost for path finding, and determines whether the target position has moved, avoiding node route redundancy. Finally, the path cost is calculated to prune routes with excessively high costs. Therefore, according to the performance comparison table, compared with the other three, its success rate is higher, the running time and number of iterations are also the shortest, and the average path length does not change much compared with the shortest path.

[0176] Furthermore, since most robotic arms are multi-input, multi-output, and strongly nonlinear control systems, when completing a task, the multiple axes of the robotic arm cooperate with each other to ensure that the position of the end of the robotic arm meets the target requirements. Especially when completing tasks with high precision requirements, oscillation or overshoot during control will cause operational errors. It can be seen that a fast, stable, and high-precision control strategy is particularly important for the control of the robotic arm. Therefore, this application proposes a recurrent neural network with improved adaptive search learning to improve the PID control algorithm of the robotic arm movement, conduct network training for the PID control parameters separately, accelerate the convergence of PID control, and thus improve the control accuracy of the robotic arm.

[0177] For the recurrent neural network model, parameters such as the number of neurons in each hidden layer, learning rate, and number of iterations have a great impact on its performance; therefore, this application uses an improved adaptive search learning algorithm to optimize the model parameters of the recurrent neural network, thereby improving the diagnostic performance of the neural network, and optimizes the three parameters of the robot arm movement PID as network samples, such as Figure 3 As shown, it includes the following steps:

[0178] (1) Initialize the parameters of the recurrent neural network, including population size, maximum number of iterations, scale factor, perturbation factor, and initial learning rate;

[0179] (2) Obtain historical samples of the robot arm's PID control parameters and divide them into training samples and test samples in proportion;

[0180] (3) Input the training samples into the recurrent neural network and use the improved adaptive search learning algorithm to optimize the number of hidden layer nodes, learning rate and number of iterations of the recurrent neural network until the stopping condition is reached;

[0181] (4) The test samples are input into the recurrent neural network with the optimal parameters to obtain the optimal PID parameters of the robotic arm motion.

[0182] Furthermore, traditional recurrent neural networks such as LSTM and GRU have certain autonomous learning capabilities, but they have problems such as slow convergence speed and low accuracy, and cannot be directly applied to robotic arm movement. Therefore, the traditional adaptive algorithm is improved and applied to the recurrent neural network to improve the overall adaptive learning ability. Figure 4 As shown, the optimization steps of the improved adaptive search learning algorithm proposed in this application are:

[0183] (1) Initialize the adaptive search learning algorithm, calculate the fitness value of each individual in the population and sort it, record the current best and worst individual information, and divide it into discoverers, followers, and vigilants;

[0184] Suppose the population of N is expressed as:

[0185] X=[X1,X2,…,X N ] T ,X i =[X i,1 ,X i,2 ,…,X i,m ] (9)

[0186] In formula (9), X i represents the i-th population, m is the number of individuals in the population; the position of an individual in the population is expressed as: Among them, h1,h2,…,h n are the number of neurons in the n hidden layers of the recurrent neural network; ψ are the learning rate and number of iterations of the recurrent neural network;

[0187] The fitness calculation formula is:

[0188]

[0189] In formula (10), f() is the fitness function;

[0190] Individuals in the population are divided into three categories: discoverers, followers, and sentinels. Discoverers are individuals ranked in the top 10%-20% of fitness values. All other individuals are classified as followers before the start of an iteration, and there are no sentinels. After the iteration begins, fitness values ​​change. In each iteration, the top 10%-20% of individuals are selected as new discoverers, and 10%-20% of the remaining individuals are randomly selected as sentinels. In the event of an abnormal iteration, a warning is issued, terminating the iteration. Except for discoverers and sentinels, all others are followers.

[0191] (2) According to the traditional algorithm, the number of discoverers and followers is set at the beginning and will not change. This can easily lead to insufficient number of discoverers in the early stage of the algorithm, making it impossible to conduct a full search within a wide search range; in the later stage of the algorithm, there are too many discoverers and insufficient number of followers, making it impossible to conduct a detailed search within a smaller search range. To this end, it is necessary to adaptively adjust the number of discoverers and followers according to the iterative process to improve the algorithm's search accuracy. The adjustment formula is:

[0192]

[0193] In formula (11), F num and G num are the number of discoverers and followers respectively; N is the population size; b is the proportional coefficient, which is used to control the ratio of discoverers to followers. In order to ensure that the number of followers is greater than the number of discoverers, its value is (0,0.5); iter max is the maximum number of iterations; ρ is the perturbation factor, which takes a value of [0, 0.1] and is used to slightly perturb the nonlinear decreasing r; t is the current number of iterations;

[0194] (3) Mutate the discoverer and follower, and update the positions of the discoverer, follower, and sentinel;

[0195] In traditional algorithms, the positions of discoverers and followers are relatively fixed and unique, lacking a mutation mechanism. This can easily lead to insufficient population diversity and trapping in local optimal solutions. Therefore, it is necessary to increase the mutation behavior of individuals in the population to ensure population diversity. The Cauchy distribution is a distribution well suited for perturbing data. This paper uses it to perturb the positions of discoverers and followers in traditional algorithms to achieve position mutation. The Cauchy mutation distribution function is expressed as:

[0196]

[0197] In formula (12), d is a random number that is randomly updated and has a value between positive and negative infinity. Inf represents infinity and is the abbreviation of infinity. After the position mutation, the new positions obtained by the discoverer and follower are expressed as:

[0198]

[0199] In formula (13), is the individual position of the i-th individual in the discoverer or follower after the position mutation in the t-th iteration, is the individual position of the i-th individual in the discoverer or follower before the mutation in the t-th iteration.

[0200] Then update the positions of the discoverer, follower, and sentinel. The formula for updating the discoverer's position is:

[0201]

[0202] In formula (14), Indicates the position of the i-th individual in the discoverer at the t+1th iteration, t and iter max Represent the current number of iterations and the maximum number of iterations respectively; represents the position of the i-th individual in the discoverer after the t-th iteration; δ is a random number in the range (0,1); R2 represents the warning value, which is in the range [0,1]; ST represents the safety value, which is in the range [0.5,1]; Q represents a random number that obeys the [0,1] normal distribution; L represents a matrix whose elements are all 1;

[0203] The follower's position update formula is:

[0204]

[0205] In formula (15), is the position of the i-th individual in the follower at the t+1th iteration; is the position of the individual with the worst fitness value in the population after the tth iteration; is the position of the i-th individual in the follower after the t-th iteration; is the position of the p-th individual in the discoverer at the t+1th iteration; A represents a matrix in which each element is randomly assigned a value of -1 or 1, and satisfies A * =A * (AA T ) -1 ;

[0206] The position update formula of the sentinel is:

[0207]

[0208] In formula (16), is the position of the i-th individual in the vigilance at the t+1th iteration, is the position of the individual with the best fitness value among the vigilants after the tth iteration; ∈ is the parameter for controlling the step length, which obeys the normal distribution with mean 0 and variance 1; k is a random number in the range of [-1,1], which determines the movement direction of the individual in the population and is also the step length control parameter; f i represents the fitness value of individual i in the vigilance after the tth iteration; f g represents the fitness value of the best individual among the vigilants after the tth iteration; f w Represents the fitness value of the worst individual among the vigilants; μ is a small constant used to avoid the denominator being 0.

[0209] In this way, the positions of the discoverer and follower are updated through Cauchy mutation, ensuring that the algorithm has the ability to escape from the local optimality.

[0210] (4) Determine whether the maximum number of iterations has been reached or whether the individual fitness value has reached the expected limit threshold. If not, recalculate the fitness value of each individual in the population and adjust the number of discoverers and followers for iterative learning; otherwise, output the number of discoverers, followers, and alerters.

[0211] Furthermore, the method also includes the steps of setting the robot arm movement constraints, introducing sparse rewards to set the reward function, evaluating the final movement path and recording the practicality of the path, so as to facilitate the subsequent optimization and improvement of the robot arm path planning.

[0212] Specifically, there are infinite speed pairs (v, ω) in the speed space of the robot arm's movement process. The robot arm is constrained according to its own characteristic limitations and the factors of the surrounding environment. The maximum operating speed V of the robot arm can be known from the robot arm operation manual. max and the maximum angular velocity ω max In order to ensure the stable operation of the robot arm during operation, it is necessary to set the maximum operating speed V1 and angular velocity ω1 of the robot arm in advance and do the following:

[0213]

[0214] The speed and angular velocity of the operation are tracked according to the speed sensing sensor bound to the robotic arm. However, if it is detected that the maximum operating speed V1 and angular velocity ω1 set are exceeded, an alarm warning will be issued and the operation of the robotic arm will be terminated in advance.

[0215] Since the robot arm often needs to make multiple autonomous decisions during continuous operations when performing tasks, and the motion reward signal may only be given at certain specific nodes, it is difficult to design the robot arm's reward and punishment function. Therefore, the idea of ​​sparse rewards is introduced, and a reward function is set up, which consists of a sparse reward κ1, a distance reward κ2, and a motion angle reward κ3, which can be expressed as:

[0216] K=κ1+κ2+κ3 (18).

[0217] The sparse reward κ1 is obtained at the last step of each exploration round. If the arrival task is completed within the maximum movement time, a positive reward stimulus is obtained. The maximum movement time is pre-set based on the robot arm's movement distance. If it is not achieved, no reward is given. The essence of sparse rewards is that only a small portion of positive rewards and negative penalties can provide a reward stimulus. Therefore, when manually designing rewards, it is necessary to modify the design based on actual conditions and experience. The formula is:

[0218]

[0219] After the robot arm completes the action and reaches the next node, it will receive a distance reward κ2 related to its current position. In addition, the difference in the movement joint angle is also included in the reward function, that is, a movement angle reward κ3 is determined, which is formulated as follows:

[0220]

[0221] In formula (20), o i is the i-th joint angle in the current motion of the robot arm, θ i is the i-th joint angle of the end of the robot arm at the target object, and k is the number of joint angles of the robot arm.

[0222] To sum up, through the method proposed in this application, the operation trajectory of the robot arm can be reasonably planned first, the turning points in the path movement process can be reduced, the cost of the robot arm operation can be reduced, and the position of the target node can be verified for the second time, thereby improving the accuracy of the robot arm movement precision; and in the process of robot arm movement, with the help of a recurrent neural network with self-learning function, the convergence of the network can be accelerated, and the PID parameter value of the robot arm operation can be quickly determined, thereby improving the stability of the robot arm operation, and using a reasonable reward function to reasonably evaluate the final moving path and record the practicality of the path, so as to facilitate the subsequent optimization and improvement of the robot arm path planning operation (such as adjustment of the intelligent sampling factor, etc.).

[0223] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0224] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A robotic arm path planning and control method based on dual planning and autonomous learning, characterized in that: The steps include: Step 1: Obtain the robot's location coordinates, movement speed, yaw angular velocity, and obstacle information to create a factory grid map. Step 2: Simplify the factory grid map, remove irrelevant obstacle coordinate points, and obtain a simplified grid map; Step 3: Use random sampling in the simplified grid map for initial planning and search to generate the initial movement path, specifically: Set the node set of the initial moving path and set it to an empty set; The coordinates of the robot's position are used as the starting node and stored sequentially in the node set of the initial movement path; Search for a random node q in the simplified grid map rand , the search formula is: Among them, q goal is the target node at the target location, P is a random number in [0,1], and a is a custom constant whose value is determined by the number of obstacles; Get a random node q in a simplified grid map rand The nearest neighbor node q nearst , calculate the new node q new , the formula is: Among them, d represents the step size or increment, which is used to determine the nearest neighbor node q nearst Expand to new node q new distance and direction; After generating a new node, take the new node as the starting node and store it in the node set of the initial moving path in sequence. Continue to randomly sample and obtain random nodes for iterative search until P>1-a appears. Then, the random node q rand As the target node q goal And sequentially store them in the node set of the initial moving path, ending the search to generate the initial moving path; Step 4: Calculate the path distance cost of the initial moving path to select the optimal moving path; if the path distance cost of the optimal moving path is too high, re-search and generate the initial moving path; Step 5: Determine whether the target position has moved. If so, change the target position and re-plan the initial movement path. Step 6: If the target position has not moved, perform secondary planning and introduce the intelligent sampling factor to expand the guided path to obtain the intelligent sampling area. Combine the factory grid map with the intelligent sampling area to generate a new sampling area. Continuously iterate the search within the new sampling area to optimize the optimal movement path and obtain the final movement path. Step 7: Use the recurrent neural network based on improved adaptive search learning to improve the PID control algorithm for the robot arm movement, perform network training on the PID control parameters separately, accelerate the convergence of PID control, and realize the control of the robot arm.

2. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 1, characterized in that: The base of the robotic arm is equipped with a position sensing sensor, which senses its position coordinates (x, y, z) with the help of a wireless sensor network composed of routers; A plurality of speed sensing sensors are placed on the robotic arm to obtain the moving speed v and the yaw angular velocity ω of the robotic arm; The robotic arm is also equipped with a laser radar as an obstacle sensing sensor of the robotic arm. The laser radar point cloud is clustered using the Euclidean distance clustering method based on the KD tree to obtain the coordinates and length, width and height of the obstacle relative to the robotic arm itself.

3. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 1, characterized in that: In step 4, the path distance cost calculation formula of the initial moving path is: Where d1 is the Euclidean distance in the joint space of the manipulator, d2 is the Euclidean distance in the Cartesian space of the manipulator, and Δθ is the Euclidean distance of the manipulator posture; α, β, and γ are the weights of d1, d2, and Δθ respectively; The Euclidean distance d between two adjacent nodes in the initial moving path in the manipulator joint space j The calculation formula is: Where k is the angle in the joint space of the robot arm, q 1i is the angle of the previous node i, q 2i is the i-th angle of the next node; Sum the Euclidean distances between all adjacent nodes in the initial movement path in the manipulator joint space to obtain the Euclidean distance d1 of the initial movement path in the manipulator joint space; The Euclidean distance d between two adjacent nodes in the initial moving path in the Cartesian space of the robot arm d The calculation formula is: Among them, (x1, y1, z1) is the vector representation of the previous node in Cartesian space, and (x2, y2, z2) is the vector representation of the next node in Cartesian space; Sum the Euclidean distances between all adjacent nodes in the initial moving path in the robotic arm Cartesian space to obtain the Euclidean distance d2 of the initial moving path in the robotic arm Cartesian space; The calculation formula of the Euclidean distance Δa between two adjacent nodes in the initial moving path in the robot arm posture is: Among them, r, p, are the rotation angles around the x, y, and z axes in Cartesian space, r1, p1, is the rotation angle of the previous node around the x, y, z axis in Cartesian space, r2, p2, is the rotation angle of the latter node around the x, y, z axis in Cartesian space; Sum the Euclidean distances between all adjacent nodes in the initial moving path at the robot arm posture to obtain the Euclidean distance Δθ of the initial moving path at the robot arm posture; After obtaining the path distance costs of each initial moving path, compare them and select the initial moving path with the minimum path distance cost as the optimal moving path; Set a path distance cost threshold. If the calculated optimal moving path's path distance cost is greater than the path distance cost threshold, re-search and generate the initial moving path. If no initial moving path that meets the requirements is found after multiple searches, the search is stopped, a null value is output, and the on-site personnel are prompted.

4. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 1, characterized in that: In step 6, the secondary planning is specifically as follows: If the target position does not move, the turning node in the optimal moving path is set as the beacon node q brand , introducing the intelligent sampling factor τ to expand the path and obtain the intelligent sampling area; Combine the factory grid map with the smart sampling area to generate new sampling areas; Customize the intelligent sampling factor τ to determine whether P>1-τ occurs. If so, select the beacon node q brand Start searching for the best moving path from the starting node to the beacon node q brand The path of the points remains unchanged; If this does not happen, the starting node coordinates of the optimal moving path are used as the starting node and the search is iterated in the new sampling area; When searching, a random node q rand The search formula is: Where P is a random number in [0,1], τ is a custom intelligent sampling factor, and its value is [0,1]. During the random node search process, when P>1-0.5τ occurs, the node q finally searched will be final As the final node of this search; Get the final node q of this search in the new sampling area final The nearest neighbor node q nearst , calculate the new node q new , the formula is: Among them, d represents the step size or increment, which is used to determine the nearest neighbor node q nearest Expand to new node q new distance and direction; Generate a new node q new After that, calculate and compare the new node q new Or the nearest neighbor node q nearst The path distance cost as the end point; If the nearest neighbor node q nearst The path distance cost as the end point is less than the path distance starting from the new node q new As the path distance cost of the end point, the nearest neighbor node q nearst As the final node, generate the final moving path; otherwise, use the new node q new As the final node, the final movement path is generated.

5. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 1, characterized in that: In step 7, the training steps of the improved adaptive search learning recurrent neural network are: Initialize the parameters of the recurrent neural network, including population size, maximum number of iterations, scaling factor, perturbation factor, and initial learning rate; Obtain historical samples of the robot arm's PID control parameters and divide them into training samples and test samples in proportion; The training samples are input into the recurrent neural network, and the improved adaptive search learning algorithm is used to optimize the number of hidden layer nodes, learning rate and number of iterations of the recurrent neural network until the stopping condition is reached; The test samples are input into the recurrent neural network with the optimal parameters to obtain the optimal PID parameters of the robotic arm motion.

6. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 5, characterized in that: The optimization steps of the improved adaptive search learning algorithm are: Initialize the adaptive search learning algorithm, calculate the fitness value of each individual in the population and sort it, record the current best and worst individual information, and divide the discoverers, followers, and guards; Adaptively adjust the number of discoverers and followers. The adaptive adjustment formula is: Among them, F num and G num are the number of discoverers and followers respectively; N is the population size; b is the proportional coefficient, which is used to control the ratio of discoverers and followers, and its value is (0,0.5); iter max is the maximum number of iterations; ρ is the perturbation factor, which takes a value of [0, 0.1] and is used to slightly perturb the nonlinear decreasing r; t is the current number of iterations; Mutate the discoverer and follower, and update the positions of the discoverer, follower, and sentinel; Determine whether the maximum number of iterations has been reached or whether the individual fitness value has reached the expected limit threshold. If not, recalculate the fitness value of each individual in the population and adjust the number of discoverers and followers for iterative learning; otherwise, output the number of discoverers, followers and alerts.

7. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 6, characterized in that: Suppose the population of N is expressed as: X=[X1,X2,…,X N ] T ,X i [X i,1 ,X i,2 ,…,X i,m ] Among them, X i represents the i-th population, m is the number of individuals in the population; The position of an individual in the population is expressed as: Among them, h1,h2,…,h n are the number of neurons in the n hidden layers of the recurrent neural network; ψ are the learning rate and number of iterations of the recurrent neural network; The fitness value calculation formula is: Among them, f() is the fitness function; The discoverers are individuals ranked in the top 10%-20% of fitness values; the vigilants are randomly selected from 10%-20% of individuals except the discoverers; the followers are individuals except the discoverers and vigilants; When mutating the discoverer and follower, the Cauchy mutation distribution function is used to perform perturbations to achieve positional variation of the discoverer and follower; The Cauchy variation distribution function f(d) is expressed as: Where d is a random number that is updated randomly and its value is between positive and negative infinity, and inf represents infinity; After the positions of the discoverer and follower are mutated, the new positions are expressed as: in, is the individual position of the i-th individual in the discoverer or follower after the position mutation in the t-th iteration, is the individual position of the i-th individual in the discoverer or follower before the mutation in the t-th iteration.

8. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 7, characterized in that: The position update formula of the discoverer is: in, Indicates the position of the i-th individual in the discoverer at the t+1th iteration, t and iter max Represent the current number of iterations and the maximum number of iterations respectively; represents the position of the i-th individual in the discoverer after the t-th iteration; δ is a random number in the range (0,1); R2 represents the warning value, which is in the range [0,1]; ST represents the safety value, which is in the range [0.5,1]; Q represents a random number that obeys the [0,1] normal distribution; L represents a matrix whose elements are all 1; The follower's position update formula is: in, is the position of the i-th individual in the follower at the t+1th iteration; is the position of the individual with the worst fitness value in the population after the tth iteration; is the position of the i-th individual in the follower after the t-th iteration; is the position of the p-th individual in the discoverer at the t+1th iteration; A represents a matrix in which each element is randomly assigned a value of -1 or 1, and satisfies A * =A * (AA T ) -1 ; The position update formula of the sentinel is: in, is the position of the i-th individual in the vigilance at the t+1th iteration, is the position of the individual with the best fitness value among the vigilants after the tth iteration; ∈ is the parameter for controlling the step length, which obeys the normal distribution with mean 0 and variance 1; k is a random number in the range of [-1,1], which determines the movement direction of the individual in the population and is also the step length control parameter; f i represents the fitness value of individual i in the vigilance after the tth iteration; f g represents the fitness value of the best individual among the vigilants after the tth iteration; f w Represents the fitness value of the worst individual among the vigilants; μ is a small constant used to avoid the denominator being 0.

9. The robot arm path planning and control method based on dual planning and autonomous learning according to claim 1, characterized in that: The method further includes: setting movement constraints for the manipulator, introducing a sparse reward setting reward function, evaluating the final movement path and recording the practicality of the path to facilitate subsequent optimization and improvement of the manipulator path planning; The robot arm movement constraint is expressed as: Among them, V max 、ω max are the maximum moving speed and maximum yaw angular velocity of the manipulator, respectively; V1 and ω1 are the set maximum moving speed and yaw angular velocity of the manipulator, respectively; The reward function is expressed as: K=κ1+κ2+κ3 Among them, κ1 is the sparse reward, κ2 is the distance reward, and κ3 is the motion angle reward; The sparse reward κ1 formula is: The formula for the motion angle reward κ3 is: Among them, i is the i-th joint angle in the current motion of the robot arm, θ i is the i-th joint angle of the end of the robot arm at the target object, and k is the number of joint angles of the robot arm.

Images