Dynamic obstacle avoidance method for robotic arms based on intelligent task supervision

Abstract

The present invention relates to a dynamic obstacle avoidance method for a manipulator based on intelligent task supervision, comprising the following steps: S1: collecting a set of position information; S2: constructing basic behaviors and composite behaviors for zero-space behavior control based on the state of the manipulator and the requirements of trajectory planning, and setting the behavior priority as action information; S3: setting a control obstacle function based on the safety distance requirements of the manipulator; S4: constructing a deep reinforcement learning training model; S5: inputting the set of position information and the action information into the deep reinforcement learning training model, obtaining the corresponding reward according to the set reward function, and if the distance between the end of the manipulator and the obstacle is less than the minimum safety distance according to the information, the input of the manipulator is taken over by the control obstacle function. The present invention effectively improves the flexibility and adaptability of the manipulator in a dynamic obstacle environment and avoids collision accidents.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic obstacle avoidance of a robot arm, and in particular to a dynamic obstacle avoidance method of a robot arm based on intelligent task supervision. Background Art

[0002] In the past decade, trajectory planning has made significant progress and has been widely used in robotics, automated control, and unmanned driving, becoming one of their core technologies. With the improvement of computing power, the advancement of sensor technology, and the development of artificial intelligence algorithms, trajectory planning technology has not only made breakthroughs in theoretical research, but also shown broad prospects in practical applications, among which industrial robotic arms are the most widely used.

[0003] With the transformation and upgrading of traditional factories, in today's manufacturing workshops, the position of workpieces may change frequently, and other robotic arms or workers may also move in the work area. In the case of such dynamic obstacles, trajectory planning and obstacle avoidance tasks are prone to conflict, resulting in poor planned trajectories and even collision accidents. The zero-space behavior control method has excellent performance in dynamic environments because it can better handle multi-task conflicts.

[0004] Nevertheless, since the task priority of zero-space behavior control is fixed, it cannot be dynamically adjusted according to environmental changes during actual execution, which limits the adaptability and flexibility of the zero-space behavior control method. Reinforcement learning has the characteristics of highly autonomous learning, so reinforcement learning is used as the task supervisor of behavior control. However, although reinforcement learning can better deal with static obstacles, it still has deficiencies in the stability of obstacle avoidance when dealing with dynamic obstacles, and cannot ensure the safety of the robot arm. Summary of the invention

[0005] In order to solve the above problems, the purpose of the present invention is to provide a dynamic obstacle avoidance method for a robotic arm based on intelligent task supervision, which effectively improves the flexibility and adaptability of the robotic arm in a dynamic obstacle environment and avoids collision accidents.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision comprises the following steps:

[0008] S1: Collect the position information of the end effector of the robot arm, the position information of the target point and the position information of the obstacle to obtain a position information set;

[0009] S2: Based on the state of the robot and the requirements of trajectory planning, construct basic behaviors and composite behaviors for zero-space behavior control, and set behavior priorities as action information;

[0010] S3: Setting the control obstacle function based on the safety distance requirements of the robot arm;

[0011] S4: construct a deep reinforcement learning training model, initialize the network parameters of the deep reinforcement learning training model based on the Actor-Critic algorithm, construct a state space function corresponding to the position information and action information, and set a reward function;

[0012] S5: Input the position information set and the action information into the deep reinforcement learning training model, obtain the corresponding reward according to the set reward function, store the position information, the action information and the reward into the experience pool, and select samples from the experience pool according to the rules to update the Actor-Critic algorithm. If the distance between the end of the robot arm and the obstacle is less than the minimum safety distance according to the information, the input of the robot arm is taken over by the obstacle control function.

[0013] Furthermore, basic behaviors include motion behaviors and obstacle avoidance behaviors, specifically:

[0014] In sports behavior, definition is the controllable task variable that drives the robot arm to move along the desired trajectory. The position coordinates of the end effector of the robot arm are , the reference speed of the motion behavior for:

[0015]

[0016]

[0017] in, R represents a real number, for The pseudo-inverse matrix of , is the Jacobian matrix of the robot, is the Jacobian matrix of the motion behavior; is the expected task function, is the first-order derivative of the expected task function, representing the speed of the expected task; For task gain, is the task error; Represents a task as a three-dimensional vector, and each component is a real number.

[0018] In obstacle avoidance behavior, we define To enable the robot end effector to provide obstacle avoidance support while moving along the desired trajectory, the controllable task variables of the obstacle avoidance behavior are: for:

[0019]

[0020]

[0021] in for The pseudo-inverse matrix of , is the Jacobian matrix of the obstacle avoidance behavior; is the obstacle coordinate; For task gain; is the task error, For a safe distance.

[0022] Furthermore, the composite behavior includes a first composite behavior and a second composite behavior, which are as follows:

[0023] In the first composite behavior, the motion behavior is the main task and the obstacle avoidance behavior is the auxiliary task. In terms of priority, the motion behavior is higher than the obstacle avoidance behavior. The speed of the first composite behavior is as follows:

[0024]

[0025] in The null space for obstacle avoidance behavior; is the identity matrix;

[0026] In the second composite behavior, obstacle avoidance is the main task and motion is the auxiliary task. In terms of priority, obstacle avoidance is higher than motion. The speed of the second composite behavior is as follows:

[0027]

[0028] in is the null space of motion behavior.

[0029] Furthermore, the priority setting is as follows: when the priority is set to 0, it is the first composite behavior , when the priority is 1, it is the second composite behavior .

[0030] Furthermore, S3 is specifically:

[0031] Set two safety distances: one for , used during the Actor-Critic algorithm training to promote the learning and updating of the agent strategy, and the other is , which is used to control the obstacle function to ensure the completion of the obstacle avoidance task, where is a preset real number, specifically:

[0032] definition is a safe set, and its expression is:

[0033]

[0034] in is the control barrier function, Indicates system status is safe; safe set is the state space A subset of , containing all safe states; express for A subset of for dimensional real number space;

[0035] To ensure system status To always stay in the safe set, the equation must be satisfied:

[0036]

[0037] in is the gain parameter used to adjust Growth rate; is the input quantity; is the first-order derivative of the control barrier function, which represents the rate of change of the control barrier function.

[0038] For affine systems ,in, is the first-order derivative of the system state, is the dynamic behavior inside the system, For input The impact on the system status will be Substituting in, we get:

[0039]

[0040] Define the state quantity of the end effector of the robot arm as , the minimum safe distance is The expected speed is , the safety distance function is , define the control barrier function as:

[0041]

[0042] in are the obstacle coordinates, is the coordinate of the end effector of the robot arm;

[0043] After derivation, we get:

[0044]

[0045] In order to minimize the task error, the objective function is set as ,in is the expected input; the entire control barrier function is solved as a quadratic programming problem:

[0046] .

[0047] Furthermore, S4 is specifically:

[0048] The network parameters of the deep reinforcement learning training model are initialized based on the Actor-Critic algorithm. The state space expression of the deep learning algorithm is:

[0049]

[0050] in is the rotation angle of the seven joints of the robotic arm; is the linear velocity of the end effector of the robot arm; is the distance between the end effector of the robot and the obstacle; is the distance between the end effector of the robot and the target point;

[0051] The expression based on trajectory error, safety constraint and reward function is:

[0052]

[0053]

[0054]

[0055]

[0056] in is the expected trajectory error of the end effector of the robot arm, is the sampling time, for The position of the end effector of the robot arm at the moment, is the sum of the radius of the obstacle and the radius of the end effector of the robot arm; the entire reward function consists of three parts, namely Tracking error reward function, Minimum distance reward function, Collision penalty reward function, after adding the three reward values, we get the final reward value.

[0057] Furthermore, S5 is specifically:

[0058] definition for the environment; is the state set of the end effector of the robot arm, is the initial state; It is a set of compound behaviors. The robot needs to select compound behaviors based on the environment to complete the task. For the experience replay pool, is the capacity of the experience replay pool, is the total time step of one round, is the distance between the end effector of the robot and the obstacle, is the total training rounds;

[0059] During the entire task supervision process, the agent and the environment To interact, in state When a composite behavior is selected according to the Actor network's strategy , the agent is performing compound actions Afterwards, the environment will give the agent a reward As feedback, the agent then enters the next state , which generates a four-tuple This quadruple will be used as the experience for future learning and stored in the experience replay pool of the reinforcement learning algorithm. If the amount of experience is equal to the experience replay pool capacity , the experience replay pool will delete the earliest stored quadruple to store new experience;

[0060] In trajectory planning, the reinforcement learning task supervisor will select the task priority of zero-space behavior according to the learned strategy and guide the robot arm to perform trajectory planning. When the distance between the end effector of the robot arm and the obstacle is less than the minimum safety distance, the control obstacle function will take over the input of the robot arm to control the robot arm to avoid collision accidents.

[0061] The present invention has the following beneficial effects:

[0062] 1. This invention introduces a deep reinforcement learning model to enable the robot arm to dynamically learn and adjust strategies according to the actual environment. In a complex and uncertain environment, the robot arm can autonomously adapt to the distribution and changes of obstacles, thereby achieving efficient obstacle avoidance and trajectory planning;

[0063] 2. The present invention uses the zero-space behavior control method, combined with the priority management of basic behaviors and compound behaviors, to effectively achieve the coordination between the robot arm in meeting the main task (such as reaching the target point) and the obstacle avoidance task, thereby avoiding task conflicts. At the same time, the zero-space behavior control supports the robot arm to dynamically avoid obstacles without changing the main task, thereby improving the flexibility of the system;

[0064] 3. By setting the obstacle control function, the present invention can take over the control of the robotic arm in time when the distance between the robotic arm and the obstacle is lower than the minimum safety distance, avoiding potential collision risks, providing multiple safety protection mechanisms, and improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0066] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0067] refer to Figure 1 In this embodiment, a method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision is provided, comprising the following steps:

[0068] S1: Collect the position information of the end effector of the robot arm, the position information of the target point and the position information of the obstacle to obtain a position information set;

[0069] S2: Based on the state of the robot and the requirements of trajectory planning, construct basic behaviors and composite behaviors for zero-space behavior control, and set behavior priorities as action information;

[0070] S3: Setting the control obstacle function based on the safety distance requirements of the robot arm;

[0071] S4: construct a deep reinforcement learning training model, initialize the network parameters of the deep reinforcement learning training model based on the Actor-Critic algorithm, construct a state space function corresponding to the position information and action information, and set a reward function;

[0072] S5: Input the position information set and the action information into the deep reinforcement learning training model, obtain the corresponding reward according to the set reward function, store the position information, the action information and the reward into the experience pool, and select samples from the experience pool according to the rules to update the Actor-Critic algorithm. If the distance between the end of the robot arm and the obstacle is less than the minimum safety distance according to the information, the input of the robot arm is taken over by the obstacle control function.

[0073] Furthermore, basic behaviors include motion behaviors and obstacle avoidance behaviors, specifically:

[0074] In sports behavior, definition is the controllable task variable that drives the robot arm to move along the desired trajectory. The position coordinates of the end effector of the robot arm are , the reference speed of the motion behavior for:

[0075]

[0076]

[0077] in, R represents a real number, for The pseudo-inverse matrix of , is the Jacobian matrix of the robot, is the Jacobian matrix of the motion behavior; is the expected task function, is the first-order derivative of the expected task function, representing the speed of the expected task; For task gain, is the task error; Represents a task as a three-dimensional vector, and each component is a real number.

[0078] In obstacle avoidance behavior, we define To enable the robot end effector to provide obstacle avoidance support while moving along the desired trajectory, the controllable task variables of the obstacle avoidance behavior are: for:

[0079]

[0080]

[0081] in for The pseudo-inverse matrix of , is the Jacobian matrix of the obstacle avoidance behavior; is the obstacle coordinate; For task gain; is the task error, For a safe distance.

[0082] Furthermore, the composite behavior includes a first composite behavior and a second composite behavior, which are as follows:

[0083] In the first composite behavior, the motion behavior is the main task and the obstacle avoidance behavior is the auxiliary task. In terms of priority, the motion behavior is higher than the obstacle avoidance behavior. The speed is:

[0084]

[0085] in The null space for obstacle avoidance behavior; is the identity matrix;

[0086] In the second composite behavior, obstacle avoidance is the main task and motion is the auxiliary task. In terms of priority, obstacle avoidance is higher than motion. The speed is:

[0087]

[0088] in is the null space of motion behavior.

[0089] Furthermore, the priority setting is as follows: when the priority is set to 0, it is the first composite behavior , when the priority is 1, it is the second composite behavior .

[0090] Furthermore, S3 is specifically:

[0091] Set two safety distances: one for , used during the Actor-Critic algorithm training to promote the learning and updating of the agent strategy, and the other is , which is used to control the obstacle function to ensure the completion of the obstacle avoidance task, where is a preset real number, specifically:

[0092] definition is a safe set, and its expression is:

[0093]

[0094] in is the control barrier function, Indicates system status is safe; safe set is the state space A subset of , containing all safe states; express for A subset of for dimensional real number space;

[0095] To ensure system status To always stay in the safe set, the equation must be satisfied:

[0096]

[0097] in is the gain parameter used to adjust The growth rate, is the input quantity; is the first-order derivative of the control barrier function, which represents the rate of change of the control barrier function.

[0098] For affine systems ,in, is the first-order derivative of the system state, is the dynamic behavior inside the system, For input The impact on the system status will be Substituting in, we get:

[0099]

[0100] Define the state quantity of the end effector of the robot arm as , the minimum safe distance is The expected speed is , the safety distance function is , define the control barrier function as:

[0101]

[0102] in are the obstacle coordinates, is the coordinate of the end effector of the robot arm;

[0103] After derivation, we get:

[0104]

[0105] In order to minimize the task error, the objective function is set as ,in is the expected input; the entire control barrier function is solved as a quadratic programming problem:

[0106] .

[0107] Furthermore, S4 is specifically:

[0108] The network parameters of the deep reinforcement learning training model are initialized based on the Actor-Critic algorithm. The state space expression of the deep learning algorithm is:

[0109]

[0110] in is the rotation angle of the seven joints of the robotic arm; is the linear velocity of the end effector of the robot arm; is the distance between the end effector of the robot and the obstacle; is the distance between the end effector of the robot and the target point;

[0111] The expression based on trajectory error, safety constraint and reward function is:

[0112]

[0113]

[0114]

[0115]

[0116] in is the expected trajectory error of the end effector of the robot arm, is the sampling time, for The position of the end effector of the robot arm at the moment, is the sum of the radius of the obstacle and the radius of the end effector of the robot arm; the entire reward function consists of three parts, namely Tracking error reward function, Minimum distance reward function, Collision penalty reward function, after adding the three reward values, we get the final reward value.

[0117] Furthermore, S5 is specifically:

[0118] definition for the environment; is the state set of the end effector of the robot arm, is the initial state; It is a set of compound behaviors. The robot needs to select compound behaviors based on the environment to complete the task. For the experience replay pool, is the capacity of the experience replay pool, is the total time step of one round, is the distance between the end effector of the robot and the obstacle, is the total training rounds;

[0119] During the entire task supervision process, the agent and the environment To interact, in state When a composite behavior is selected according to the Actor network's strategy , the agent is performing compound actions Afterwards, the environment will give the agent a reward As feedback, the agent then enters the next state , which generates a four-tuple This quadruple will be used as the experience for future learning and stored in the experience replay pool of the reinforcement learning algorithm. If the amount of experience is equal to the experience replay pool capacity , the experience replay pool will delete the earliest stored quadruple to store new experience;

[0120] In trajectory planning, the reinforcement learning task supervisor will select the task priority of zero-space behavior according to the learned strategy and guide the robot arm to perform trajectory planning. When the distance between the end effector of the robot arm and the obstacle is less than the minimum safety distance, the control obstacle function will take over the input of the robot arm to control the robot arm to avoid collision accidents.

[0121] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0125] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A dynamic obstacle avoidance method for a robotic arm based on intelligent task supervision, characterized in that: The following steps are involved: S1: Collect the position information of the end effector of the robot arm, the position information of the target point and the position information of the obstacle to obtain a position information set; S2: Based on the state of the robot and the requirements of trajectory planning, construct basic behaviors and composite behaviors for zero-space behavior control, and set behavior priorities as action information; S3: Setting the control obstacle function based on the safety distance requirements of the robot arm; S4: construct a deep reinforcement learning training model, initialize the network parameters of the deep reinforcement learning training model based on the Actor-Critic algorithm, construct a state space function corresponding to the position information and action information, and set a reward function; S5: Input the position information set and the action information into the deep reinforcement learning training model, obtain the corresponding reward according to the set reward function, store the position information, the action information and the reward into the experience pool, and select samples from the experience pool according to the rules to update the Actor-Critic algorithm. If the distance between the end of the robot arm and the obstacle obtained according to the position information is less than the minimum safety distance, the input of the robot arm is taken over by the control obstacle function; The basic behaviors include motion behaviors and obstacle avoidance behaviors, specifically: In sports behavior, definition is the controllable task variable that drives the robot arm to move along the desired trajectory. The position coordinates of the end effector of the robot arm are , the reference speed of the motion behavior for: in, R represents a real number, for The pseudo-inverse matrix of , is the Jacobian matrix of the robot, is the Jacobian matrix of the motion behavior; is the expected task function, is the first-order derivative of the expected task function, representing the speed of the expected task; For task gain, is the task error; Represents that the task is a three-dimensional vector, and each component is a real number; In obstacle avoidance behavior, we define To enable the robot end effector to provide obstacle avoidance support while moving along the desired trajectory, the controllable task variables of the obstacle avoidance behavior are: for: in for The pseudo-inverse matrix of , is the Jacobian matrix of the obstacle avoidance behavior; is the obstacle coordinate; For task gain; is the task error, For a safe distance.

2. The method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision according to claim 1, characterized in that: The composite behavior includes a first composite behavior and a second composite behavior, which are as follows: In the first composite behavior, the motion behavior is the main task and the obstacle avoidance behavior is the auxiliary task. In terms of priority, the motion behavior is higher than the obstacle avoidance behavior. The speed is: in is the null space for obstacle avoidance behavior, is the identity matrix; In the second composite behavior, obstacle avoidance is the main task and motion is the auxiliary task. In terms of priority, obstacle avoidance is higher than motion. The speed is: in is the null space of motion behavior.

3. The method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision according to claim 2, characterized in that: Priority setting, specifically: Setting the priority to 0 is the first composite behavior , when the priority is 1, it is the second composite behavior .

4. The method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision according to claim 1, characterized in that: The S3 is specifically: Set two safety distances: one for , used during the Actor-Critic algorithm training to promote the learning and updating of the agent strategy, and the other is , which is used to control the obstacle function to ensure the completion of the obstacle avoidance task, where is a preset real number, specifically: definition is a safe set, and its expression is: ； in is the control barrier function, Indicates system status is safe; safe set is the state space A subset of , containing all safe states; express for A subset of for dimensional real number space; To ensure system status To always stay in the safe set, the equation must be satisfied: ； in is the gain parameter used to adjust Growth rate; is the input quantity; is the first-order derivative of the control obstacle function, which represents the rate of change of the control obstacle function; For affine systems ,in, is the first-order derivative of the system state, is the dynamic behavior inside the system, For input The impact on the system status will be Substituting in, we get: ； Define the state quantity of the end effector of the robot arm as , the minimum safe distance is The expected speed is , the safety distance function is , define the control barrier function as: ； in are the obstacle coordinates, is the coordinate of the end effector of the robot arm; After derivation, we get: ； In order to minimize the task error, the objective function is set as ,in is the expected input; the entire control barrier function is solved as a quadratic programming problem: 。 5. The method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision according to claim 4 is characterized in that: The S4 is specifically: The network parameters of the deep reinforcement learning training model are initialized based on the Actor-Critic algorithm. The state space expression of the deep learning algorithm is: ； in is the rotation angle of the seven joints of the robotic arm; is the linear velocity of the end effector of the robot arm; is the distance between the end effector of the robot and the obstacle; is the distance between the end effector of the robot and the target point; The expressions of trajectory error, safety constraint and reward function are: ； ； ； ； in is the expected trajectory error of the end effector of the robot arm, is the sampling time, for The position of the end effector of the robot arm at the moment, is the sum of the radius of the obstacle and the radius of the end effector of the robot arm; the entire reward function is given by r It consists of three parts, namely Tracking error reward function, Minimum distance reward function, Collision penalty reward function, after adding the three reward values, we get the final reward value.

6. The method for dynamic obstacle avoidance of a robotic arm based on intelligent task supervision according to claim 1, characterized in that: The S5 is specifically: definition for the environment; is the state set of the end effector of the robot arm, is the initial state; It is a set of compound behaviors. The robot needs to select compound behaviors based on the environment to complete the task. For the experience replay pool, is the capacity of the experience replay pool, is the total time step of one round, is the distance between the end effector of the robot and the obstacle, is the total training rounds; During the entire task supervision process, the agent and the environment To interact, in state When a composite behavior is selected according to the Actor network's strategy , the agent is performing compound actions Afterwards, the environment will give the agent a reward As feedback, the agent then enters the next state , which generates a four-tuple This quadruple will be used as the experience for future learning and stored in the experience replay pool of the reinforcement learning algorithm. If the amount of experience is equal to the experience replay pool capacity , the experience replay pool will delete the earliest stored quadruple to store new experience; In trajectory planning, the reinforcement learning task supervisor will select the task priority of zero-space behavior according to the learned strategy and guide the robot arm to perform trajectory planning. When the distance between the end effector of the robot arm and the obstacle is less than the minimum safety distance, the control obstacle function will take over the input of the robot arm to control the robot arm to avoid collision accidents.

Images