A method for dynamic obstacle avoidance and planning control of a full-link robot arm in a human-robot collaborative environment

Abstract

The application discloses a kind of man-machine cooperation environment under mechanical arm full connecting rod dynamic obstacle avoidance and planning control method, comprising: the joint motion information obtained is input to the motion classification model and is classified, according to classification result, through corresponding joint position prediction network model, human motion prediction is carried out, and three-dimensional joint position of future certain time step is obtained;According to the distance between man-machine sphere and capsule body model, the repulsive field of obstacle is established, and the attractive field is established according to the distance of target point to the end of mechanical arm;Desired angle of each joint is calculated;The desired trajectory of each joint in future time domain is obtained, cost function is constructed, then rolling optimization is carried out, so that the optimal joint angular velocity control sequence is obtained to control robot motion.The application can realize that mechanical arm can effectively complete full connecting rod compliant obstacle avoidance while completing corresponding task, satisfy the real-time and compliance of mechanical arm full connecting rod dynamic obstacle avoidance.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and path planning, specifically to a method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment. Background Technology

[0002] Robotics technology is experiencing an unprecedented period of rapid development, with its applications expanding from highly controllable structured industrial scenarios to open and complex unstructured fields such as collaborative manufacturing, medical assistance, home services, and field operations. This trend presents stringent challenges to the environmental perception, decision-making, planning, and real-time control capabilities of robotic arms—the core requirement being that robotic arms must operate safely and efficiently in highly dynamic, unstructured scenarios where humans and machines coexist. Unlike structured environments, unstructured environments lack reliable, precise prior models. They are filled with unpredictable moving entities whose trajectories exhibit significant uncertainty. This necessitates that robotic arms possess obstacle perception and prediction capabilities, as well as robust real-time online planning abilities, to flexibly respond to complex and ever-changing operational conditions.

[0003] Common path planning algorithms, such as Dijkstra's algorithm and A* PRM, RRT, and their improved algorithms are essentially offline planning algorithms. Before a robot performs a motion task, they generate one or more feasible paths based on known global environmental information (such as obstacle distribution and start / end coordinates). However, this method cannot meet the real-time obstacle avoidance path planning requirements of dynamic environments. In contrast to offline planning algorithms, the Artificial Potential Field (APF) algorithm is a real-time path planning method based on a virtual force field. Its core idea is to establish a virtual potential field in the robot's motion environment in real time, combining gravitational and repulsive fields. Under the influence of the potential field forces, the robot moves along the negative gradient direction of the potential field, thereby achieving real-time obstacle avoidance and moving towards the target point. However, paths planned by the APF method can be uneven, and the robot's speed cannot be planned and constrained when executing path control commands, resulting in unsmooth obstacle avoidance motion. Reinforcement learning algorithms can learn action strategies that maximize cumulative rewards through trial-and-error interaction between the robot and its environment, thereby achieving obstacle avoidance. However, the training process for this method often takes place in a simulation environment, and the trained strategies do not perform well in sim2real processes. Furthermore, current obstacle avoidance path planning for robotic arms mainly focuses on the end effector. However, a robotic arm is a high-dimensional robot composed of multiple joints and links. How to plan obstacle avoidance for all the joints of the robotic arm in a high-dimensional space is also an urgent problem to be solved.

[0004] In summary, the existing technology mainly has the following problems: Poor real-time performance of path planning: Offline algorithms lack the ability to perceive dynamic obstacles and are unable to meet the real-time obstacle avoidance requirements in unstructured environments.

[0005] Artificial potential field algorithm path planning is not smooth and control process is not smooth: Artificial potential field algorithm has difficulty constraining and optimizing the joint angular velocity of robotic arm, the planned path will have abrupt changes and the control process is not smooth, which will lead to excessive impact and reduced life of robotic arm.

[0006] Obstacle avoidance with full linkage in high-dimensional space is difficult: the planning algorithm of planar mobile robots is difficult to transfer to multi-link joint coupled robotic arms in high-dimensional space, making it difficult to achieve the effect of full linkage obstacle avoidance. Summary of the Invention

[0007] To address the technical problems existing in current technologies, this invention proposes a dynamic obstacle avoidance and planning control method for a robotic arm with all linkages in a human-machine collaborative environment. This invention integrates recurrent neural networks for dynamic obstacle perception and prediction, an APF artificial potential field algorithm and a full-link torque application strategy for real-time obstacle avoidance, and MPC model predictive control to track the desired obstacle avoidance trajectory while optimizing the robotic arm's control process. This achieves compliant obstacle avoidance control of the robotic arm's full linkages in dynamic environments, ensuring collaborative safety in human-machine collaborative scenarios.

[0008] This invention provides a method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment, comprising: S1. Obtain the three-dimensional position and velocity information of the shoulder, elbow, and wrist joints of the human upper limb using a depth camera; S2. Input the obtained motion information into the BiLSTM-Attention model to classify the motion first, and then adaptively select the prediction network model to predict human motion based on the classification results. S3. By establishing a spherical and capsule-shaped mixed envelope model of the human upper limb and the robotic arm, a repulsive force field of the obstacle is established based on the distance between the sphere and the capsule, and a gravitational field is established based on the distance of the target point to the end of the robotic arm.

[0009] S4. By using the full-link torque application strategy of the robotic arm based on the APF artificial potential field algorithm and the force Jacobian matrix of each joint of the robotic arm, the desired angle of each joint can be calculated in real time. S5. By combining the prediction information of dynamic obstacles and APF path planning, the expected trajectory of each joint in the future time domain can be obtained. Finally, the expected trajectory tracking error, joint angular velocity, and joint angular acceleration are introduced into the cost function through the MPC model predictive control algorithm. Then, rolling optimization is performed to obtain the optimal joint angular velocity control sequence to control the robot's motion.

[0010] Furthermore, the Kinect v2.0 depth camera is used to capture motion information of the shoulder, elbow, and wrist joints of the human upper limbs, including the position and velocity information of the joints.

[0011] Furthermore, the BiLSTM-Attention model training process includes: splitting the dataset into a sliding window, normalizing the data, dividing it into training and test sets, and training and iterating the model parameters using the BiLSTM-Attention network model. During real-time inference deployment, future keypoint trajectories are predicted using motion information from the historical sliding window. obs t+1 ,obs t+2 ..., obs t+N}

[0012] Furthermore, the 3D positions of each joint of the robotic arm in space are calculated using the DH parameters of the robotic arm, along with the positions of the shoulder, elbow, and wrist joints captured by the depth camera. A repulsive potential field and repulsive force are established by reading the distances between the various spheres and capsules, expressed as:

[0013]

[0014] in, Let be the repulsive potential field function; It is a repulsive force; Indicates the repulsion factor; This indicates the spatial distance between the obstacle and the current position of the robotic arm; This indicates the maximum distance affected by the repulsive potential field of the obstacle. It is the vector pointing from the robot to the obstacle; The gravitational potential field and the expression for gravity are:

[0015]

[0016] in, Indicates the gravitational factor; This represents the spatial distance between the target point and the robot's current position; The power coefficient representing the gravitational distance. It is the vector pointing from the robot to the target point.

[0017] The distance between the robotic arm and human obstacles is detected in real time using human-machine sphere and capsule envelope models to generate repulsive force on the linkage.

[0018] Furthermore, the three-dimensional gravitational force generated by the target point acts on the end effector of the robotic arm, and the gravitational torque on each joint of the robotic arm can be calculated using the force Jacobian matrix of the end effector:

[0019]

[0020] in The Jacobian matrix at the end effector of the robotic arm. The gravitational torques of each joint; The repulsive force acts at the point closest to the robotic arm link. To calculate the repulsive torque, the repulsive force is transferred to the previous joint, and the force Jacobian matrix at that point is used to calculate the repulsive torque in real time.

[0021]

[0022] in Let be the Jacobian matrix of the force-bearing joints on the robotic arm. The repulsive torque of each joint; The resultant torque of all joints is:

[0023] Setting the joint angle step size through the resultant torque Calculate the desired joint angle. Combine this with the obstacle position prediction sequence obtained in the previous step. obs t+1 ,obs t+2 ..., obs t+N The expected trajectory sequence of each joint in the next N steps can be obtained recursively. , … },in Represents which joint of the robotic arm ={1, 2, 3, 4, 5, 6}.

[0024] Furthermore, the MPC model predictive control algorithm is used to track the desired trajectory, and the model used is a discrete kinematic model of the joint:

[0025] in, Joint angular velocity is also a control input. It is the sampling control period; Incorporating velocity and acceleration into the cost function:

[0026] in That is, the expected trajectory sequence of the joints { , … }, , This is the weight matrix for the state tracking error. W is the weight matrix for controlling the input regularization term, and W is the weight matrix for the acceleration smoothing term.

[0027] Optimal speed control input is obtained in real time through rolling optimization. Control commands are sent to the robotic arm to achieve compliant obstacle avoidance control.

[0028] This invention also provides a dynamic obstacle avoidance and planning control system for a robotic arm with all linkages in a human-machine collaborative environment.

[0029] The present invention also provides a computer device.

[0030] The present invention also provides a computer-readable storage medium.

[0031] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention utilizes a depth camera and a BiLSTM-Attention neural network human prediction model to perceive human obstacles and predict future trajectories. Combining sphere and capsule envelope models with an artificial potential field algorithm, it proposes a full-link obstacle avoidance strategy to plan joint paths in the joint space of a robotic arm, thereby achieving full-link obstacle avoidance in dynamic environments. Simultaneously, addressing issues such as non-compliance and abrupt changes during obstacle avoidance, it employs an MPC model predictive control algorithm to track the expected joint trajectory over a future time domain. Furthermore, it incorporates joint angular velocity and angular acceleration into the MPC cost function, achieving compliant obstacle avoidance control. The entire obstacle avoidance process includes a human perception and prediction module, an artificial potential field path planning module, and an MPC trajectory tracking and control module, enabling dynamic obstacle avoidance and planning control of the robotic arm in a human-machine collaborative environment, ensuring the safety of collaborating personnel. Attached Figure Description

[0032] Figure 1 This is a flowchart of the implementation method of the present invention.

[0033] Figure 2 This is a flowchart of the key point prediction process based on the BiLSTM-Attention model.

[0034] Figure 3 Real-time capture and trajectory prediction of the human wrist joint position.

[0035] Figure 4 This is a model diagram of the human upper limb and the sphere and capsule of a robotic arm.

[0036] Figure 5 This is a diagram of the potential field of attraction and repulsion.

[0037] Figure 6 Strategy for applying torque to the entire linkage.

[0038] Figure 7 This is a simulation diagram based on traditional APF obstacle avoidance.

[0039] Figure 8 The image shows a simulation diagram of obstacle avoidance based on BiLSTM-Attention-APF-MPC. Detailed Implementation

[0040] This section will provide a more detailed explanation of the embodiments of the present invention in conjunction with the accompanying drawings.

[0041] like Figure 1 As shown in the embodiment of the present invention, the method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment includes the following steps: S1. Obtain joint motion information of the shoulder, elbow, and wrist joints of the human upper limb through a depth camera for data acquisition and real-time inference of the neural network.

[0042] In one embodiment, a Kinect v2.0 depth camera is used to acquire joint motion information. The joint motion information includes three-dimensional position information and velocity information.

[0043] S2. Input the obtained joint motion information into the motion classification model to classify the motion. Based on the classification results, adaptively select the joint position prediction network model to predict human motion and obtain the three-dimensional joint position at a future time step. Use the predicted three-dimensional position of the joint as the center of the dynamic obstacle and regard the sphere and capsule formed by the human upper limb as the dynamic obstacle.

[0044] In one embodiment, the obtained joint motion information is preprocessed and normalized, and a sliding window input motion classification model is set up, such as using a BiLSTM-Attention model. This model uses BiLSTM to simultaneously model the sequential dependencies of the time-series data and fuse their hidden states. A multi-head attention mechanism is also added to enhance the weights of key time steps, resulting in more accurate classification. The overall network flow is as follows: Figure 2 As shown, Figure 3As shown, when implementing the deployment model, the motion is first classified, and then the prediction network model is adaptively selected based on the classification result (for example, if the classification result obtained by the network in the first stage is "motion type one", then the second stage automatically calls the joint position prediction network model corresponding to motion type one (model 1 in the figure) to predict the joint movement, thereby realizing the prediction of the motion trajectory of the human upper limb joints (taking the upper limb wrist joint as an example).

[0045] Figure 2 The neural network prediction part of the network consists of two stages: The first stage of the motion classification model uses a two-layer BiLSTM-Attention model to identify the types of motion. LSTM is an improvement on RNN (Recurrent Neural Network). The LSTM network model selectively retains and filters key information in temporal data through gating mechanisms (including input gate, forget gate, and output gate). However, the features of an action sequence often depend not only on the preceding temporal features but also on the following temporal features (for example, to determine "stand up," it is necessary to see if the "upright" state appears later). Therefore, BiLSTM is chosen to simultaneously model the sequential dependencies of temporal data and fuse the hidden states of both. Attention mechanisms are also added to enhance the weights of key time steps, thereby achieving more accurate classification results. In one embodiment, there are four types of motion, including arm swinging up and down, arm swinging forward and backward, arm movement to the left and right, and arm movement to the right and backward.

[0046] The second stage involves selecting a prediction network model specific to the identified motion type to predict the motion trajectory. Figure 2 In the motion trajectory prediction section, Model 1, Model 2, Model 3, and Model 4 represent models for predicting joint positions under different motion types. Correspondingly, all joint position prediction models for each motion type use a two-layer BiLSTM-Attention model. The input to the BiLSTM-Attention model is the 3D position and velocity of the shoulder, elbow, and wrist joints over the past M time steps, totaling 18... With M features, the classification network outputs a one-dimensional motion type, while the position prediction network outputs a three-dimensional joint position at a future time step. Then, through interpolation, the predicted trajectory of the joint's three-dimensional position for the next N steps is obtained. obs t+1 ,obs t+2 ..., obs t+N The predicted three-dimensional positions of the joints are used as the centers of the dynamic obstacles, and the spheres and capsules formed by the human upper limbs are considered as dynamic obstacles. obs tThe three-dimensional position of the most recently observed joint. obs t+N This represents the three-dimensional location of the joint in the Nth step of the future prediction.

[0047] In one embodiment, the neural network model is trained in the following way: This invention uses a depth camera to capture the movement of the shoulder, elbow, and wrist joints. Four motion modes were used to verify the algorithm. The motion information of the joints is labeled with motion types, preprocessed, normalized, and segmented by sliding window, and the training set and test set are divided accordingly. The network parameters are initialized using a BiLSTM recurrent neural network combined with an attention mechanism. The output data obtained from training a long short-term memory neural network is used. During training, the Euclidean distance between the predicted and actual values ​​of key points is used as the network loss. Parameter learning and iteration are performed during training. The test set is input into the trained model to obtain the types of motions output by the model. The model then automatically selects the joint position prediction model based on these motion types to obtain the final joint position prediction values, thus generating a sequence of predicted joint obstacle positions for the next N steps. obs t+1 ,obs t+2 ..., obs t+N}

[0048] S3. Establish a spherical and capsule-shaped mixed envelope model of the human upper limb and the robotic arm. Based on the surface distance between the upper limb and the spherical and capsule-shaped envelope models of the robotic arm, establish a repulsive force field for the obstacle. Based on the distance between the target point and the end of the robotic arm, establish a gravitational field, thereby driving the robotic arm to move towards the target point while achieving obstacle avoidance.

[0049] By using a human-machine sphere and capsule envelope model to detect the distance between the robotic arm and human obstacles in real time, a repulsive force is generated on the linkage. The three-dimensional gravitational force generated at the target point acts on the end of the robotic arm, and the gravitational torque on each joint of the robotic arm can be calculated using the force Jacobian matrix of the end.

[0050] Gravitational potential field and gravity The expression is:

[0051]

[0052] in, Indicates the gravitational factor; This represents the spatial distance between the target point and the robot's current position; The power coefficient representing the gravitational distance. It is the vector pointing from the robot to the target point; Then, the gravitational torque is calculated based on the forces acting on the end effector of the robotic arm and the Jacobian matrix. :

[0053] in The Jacobian matrix at the end effector of the robotic arm. Represents the transpose of a matrix. It is a six-dimensional vector representing the gravitational torque acting on the joint; In one embodiment, using the joints of the human upper limb and the robotic arm as center points, spheres and capsules are used to envelop the robotic arm and the human upper limb. The established sphere and capsule envelopment model is as follows: Figure 4 As shown, the corresponding gravitational field is as follows: Figure 5 As shown.

[0054] S4. Using the artificial potential field (APF) algorithm-based full-link torque application strategy for the robotic arm, the force situation of each joint of the robotic arm can be calculated in real time. Then, combined with the force Jacobian matrix of each joint point, the torque of each joint can be calculated in real time. The torque of each joint is normalized and calculated with the driving step size, so that the desired joint torque can be mapped to the desired joint angle. The full-link torque application strategy of the robotic arm is to transfer the force to the previous joint point of the link according to the actual force point of the link.

[0055] In one embodiment, Figure 6 The system calculates the distances between the center points and the central axis of capsules, spheres, and other objects by using the distances between points and lines in three-dimensional space, as well as the distances between lines. Then, the radius of the capsule or sphere is subtracted to obtain the closest human-machine distance. Taking a spherical obstacle created by the wrist joint as an example, the point closest to the center of the obstacle and the central axis of the robotic arm link is point p, and the closest distance is... ,use Figure 5 Calculate the repulsive force in the repulsive potential field:

[0056]

[0057] Let be the repulsive potential field function; It is a repulsive force; Indicates the repulsion factor; This indicates the spatial distance between the obstacle and the current position of the robotic arm; This indicates the maximum distance affected by the repulsive potential field of the obstacle. It is the vector pointing from the robot to the obstacle; Based on the position of point P on link ab, the force... Mapped to the previous joint position:

[0058] in The length of the link. Let p be the distance from the end of the current link.

[0059] Then, based on the current stress on each joint, the repulsive torque of each joint is calculated using the Jacobian matrix. :

[0060] in The Jacobian matrix of the stress joints. Represents the transpose of a matrix. It is a six-dimensional vector representing the repulsive torque acting on the joint; The resultant torque of all joints is:

[0061] in This represents the resultant torque acting on each joint; Next, the joint torque is normalized and multiplied by the maximum joint step length. The desired joint angle is obtained:

[0062] in The current angles of each joint of the robotic arm. This is the desired angle for each joint of the robotic arm in the next step.

[0063] S5. Combining the predicted information of dynamic obstacles obtained in step S2 and the calculation method of joint expected angles in step S4, the expected trajectories of each joint in the future time domain are calculated. Then, the joint expected trajectory tracking error, joint angular velocity, and joint angular acceleration are introduced into the cost function through the MPC model predictive control algorithm. The joint angular velocity is used as the control variable with preset weights (setting the weight matrix). , , Rolling optimization is performed to obtain the optimal joint angular velocity control sequence. Finally, the joint angular velocity is multiplied by the control cycle to obtain the joint angle increment, which is then transmitted to the robotic arm to achieve obstacle avoidance compliant control.

[0064] In one embodiment of the present invention, the predicted trajectory of the three-dimensional position of the joint in step S2 is used. obst+1 ,obs t+2 ..., obs t+N The method for calculating the expected joint angles in step S4 can be used to calculate the expected trajectory formed by the joint angles of the robotic arm in the next N steps. , … } This represents which joint of the robotic arm. Representing the In one embodiment, the desired joint angle of a joint at the Nth step in the future is... ={1, 2, 3, 4, 5, 6}.

[0065] Next, the MPC model predictive control algorithm is used to track the desired trajectory of the joint angle. , … Taking velocity and acceleration into account in the cost function, the cost function of the MPC model predictive control algorithm is set as follows:

[0066] in The cost function representing the predictive control algorithm of the MPC model. Represents a sequence of states. Represents a control sequence. Represents the current step number. For the first The joint angle values ​​of the robotic arm in each step. The expected trajectory representing the joint angle { , … }, This means the control input is also the joint angular velocity. Represents the angular acceleration of the robotic arm joints. This represents the joint angle state at the end of the optimization process. This represents the final joint angle state of the joint angle prediction sequence. , This is the weight matrix for the state tracking error. To control the weight matrix of the input regularization term, This is the weight matrix for the acceleration smoothing term. The control input for the entire process is optimized by introducing velocity and acceleration terms. The MPC model predictive control algorithm uses discrete joint dynamics as the joint recursion:

[0067] in, This refers to the joint angular velocity, which is also the control input. It is the sampling control period, and constraints are introduced at the same time:

[0068]

[0069] This is a constraint on the minimum joint angle of the robotic arm. This is a constraint on the maximum value of the robot arm joint angles. For the first The joint angle values ​​of the robotic arm in each step. This is a constraint on the minimum angular velocity of the robotic arm joints. This is a constraint on the maximum value of the angular velocity of the robotic arm joints.

[0070] Optimal speed control input is obtained in real time through rolling optimization. Control commands are sent to the robotic arm to achieve compliant obstacle avoidance control. After obtaining the optimal speed control signal, it is then used to:

[0071] To obtain the desired optimal joint angle increment The optimal joint angle increment is sent to the robotic arm's controller to achieve compliant full-link obstacle avoidance control. Complete closed-loop control of the robotic arm is achieved by real-time reading of state observation data fed back from the robotic arm controller and capturing human motion data, thus realizing real-time compliant full-link obstacle avoidance.

[0072] In one embodiment, such as Figure 7 , 8 As shown, Figure 7 Using a traditional artificial potential field algorithm, it can be observed that during obstacle avoidance, the lack of velocity and acceleration constraints leads to abrupt changes in joint angular velocity, and also in the joint angle trajectory. For example... Figure 8 As shown, by introducing BiLSTM-Attention neural network prediction and MPC model prediction control algorithm to track the desired trajectory, a compliant obstacle avoidance process can be obtained. While ensuring safety, a smoother process is achieved, reducing abrupt changes and extending the life of the robotic arm.

[0073] In summary, the embodiments of the present invention extract the coordinates of the joints of the human upper limb using a depth camera and predict the human arm movement using a BiLSTM-Attention model. By combining the predicted human arm movement information with a full-link obstacle avoidance strategy based on an artificial potential field algorithm, the expected trajectory of the joints in the future time domain can be obtained. Then, the expected trajectory of the joints is input into the MPC model predictive control algorithm. By introducing a cost function and constraints, the entire control process is optimized to obtain a real-time, safe, and compliant obstacle avoidance effect.

[0074] The method described in the foregoing embodiments of this invention uses a depth camera to capture the joints of the human upper limb as dynamic obstacles in the environment. A sliding window is established, and historical joint positions and velocity features within the sliding window are used to predict the future trajectories of joints via a recurrent neural network, achieving the perception and prediction of dynamic obstacles. Next, a hybrid sphere and capsule envelope is established for the robotic arm and the human upper limb to achieve efficient collision detection. Finally, an artificial potential field algorithm is used to establish a full-link torque application strategy for the robotic arm, combined with a model predictive control (MPC) algorithm. The angular velocity and angular acceleration of the robotic arm joints are used as part of the cost function for real-time rolling optimization to obtain the optimal velocity control sequence, thereby satisfying the compliance requirements of the robotic arm in obstacle avoidance.

[0075] In one embodiment, a dynamic obstacle avoidance and planning control system for a robotic arm with all linkages in a human-machine collaborative environment is provided to implement the method described in the foregoing embodiment. The system includes the following modules: The human body perception and prediction module is used to acquire joint motion information of the shoulder, elbow and wrist joints of the human upper limb in real time, and input the acquired joint motion information into the motion classification model to classify the movement. Based on the classification results, the module uses the corresponding joint position prediction network model to predict human motion and obtain the three-dimensional joint position at a future time step. The Artificial Potential Field (APF) path planning module is used to establish a hybrid envelope model of the human upper limb and the robotic arm consisting of spheres and capsules. It establishes a repulsive field for obstacles based on the distance between the spheres and capsules, and an attractive field based on the distance between the target point and the end of the robotic arm. It executes the attractive and repulsive potential fields and the full-link obstacle avoidance strategy, and plans the expected future trajectory of each joint of the robotic arm by combining the predicted values ​​obtained from the human perception and prediction module. The MPC model predictive control module is used to incorporate the desired trajectory tracking error, joint angular velocity, and joint angular acceleration into the cost function through the MPC model predictive control algorithm, and perform rolling optimization to obtain the optimal joint angular velocity control sequence to control the robot's motion.

[0076] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the foregoing embodiments.

[0077] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described in the foregoing embodiments.

[0078] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment, characterized in that, Includes the following steps: The motion information of the shoulder, elbow, and wrist joints of the human upper limbs is obtained by using a depth camera. The obtained joint motion information is input into the motion classification model to classify the movement. Based on the classification results, the human motion is predicted through the corresponding joint position prediction network model to obtain the three-dimensional joint position at a future time step. Establish a hybrid envelope model of the human upper limb and the robotic arm consisting of spheres and capsules. Establish a repulsive force field for obstacles based on the distance between the spheres and capsules, and establish a gravitational field based on the distance between the target point and the end of the robotic arm. The robotic arm uses an APF algorithm based on artificial potential field to apply torque to all links and combines the force Jacobian matrix of each joint to calculate the expected angle of each joint in real time. By combining the predicted information of dynamic obstacles and the expected angles of each joint, the expected trajectory of each joint in the future time domain is obtained. The expected trajectory tracking error, joint angular velocity, and joint angular acceleration are introduced into the cost function through the MPC model predictive control algorithm to perform rolling optimization, thereby obtaining the optimal joint angular velocity control sequence to control the robot's motion.

2. The method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment according to claim 1, characterized in that, The BiLSTM-Attention model takes the motion information of joints from a historical sliding window as input and outputs the predicted trajectory of joints in the future time domain.

3. The method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment according to claim 1, characterized in that, Both the motion classification model and the joint position prediction network model use a two-layer BiLSTM-Attention model.

4. The method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment according to claim 1, characterized in that, Using the joints of the human body and the joints of the robotic arm as the center points, spheres and capsules are used to envelop the robotic arm and the human upper limb. The distance between the sphere and capsule enveloping models is calculated in real time to calculate the human-machine distance and perform collision detection. Then, a repulsive field is introduced based on the human-machine distance, and an attractive field is introduced based on the distance between the robotic arm end and the target point.

5. The method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment according to claim 1, characterized in that, The robotic arm's full-link torque application strategy converts the force to the previous joint point of the link based on the actual force point of the link. At the same time, it uses the Jacobian matrix of the current joint point and the three-dimensional force to calculate the expected joint torque for obstacle avoidance and map it to the expected joint angle.

6. The method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment according to claim 1, characterized in that, The cost function is as follows: in, The cost function representing the predictive control algorithm of the MPC model. Represents a sequence of states. Represents a control sequence. Represents the current step number. For the first The joint angle values ​​of the robotic arm in each step. This represents the desired trajectory of the joint angles, i.e., the joint prediction sequence. , This is the weight matrix for the state tracking error. To control the weight matrix of the input regularization term, This is the weight matrix for the acceleration smoothing term.

7. A method for dynamic obstacle avoidance and planning control of a robotic arm with all linkages in a human-machine collaborative environment, as described in any one of claims 1-6, is characterized in that... The expression for the desired joint angle is: The current angles of each joint of the robotic arm. This is to determine the desired angles for each joint of the robotic arm in the next step. For the maximum joint stride length, This represents the resultant torque acting on each joint. The resultant torque of all joints, For joint repulsive torque, This is the gravitational torque.

8. A dynamic obstacle avoidance and planning control system for a robotic arm with all linkages in a human-machine collaborative environment, characterized in that, The system for implementing the method according to any one of claims 1-7 includes the following modules: The human body perception and prediction module is used to acquire joint motion information of the shoulder, elbow and wrist joints of the human upper limb in real time, and input the acquired joint motion information into the motion classification model to classify the movement. Based on the classification results, the module uses the corresponding joint position prediction network model to predict human motion and obtain the three-dimensional joint position at a future time step. The Artificial Potential Field (APF) path planning module is used to establish a hybrid envelope model of the human upper limb and the robotic arm consisting of spheres and capsules. It establishes a repulsive field for obstacles based on the distance between the spheres and capsules, and an attractive field based on the distance between the target point and the end of the robotic arm. It executes the attractive and repulsive potential fields and the full-link obstacle avoidance strategy, and plans the expected future trajectory of each joint of the robotic arm by combining the predicted values ​​obtained from the human perception and prediction module. The MPC model predictive control module is used to incorporate the desired trajectory tracking error, joint angular velocity, and joint angular acceleration into the cost function through the MPC model predictive control algorithm, and perform rolling optimization to obtain the optimal joint angular velocity control sequence to control the robot's motion.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images