Seven-axis robot autonomous obstacle avoidance motion control method based on improved SAC algorithm

Abstract

The application relates to a seven-axis mechanical arm autonomous obstacle avoidance motion control method based on an improved SAC algorithm, and the method comprises the following steps: S1, camera calibration and mechanical arm hand-eye calibration are performed on a real seven-axis mechanical arm; S2, a working environment is arranged in a real working space; S3, according to the calibration result, a target point and various obstacles are mapped into a training environment; S4, a three-dimensional simulation environment is established, specifically comprising the establishment of a corresponding mechanical arm model, a working space, a target point, various obstacles and the setting of a training process; S5, an improved SAC algorithm is used to train the mechanical arm in a motion planning task of autonomous obstacle avoidance; S6, the SAC algorithm after the training is completed is selected to select an optimal network model for evaluation, and after the evaluation is completed, the application of a real mechanical arm is performed, and the motion control of the seven-axis mechanical arm is completed. The application adds a feature extraction network of a fusion gating mechanism, so that the mechanical arm can effectively make decisions and perform autonomous obstacle avoidance motion.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control, specifically to a method for autonomous obstacle avoidance motion control of a seven-axis robotic arm based on an improved SAC algorithm. Background Technology

[0002] Seven-axis robotic arms are a common type of robot in production and daily life, and their intelligent control has been a key research focus in recent years. Deep reinforcement learning algorithms, with their self-learning capabilities, adapt to the environment through online or offline learning and continuous trial and error, enabling seven-axis robotic arms to work effectively in unknown environments. They achieve automated control of complex tasks on inexpensive hardware, eliminating the need for sophisticated and expensive sensors and actuators, thus significantly reducing costs. Currently, research on obstacle avoidance in static environments is relatively extensive, but research on obstacle avoidance in dynamic environments is less common.

[0003] Therefore, there is an urgent need to design an autonomous obstacle avoidance motion control method that can perform motion planning tasks in dynamic environments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide an autonomous obstacle avoidance motion control method for a seven-axis robotic arm based on an improved SAC algorithm. It incorporates a feature extraction network with a fusion gating mechanism, thereby enabling the seven-axis robotic arm to make effective decisions and perform autonomous obstacle avoidance motion.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is: a seven-axis robotic arm autonomous obstacle avoidance motion control method based on an improved SAC algorithm, the method comprising:

[0006] S1, perform camera calibration and robotic arm hand-eye calibration on a real seven-axis robotic arm;

[0007] S2, setting up the work environment in a real workspace;

[0008] S3, based on the calibration results, maps the target point and various obstacles into the training environment;

[0009] S4. Establish a three-dimensional simulation environment, which includes creating a corresponding seven-axis robotic arm model, workspace, target point, various obstacles, and setting up the training process.

[0010] S5 uses an improved SAC algorithm to train a seven-axis robotic arm model for autonomous obstacle avoidance motion planning in a 3D simulation environment. The network model based on the improved SAC algorithm includes a backbone network, a policy network, and an evaluation network. The backbone network includes a decoupled feature extraction network and an adaptive weight gating. The decoupled feature extraction network is used to extract features for each state variable in the state space, and the adaptive weight gating is used to measure the state variables.

[0011] S6. Select the optimal network model from the trained SAC algorithm for evaluation. After evaluation, apply it to a real robotic arm to complete the motion control of the seven-axis robotic arm.

[0012] Furthermore, S1 specifically includes:

[0013] S11, camera calibration for a real seven-axis robotic arm;

[0014] S12, perform hand-eye calibration on a real seven-axis robotic arm;

[0015] S13, calculate the transformation matrix between the coordinate system calibrated by the camera and the coordinate system calibrated by the hand and eye, calculate the reprojection error based on the transformation matrix, and complete the calibration after the error meets the threshold requirement.

[0016] Furthermore, S3 specifically includes:

[0017] S31, determine the actual seven-axis robotic arm model used and its supported communication interfaces;

[0018] S32, install the corresponding driver and library files according to the seven-axis robotic arm model and interface;

[0019] S33, write control scripts in CoppeliaSim to convert the seven-axis robotic arm movements in the simulation environment into instructions that can be executed by a real seven-axis robotic arm.

[0020] S34, using the communication interface to connect the seven-axis robotic arm to CoppeliaSim;

[0021] After successfully connecting to the S35, perform testing and debugging.

[0022] Furthermore, S4 specifically includes:

[0023] S41, set the O-XYZ coordinate system, set the corresponding camera position in the simulation environment for easy observation, and set the target point position and various obstacles;

[0024] S42, import the seven-axis robotic arm model, support table model, target model, and obstacle model, set the workspace, and then restore all the joints of the seven-axis robotic arm to their initial state.

[0025] Furthermore, S5 includes:

[0026] S51, Construct an improved SAC algorithm model;

[0027] S52, Constructing the state space of the improved SAC algorithm model:

[0028]

[0029] This refers to the current position of the seven joints. ; It is the random target position that the end effector must reach; This is the current location of the end effector; It indicates the location of the obstacle in the current round; This is the current attitude information of the end effector; These are the angular velocities currently being executed by the seven joints. ;

[0030] S53, Constructing the action space of the improved SAC algorithm model:

[0031]

[0032] These are the angular velocities currently being executed by the seven joints. .

[0033] S54, clear the experience pool, which stores state, action, reward, and next state information. Data is retrieved from the experience pool using a minimum sampling size of 64, and the batch data is stored as a dictionary. Upon retrieval, it is converted into a vector or tensor. The form of stored experience is B = { , , , , done}, whose parameters correspond to the state, action, reward, next state, and whether the round is completed, respectively;

[0034] S55, initialize random seed, set parameters;

[0035] S56, Construct the reward function for the improved SAC algorithm model; where, the reward function... Including initial rewards and directional rewards ;Right now

[0036] =

[0037] Initial reward function for:

[0038] =

[0039] The task status function is 1 if completed and 0 if not completed. For distance reward function; For obstacle avoidance reward function; This represents the additional fixed reward value obtained by completing the task within the specified time step;

[0040]

[0041] In the formula, dis is the Manhattan distance in three-dimensional space, specifically:

[0042]

[0043] This represents the current position of the end effector in three-dimensional space. For the final target location;

[0044]

[0045] The Manhattan distance in three-dimensional space is specifically:

[0046]

[0047] These are the current position coordinates of the end effector in three-dimensional space. The coordinates of the obstacle's location;

[0048]

[0049] in,

[0050]

[0051]

[0052]

[0053] The gripper's attitude information can be represented by a quaternion "q", which consists of a real part... and three imaginary parts , , The component, "q", can be obtained from the API in CoppeliaSim;

[0054] S57, Construct a random noise that follows a normal distribution. By interfering with the motion, the seven-axis robotic arm can be ensured to continuously explore the workspace and avoid getting trapped in local optima.

[0055] S58, a seven-axis robotic arm in a simulation environment performs actions and determines rewards.

[0056] S59, when the end effector of the seven-axis robotic arm reaches the target area, exceeds the workspace, collides, or the step count reaches the maximum number of rounds, the round ends, the robotic arm returns to the initial pose, and the next training round begins.

[0057] Furthermore, in S58, the seven-axis robotic arm in the simulation environment performs actions, specifically including:

[0058] S581, Improved SAC algorithm initialization;

[0059] S582, obtain current environmental observation values;

[0060] S583, based on the input, improves the SAC algorithm to give the action;

[0061] S584, a seven-axis robotic arm performs actions in the current environment.

[0062] Furthermore, the policy network includes a real-time action network and a target action network, and the entire process is as follows:

[0063] Real-time action networks rely on state transmission from the backbone network. Get action strategy This action Given a random noise that follows a normal distribution Interference is introduced to ensure that the seven-axis robotic arm can continuously explore the workspace and thus avoid getting trapped in local optima;

[0064] +

[0065] Real-time action networks execute actions Obtain the next state ,award 1. Whether the task for this round is completed (done) will be indicated. , , , , done} is stored in the experience pool;

[0066] The environment will be in the next state. The data is transmitted from the backbone network to the target action network to obtain the next action. .

[0067] Furthermore, the evaluation network includes a real-time evaluation network and a target evaluation network, with the real-time evaluation network and the target evaluation network performing the current action respectively. and the next step Valuation calculation;

[0068]

[0069] Indicating in strategy The expected reward Q obtained by choosing action a when the state is s; This is the next state; It's the next move; It is a discount factor. It is the reward value obtained in this round; This refers to an agent receiving a reward for taking action a in state s. and the next state The probability of.

[0070] Furthermore, during training, the network parameters are updated using the loss function derived from network evaluation;

[0071] Perform backbone network parameter analysis The update is as follows:

[0072] ←

[0073] It is the learning rate, used to control each parameter update; Regarding parameters The loss function; It is a loss function Regarding parameters The gradient is used to guide the direction of parameter updates;

[0074] Real-time evaluation of network parameters The update is as follows:

[0075] ←

[0076] It is the learning rate, used to control each parameter update; Regarding parameters The loss function is used to measure the loss between the current real-time value evaluation network Q-value and the target Q-value; It is the loss function with respect to the parameters The estimated gradient is used to guide the direction of parameter updates;

[0077] Update the parameters of the Online Actor network. Specifically:

[0078] ←

[0079] It is the learning rate, which controls each parameter update; Regarding parameters The objective function for policy performance is used to measure the effectiveness of the policy function under the current parameters. It's about parameters. The policy gradient estimate is used to guide the direction of parameter updates;

[0080] Regularization term for updating entropy Specifically:

[0081] ←

[0082] It is the learning rate, which controls the step size of each parameter update; It's about parameters. The loss function is used to measure the current... With the goal The gap between them; It's about parameters. The estimated gradient is used to guide the direction of parameter updates;

[0083] Update target evaluation network parameters Specifically:

[0084] ←

[0085] It is a smoothing coefficient that controls the weighted average ratio of the current parameters and the target parameters;

[0086] Update Target Actor network parameters Specifically:

[0087] ←

[0088] It is a smoothing coefficient that controls the weighted average ratio of the current parameter and the target parameter.

[0089] Furthermore, the decoupled feature extraction network includes MLP networks and CNN networks. One-dimensional vectors are processed by MLP networks, and multi-dimensional vectors are processed by CNN networks.

[0090] Furthermore, the improved SAC algorithm was trained in a simulation environment until it converged.

[0091] By adopting the above technical solution, the present invention has the following beneficial effects:

[0092] 1. This invention constructs a feature extraction network with an integrated gating mechanism based on the SAC algorithm. This feature extraction network can change with the input quantity. That is, for each input in the state space, feature extraction is performed first, then the input is passed to the weight gating to calculate the weight value, and finally sent to the fully connected layer for integrated output, which promotes the effective decision-making of the robotic arm.

[0093] 2. This invention designs a reward function that facilitates rapid convergence. Obtaining this reward function improves the robot's understanding of the environment, reduces the exploration cycle, and increases the exploration enthusiasm. This not only greatly increases the flexibility of the robotic arm but also improves the robustness of the algorithm.

[0094] 3. The present invention can reduce the wear and tear on the physical robotic arm and lower the cost by training it in simulation software and then transferring it to the real robotic arm. Attached Figure Description

[0095] Figure 1 This is a schematic diagram of the improved SAC algorithm of the present invention;

[0096] Figure 2 This is a structural diagram of the backbone network of the present invention;

[0097] Figure 3 This is a schematic diagram of the simulation experiment training process of the present invention;

[0098] Figure 4 This is a schematic diagram of the motion control of a real seven-axis robotic arm;

[0099] Figure 5 This diagram illustrates a comparison of the success rates of the improved SAC algorithm and the original SAC algorithm in completing a path planning training task in an accessible environment.

[0100] Figure 6 This is a schematic diagram showing the training comparison between the reward function proposed in this invention and the original reward function;

[0101] Figure 7 This diagram illustrates the success rate of evaluating the optimal control strategy selected by the present invention and different models.

[0102] in, Figure 5 In the diagram, (a) shows the training success rate of different models, and (b) shows the convergence of the reward function of different models.

[0103] Figure 6 In the diagram, (a) shows the training success rate under different reward functions, and (b) shows the convergence of the reward functions under different reward functions. Detailed Implementation

[0104] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0105] like Figure 1 , 2 As shown in Figures 3 and 4, a method for autonomous obstacle avoidance motion control of a seven-axis robotic arm based on an improved SAC algorithm is described. The method includes:

[0106] S1, calibrating the actual seven-axis robotic arm; specifically:

[0107] S11, performing camera calibration on a real seven-axis robotic arm; specifically:

[0108] S111, a camera is installed in the workspace of a real seven-axis robotic arm, and the coordinate system is ensured to be consistent;

[0109] S112, start the camera (using KinectV2 camera as an example), and then use the bridging tool in the camera toolkit to connect it to the CoppeliaSim simulation software;

[0110] S113 sets the camera's default image acquisition resolution;

[0111] S114, run kinect2_calibration in the toolkit to record image information;

[0112] S115, test the camera by placing the calibration board within the camera's field of view, moving the calibration board to different poses, and taking pictures of the corner points;

[0113] S116, use kinect2_calibration from the toolkit to calibrate the obtained image data;

[0114] S117, Save and load the calibration results, and confirm whether it has taken effect;

[0115] S12, perform hand-eye calibration on a real seven-axis robotic arm; specifically:

[0116] S121, turn on the camera and load the intrinsic parameter calibration results.

[0117] S122, Operate the seven-axis robotic arm to move it within the camera's field of view to detect at least 25 different poses of the markers, record the image coordinates of the marker center points and their poses in the seven-axis robotic arm's base coordinate system, and save the records;

[0118] S13: Record the data, input the corresponding parameters, calculate the transformation matrix between the coordinate system calibrated by the camera and the coordinate system calibrated by the hand and eye, calculate the reprojection error based on the transformation matrix, and complete the calibration after the error meets the threshold requirement.

[0119] S2, setting up the work environment in a real workspace;

[0120] S3, based on the calibration results, maps the target point and various obstacles into the training environment; specifically:

[0121] S31, Determine the actual seven-axis robotic arm model used and its supported communication interfaces, such as RS232, TCP / IP, etc.

[0122] S32, based on the seven-axis robotic arm model and interface, install the corresponding driver and library files so that V-REP can communicate with the seven-axis robotic arm;

[0123] S33. Write Lua control scripts in CoppeliaSim to translate the movements of a seven-axis robotic arm in a simulated environment into commands that can be executed by a real seven-axis robotic arm. The control scripts typically use CoppeliaSim's API interface to call the seven-axis robotic arm driver or library files and send commands to the seven-axis robotic arm.

[0124] S34 uses the communication interface to connect the seven-axis robotic arm to CoppeliaSim. During the connection process, it is necessary to ensure that parameters such as the communication protocol and baud rate are set correctly.

[0125] After the S35 is successfully connected, testing and debugging can be carried out. By observing the movement and status of the seven-axis robotic arm and comparing it with the seven-axis robotic arm in the simulation environment, the control script and the parameters of the seven-axis robotic arm can be gradually adjusted to achieve better performance and accuracy.

[0126] S4. Establish a 3D simulation environment, specifically including creating a corresponding seven-axis robotic arm model, workspace, target point, various obstacles, and setting the training process; specifically:

[0127] S41, Set the O-XYZ coordinate system, set the corresponding camera position in the simulation environment for easy observation, and set the target point position and various obstacles;

[0128] S42: Import the seven-axis robotic arm model, support table model, target model, and obstacle model; set the workspace; and then restore all joints to their initial state.

[0129] S5 uses an improved SAC algorithm to train a seven-axis robotic arm for autonomous obstacle avoidance motion planning tasks, and records the total reward value, the number of steps completed in each round, and the average single step score; the network model based on the improved SAC algorithm includes a backbone network, a policy network, and an evaluation network.

[0130] Traditional SAC algorithms integrate all state variables in the state space and then input them into the network for feature extraction. However, feature extraction for multi-dimensional state variables is quite challenging. This embodiment designs a backbone network. The backbone network includes a decoupled feature extraction network and an adaptive weight gating system. The decoupled feature extraction network extracts features for each state variable in the state space. This network includes an MLP network and a CNN network; one-dimensional vectors are processed by the MLP network, and multi-dimensional vectors are processed by the CNN network, the specific number depending on the number of input state variables. The adaptive weight gating system has self-optimization capabilities. Adding adaptive weight gating after the decoupled feature extraction network allows for the measurement of the impact of state variables on the overall improved SAC algorithm, facilitating adjustments to the weight ratios of different state variables for adaptive improvement. As training progresses, the backbone network updates its parameters through backpropagation, enabling rapid adaptation to environmental changes and continuously providing optimized action strategies based on feature weights to improve the algorithm's convergence speed. The more variables in the state space, the more significant the improvement.

[0131] S5 specifically includes:

[0132] S51, Constructing a network to improve the SAC algorithm:

[0133] Perform random initialization on the backbone network, i.e., initialize the network parameters. ← ( The policy network is randomly initialized, that is, the parameters of the real-time action network are initialized. ← Initialize the target action network parameters: ← Randomly initialize the evaluation network, i.e., initialize the parameters of the real-time evaluation network. ← Initialize target evaluation network parameters ← ;

[0134] S52, Constructing the state space of the improved network model:

[0135]

[0136] This refers to the current position of the seven joints. ; It is the random target position that the end effector must reach; This is the current location of the end effector; It indicates the location of the obstacle in the current round; This is the current attitude information of the end effector; These are the angular velocities currently being executed by the seven joints. ;

[0137] S53, Constructing the action space of the improved SAC algorithm:

[0138]

[0139] These are the angular velocities currently being executed by the seven joints. .

[0140] S54, clear the experience pool, which stores state, action, reward, and next state information. Data is retrieved from the experience pool using a minimum sampling size of 64, and the batch data is stored as a dictionary. Upon retrieval, it is converted into a vector or tensor. The form of stored experience is B = { , , , , done}, whose parameters correspond to the state, action, reward, next state, and whether the round is completed, respectively;

[0141] S55: Initialize the random seed and set other parameters. During training, the maximum number of steps per episode is 50, the number of episodes is 10000, and the experience pool size is 10. 6 The sampling size is 64, the decay factor γ in the Bellman equation is 0.99, and the soft update parameters are... The policy network learning rate is 0.0005, and the evaluation network learning rate is 0.001. At the beginning of each round, the seven-axis robotic arm resets to its initial set position. During evaluation, the number of episodes is 500, and the remaining parameters are determined by the optimal episode.

[0142] S56. Construct an improved reward function for the SAC algorithm. A suitable reward function can guide the agent's behavior, enabling the agent to adapt to an environment full of obstacles more quickly and reach the target point.

[0143] reward function Including initial rewards and directional rewards ;Right now

[0144] =

[0145] Initial reward function for:

[0146] =

[0147] The task status function is 1 if completed and 0 if not completed. For distance reward function; For obstacle avoidance reward function; This represents the additional fixed reward value obtained by completing the task within the specified time step;

[0148]

[0149] In the formula, dis is the Manhattan distance in three-dimensional space, specifically:

[0150]

[0151] This represents the current position of the end effector in three-dimensional space. The final target location.

[0152]

[0153] The Manhattan distance in three-dimensional space is specifically:

[0154]

[0155] These are the current position coordinates of the end effector in three-dimensional space. The coordinates of the obstacle's location;

[0156] If only Setting the obstacle avoidance reward to -20 is insufficient for the agent to achieve full-map obstacle perception, which will affect the training of the seven-axis robotic arm, making it difficult to complete tasks and converge in the early stages. Therefore, the obstacle avoidance reward designed in this paper... It can improve the perception capabilities of a seven-axis robotic arm, enabling it to avoid obstacles effectively and converge faster.

[0157]

[0158] The reward increases by 20 for each completed task; otherwise, it is 0.

[0159]

[0160] in,

[0161]

[0162]

[0163]

[0164] The gripper's attitude information can be represented by a quaternion "q", which consists of a real part... and three imaginary parts , , The component, "q", can be obtained from the API in CoppeliaSim.

[0165] S57, Construct a random noise that follows a normal distribution. By interfering with the motion, the seven-axis robotic arm can be ensured to continuously explore the workspace and avoid getting trapped in local optima.

[0166] S58, a seven-axis robotic arm in a simulation environment performs actions and determines rewards; among them,

[0167] The seven-axis robotic arm performs actions in a simulation environment, specifically including:

[0168] S581, Improved SAC algorithm initialization;

[0169] S582, obtain current environmental observation values;

[0170] S583, based on the input, improves the SAC algorithm to give the action;

[0171] S584, a seven-axis robotic arm performs actions in the current environment;

[0172] S59: When the end effector of the seven-axis robotic arm reaches the target area, exceeds the workspace, collides, or the step count reaches the maximum number of rounds, the round ends and the robotic arm returns to its initial posture.

[0173] During training, various parameters are updated, specifically as follows:

[0174] Perform backbone network parameter analysis The update is as follows:

[0175] ←

[0176] It is the learning rate, used to control each parameter update; Regarding parameters The loss function; It is a loss function Regarding parameters The gradient is used to guide the direction of parameter updates;

[0177] Real-time evaluation of network parameters The update is as follows:

[0178] ←

[0179] It is the learning rate, used to control each parameter update; Regarding parameters The loss function is used to measure the loss between the current real-time value evaluation network Q-value and the target Q-value; It is the loss function with respect to the parameters The estimated gradient is used to guide the direction of parameter updates;

[0180] Update the parameters of the Online Actor network. Specifically:

[0181] ←

[0182] It is the learning rate, which controls each parameter update; Regarding parameters The objective function for policy performance is used to measure the effectiveness of the policy function under the current parameters. It's about parameters. The policy gradient estimate is used to guide the direction of parameter updates;

[0183] Regularization term for updating entropy Specifically:

[0184] ←

[0185] It is the learning rate, which controls the step size of each parameter update; It's about parameters. The loss function is used to measure the current... With the goal The gap between them; It's about parameters. The estimated gradient is used to guide the direction of parameter updates;

[0186] Update target evaluation network parameters Specifically:

[0187] ←

[0188] It is a smoothing coefficient that controls the weighted average ratio of the current parameters and the target parameters;

[0189] Update Target Actor network parameters Specifically:

[0190] ←

[0191] It is a smoothing coefficient that controls the weighted average ratio of the current parameter and the target parameter.

[0192] S6. Select the optimal network model from the trained SAC algorithm for evaluation. After evaluation, apply it to a real robotic arm to complete the motion control of the seven-axis robotic arm.

[0193] like Figure 1 As shown, the policy network includes a real-time action network and a target action network. The entire working process is as follows:

[0194] Real-time action networks rely on state transmission from the backbone network. Get action strategy This action Given a random noise that follows a normal distribution Interference is introduced to ensure that the seven-axis robotic arm can continuously explore the workspace and thus avoid getting trapped in local optima;

[0195] +

[0196] Real-time action networks execute actions Obtain the next state ,award 1. Whether the task for this round is completed (done) will be indicated. , , , , done} is stored in the experience pool;

[0197] The environment will be in the next state. The data is transmitted from the backbone network to the target action network to obtain the next action. .

[0198] like Figure 1 As shown, the evaluation network includes a real-time evaluation network and a target evaluation network, with the real-time evaluation network and the target evaluation network performing the current action respectively. and the next step Valuation calculation;

[0199]

[0200] Indicating in strategy The expected reward Q obtained by choosing action a when the state is s; This is the next state; It's the next move; It is a discount factor. It is the reward value obtained in this round; This refers to an agent receiving a reward for taking action a in state s. and the next state The probability is used to update the network parameters through the loss function derived from evaluating the network, until convergence.

[0201] The experimental results of the schemes in the above embodiments are described below.

[0202] The experimental data was obtained by selecting the optimal parameters from 10,000 training iterations in CoppeliaSim and evaluating them over 500 iterations. The seven-axis robotic arm was required to avoid dynamic obstacles and reach a random target area within 500 iterations. The experimental results are shown in the table below:

[0203] Training and comparison experiments were conducted on several different models in the table above. The experimental results show that the success rates of SAC and DDPG are not as good as the method in this embodiment, and the success rate is poor. In contrast, the method in this embodiment has a certain robustness to interference from obstacles in the working environment, has a high success rate, and the effect is better.

[0204] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for autonomous obstacle avoidance motion control of a seven-axis robotic arm based on an improved SAC algorithm, characterized in that, The methods include: S1, perform camera calibration and robotic arm hand-eye calibration on a real seven-axis robotic arm; S2, setting up the work environment in a real workspace; S3, based on the calibration results, maps the target point and various obstacles into the training environment; S4. Establish a three-dimensional simulation environment, which includes creating a corresponding seven-axis robotic arm model, workspace, target point, various obstacles, and setting up the training process. S5 uses an improved SAC algorithm to train a seven-axis robotic arm model for autonomous obstacle avoidance motion planning in a 3D simulation environment. The network model based on the improved SAC algorithm includes a backbone network, a policy network, and an evaluation network. The backbone network includes a decoupled feature extraction network and an adaptive weight gating. The decoupled feature extraction network is used to extract features for each state variable in the state space, and the adaptive weight gating is used to measure the state variables. S6. Select the optimal network model from the trained SAC algorithm for evaluation. After evaluation, apply it to a real robotic arm to complete the motion control of the seven-axis robotic arm.

2. The autonomous obstacle avoidance motion control method for a seven-axis robotic arm based on the improved SAC algorithm according to claim 1, characterized in that, The reward function of the improved SAC algorithm model is: reward function Including initial rewards and directional rewards ;Right now = ； Initial reward function for: = ； The task status function is 1 if completed and 0 if not completed. For distance reward function; For obstacle avoidance reward function; This represents the additional fixed reward value obtained by completing the task within the specified time step; ； In the formula, dis is the Manhattan distance in three-dimensional space, specifically: ； This represents the current position of the end effector in three-dimensional space. For the final target location; ； The Manhattan distance in three-dimensional space is specifically: ； These are the current position coordinates of the end effector in three-dimensional space. The coordinates of the obstacle's location; ； in, ； ； ； The gripper's attitude information is represented by a quaternion "q", which consists of a real part... and three imaginary parts , , The "q" is derived from the API in CoppeliaSim.

3. The autonomous obstacle avoidance motion control method for a seven-axis robotic arm based on the improved SAC algorithm according to claim 1, characterized in that, The policy network consists of a real-time action network and a target action network. The entire process is as follows: Real-time action networks rely on state transmission from the backbone network. Get action ; The target action network transmits state based on the backbone network. Get action .

4. The autonomous obstacle avoidance motion control method for a seven-axis robotic arm based on the improved SAC algorithm according to claim 3, characterized in that, The evaluation network includes a real-time evaluation network and a target evaluation network, with the real-time evaluation network and the target evaluation network performing the current action respectively. and the next step The estimation is calculated, the training network is updated, and the process is repeated until convergence.

5. The autonomous obstacle avoidance motion control method for a seven-axis robotic arm based on the improved SAC algorithm according to claim 1, characterized in that, Decoupled feature extraction networks include MLP networks and CNN networks. One-dimensional vectors correspond to one MLP network for processing, and one multi-dimensional vectors correspond to one CNN network for processing.

Images