Door opening method for robot arm based on improved SAC algorithm
By combining the improved SAC algorithm with inverse kinematics, a door-opening task scenario for the robotic arm is constructed. By employing a dense reward function and prioritizing experience replay, the problems of policy instability and insufficient motion planning during the door-opening process of the robotic arm are solved, thereby improving the stability and reliability of the door-opening process of the robotic arm.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING TECH & BUSINESS UNIV
- Filing Date
- 2026-01-27
- Publication Date
- 2026-06-02
AI Technical Summary
Existing reinforcement learning-based robotic arm door-opening technologies suffer from significant training noise, unstable policy convergence, and insufficient motion planning and control, making it difficult to guarantee the feasibility and success rate of the door-opening process.
An improved SAC algorithm is adopted. By constructing a door-opening task scenario of a robotic arm in a simulation environment, the pose increment of the robotic arm end effector is used as the motion space. Motion control is performed by combining inverse kinematics. The optimal policy model is trained by using a dense reward function and a priority experience replay mechanism, and then deployed to a real robotic arm system to perform door-opening operations.
It improves the stability and reliability of the robotic arm's door-opening process, enhances its adaptability to different environments, achieves stable strategy convergence and smooth motion, and increases the success rate of door-opening tasks.
Smart Images
Figure CN122133439A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent motion, and specifically to a method for a robotic arm to open a door based on an improved SAC algorithm. Background Technology
[0002] With the development of service robots and intelligent manufacturing technologies, the demand for robotic arms in scenarios such as home services and public facility operation is increasing. Among these, autonomous door opening by robotic arms, as a fundamental link in human-computer interaction and complex task execution, has received widespread attention. Currently, reinforcement learning, due to its autonomous exploration and strategy optimization capabilities, is gradually being introduced into research on robotic arm door opening tasks, aiming to improve the robot's adaptability to different types of doors or unknown environments.
[0003] However, existing reinforcement learning-based robotic arm door-opening technologies still have certain shortcomings. On the one hand, most reinforcement learning algorithms rely on random policies for action exploration. While this can enhance exploration capabilities, it can also lead to large fluctuations in reward signals and significant training noise during training, thus affecting the stability and efficiency of policy convergence. On the other hand, existing research focuses primarily on completing door-opening tasks through pushing, pulling, or passing through door frames, paying less attention to the motion planning and control of the robotic arm during the door-opening process, making it difficult to fully guarantee the feasibility and success rate of the door-opening process.
[0004] Therefore, in order to solve the above problems, there is an urgent need for a new method for robotic arms to open doors, which can achieve stable motion control of the robotic arm during the door opening process and improve the reliability and practicality of the robotic arm in performing door opening tasks. Summary of the Invention
[0005] In view of this, the purpose of this invention is to overcome the defects in the prior art and provide a robotic arm door opening method based on the improved SAC algorithm, which can realize stable motion control of the robotic arm during the door opening process and improve the reliability and practicality of the robotic arm in performing the door opening task.
[0006] The robotic arm door opening method based on the improved SAC algorithm of the present invention includes:
[0007] A door-opening task scenario with a robotic arm is constructed in a simulation environment, and the pose increment of the robotic arm's end effector is used as the action space for reinforcement learning.
[0008] Based on the motion space, the end-effector pose increment is converted into a joint space target motion quantity, and the robotic arm is controlled to perform the corresponding action.
[0009] An observation space for reinforcement learning is constructed, and an improved SAC algorithm is used to train the robotic arm to open a door based on the observation space and action space to obtain the optimal policy model.
[0010] The optimal strategy model is deployed to a real robotic arm system, and the robotic arm is controlled to complete the door opening operation based on the end-effector pose increment output by the optimal strategy model.
[0011] Furthermore, construct a scenario for the robotic arm to open a door, specifically including:
[0012] Before each round of reinforcement learning training begins, the positions of the door and the door handle are randomly set, and the initial end pose of the robotic arm when it grasps the door handle is determined by the inverse kinematics algorithm, so that the robotic arm is in the state of grasping the door handle before the training begins.
[0013] Furthermore, the end-effector pose increment includes position increment and attitude angle increment; the motion space is a continuous motion space.
[0014] Furthermore, the observation space includes the real-time opening and closing angle of the door and the target opening and closing angle, the real-time position of the end effector of the robotic arm, the real-time position of the gripping point on the door handle, and the position of the gripping point on the door handle during environmental initialization.
[0015] Furthermore, a dense reward function is used in the reinforcement learning training, and the reward function is determined according to the following formula:
[0016] ;
[0017] in, For the overall reward function; Rewards are given for the pose changes of the robotic arm's end effector and the door handle gripping point. The reward is the change in pose between the initial position and the actual position of the door handle. Rewards are given for changes in the opening and closing angle of the door. , as well as These represent the corresponding reward weight coefficients.
[0018] Furthermore, the pose change reward between the robotic arm's end effector and the door handle gripping point is determined according to the following formula. :
[0019] ;
[0020] in, , ;
[0021] The L2 norm of the difference between the end effector position and the gripping point position of the robotic arm. and These represent the position of the robotic arm's end effector and the position of the gripping point, respectively.
[0022] This represents the minimum angular error between the end-effector posture and the gripper posture. and These represent the rotation matrices obtained by Euler angle transformation of the robotic arm's end effector and gripping point, respectively.
[0023] The pose change reward between the initial position and the actual position of the door handle is determined according to the following formula. :
[0024] ;
[0025] in, ;
[0026] ;
[0027] Let L2 be the difference between the initial door handle position and the actual door handle position. and These represent the initial door handle position and the actual door handle position, respectively.
[0028] This represents the minimum angular error between the initial door handle posture and the actual door handle posture. and These represent the rotation matrices obtained by transforming the initial door handle and the actual door handle using Euler angles, respectively.
[0029] The reward is determined by the following formula to determine the change in the opening and closing angle of the door. :
[0030] ;
[0031] in, and These represent the real-time opening / closing angle of the door and the target opening / closing angle, respectively.
[0032] Furthermore, the improved SAC algorithm includes:
[0033] The uniform random sampling replay in the SAC algorithm is replaced with priority experience replay, and the sampling probability is assigned according to the importance of the experience.
[0034] Furthermore, the sampling probability of each sample is determined according to the following formula:
[0035] ;
[0036] in, Indicates sample The probability of being sampled. The TD error magnitude represents the Q value. This emphasizes how much importance is attached to this priority. This represents the total number of samples.
[0037] Furthermore, when deploying the optimal strategy model to the real robotic arm system, the motion space and observation space of the simulation environment are aligned with those of the real robotic arm system.
[0038] The beneficial effects of this invention are as follows: This invention discloses a robotic arm door-opening method based on an improved SAC algorithm. It uses the robotic arm's end-effector pose increment as the action space for reinforcement learning and combines inverse kinematics for motion control, avoiding the instability caused by direct random exploration of joint angles. The use of a prioritized experience replay mechanism effectively reduces reward signal fluctuations, thereby improving the stability and convergence speed of reinforcement learning training. By using the improved SAC algorithm to train the robotic arm for door-opening, it balances exploration capability and policy stability. Through training in a simulation environment and deploying it to a real robotic arm system, it achieves effective transfer of the policy model between simulation and real-world environments, thus providing reliable technical support for the robotic arm to perform door-opening and subsequent continuous tasks. Attached Figure Description
[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0040] Figure 1 This is a schematic diagram of the robotic arm door opening method of the present invention;
[0041] Figure 2 This is a schematic diagram illustrating the randomized processing of the positions of the refrigerator door and door handle according to the present invention;
[0042] Figure 3 A schematic diagram illustrating the reward function settings of this invention;
[0043] Figure 4 This is a schematic diagram illustrating the principle of the improved SAC algorithm of the present invention.
[0044] Figure 5a This is a schematic diagram of the simulation experiment scenario of the present invention;
[0045] Figure 5b This is a schematic diagram of the physical experiment scenario of the present invention;
[0046] Figure 6 This is a schematic diagram of the reward transformation curves for different algorithms of the present invention;
[0047] Figure 7 This is a simulation diagram of the door opening effect of the MuJoCo invention;
[0048] Figure 8a This is a first-person perspective illustration of the actual physical door opening effect of the present invention.
[0049] Figure 8bThis is a second-view illustration of the actual physical door opening effect of the present invention;
[0050] Figure 9a This is a schematic diagram comparing the simulated motion trajectories of the present invention;
[0051] Figure 9b This is a schematic diagram comparing the actual motion trajectories of the present invention. Detailed Implementation
[0052] The present invention will be further described below with reference to the accompanying drawings, as shown in the figures:
[0053] This embodiment discloses a robotic arm door opening method based on an improved SAC algorithm, including the following steps:
[0054] A door-opening task scenario with a robotic arm is constructed in a simulation environment, and the pose increment of the robotic arm's end effector is used as the action space for reinforcement learning.
[0055] Based on the motion space, the end-effector pose increment is converted into a joint space target motion quantity, and the robotic arm is controlled to perform the corresponding action.
[0056] An observation space for reinforcement learning is constructed, and an improved SAC algorithm is used to train the robotic arm to open a door based on the observation space and action space to obtain the optimal policy model.
[0057] The optimal strategy model is deployed to a real robotic arm system, and the robotic arm is controlled to complete the door opening operation based on the end-effector pose increment output by the optimal strategy model.
[0058] In this embodiment, the MuJoCo platform is used to provide a simulation environment and construct a scenario for a robotic arm opening a door. MuJoCo is a high-performance physics engine for robotics and biomechanics, which uses generalized coordinates, implicit integration, and convex optimization contact solvers to ensure dynamic stability and accuracy, and will not be elaborated further here.
[0059] Construct a robotic arm door-opening task scenario in a simulation environment, specifically including:
[0060] Before each round of reinforcement learning training, the positions of the door and door handle are randomly set, and the initial end effector pose of the robotic arm when grasping the door handle is determined by an inverse kinematics algorithm, so that the robotic arm is in the door handle grasping state before the training begins. Figure 2 As shown, the positions of the refrigerator door and door handle in the simulation training are randomized in the domain. The double arrows in the figure represent motion. The refrigerator door will randomly change its position within a range of 0.1 meters in front and back; while the door handle will randomly change its position within a range of 0.1 meters in the up, down, left and right.
[0061] In the simulation environment, the door handle is set to be able to change pose relative to the door body to prevent unrealistic situations that rely on fixed constraints during reinforcement learning training, expand the diversity of training scenarios, reduce dependence on a single initial condition, and improve the ability of reinforcement learning to adapt to different door body position changes. Moreover, by ensuring that the robotic arm has completed the grasping before training, invalid exploration and training interference can be effectively avoided, thereby improving the stability of the later training process.
[0062] In this embodiment, the pose increment of the robotic arm's end effector is determined by the displacement increment. , , and attitude Euler angle increment , , This indicates that the end-effector pose increment includes both position increment and attitude angle increment.
[0063] The action space is a continuous action space, and the action output of reinforcement learning is achieved by using end-effector pose increments instead of joint angles, which can improve the transferability between different robotic arms.
[0064] The above settings allow direct application to the end-effector rather than specific joint angle configurations, thereby reducing dependence on the robotic arm's structural parameters and improving adaptability and portability to different robotic arm models. At the same time, they reduce control instability caused by random exploration of joint space, thus improving the smoothness and reliability of the door opening process.
[0065] Specifically, the inverse kinematics solver is used to convert the end-effector pose increment into the target motion quantity in the joint space, and the manipulator is controlled by the manipulator to perform the corresponding action; that is, the mapping from the end-effector pose to the joint space is realized by the inverse kinematics solver, and the manipulator is stabilized by the manipulator to achieve accurate tracking of the end-effector pose.
[0066] In this embodiment, the observation space includes the real-time opening and closing angle of the door and the target opening and closing angle, the real-time position of the robotic arm's end effector, the real-time position of the gripping point on the door handle, and the position of the gripping point on the door handle during environment initialization. The specific number of dimensions in the observation space is shown in Table 1.
[0067] Table 1
[0068]
[0069] By simultaneously introducing the real-time opening and closing angles of the door and the target opening and closing angle, the position of the robotic arm's end effector, and the real-time position information of the door handle's gripping point into the observation space, reinforcement learning can comprehensively perceive the progress of task execution and the spatial relationship between the robotic arm and the door handle, thereby improving the completeness and accuracy of the state representation, enhancing the ability to judge the door opening process, and improving the stability and success rate of the door opening action.
[0070] In this embodiment, reward functions are generally classified into dense reward functions, sparse reward functions, and mixed reward functions. The reinforcement learning training of this invention uses a dense reward function, which is determined according to the following formula:
[0071] ;
[0072] in, For the overall reward function; Rewards are given for the pose changes of the robotic arm's end effector and the door handle gripping point. The reward is the change in pose between the initial position and the actual position of the door handle. Rewards are given for changes in the opening and closing angle of the door; , as well as These represent the corresponding reward weight coefficients, which can be set according to actual working conditions, and will not be elaborated here.
[0073] The reward for the pose change of the robotic arm's end effector relative to the door handle gripping point is determined by the following formula. :
[0074] ;
[0075] in, , ;
[0076] The L2 norm of the difference between the end effector position and the gripping point position of the robotic arm. and These represent the position of the robotic arm's end effector and the position of the gripping point, respectively.
[0077] This represents the minimum angular error between the end-effector posture and the gripper posture. and These represent the rotation matrices obtained by Euler angle transformation of the robotic arm's end effector and gripping point, respectively.
[0078] The pose change reward between the initial position and the actual position of the door handle is determined according to the following formula. :
[0079] ;
[0080] in, ;
[0081] ;
[0082] Let L2 be the difference between the initial door handle position and the actual door handle position. and These represent the initial door handle position and the actual door handle position, respectively. Therefore, in the simulation environment, the door handle is set to be able to move freely on the door to prevent cheating during training.
[0083] This represents the minimum angular error between the initial door handle posture and the actual door handle posture. and These represent the rotation matrices obtained by transforming the initial door handle and the actual door handle using Euler angles, respectively.
[0084] The reward is determined by the following formula to determine the change in the opening and closing angle of the door. :
[0085] ;
[0086] in, The value is the L2 norm of the difference between the real-time opening / closing angle of the gate and the target opening / closing angle. and These represent the real-time opening / closing angle of the door and the target opening / closing angle, respectively.
[0087] Specific reward function settings, such as Figure 3 As shown. By setting In reinforcement learning training, the robotic arm can maintain a relatively reasonable end-effector posture when grasping a door handle to open the door, making it easier and faster to open the door. This can be achieved by setting... It can generate high-quality data when interacting in a simulation environment. It enables reinforcement learning to obtain important rewards for good policies and can guide the policy network to converge.
[0088] In this embodiment, the SAC (Soft Actor-Critic) algorithm evaluates the merits of actions based on a value algorithm in the improved SAC. The expression is:
[0089] ;
[0090] In the formula, Indicates the current state and actions Q value under, Indicates an immediate reward. Indicates the discount factor. Indicates in The maximum Q value in the state, It is to perform an action The next state that is reached later. It is in state The action that maximizes the Q value among all available actions.
[0091] In the improved SAC based on policy algorithms, the probability distribution of the output action is determined by parameterizing the action policy, rather than first solving for the Q-value and then using a greedy threshold to determine the action. The expression is:
[0092] ;
[0093] in, Indicates the output action. and These represent the input states in the policy network. The mean and logarithm are then output. This indicates that the noise variable is obtained through a normal distribution (mean 0, standard deviation 1), and then... The hyperbolic tangent function is used for nonlinear mapping. The policy network is an improved actor network in the SAC algorithm.
[0094] The SAC algorithm employs the maximum entropy reinforcement learning approach, introducing entropy constraints during policy optimization to adaptively control the exploration intensity. By incorporating entropy maximization into the optimization objective, it maintains the randomness and diversity of the policy while ensuring algorithm convergence. The policy network (also known as the actor network) outputs actions, while the critic network evaluates the value of those actions. Through mutual cooperation and joint optimization, they drive continuous policy updates and improvements. Therefore, the target policy of the SAC algorithm is as follows:
[0095] ;
[0096] in, Represents a random policy. Indicates the optimal strategy; The strategy that maximizes the objective function value ; Indicates the strategy The expected distribution of the induced state-motion trajectory is taken; Indicates in Instant rewards for each moment; Indicates the policy in the state Entropy below; This is the entropy-temperature parameter, used to balance reward maximization and exploration. The parameters are also the learning parameters. Therefore, when training the robotic arm to open a door using the improved SAC algorithm, in addition to training a policy network and two evaluation networks, it is also necessary to update... parameter.
[0097] In existing SAC algorithms, uniform random sampling is performed from the experience replay pool at each parameter update to form a mini-batch for training the network and parameters. This method has low sample utilization efficiency, treats relatively important experience equally with ordinary experience, and often converges slowly in complex tasks.
[0098] Therefore, the improved SAC algorithm replaces the uniform random sampling replay in the SAC algorithm with priority empirical replay, and assigns sampling probabilities based on the importance of the experience, typically using the TD error of the Q value as the weight; a larger error indicates that the data is more helpful for updating the network parameters. The sampling probability of each sample is determined according to the following formula:
[0099] ;
[0100] in, Indicates sample The probability of being sampled; The TD error magnitude, which represents the Q value, is usually... , Represents a very small positive constant, preventing [the occurrence of certain variables]. When the value is 0, the sample will not be accessed; This emphasizes how much importance is attached to this priority. This represents the total number of samples.
[0101] Therefore, the principle of the improved SAC algorithm is as follows: Figure 4 As shown. The optimal strategy model is a strategy network model trained by an improved SAC algorithm, which is used to output the pose increment of the robotic arm end effector given the current observation space state as input, so as to control the door opening action of the robotic arm.
[0102] In this embodiment, the optimal strategy model is deployed to a real robotic arm system. Specifically, by aligning the motion space and observation space with the simulation environment, the foundation for reproducing the effect of the strategy network is laid. The relative pose changes of the door handle and the robotic arm's end effector are relatively small, and the relative pose change of the door handle from its initial position is almost constant; therefore, a small reward value is sufficient for these parts. During the door opening process, the robotic arm's end effector is essentially perpendicular to the refrigerator door. Therefore, the change in the door's opening angle approximates the change in the robotic arm's end effector's rotation relative to the z-axis in world coordinates, and this is used as the reward value for the door's opening angle. Finally, the motion space is aligned with the actual changes in the robotic arm's end effector, and the robotic arm uses the moveit library for motion control.
[0103] To better understand the robotic arm door-opening method of the present invention, further experimental comparison and analysis were conducted:
[0104] The training process was performed on an Ubuntu 18.04 operating system. This training machine was equipped with an Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz x 16 processor, a GeForce GTX 1650 GPU with 4GB of video memory, and 16GB of RAM. Training and testing were conducted using the PyTorch deep learning framework. The robotic arm system used was a Kinova J2S7S300 mobile robot equipped with a J2S7S300 robotic arm and an omnidirectional mobile chassis. The simulation and physical experiment scenarios are as follows: Figure 5a and Figure 5b As shown.
[0105] The door-opening task in MuJoCo is used as the object. Before task training, the door handle and refrigerator are randomly initialized to achieve domain randomization. The original SAC, DDPG, TD3, and PPO algorithms are compared with the improved SAC algorithm. The hyperparameters of the algorithms during training are basically consistent, as shown in Table 2:
[0106] Table 2
[0107]
[0108] After determining the hyperparameters, training can be used to obtain the change in the reward value for each algorithm as the step increases, such as... Figure 6 As shown in the image, analysis of the reward change curves reveals that the PPO algorithm exhibits drastic reward fluctuations and a relatively low final reward value. In contrast, the improved SAC algorithm (SAC-PER) presented in this paper demonstrates more stable rewards and the highest final reward value, leading to better action strategies and providing a foundation for subsequent algorithm deployment.
[0109] Furthermore, Figure 7 Figure 8 shows the simulated and real physical door opening processes, respectively. The real physical door opening process has two different perspectives of the opening effect, such as... Figure 8a The first-person perspective shown and Figure 8b The second perspective is shown. In addition, the object pose detection work from previous phases was incorporated into the real-world physics experiment, enabling the opening of the refrigerator door and the grasping of an object, completing a relatively complete and continuous task flow. Specifically, the door handle is first identified using an object pose detection algorithm, and then the robotic arm is controlled to grasp the handle; next, the reinforcement learning method of this invention is used to open the door, moving it to the target position; the position of the robotic arm is adjusted, and the pose of the object inside the refrigerator is detected again; finally, the object is grasped and placed on the table. It can be seen that the method of this invention has good practical applicability in both the first and second perspectives.
[0110] To better illustrate the feasibility of the door-opening method of this invention, the trajectory diagrams generated by the traditional motion planning method and the reinforcement learning method of this invention in the door-opening task were compared. As shown in Figure 9, observing the comparison in the figure reveals that the method of this invention performs better in simulations (such as...). Figure 9a The simulated motion trajectory shown is compared with the real world (such as...). Figure 9b The blue trajectory shown in the comparison with the actual motion trajectory can better fit the ideal green trajectory. In contrast, the traditional method produces a large error, ultimately leading to the failure of opening the door.
[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for opening a door with a robotic arm based on an improved SAC algorithm, characterized in that: include: A door-opening task scenario with a robotic arm is constructed in a simulation environment, and the pose increment of the robotic arm's end effector is used as the action space for reinforcement learning. Based on the motion space, the end-effector pose increment is converted into a joint space target motion quantity, and the robotic arm is controlled to perform the corresponding action. An observation space for reinforcement learning is constructed, and an improved SAC algorithm is used to train the robotic arm to open a door based on the observation space and action space to obtain the optimal policy model. The optimal strategy model is deployed to a real robotic arm system, and the robotic arm is controlled to complete the door opening operation based on the end-effector pose increment output by the optimal strategy model.
2. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: Constructing a scenario for a robotic arm to open a door, specifically including: Before each round of reinforcement learning training begins, the positions of the door and the door handle are randomly set, and the initial end pose of the robotic arm when it grasps the door handle is determined by the inverse kinematics algorithm, so that the robotic arm is in the state of grasping the door handle before the training begins.
3. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: The end-effector pose increment includes position increment and attitude angle increment; the motion space is a continuous motion space.
4. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: The observation space includes the real-time opening and closing angle of the door and the target opening and closing angle, the real-time position of the end effector of the robotic arm, the real-time position of the gripping point on the door handle, and the position of the gripping point on the door handle during environmental initialization.
5. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: The reinforcement learning training employs a dense reward function, which is determined according to the following formula: ； in, For the overall reward function; Rewards are given for the pose changes of the robotic arm's end effector and the door handle gripping point. The reward is the change in pose between the initial position and the actual position of the door handle. Rewards are given for changes in the opening and closing angle of the door. , as well as These represent the corresponding reward weight coefficients.
6. The robotic arm door opening method based on the improved SAC algorithm according to claim 5, characterized in that: The reward for the pose change of the robotic arm's end effector relative to the door handle gripping point is determined by the following formula. : ； in, , ; The L2 norm of the difference between the end effector position and the gripping point position of the robotic arm. and These represent the position of the robotic arm's end effector and the position of the gripping point, respectively. This represents the minimum angular error between the end-effector posture and the gripper posture. and These represent the rotation matrices obtained by Euler angle transformation of the robotic arm's end effector and gripping point, respectively. The pose change reward between the initial position and the actual position of the door handle is determined according to the following formula. : ； in, ; ； Let L2 be the difference between the initial door handle position and the actual door handle position. and These represent the initial door handle position and the actual door handle position, respectively. This represents the minimum angular error between the initial door handle posture and the actual door handle posture. and These represent the rotation matrices obtained by transforming the initial door handle and the actual door handle using Euler angles, respectively. The reward is determined by the following formula to determine the change in the opening and closing angle of the door. : ； in, and These represent the real-time opening / closing angle of the door and the target opening / closing angle, respectively.
7. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: The improved SAC algorithm includes: The uniform random sampling replay in the SAC algorithm is replaced with priority experience replay, and the sampling probability is assigned according to the importance of the experience.
8. The robotic arm door opening method based on the improved SAC algorithm according to claim 7, characterized in that: The sampling probability of each sample is determined according to the following formula: ； in, Indicates sample The probability of being sampled. The TD error magnitude represents the Q value. This emphasizes how much importance is attached to this priority. This represents the total number of samples.
9. The robotic arm door opening method based on the improved SAC algorithm according to claim 1, characterized in that: When deploying the optimal strategy model to a real robotic arm system, the simulation environment is aligned with the motion space and observation space of the real robotic arm system.