A method for accurately controlling the opening and closing of the charging cover by a robotic arm based on deep reinforcement learning
Through the method of multi-network connection and control sequence iterative optimization, a robot arm precision control charging cover opening and closing model was constructed, which solved the problem of excessive action space in robot arm control by deep reinforcement learning, and realized the opening and closing of robot arm charging cover Efficient control and timeliness.
Patent Information
- Application Number
- CN202411346053.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-26
AI Technical Summary
In the prior art, deep reinforcement learning is too large when the robotic arm controls the opening and closing of the charging cover, which is difficult to converge, affects the timeliness, and fails to provide an accurate movement trajectory control strategy.
Multi-network connection is used to build a robot arm precisely controlled charging cover opening and closing model, build a neural network for each section according to the inherent morphology of the robot arm, and build a control sequence relationship to output the movement trajectory control strategy of each section; through control sequence iteration optimization, calculate Loss functions and update neural network parameters to provide accurate movement trajectory control strategies.
The movement space of the robotic arm movement path is reduced, the convergence speed is accelerated, the timeliness of the robotic arm control charging cover opening and closing is improved, and an accurate movement trajectory control strategy is provided.
Smart Images

Figure CN118848997B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of robotic arm control, and specifically relates to a method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning. Background Art
[0002] With the rapid development of the electric vehicle industry, the market demand and user ownership of electric vehicles have shown a rapid upward trend. However, despite the rapid growth of the electric vehicle market, the construction of supporting charging infrastructure has failed to develop synchronously, resulting in an extremely tight supply of charging facilities. This mismatch between supply and demand has, to a certain extent, hindered the popularization and application of electric vehicles. In addition, although autonomous driving technology has made breakthroughs in many fields, the charging process of electric vehicles still mainly relies on manual operation. At present, there is no mature automated charging solution, and the opening and closing of the charging cover cannot be accurately controlled, which not only increases the operational complexity of electric vehicle users, but also limits the convenience and efficiency of electric vehicle use.
[0003] Using deep reinforcement learning to enable a robotic arm to precisely control the opening and closing of the charging cover is a key technology in the automated charging solution for electric vehicles. By exploring the movement path of the robotic arm through deep reinforcement learning, the robotic arm can automatically and precisely control the opening and closing of the charging cover of the electric vehicle, which can improve the charging efficiency of electric vehicles and reduce labor costs to meet the needs of electric vehicles for fast, efficient and automated charging.
[0004] CN115179280A proposes a magnetic field-based reward shaping method for reinforcement learning manipulator control. First, the task environment and the relevant parameters of the manipulator, target, and obstacle are designed, and the hyperparameters of the reinforcement learning algorithm are set; secondly, the manipulator interacts with the environment, collects training data, calculates the magnetic field strength of the manipulator's end coordinates in the target and obstacle, and obtains the reward function; finally, a batch of data is collected from the experience replay pool, and the deep reinforcement learning algorithm is used to train the manipulator to complete the specified task. Although this method can provide the manipulator with more abundant orientation information about the target and obstacles while ensuring that the optimal strategy remains unchanged, thereby effectively improving the learning efficiency and convergence speed of the reinforcement learning algorithm in a complex dynamic environment, this method does not consider the construction of multiple neural networks according to the inherent morphology of the manipulator, and does not construct the sequence relationship of the sections of the manipulator, resulting in the action space of deep reinforcement learning being too large, difficult to converge, and affecting the timeliness of the manipulator control.
[0005] CN115091469A proposes a deep reinforcement learning manipulator motion planning method based on the maximum entropy framework. First, collect environmental information, including the manipulator state information in the initial state, the moving target point and the intermediate obstacle information; secondly, establish a manipulator kinematic model, and obtain the manipulator end point position information through positive kinematics operation; finally, obtain the manipulator velocity increment according to the state quantity. This method defines the environment in deep reinforcement learning, defines the state, action and reward function in the manipulator path planning environment in continuous space, and dynamically adjusts the range of Q by embedding the return distribution function into the maximum entropy in the algorithm, which alleviates the overestimation of the Q value in traditional reinforcement learning; at the same time, the method of reorganizing trajectory experience replay is used to fully explore the restricted space and improve the convergence speed of the algorithm. However, this method does not consider the calculation of the loss function based on the feedback of the moving path length, obstacle collision, friction of each joint, position error, etc., and ignores the iterative update of each neural network based on the loss function, and cannot provide accurate movement trajectory control strategies for each section of the manipulator. Summary of the invention
[0006] In view of the problems existing in the prior art, the present invention proposes a method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning, including a method for constructing a model for accurately controlling the opening and closing of a charging cover by a robotic arm based on multi-network connection, and a method for training a model for accurately controlling the opening and closing of a charging cover by a robotic arm based on iterative optimization of a control sequence. The present invention reduces the action space of the robotic arm's moving path, accelerates the convergence speed, and provides an accurate moving trajectory control strategy for each section of the robotic arm.
[0007] The first aspect of the present invention proposes a method for constructing a model of a charging cover opening and closing controlled by a robotic arm, using the following technical solutions:
[0008] The construction method of the model of the robot arm accurately controlling the opening and closing of the charging cover is based on multi-network connection. The specific steps are as follows:
[0009] (A1) According to the inherent shape of the robot arm, a neural network is constructed for each section of the robot arm;
[0010] (A2) Construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network;
[0011] (A3) The final destination position, the preceding robot arm position, and all fault positions are input into each neural network. Each neural network outputs the movement trajectory control strategy of the corresponding robot arm. The movement trajectory control strategy output by the preceding neural network is input into the succeeding neural network to obtain a model for the robot arm to accurately control the opening and closing of the charging cover.
[0012] The specific process of step (A1) is as follows:
[0013] (A1-1) Consider that the robot arm has Section, the set is represented as:
[0014] ;
[0015] in, This is the robot arm closest to the charging cover. It is the robot arm closest to the robot arm base;
[0016] (A1-2) corresponds to the construction of the neural network of each robotic arm, expressed as:
[0017] ;
[0018] in, For the Sectional Robotic Arm The corresponding neural network, , are neural network parameters, Generate functions for neural networks.
[0019] The specific process of step (A2) is as follows:
[0020] (A2-1) To construct the control sequence relationship of each section of the robotic arm, the control sequence relationship of the robotic arm is constructed with the robotic arm base to the robotic arm end of the charging cover as the sequence direction, and then the sequence relationship of each neural network is obtained; for The subsequent robot arm, then the corresponding for Post-sequential neural network.
[0021] In step (A3), the movement trajectory control strategy of the corresponding robot arm is expressed as:
[0022] ;
[0023] in, For the Sectional Robotic Arm Corresponding movement trajectory control strategy; For the Sectional Robotic Arm The corresponding neural network; is the final destination position of the robot arm; For the Sectional Robotic Arm The preceding robot arm, i.e. Sectional Robotic Arm location; is the set of all fault positions of the robot.
[0024] The method for constructing a model of the precise control of the opening and closing of the charging cover by a robotic arm proposed in the present invention reduces the action space of the moving path of the robotic arm and accelerates the convergence speed. First, a neural network is constructed for each section of the robotic arm according to the inherent form of the robotic arm; secondly, the control sequence relationship of the robotic arm and the sequence relationship of the neural network are constructed with the sequence direction from the robotic arm base to the robotic arm end of the charging cover; then, the final destination position, the position of the preceding robotic arm, and all fault positions are input into the neural network; finally, the neural network outputs the movement trajectory control strategy of the corresponding section of the robotic arm, and the movement trajectory control strategy output by the preceding neural network is input into the succeeding neural network, thereby realizing precise control of the opening and closing of the charging cover controlled by the robotic arm, accelerating the convergence rate of deep reinforcement learning, and effectively improving the timeliness of the opening and closing of the charging cover controlled by the robotic arm.
[0025] On the other hand, the present invention also proposes a training method for a robot arm to accurately control the opening and closing model of a charging cover, based on iterative optimization of a control sequence, and the specific steps are as follows:
[0026] (B1) Construct the overall loss function of the robot arm's precise control of the charging cover opening and closing model and determine whether it is lower than the preset threshold; if so, the model outputs the optimal robot arm movement trajectory movement strategy; if not, execute (B2)-(B3);
[0027] (B2) Divide the overall loss function of the model of the robot arm precisely controlling the opening and closing of the charging cover into two equal parts to obtain the average loss function of the robot arm. Based on the average loss function of the last section of the robot arm, use the gradient descent method to update its neural network parameters.
[0028] (B3) Back-stepping the optimal movement trajectory strategy of the preceding robot arm based on the subsequent robot arm, and calculating the deviation between the back-stepped updated movement trajectory strategy and the original movement trajectory strategy; adjusting the robot arm trajectory control loss function based on the deviation and the robot arm average loss function, and updating each neural network parameter based on the robot arm trajectory control loss function.
[0029] In step (B1), the overall moving path length error of the charging cover opening and closing model is precisely controlled by the robotic arm , obstacle collision Overall friction , Position error Construct the overall loss function of the model, expressed as:
[0030] ;
[0031] in, is the overall loss function of the model, is the overall moving path length error, is the obstacle collision weight, is the friction weight, is the position error weight. This formula means that the greater the overall moving path length error and position error, the more serious the obstacle collision and overall friction, the greater the overall loss function of the model.
[0032] Considering that each section of the robot has different movement control strategies, based on the overall loss function of the model The deviation of updating the parameters of each neural network is large. In order to improve the accuracy of the robot arm movement trajectory control strategy, the overall loss function of the model is divided into two halves The robot arm average loss function is obtained. The mobile trajectory control strategy of will not affect the mobile trajectory control strategy of other robotic arms, and can be directly based on the robot arm average loss function , update the neural network using gradient descent Parameters.
[0033] In step (B2), the robot arm average loss function is:
[0034] ;
[0035] in, is the overall loss function of the model, is the robot arm average loss function, M is the number of robot arm sections, unit: section.
[0036] For Sectional Robotic Arm , The subsequent robotic arm The movement trajectory control strategy will affect Therefore, the update is located in the sequence Neural Network Afterwards, based on Inverse calculation of the mobile trajectory control strategy The optimal moving trajectory movement strategy is used to move the robotic arm based on the updated movement trajectory movement strategy of each robotic arm.
[0037] In step (B3), the deviation between the updated movement trajectory strategy and the original movement trajectory strategy is expressed as:
[0038] ;
[0039] in, is the overall moving path length of the original moving trajectory moving strategy, m; is the overall moving path length of the updated moving trajectory movement strategy, m; is the final robot position of the original moving trajectory strategy, is the final robot position of the updated movement trajectory strategy, is the final robot position deviation weight.
[0040] To improve the reliability of the loss function, based on the calculated deviation And the robot arm average loss function Adjust its robot arm trajectory control loss function In step (B3), the robot trajectory control loss function is , expressed as:
[0041] ;
[0042] in, For robotic arm The bias weight, is the deviation between the updated mobile trajectory strategy and the original mobile trajectory strategy, is the robot arm average loss function. Based on the updated robot arm trajectory control loss function Update Sectional Robotic Arm The corresponding neural network .
[0043] The present invention proposes a training method for a robot arm precise control charging cover opening and closing model based on iterative optimization of a control sequence. First, the overall loss function of the model is constructed according to feedback conditions such as the moving path length, obstacle collision conditions, friction of each joint, position error, etc.; secondly, the overall loss function of the model is divided equally to obtain the robot arm average loss function; then for the last section of the robot arm, based on the robot arm average loss function, the gradient descent method is used to update the parameters of the neural network. Finally, the optimal movement trajectory movement strategy of the preceding robot arm is inferred based on the movement trajectory control strategy of the subsequent robot arm, and the robot arm is moved based on the updated movement trajectory movement strategy of each section of the robot arm. The deviation between the updated movement trajectory movement strategy and the original movement trajectory movement strategy is calculated, and the robot arm trajectory control loss function is adjusted based on the calculated deviation and the average loss function. The neural network is updated based on the robot arm trajectory control loss function, thereby realizing reverse iterative optimization based on the control sequence.
[0044] The present invention also proposes a method for a robotic arm to accurately control the opening and closing of a charging cover based on deep reinforcement learning, including a method for constructing the above-mentioned model of a robotic arm to accurately control the opening and closing of a charging cover and a model training method.
[0045] Compared with existing technologies:
[0046] (1) The present invention proposes a method for constructing a model of a robotic arm that accurately controls the opening and closing of a charging cover based on multi-network connection. It considers constructing multiple neural networks in sections according to the inherent shape of the robotic arm, and constructs the sequence relationship of each section of the robotic arm, thereby avoiding the problem that the action space of deep reinforcement learning is too large, difficult to converge, and affects the timeliness of the robotic arm control.
[0047] (2) The present invention proposes a training method for a robot arm's precise control model of the opening and closing of the charging cover based on iterative optimization of the control sequence. The loss function is calculated based on feedback such as the length of the moving path, obstacle collision, friction of each joint, position error, etc., and each neural network is iteratively updated based on the loss function, thereby providing a precise movement trajectory control strategy for each section of the robot arm. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION
[0049] Example 1
[0050] The construction method of the model of the robot arm accurately controlling the opening and closing of the charging cover is based on multi-network connection. The specific steps are as follows:
[0051] (A1) According to the inherent shape of the robot arm, a neural network is constructed for each section of the robot arm;
[0052] (A2) Construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network;
[0053] (A3) The final destination position, the previous robot arm position, and all fault positions are input into each neural network. Each neural network outputs the movement trajectory control strategy of the corresponding robot arm. The movement trajectory control strategy output by the previous neural network is input into the subsequent neural network to obtain the robot arm precise control charging cover opening and closing model.
[0054] The specific process of step (A1) is as follows:
[0055] (A1-1) Consider that the robot arm has Section, the set is represented as:
[0056] ;
[0057] in, This is the robot arm closest to the charging cover. It is the robot arm closest to the robot arm base;
[0058] (A1-2) corresponds to the construction of the neural network of each robotic arm, expressed as:
[0059] ;
[0060] in, For the Sectional Robotic Arm The corresponding neural network, , are neural network parameters, Generate functions for neural networks.
[0061] The specific process of step (A2) is as follows:
[0062] (A2-1) Taking the direction from the base of the robot arm to the end of the robot arm of the charging cover as the sequence direction, construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network; for The subsequent robot arm, then the corresponding for Post-sequential neural network.
[0063] In step (A3), the movement trajectory control strategy of the corresponding robot arm is expressed as:
[0064] ;
[0065] in, For the Sectional Robotic Arm Corresponding movement trajectory control strategy; is the final destination position of the robot arm; For the Sectional Robotic Arm The preceding robot arm, i.e. Sectional Robotic Arm location; is the set of all fault positions of the robot.
[0066] Example 2
[0067] The training method of the robot arm's precise control of the charging cover opening and closing model is based on iterative optimization of the control sequence. The specific steps are as follows:
[0068] (B1) Construct the overall loss function of the robot arm's precise control of the charging cover opening and closing model and determine whether it is lower than the preset threshold; if so, the model outputs the optimal robot arm movement trajectory movement strategy; if not, execute (B2)-(B3);
[0069] (B2) Divide the overall loss function of the model of the robot arm precisely controlling the opening and closing of the charging cover into two equal parts to obtain the average loss function of the robot arm. Based on the average loss function of the last section of the robot arm, use the gradient descent method to update its neural network parameters.
[0070] (B3) Back-stepping the optimal movement trajectory strategy of the preceding robot arm based on the subsequent robot arm, and calculating the deviation between the back-stepped updated movement trajectory strategy and the original movement trajectory strategy; adjusting the robot arm trajectory control loss function based on the deviation and the robot arm average loss function, and updating each neural network parameter based on the robot arm trajectory control loss function.
[0071] In step (B1), the overall moving path length error of the charging cover opening and closing model is precisely controlled by the robotic arm , obstacle collision Overall friction , Position error Construct the overall loss function of the model, expressed as:
[0072] ;
[0073] in, is the overall loss function of the model, is the overall moving path length error, is the obstacle collision weight, is the friction weight, is the position error weight.
[0074] In step (B2), the robot arm average loss function is:
[0075] ;
[0076] in, is the overall loss function of the model, is the robot arm average loss function, M is the number of robot arm sections, unit: section.
[0077] In step (B3), the deviation between the updated movement trajectory strategy and the original movement trajectory strategy is expressed as:
[0078] ;
[0079] in, is the overall moving path length of the original moving trajectory moving strategy, m; is the overall moving path length of the updated moving trajectory movement strategy, m; is the final robot position of the original moving trajectory strategy, is the final robot position of the updated movement trajectory strategy, is the final robot position deviation weight.
[0080] In step (B3), the robot trajectory control loss function is , expressed as:
[0081] ;
[0082] in, For robotic arm The bias weight, is the deviation between the updated mobile trajectory strategy and the original mobile trajectory strategy, is the robot arm average loss function. Based on the updated robot arm trajectory control loss function Update Sectional Robotic Arm The corresponding neural network .
[0083] Example 3
[0084] A method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning, including a method for constructing a model for accurately controlling the opening and closing of a charging cover by a robotic arm and a method for training the model for accurately controlling the opening and closing of a charging cover by a robotic arm.
[0085] The construction method of the model of the robot arm accurately controlling the opening and closing of the charging cover is based on multi-network connection. The specific steps are as follows:
[0086] (A1) According to the inherent form of the robotic arm, a neural network is constructed for each section of the robotic arm; specifically:
[0087] (A1-1) Consider that the robot arm has Section, the set is represented as:
[0088] ;
[0089] in, This is the robot arm closest to the charging cover. It is the robot arm closest to the robot arm base;
[0090] (A1-2) corresponds to the construction of the neural network of each robotic arm, expressed as:
[0091] ;
[0092] in, For the Sectional Robotic Arm The corresponding neural network, , are neural network parameters, Generate functions for neural networks.
[0093] (A2) Construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network;
[0094] Taking the direction from the base of the robot arm to the end of the robot arm of the charging cover as the sequence direction, the control sequence relationship of the robot arm is constructed, and then the sequence relationship of each neural network is obtained; for The subsequent robot arm, then the corresponding for Post-sequential neural network.
[0095] (A3) The final destination position, the preceding robot arm position, and all fault positions are input into each neural network. Each neural network outputs the movement trajectory control strategy of the corresponding robot arm. The movement trajectory control strategy output by the preceding neural network is input into the succeeding neural network to obtain a model for the robot arm to accurately control the opening and closing of the charging cover.
[0096] The movement trajectory control strategy of the corresponding robot arm is expressed as:
[0097] ;
[0098] in, For the Sectional Robotic Arm Corresponding movement trajectory control strategy; is the final destination position of the robot arm; For the Sectional Robotic Arm The preceding robot arm, i.e. Sectional Robotic Arm location; is the set of all fault positions of the robotic arm. The training method of the model for accurately controlling the opening and closing of the charging cover of the robotic arm based on iterative optimization of the control sequence is as follows:
[0099] (B1) Construct the overall loss function of the model and determine whether it is lower than the preset threshold; if so, the model outputs the optimal robot arm movement trajectory movement strategy; if not, execute (B2)-(B3);
[0100] The overall moving path length error of the charging cover opening and closing model based on the robot arm , obstacle collision Overall friction , Position error Construct the overall loss function of the model, expressed as:
[0101] ;
[0102] in, is the overall loss function of the model, is the overall moving path length error, is the obstacle collision weight, is the friction weight, is the position error weight.
[0103] (B2) Divide the overall loss function of the model into two equal parts to obtain the average loss function of the robot arm. Based on the average loss function of the robot arm, use the gradient descent method to update the parameters of each neural network.
[0104] The robot arm average loss function is:
[0105] ;
[0106] in, is the overall loss function of the model, is the robot arm average loss function, M is the number of robot arm sections, unit: section.
[0107] (B3) Calculate the deviation between the updated movement trajectory strategy and the original movement trajectory strategy based on the movement trajectory control strategy of the subsequent robot arm; adjust the robot arm trajectory control loss function based on the calculated deviation and the robot arm average loss function, update each neural network parameter based on the robot arm trajectory control loss function, and return to step (A3).
[0108] The deviation between the updated mobile trajectory strategy and the original mobile trajectory strategy is expressed as:
[0109] ;
[0110] in, is the overall moving path length of the original moving trajectory moving strategy, m; is the overall moving path length of the updated moving trajectory movement strategy, m; is the final robot position of the original moving trajectory strategy, is the final robot position of the updated movement trajectory strategy, is the final robot position deviation weight.
[0111] Robotic arm trajectory control loss function , expressed as:
[0112] ;
[0113] in, For robotic arm The bias weight, is the deviation between the updated mobile trajectory strategy and the original mobile trajectory strategy, is the robot arm average loss function. Based on the updated robot arm trajectory control loss function Update Sectional Robotic Arm The corresponding neural network 。
Claims
1. A method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning, characterized in that: Including the construction of a model for the robot arm to accurately control the opening and closing of the charging cover and the training of the model for the robot arm to accurately control the opening and closing of the charging cover; The steps of constructing the model of the robot arm accurately controlling the opening and closing of the charging cover are as follows: based on multi-network connection, (A1) According to the inherent shape of the robot arm, a neural network is constructed for each section of the robot arm; (A2) Construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network; (A3) The final destination position, the preceding robot arm position, and all fault positions are input into each neural network. Each neural network outputs the movement trajectory control strategy of the corresponding robot arm. The movement trajectory control strategy output by the preceding neural network is input into the succeeding neural network to obtain a model for the robot arm to accurately control the opening and closing of the charging cover. The training steps of the robot arm precise control charging cover opening and closing model are: based on control sequence iterative optimization, (B1) Construct the overall loss function of the robot arm's precise control of the charging cover opening and closing model and determine whether it is lower than the preset threshold; if so, the model outputs the optimal robot arm movement trajectory movement strategy; if not, execute (B2)-(B3); (B2) Divide the overall loss function of the model of the robot arm precisely controlling the opening and closing of the charging cover into two equal parts to obtain the average loss function of the robot arm. Based on the average loss function of the last section of the robot arm, use the gradient descent method to update its neural network parameters. (B3) Back-stepping the optimal movement trajectory strategy of the preceding robot arm based on the subsequent robot arm, and calculating the deviation between the back-stepped updated movement trajectory strategy and the original movement trajectory strategy; adjusting the robot arm trajectory control loss function based on the deviation and the robot arm average loss function, and updating each neural network parameter based on the robot arm trajectory control loss function.
2. According to the method of claim 1, the method for accurately controlling the opening and closing of the charging cover by a robotic arm based on deep reinforcement learning is characterized in that: The specific process of step (A1) is as follows: (A1-1) Consider that the robot arm has Section, the set is represented as: ; in, This is the robot arm closest to the charging cover. It is the robot arm closest to the robot arm base; (A1-2) corresponds to the construction of the neural network of each robotic arm, expressed as: ; in, For the Sectional Robotic Arm The corresponding neural network, , are neural network parameters, Generate functions for neural networks.
3. The method for accurately controlling the opening and closing of the charging cover by a robotic arm based on deep reinforcement learning according to claim 2 is characterized in that: The specific process of step (A2) is as follows: (A2-1) Taking the direction from the base of the robot arm to the end of the robot arm of the charging cover as the sequence direction, construct the control sequence relationship of the robot arm, and then obtain the sequence relationship of each neural network; for The subsequent robotic arm, then the corresponding for Post-sequential neural network.
4. The method for accurately controlling the opening and closing of the charging cover by a robotic arm based on deep reinforcement learning according to claim 3 is characterized in that: In step (A3), the movement trajectory control strategy of the corresponding robot arm is expressed as: ; in, For the Sectional Robotic Arm Corresponding movement trajectory control strategy; For the Sectional Robotic Arm The corresponding neural network, is the final destination position of the robot arm; For the Sectional Robotic Arm The preceding robot arm, i.e. Sectional Robotic Arm location; is the set of all fault positions of the robot.
5. The method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning according to claim 1, characterized in that: In step (B1), the overall moving path length error of the charging cover opening and closing model is precisely controlled by the robotic arm , obstacle collision Overall friction , Position error Construct the overall loss function of the model, expressed as: ; in, is the overall loss function of the model, is the overall moving path length error, is the obstacle collision weight, is the friction weight, is the position error weight.
6. The method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning according to claim 5 is characterized in that: In step (B2), the robot arm average loss function is: in, is the overall loss function of the model, is the robot arm average loss function, M is the number of robot arm sections, unit: section.
7. The method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning according to claim 6, characterized in that: In step (B3), the deviation between the updated movement trajectory strategy and the original movement trajectory strategy is expressed as: ; in, is the overall moving path length of the original moving trajectory moving strategy, m; is the overall moving path length of the updated moving trajectory movement strategy, m; is the final robot position of the original moving trajectory strategy, is the final robot position of the updated movement trajectory strategy, is the final robot position deviation weight.
8. The method for accurately controlling the opening and closing of a charging cover by a robotic arm based on deep reinforcement learning according to claim 7, characterized in that: In step (B3), the robot trajectory control loss function is , expressed as: in, For robotic arm The bias weight, is the deviation between the updated mobile trajectory strategy and the original mobile trajectory strategy, is the robot arm average loss function.
Citation Information
Patent Citations
Deep reinforcement learning mechanical arm motion planning method based on maximum entropy frame
CN115091469A
Magnetic field-based reward shaping method for reinforcement learning mechanical arm control
CN115179280A
Exoskeleton robot system for reloading batteries of electric vehicle
CN102922508A
Industrial mechanical arm automatic control method based on deep reinforcement learning
CN108052004A
Electric vehicle layered charging strategy method based on probability transfer matrix
CN113919229A
