A mechanical arm safety control method and system based on tail risk constraint

Abstract

The application discloses a kind of mechanical arm safety control method and system based on tail risk constraint, which comprises the following steps: collecting the current state vector data of mechanical arm and carrying out data preprocessing and action generation processing, to obtain the displacement increment action of the end of mechanical arm;According to the displacement increment action of the end of mechanical arm, control mechanical arm to execute safety cost calculation, and carry out reward and cost evaluation, to obtain cumulative task income estimate value and cumulative safety cost estimate value;Screening the success trajectory of the current action of mechanical arm execution carries out tail risk statistics, and dynamically updates Lagrange multiplier, outputs updated Lagrange multiplier;Carry out strategy optimization, output the safety control action of mechanical arm.The application can reduce the sudden change of mechanical arm action and control jitter, improve the stability of mechanical arm operation.The application can be widely applied in the field of robot control technology as a kind of mechanical arm safety control method and system based on tail risk constraint.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a method and system for safety control of a robotic arm based on tail risk constraints. Background Technology

[0002] Robotic arms are widely used in industrial assembly, material handling, welding, grinding, sorting, and collaborative operations. In actual production processes, robotic arms not only need to complete target arrival or trajectory tracking tasks, but also must ensure the smoothness, safety, and lifespan of the mechanism during execution. However, traditional robotic arm control methods mostly focus on minimizing target error or maximizing task completion rate as the primary optimization objective, lacking unified modeling and constraints for speed fluctuations, acceleration impacts, and long-term wear during the control process.

[0003] In recent years, deep reinforcement learning has achieved good results in robotic arm control. Deep reinforcement learning-based robotic arm control methods can directly establish a mapping relationship from state input to action output, exhibiting strong nonlinear expressive power and adaptability. However, most existing reinforcement learning control methods typically only drive policy optimization through reward functions. While they are effective in improving task success rates, they are prone to generating large velocity and acceleration spikes during the control process, leading to overly aggressive robotic arm actions, which in turn exacerbates mechanism wear and safety risks.

[0004] To address these issues, some research has begun to introduce safety constraints or cost-penalty mechanisms into reinforcement learning. However, existing constraint methods typically rely on average costs as the primary adjustment basis, reflecting only the overall risk level and failing to effectively characterize and suppress the small number of high-amplitude tail-risk actions during the control process. Although these high-risk tail-risk actions occur less frequently statistically, they are more likely to cause mechanical shocks, exceed limits, equipment vibrations, and component fatigue, significantly impacting the stable operation of industrial sites. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a robotic arm safety control method and system based on tail risk constraints, which can reduce abrupt changes in robotic arm movements and control jitter, thereby improving the operational stability of the robotic arm.

[0006] The first technical solution adopted in this invention is: a robotic arm safety control method based on tail risk constraints, comprising the following steps:

[0007] The current state vector data of the robotic arm is collected and preprocessed with data and motion generation to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space.

[0008] Based on the displacement increment of the robotic arm end effector in three-dimensional Cartesian space, the safety cost of the robotic arm is calculated, and the reward and cost are evaluated to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm.

[0009] Successful trajectories of the robotic arm executing the current action are selected for tail risk statistics, and the Lagrange multipliers are dynamically updated, outputting the updated Lagrange multipliers;

[0010] By combining the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier, the strategy is optimized, and the safe control action of the robotic arm is output.

[0011] Furthermore, the step of acquiring the current state vector data of the robotic arm and performing data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm's end effector in three-dimensional Cartesian space specifically includes:

[0012] The current state vector data of the robotic arm is acquired and preprocessed to obtain the preprocessed current state vector of the robotic arm. The current state vector data of the robotic arm includes the current end position, joint position, joint velocity, target position, and the relative error between the current position and the target position.

[0013] Based on the policy network, the preprocessed current state vector of the robotic arm is fused, encoded and nonlinearly mapped to extract feature information reflecting the current control state of the robotic arm.

[0014] The displacement increment motion of the robotic arm end effector in three-dimensional Cartesian space is determined based on the characteristic information of the current control state of the robotic arm.

[0015] Furthermore, the step of calculating the safety cost of controlling the robotic arm to perform actions based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space, and evaluating rewards and costs to obtain the estimated cumulative task benefit and the estimated cumulative safety cost of the robotic arm's current state action pair, specifically includes:

[0016] The total safety cost of the robotic arm's actions is calculated by controlling the robotic arm's execution safety cost based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space.

[0017] Based on the total safety cost of the robotic arm's actions, a reward and cost assessment is performed on the robotic arm's current actions to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the robotic arm's current state action pair.

[0018] Furthermore, the step of calculating the total safety cost of the robotic arm's actions based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space specifically includes:

[0019] The robot arm is controlled to perform the current action based on the displacement increment of the end effector in three-dimensional Cartesian space. The current action vector of the robot arm is collected, which includes the joint velocity of each joint of the robot arm at each time and the displacement change of each joint of the robot arm at each time.

[0020] Based on the joint velocities of each joint of the robotic arm at each moment, the difference between the joint velocities at the current moment and the previous moment is calculated to obtain the acceleration of each joint of the robotic arm.

[0021] The cumulative displacement changes of each joint of the robotic arm at each moment are calculated to obtain the wear amount of each joint of the robotic arm.

[0022] The robot arm joint velocity cost is calculated based on the mean of the squared terms of the joint velocities of each joint at each moment; the robot arm joint acceleration cost is calculated based on the mean of the squared terms of the accelerations of each joint; and the robot arm joint wear cost is calculated based on the ratio of the wear amount of each joint to the preset wear budget.

[0023] The total safety cost of the robotic arm's joints is obtained by summing the joint speed cost, joint acceleration cost, and joint wear cost.

[0024] Furthermore, the step of evaluating the reward and cost of the robotic arm's current action based on the total safety cost of the robotic arm's actions, to obtain the estimated cumulative task benefit and the estimated cumulative safety cost of the robotic arm's current action pair, specifically includes:

[0025] The preprocessed current state vector of the robotic arm is concatenated with the current action vector of the robotic arm to obtain the joint vector of the current state and action of the robotic arm.

[0026] The joint vector of the current state action of the robotic arm is input into the benefit evaluation network to evaluate the contribution of the robotic arm to the goal of reaching the task after performing the current action, and outputs the cumulative task benefit estimate of the current state action pair of the robotic arm.

[0027] The joint vector of the current state action of the robotic arm is input into the cost evaluation network. Based on the total safety cost of the robotic arm's action, the network evaluates the level of safety risk generated after the robotic arm performs the current action and outputs the cumulative safety cost estimate of the current state action pair of the robotic arm.

[0028] Furthermore, the step of performing tail risk statistics on the successful trajectory of the screening robot arm executing the current action, dynamically updating the Lagrange multipliers, and outputting the updated Lagrange multipliers specifically includes:

[0029] Successful trajectories of the robotic arm executing the current action are selected and tail risk statistics are performed in combination with the total safety cost of the robotic arm's actions to obtain the tail risk index of the robotic arm.

[0030] The Lagrange multipliers are dynamically updated based on the tail risk index of the robotic arm, and the updated Lagrange multipliers are output.

[0031] Furthermore, the step of selecting the successful trajectory of the robotic arm executing the current action and combining it with the total safety cost of the robotic arm's actions to perform tail risk statistics and obtain the tail risk index of the robotic arm specifically includes:

[0032] Select successful trajectories of the robotic arm performing the current action. The successful trajectory represents the trajectory that completes the target arrival task under the drive of the action output by the policy network, and the distance between the end effector and the target position is less than a preset success threshold when the round ends.

[0033] Extract the total safety cost of the robotic arm's current action corresponding to the successful trajectory of the robotic arm's current action, and construct the activity cost sequence of the robotic arm;

[0034] Based on the activity cost sequence of the robotic arm, the mean activity cost and the high quantile activity cost are extracted. The mean activity cost is used to characterize the average safety risk level of the successful trajectory during the effective movement phase, and the high quantile activity cost is used to characterize the high-risk spike level at the tail of the activity cost sequence.

[0035] The tail risk index of the robotic arm is obtained by weighted summation of the mean activity cost and the high quantile activity cost.

[0036] Furthermore, the step of dynamically updating the Lagrange multipliers based on the tail risk index of the robotic arm and outputting the updated Lagrange multipliers specifically includes:

[0037] The tail risk index of the robotic arm is compared with a preset safety threshold to obtain the tail risk deviation;

[0038] The Lagrange multipliers are dynamically updated based on the tail risk bias. If the tail risk index is higher than the preset safety threshold, the Lagrange multipliers are increased to enhance the safety cost constraint strength. If the tail risk index is lower than the preset safety threshold, the Lagrange multipliers are decreased to reduce the degree of inhibition on task benefit optimization. The updated Lagrange multipliers are then output.

[0039] Furthermore, the step of combining the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to perform strategy optimization and output the safe control action of the robotic arm specifically includes:

[0040] The task reward is determined based on the cumulative task benefit estimate of the current state action pair of the robotic arm, and the safety cost is determined based on the cumulative safety cost estimate of the current state action pair of the robotic arm and the updated Lagrange multiplier.

[0041] Obtain the current motion amplitude of the robotic arm and the change in motion at adjacent time points to determine the motion smoothing term;

[0042] A policy optimization objective function is constructed by combining task reward terms, safety cost terms, and action smoothness terms, and the parameters of the policy network are updated accordingly.

[0043] The updated policy network is deployed at the robotic arm control terminal, and outputs the robotic arm's safety control actions based on the real-time status information input during actual operation.

[0044] The second technical solution adopted in this invention is: a robotic arm safety control system based on tail risk constraints, comprising:

[0045] The first module is used to collect the current state vector data of the robotic arm and perform data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space.

[0046] The second module is used to control the robotic arm to perform safety cost calculation based on the displacement increment of the robotic arm end in three-dimensional Cartesian space, and to evaluate the reward and cost, so as to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm.

[0047] The third module is used to screen the successful trajectories of the robotic arm executing the current action, perform tail risk statistics, dynamically update the Lagrange multipliers, and output the updated Lagrange multipliers.

[0048] The fourth module is used to combine the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to optimize the strategy and output the safe control action of the robotic arm.

[0049] The beneficial effects of the method and system of this invention are as follows: This invention obtains the displacement increment action of the robotic arm end effector in three-dimensional Cartesian space by collecting the current state vector data of the robotic arm and performing data preprocessing and motion generation processing; then, based on the displacement increment action of the robotic arm end effector in three-dimensional Cartesian space, it controls the robotic arm to perform safety cost calculation, and performs reward and cost evaluation to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm. By constructing a multi-source safety cost model composed of joint velocity cost, joint acceleration cost, and joint wear cost, it can more comprehensively reflect the actual situation in the robotic arm control process. Risk assessment: Further screening of successful trajectories for the robotic arm's current actions is performed to statistically analyze tail risks. The Lagrange multipliers are dynamically updated, and the updated Lagrange multipliers are output. Multi-source safety cost modeling and a tail risk-aware Lagrange adjustment mechanism are introduced to improve the safety and smoothness of the robotic arm control process. Finally, combining the cumulative task reward estimate, the cumulative safety cost estimate, and the updated Lagrange multipliers of the robotic arm's current state action pair, strategy optimization is performed to output the safe control actions of the robotic arm. An action smoothing penalty term is added to the strategy optimization objective function to reduce action abrupt changes and control jitter, improving the robotic arm's operational stability. This approach effectively reduces velocity and acceleration spikes, decreases over-limit rates, and delays mechanism wear while ensuring target achievement performance. It is suitable for safe motion control of industrial robotic arms, collaborative robotic arms, and articulated robot systems. Attached Figure Description

[0050] Figure 1 This is a flowchart of the steps of a robotic arm safety control method based on tail risk constraints according to the present invention.

[0051] Figure 2 This is a structural block diagram of a robotic arm safety control system based on tail risk constraints according to the present invention.

[0052] Figure 3 This is a schematic diagram of the overall framework for robotic arm safety control based on tail risk constraints provided in a specific embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the tail risk indicator construction and Lagrange multiplier dynamic update process provided in a specific embodiment of the present invention. Detailed Implementation

[0054] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0055] First, it needs to be explained that, as Figure 3 The diagram shown is the overall flowchart of this invention. Based on the deep reinforcement learning robotic arm control framework, a multi-source safety cost model consisting of joint velocity cost, joint acceleration cost, and joint wear cost is introduced. At the same time, a tail risk index is used to dynamically adjust the Lagrange multiplier to suppress high-risk control actions and improve the safety and smoothness of the robotic arm execution process.

[0056] Deep reinforcement learning-based robotic arm control is an intelligent optimization method that obtains control strategies through the interaction of state, action, reward, and environment. Its basic principle is as follows: the robotic arm receives the current state input at each control moment, the policy network outputs a control action based on the current state, and the environment, after executing the action, provides feedback on the new state, reward, and safety cost. The policy network continuously updates its parameters accordingly, thereby learning a control strategy that enables the robotic arm to complete the target task. Traditional methods primarily aim to maximize task reward, while this invention introduces a cost evaluation network and Lagrangian constraint adjustment, making the control strategy consider not only task completion capability but also motion safety and equipment lifespan.

[0057] Based on this, refer to Figure 1 This invention provides a safety control method for a robotic arm based on tail risk constraints, the method comprising the following steps:

[0058] S100: Collect the current state vector data of the robotic arm and perform data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space.

[0059] S110. Obtain the current state vector data of the robotic arm and perform data preprocessing to obtain the preprocessed current state vector of the robotic arm. The current state vector data of the robotic arm includes the current end position, joint position, joint speed, target position, and relative error between the current position and the target position.

[0060] In this embodiment, state acquisition and data preprocessing are performed. The current end effector position, joint position, joint velocity, target position, and relative error between the current and target positions of the robotic arm are acquired and constructed into a unified state input vector. Subsequently, the state input is normalized to reduce the differences between different units and improve the training stability of the policy network and evaluation network.

[0061] S120. Based on the policy network, the preprocessed current state vector of the robotic arm is fused, encoded and nonlinearly mapped to extract feature information reflecting the current control state of the robotic arm.

[0062] S130. Determine the displacement increment action of the robotic arm end in three-dimensional Cartesian space based on the characteristic information of the current control state of the robotic arm.

[0063] In this embodiment, motion generation involves feeding a normalized state input vector into a policy network. The policy network fuses and encodes information such as end-effector position, joint position, joint velocity, target position, and relative error, performing nonlinear mapping to extract feature information reflecting the current control state. Based on this, it outputs the incremental displacement motion of the robotic arm's end-effector in three-dimensional Cartesian space. This motion is then mapped to the robotic arm controller to drive the end-effector to move towards the target position. The incremental displacement motion includes X-direction increments, Y-direction increments, and Z-direction increments.

[0064] S200. Based on the displacement increment of the robotic arm end in the three-dimensional Cartesian space, the robotic arm is controlled to perform safety cost calculation, and rewards and costs are evaluated to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm.

[0065] S210. Calculate the safety cost of the robotic arm's actions based on the displacement increment of the robotic arm's end in three-dimensional Cartesian space, and obtain the total safety cost of the robotic arm's actions.

[0066] Specifically, the robotic arm is controlled to execute the current action based on the displacement increment of its end effector in three-dimensional Cartesian space. The current action vector of the robotic arm is acquired, including the joint velocities and displacement changes of each joint at each moment. Based on the joint velocities at each moment, the difference between the current and previous joint velocities is calculated to obtain the acceleration of each joint. The displacement changes of each joint at each moment are accumulated to obtain the wear amount of each joint. The joint velocity cost is calculated based on the mean of the squared terms of the joint velocities at each moment; the joint acceleration cost is calculated based on the mean of the squared terms of the accelerations of each joint; and the joint wear cost is calculated based on the ratio of the wear amount to the preset wear budget. Finally, the joint velocity cost, joint acceleration cost, and joint wear cost are summed to obtain the total safety cost of the robotic arm's action.

[0067] In this embodiment, the robotic arm executes actions and calculates safety costs. The robotic arm executes actions output by the strategy network, the environment records the joint velocity of each joint at the current moment, and calculates the acceleration of each joint based on the difference between the joint velocity at the current moment and the previous moment. Simultaneously, the joint wear amount is obtained by accumulating the displacement changes of the joints at each moment. Further, the joint velocity cost is calculated based on the mean of the squared terms of each joint velocity, the joint acceleration cost is calculated based on the mean of the squared terms of each joint acceleration, and the joint wear cost is calculated based on the accumulated joint displacement and its proportion relative to the preset wear budget. The three costs are then summed to obtain the total safety cost.

[0068] S220. Based on the total safety cost of the robotic arm's actions, evaluate the reward and cost of the robotic arm's current actions to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the robotic arm's current state action pair.

[0069] Specifically, the preprocessed current state vector of the robotic arm is concatenated with the current action vector of the robotic arm to obtain the joint vector of the current state and action of the robotic arm. The joint vector of the current state and action of the robotic arm is input into the benefit evaluation network to evaluate the contribution of the robotic arm to the goal of reaching the target after performing the current action, and outputs the cumulative task benefit estimate of the current state and action pair of the robotic arm. The joint vector of the current state and action of the robotic arm is input into the cost evaluation network to evaluate the safety risk level generated by the robotic arm after performing the current action based on the total safety cost of the robotic arm performing the action, and outputs the cumulative safety cost estimate of the current state and action pair of the robotic arm.

[0070] In this embodiment, reward and cost evaluation are performed. The preprocessed state vector at the current moment is concatenated with the current action vector, and then input into the reward evaluation network and the cost evaluation network, respectively. The reward evaluation network outputs the cumulative task reward estimate of the current state-action pair based on the contribution of the robotic arm to the goal of reaching the target after performing the current action. The cost evaluation network outputs the cumulative safety cost estimate of the current state-action pair based on the level of safety risk generated by the robotic arm performing the current action, thereby achieving parallel evaluation of task completion capability and execution risk.

[0071] S300: Screen the successful trajectories of the robotic arm executing the current action, perform tail risk statistics, dynamically update the Lagrange multipliers, and output the updated Lagrange multipliers;

[0072] S310. Filter the successful trajectory of the robotic arm performing the current action and combine it with the total safety cost of the robotic arm performing the action to perform tail risk statistics and obtain the tail risk index of the robotic arm.

[0073] Specifically, successful trajectories of the robotic arm executing the current action are selected. A successful trajectory represents a trajectory that completes the target arrival task under the drive of the action output by the policy network, and the distance between the end effector and the target position is less than a preset success threshold at the end of the round. The total safety cost of the robotic arm's actions corresponding to the successful trajectories of the current action is extracted to construct the activity cost sequence of the robotic arm. Based on the activity cost sequence of the robotic arm, the mean activity cost and the high quantile activity cost are extracted. The mean activity cost is used to characterize the average safety risk level of the successful trajectory within the effective movement phase, and the high quantile activity cost is used to characterize the high-risk spike level at the tail of the activity cost sequence. The mean activity cost and the high quantile activity cost are weighted and summed to obtain the tail risk index of the robotic arm.

[0074] In this embodiment, tail risk statistics are performed. Within a training cycle, successful trajectories that meet the target arrival conditions are selected. A successful trajectory refers to a trajectory in which the robotic arm starts from the initial state within a training round, completes the target arrival task under the drive of the policy network's output action, and the distance between the end effector and the target position is less than a preset success threshold at the end of the round. For each successful trajectory, the total safety cost corresponding to each moment within the effective movement phase is extracted to form an activity cost sequence. The mean activity cost and the high quantile activity cost are calculated based on the activity cost sequence, where the mean activity cost is used to characterize the average safety risk level of the successful trajectory within the effective movement phase, and the high quantile activity cost is used to characterize the tail high-risk spike level in the activity cost sequence. Finally, the mean activity cost and the high quantile activity cost are weighted according to preset weights to construct a tail risk index.

[0075] The high-quantile activity cost is any one of the P90, P95, or P99 statistics of the activity cost sequence.

[0076] S320: Based on the tail risk index of the robotic arm, the Lagrange multiplier is dynamically updated, and the updated Lagrange multiplier is output.

[0077] Specifically, the tail risk index of the robotic arm is compared with a preset safety threshold to obtain the tail risk deviation. The Lagrange multiplier is dynamically updated based on the tail risk deviation. If the tail risk index is higher than the preset safety threshold, the Lagrange multiplier is increased to enhance the safety cost constraint strength. If the tail risk index is lower than the preset safety threshold, the Lagrange multiplier is decreased to reduce the degree of inhibition on task benefit optimization. The updated Lagrange multiplier is then output.

[0078] In this embodiment, as Figure 4As shown, the Lagrange multiplier is updated by comparing the tail risk index with a preset safety threshold to obtain the tail risk deviation, and then dynamically updating the Lagrange multiplier based on the tail risk deviation. When the tail risk index is higher than the preset safety threshold, the Lagrange multiplier is increased to strengthen the safety cost constraint; when the tail risk index is lower than the preset safety threshold, the Lagrange multiplier is decreased to reduce the inhibition of task benefit optimization. To reduce the impact of statistical noise on the update results, the tail risk index can be first processed by exponential moving average, and a dead zone interval can be set to reduce frequent oscillations of the Lagrange multiplier.

[0079] The Lagrange multiplier update employs exponential moving average and dead zone adjustment mechanisms to reduce constraint parameter update jitter and improve training stability.

[0080] S400 combines the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to optimize the strategy and output the safe control action of the robotic arm.

[0081] S410. Determine the task reward item based on the cumulative task benefit estimate of the current state action pair of the robotic arm, and determine the safety cost item based on the cumulative safety cost estimate of the current state action pair of the robotic arm and the updated Lagrange multiplier.

[0082] S420. Obtain the current motion amplitude of the robotic arm and the change in motion at adjacent time points to determine the motion smoothing term;

[0083] S430: Combine task reward items, safety cost items, and action smoothing items to construct a policy optimization objective function, and update the parameters of the policy network.

[0084] In this embodiment, strategy optimization involves determining a task reward term based on the cumulative task reward estimate output by the reward evaluation network, a safety cost term based on the cumulative safety cost estimate output by the cost evaluation network and the Lagrange multiplier, and a motion smoothing term based on the current motion amplitude and the motion change at adjacent time points. The task reward term, safety cost term, and motion smoothing term are combined to form the strategy optimization objective function, and the strategy network parameters are updated. By iteratively training and updating the strategy network parameters, the robotic arm can reduce speed and acceleration spikes during the control process while ensuring target arrival performance.

[0085] S440: Deploy the updated policy network to the robotic arm control terminal, and output the robotic arm's safety control actions based on the real-time status information input during actual operation.

[0086] In this embodiment, the system is deployed online. The trained policy network is deployed at the control end of the robotic arm. During actual operation, real-time status information is input, and safe control actions are output to achieve smooth control when the target arrives at the task.

[0087] In summary, the embodiments of this invention aim to address the problem that robotic arms focus solely on task success rate during target arrival and task execution, neglecting velocity spikes, acceleration spikes, and long-term wear accumulation. Based on a deep reinforcement learning robotic arm control framework, this method constructs a multi-source safety cost model composed of joint velocity cost, joint acceleration cost, and joint wear cost, and introduces a tail risk constraint mechanism to suppress high-risk tail movements during the control process. Specifically, the normalized state input vector is fed into a policy network, which outputs the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space. The current state vector and action vector are concatenated and input into the benefit evaluation network and cost evaluation network, respectively. Successful trajectories are selected during the training period, and the total safety cost corresponding to each moment within the effective motion phase is extracted to form an activity cost sequence. The mean activity cost and high-quantile activity cost are further calculated and weighted according to preset weights to construct a tail risk index for dynamically adjusting the Lagrange multiplier. Furthermore, a motion smoothing penalty term is added to the policy optimization objective function to reduce abrupt motion changes and control jitter, thereby improving the stability of the robotic arm operation. It can effectively reduce speed and acceleration spikes, reduce over-limit rate and delay mechanism wear while ensuring target arrival performance. It is suitable for safe motion control of industrial robotic arms, collaborative robotic arms and articulated robot systems.

[0088] Therefore, the embodiments of the present invention have the following advantages compared with the prior art:

[0089] In summary, the robotic arm safety control method based on tail risk constraints in this invention has the following beneficial effects:

[0090] 1) This invention constructs a multi-source safety cost model consisting of joint speed cost, joint acceleration cost, and joint wear cost, which can more comprehensively reflect the actual risks in the control process of the robotic arm.

[0091] 2) By introducing a tail risk indicator, this invention incorporates the cost of high-quantile activities into the constraint factor update process, which can effectively suppress a small number of dangerous actions with large amplitudes, thereby reducing speed spikes and acceleration spikes.

[0092] 3) This invention adopts a dynamic Lagrangian adjustment method, which adaptively adjusts the constraint strength according to the tail risk level in the control process, and has better flexibility and stability compared with the fixed penalty method.

[0093] 4) This invention adds a motion smoothing penalty term to the strategy optimization objective, which can effectively reduce control output jitter and improve the continuity and smoothness of the robotic arm's motion.

[0094] 5) While ensuring target achievement performance, this invention can reduce the control over-limit rate and slow down long-term wear accumulation, making it suitable for widespread application in industrial robotic arms and collaborative robot systems.

[0095] Finally, experimental verification is provided to illustrate the embodiments of the present invention:

[0096] This invention uses the UR5eReach-v1 six-DOF robotic arm end-effector arrival task as the experimental verification platform. The environment is built based on the PyBullet physics engine and Gymnasium interface, and the control algorithm is implemented based on the PyTorch deep learning framework. In the experiment, a fixed test set offline evaluation method is used, uniformly testing 100 fixed target points and fixed initial pose samples. The evaluation seed is fixed at 100, and a fixed initial state is used to avoid interference from random initialization.

[0097] To evaluate the effectiveness of the tail risk constraint-based robotic arm safety control method provided by this invention, the following methods were selected for comparison: an unconstrained control method, which aims only at maximizing task benefits without explicitly introducing safety constraints; a fixed penalty method, which introduces a fixed-weight safety penalty term into the strategy optimization objective; and a mean Lagrange constraint method, which uses the mean activity cost to drive the Lagrange multiplier update; while the method of this invention uses a tail risk index to drive dynamic constraint updates.

[0098] The comparative experiment used success rate, terminal error, average activity cost, over-limit rate, total activity cost P95, and total activity cost P99 as evaluation metrics. Success rate measures the robotic arm's ability to complete tasks on a fixed test set; terminal error measures the distance between the end effector and the target point when the end effector stops; average activity cost reflects the average safety cost of the effective movement phase; over-limit rate characterizes the proportion of safety costs exceeding a threshold during the activity phase; and P95 and P99 are used to characterize high-risk tail cost levels.

[0099] Based on the comparative experimental results using a fixed test set, under the same test conditions, the robotic arm safety control method based on tail risk constraints provided by this invention exhibits superior overall performance. Specifically, the success rate of this invention reaches 0.98, significantly higher than the 0.70 of the unconstrained method, 0.76 of the fixed penalty method, and 0.81 of the mean Lagrange constraint method; the average activity cost of this invention is only 0.0102, significantly lower than the 0.5293 of the unconstrained method, 0.3556 of the fixed penalty method, and 0.2216 of the mean Lagrange constraint method; the over-limit rate of this invention is only 0.0090, far lower than the 0.7423 of the unconstrained method, 0.6066 of the fixed penalty method, and 0.3695 of the mean Lagrange constraint method. Furthermore, this invention also demonstrates a significant advantage in terms of overall cost tail risk, with an overall activity cost P95 of 0.0334 and an overall activity cost P99 of 0.0480, both significantly lower than other comparative methods.

[0100] Experimental results show that the method provided by this invention can not only maintain a higher task success rate while ensuring the end-effector performance, but also significantly suppress speed spikes, acceleration spikes and high-risk tail events, reduce the probability of exceeding safety limits and slow down the cumulative wear growth rate, thereby verifying the effectiveness and practicality of this invention in the safety control of robotic arms.

[0101] Reference Figure 2 A safety control system for a robotic arm based on tail risk constraints, comprising:

[0102] The first module 201 is used to collect the current state vector data of the robotic arm and perform data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space.

[0103] The second module 202 is used to control the robotic arm to perform safety cost calculation based on the displacement increment of the robotic arm end in the three-dimensional Cartesian space, and to evaluate the reward and cost, so as to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm.

[0104] The third module 203 is used to screen the successful trajectories of the robotic arm executing the current action, perform tail risk statistics, dynamically update the Lagrange multipliers, and output the updated Lagrange multipliers.

[0105] The fourth module 204 is used to combine the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to optimize the strategy and output the safe control action of the robotic arm.

[0106] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0107] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this is not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for safety control of a robotic arm based on tail risk constraints, characterized in that, Includes the following steps: The current state vector data of the robotic arm is collected and preprocessed with data and motion generation to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space. Based on the displacement increment of the robotic arm end effector in three-dimensional Cartesian space, the safety cost of the robotic arm is calculated, and the reward and cost are evaluated to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm. Successful trajectories of the robotic arm executing the current action are selected for tail risk statistics, and the Lagrange multipliers are dynamically updated, outputting the updated Lagrange multipliers; By combining the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier, the strategy is optimized, and the safe control action of the robotic arm is output.

2. The robotic arm safety control method based on tail risk constraints according to claim 1, characterized in that, The step of acquiring the current state vector data of the robotic arm and performing data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm's end effector in three-dimensional Cartesian space specifically includes: The current state vector data of the robotic arm is acquired and preprocessed to obtain the preprocessed current state vector of the robotic arm. The current state vector data of the robotic arm includes the current end position, joint position, joint velocity, target position, and the relative error between the current position and the target position. Based on the policy network, the preprocessed current state vector of the robotic arm is fused, encoded and nonlinearly mapped to extract feature information reflecting the current control state of the robotic arm. The displacement increment motion of the robotic arm end effector in three-dimensional Cartesian space is determined based on the characteristic information of the current control state of the robotic arm.

3. The robotic arm safety control method based on tail risk constraints according to claim 2, characterized in that, The step of calculating the safety cost of controlling the robotic arm to perform actions based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space, and evaluating the rewards and costs to obtain the estimated cumulative task benefit and the estimated cumulative safety cost of the robotic arm's current state action pair, specifically includes: The total safety cost of the robotic arm's actions is calculated by controlling the robotic arm's execution safety cost based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space. Based on the total safety cost of the robotic arm's actions, a reward and cost assessment is performed on the robotic arm's current actions to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the robotic arm's current state action pair.

4. The robotic arm safety control method based on tail risk constraints according to claim 3, characterized in that, The step of calculating the total safety cost of the robotic arm's actions based on the displacement increment of the robotic arm's end effector in three-dimensional Cartesian space specifically includes: The robot arm is controlled to perform the current action based on the displacement increment of the end effector in three-dimensional Cartesian space. The current action vector of the robot arm is collected, which includes the joint velocity of each joint of the robot arm at each time and the displacement change of each joint of the robot arm at each time. Based on the joint velocities of each joint of the robotic arm at each moment, the difference between the joint velocities at the current moment and the previous moment is calculated to obtain the acceleration of each joint of the robotic arm. The wear amount of each joint of the robotic arm is obtained by cumulatively calculating the displacement changes of each joint at each time moment. The robot arm joint velocity cost is calculated based on the mean of the squared terms of the joint velocities of each joint at each moment; the robot arm joint acceleration cost is calculated based on the mean of the squared terms of the accelerations of each joint; and the robot arm joint wear cost is calculated based on the ratio of the wear amount of each joint to the preset wear budget. The total safety cost of the robotic arm's joints is obtained by summing the joint speed cost, joint acceleration cost, and joint wear cost.

5. The robotic arm safety control method based on tail risk constraints according to claim 4, characterized in that, The step of evaluating the reward and cost of the robotic arm's current action based on the total safety cost of the robotic arm's actions, to obtain the estimated cumulative task benefit and the estimated cumulative safety cost of the robotic arm's current action pair, specifically includes: The preprocessed current state vector of the robotic arm is concatenated with the current motion vector of the robotic arm to obtain the joint vector of the current state and motion of the robotic arm. The joint vector of the current state action of the robotic arm is input into the benefit evaluation network to evaluate the contribution of the robotic arm to the goal of reaching the task after performing the current action, and outputs the cumulative task benefit estimate of the current state action pair of the robotic arm. The joint vector of the current state action of the robotic arm is input into the cost evaluation network. Based on the total safety cost of the robotic arm's action, the network evaluates the level of safety risk generated after the robotic arm performs the current action and outputs the cumulative safety cost estimate of the current state action pair of the robotic arm.

6. The robotic arm safety control method based on tail risk constraints according to claim 5, characterized in that, The step of performing tail risk statistics on the successful trajectory of the screening robot arm executing the current action, dynamically updating the Lagrange multipliers, and outputting the updated Lagrange multipliers specifically includes: Successful trajectories of the robotic arm executing the current action are selected and tail risk statistics are performed in combination with the total safety cost of the robotic arm's actions to obtain the tail risk index of the robotic arm. The Lagrange multipliers are dynamically updated based on the tail risk index of the robotic arm, and the updated Lagrange multipliers are output.

7. The robotic arm safety control method based on tail risk constraints according to claim 6, characterized in that, The step of selecting the successful trajectory of the robotic arm performing the current action and combining it with the total safety cost of the robotic arm's action to perform tail risk statistics and obtain the tail risk index of the robotic arm specifically includes: Select successful trajectories of the robotic arm performing the current action. The successful trajectory represents the trajectory that completes the target arrival task under the drive of the action output by the policy network, and the distance between the end effector and the target position is less than a preset success threshold when the round ends. Extract the total safety cost of the robotic arm's current action corresponding to the successful trajectory of the robotic arm's current action, and construct the activity cost sequence of the robotic arm; Based on the activity cost sequence of the robotic arm, the mean activity cost and the high quantile activity cost are extracted. The mean activity cost is used to characterize the average safety risk level of the successful trajectory during the effective movement phase, and the high quantile activity cost is used to characterize the high-risk spike level at the tail of the activity cost sequence. The tail risk index of the robotic arm is obtained by weighted summation of the mean activity cost and the high quantile activity cost.

8. The robotic arm safety control method based on tail risk constraints according to claim 7, characterized in that, The step of dynamically updating the Lagrange multipliers based on the tail risk index of the robotic arm and outputting the updated Lagrange multipliers specifically includes: The tail risk index of the robotic arm is compared with a preset safety threshold to obtain the tail risk deviation; The Lagrange multipliers are dynamically updated based on the tail risk bias. If the tail risk index is higher than the preset safety threshold, the Lagrange multipliers are increased to enhance the safety cost constraint strength. If the tail risk index is lower than the preset safety threshold, the Lagrange multipliers are decreased to reduce the degree of inhibition on task benefit optimization. The updated Lagrange multipliers are then output.

9. The robotic arm safety control method based on tail risk constraints according to claim 8, characterized in that, The step of combining the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to optimize the strategy and output the safe control action of the robotic arm specifically includes: The task reward is determined based on the cumulative task benefit estimate of the current state action pair of the robotic arm, and the safety cost is determined based on the cumulative safety cost estimate of the current state action pair of the robotic arm and the updated Lagrange multiplier. Obtain the current motion amplitude of the robotic arm and the change in motion at adjacent time points to determine the motion smoothing term; A policy optimization objective function is constructed by combining task reward terms, safety cost terms, and action smoothness terms, and the parameters of the policy network are updated accordingly. The updated policy network is deployed at the robotic arm control terminal, and outputs the robotic arm's safety control actions based on the real-time status information input during actual operation.

10. A safety control system for a robotic arm based on tail risk constraints, characterized in that, Includes the following modules: The first module is used to collect the current state vector data of the robotic arm and perform data preprocessing and motion generation processing to obtain the displacement increment motion of the robotic arm end in three-dimensional Cartesian space. The second module is used to control the robotic arm to perform safety cost calculation based on the displacement increment of the robotic arm end in three-dimensional Cartesian space, and to evaluate the reward and cost, so as to obtain the cumulative task benefit estimate and the cumulative safety cost estimate of the current state action pair of the robotic arm. The third module is used to screen the successful trajectories of the robotic arm executing the current action, perform tail risk statistics, dynamically update the Lagrange multipliers, and output the updated Lagrange multipliers. The fourth module is used to combine the cumulative task benefit estimate of the current state action pair of the robotic arm, the cumulative safety cost estimate of the current state action pair of the robotic arm, and the updated Lagrange multiplier to optimize the strategy and output the safe control action of the robotic arm.

Images