Obstacle avoidance control method for mobile manipulator

Abstract

The application relates to a kind of mobile mechanical arm obstacle avoidance control methods, which include establishing dynamic priority allocation mechanism, adjusting the weight matrix of chassis and mechanical arm based on environmental threat and mechanical arm state in real time, balancing obstacle avoidance and joint optimization;Improve the repulsive potential field model, combine visual Transformer to predict the trajectory of dynamic obstacles, enhance short-term repulsive force;Design space-time constraint model predictive control, generate smooth trajectory through quaternion attitude tracking and dynamic constraint;Adopt double-delay deep deterministic policy gradient algorithm to globally optimize potential field strength, control weight and priority parameter.The method of the application can improve the success rate of obstacle avoidance, smooth the motion trajectory of robot, and avoid singular configuration risk.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of robots, and particularly relates to a barrier avoidance control method for a mobile manipulator. BACKGROUND

[0002] As the core equipment in industrial automation, warehouse logistics and other scenarios, the mobile manipulator needs to achieve efficient obstacle avoidance and precise operation in a dynamic and complex environment. However, the existing technology faces multiple challenges: firstly, the trajectory prediction accuracy of dynamic obstacles is insufficient, and the traditional potential field method only relies on static environment modeling, which is difficult to cope with the sudden interference of high-speed moving obstacles; secondly, the priority allocation of manipulator obstacle avoidance and chassis movement is fixed, which leads to the manipulator being easily trapped in singular configuration or the chassis obstacle avoidance response lagging behind; thirdly, although model predictive control can plan smooth trajectories, the fixed parameters are difficult to balance the multi-objective conflicts of posture tracking, energy optimization and real-time obstacle avoidance; fourthly, traditional reinforcement learning algorithms are prone to unstable control due to policy shock in dynamic environments.

[0003] Current research mainly optimizes local problems through a single technology, such as improving the potential field model to enhance repulsive force, or introducing adaptive weights to adjust the priority of manipulator joints, but has not yet achieved global parameter collaborative optimization and efficient coupling with dynamic environments. In addition, the spatiotemporal feature extraction of dynamic obstacles relies on fixed rules, and lacks precise modeling of the coupling relationship of multiple obstacle movements. Therefore, there is an urgent need for a method that integrates dynamic priority allocation, high-precision obstacle prediction, spatiotemporal constraint optimization and adaptive parameter collaboration to improve the obstacle avoidance robustness, motion smoothness and system energy efficiency of mobile manipulators in complex scenarios. SUMMARY

[0004] The application provides a dynamic obstacle avoidance control method for a mobile manipulator, which comprises the following steps:

[0005] Based on the environmental threat and the motion state of the manipulator, a dynamic priority allocation mechanism is used to adjust the dynamic weight matrix of the chassis movement and the manipulator joint optimization in real time;

[0006] The spatiotemporal features of dynamic obstacles are extracted using a visual Transformer, their motion trajectories are predicted, and an improved repulsive potential field model is used to generate obstacle avoidance guiding force;

[0007] A spatiotemporal constraint model predictive control is used to embed the objective function and the constraint condition with the dynamic weight matrix, and to generate a smooth motion trajectory;

[0008] A double-delay deep deterministic policy gradient algorithm is used to optimize the dynamic priority parameters, the repulsive potential field strength and the weight parameters of the spatiotemporal constraint model predictive control;

[0009] Joint angular velocity and chassis movement velocity instructions are generated.

[0010] Specifically, the dynamic priority allocation mechanism is specifically: constructing a dynamic weight matrix , where ζ is a forgetting factor, σ base , σ arm are chassis movement priority and robot arm joint movement priority respectively, R base is a chassis motion reachability matrix, R i (q) is a robot arm joint reachability index.

[0011] Specifically, characterized in that the priority coefficient adjustment rule is: when the obstacle distance is less than a safety threshold, it is updated as σ base ←σ base +0.7×(1-d / d0); when the joint approaches a singular configuration, it is updated as σ arm ←σ arm +0.5×(1-||Ri(q)||), where d is the current distance from the obstacle, d0 is the safety threshold.

[0012] Specifically, the improved repulsive potential field model is: the repulsive potential field function , where η is a potential field strength coefficient, λ is a velocity decay factor, v obs is an obstacle relative speed prediction vector, d is the current distance from the obstacle, and d0 is the safety threshold.

[0013] Specifically, the spatio-temporal feature extraction of the visual Transformer includes:

[0014] Divide the environment point cloud into 16×16×16 grid sub-regions, extract the geometric center coordinates, point cloud density and normal vector direction features of each sub-region, calculate the motion correlation degree between blocks through a multi-head attention mechanism, and output the predicted speed vector of the dynamic obstacle.

[0015] Specifically, the objective function of the spatio-temporal constraint model predictive control is , where Q des , Q act are expected and actual attitude quaternions respectively, τ is a joint torque, W τ is a torque weight matrix τ max is the maximum joint torque, θ i is the joint angle, θ safe is the joint safety threshold, k is the steepness coefficient, u is the control input vector, t0, t f are the starting and ending times of the prediction time domain respectively, α, β, γ are the kinematics item weight, the dynamics item weight and the ergonomics item weight respectively.

[0016] Specifically, the constraint condition of the spatio-temporal constraint model predictive control is .

[0017] Specifically, the double-delay deep deterministic policy gradient algorithm specifically comprises:

[0018] The state space is defined to include joint states, dynamic weights, reachability indexes, and obstacle velocities, the action space is a parameter set, and the reward function is designed as

[0019] wherein, Δt collision is a collision time interval, E consumption is energy consumption calculated by motor current integration, θ tool is an angle between a tool coordinate system and a target direction, sigma τ is a joint torque standard deviation, A, B, C, and D are weights of respective items.

[0020] Specifically, the double-delay deep deterministic policy gradient algorithm adopts a double Critic network to suppress overestimation, a target network realizes parameter smooth transition through soft update phi' <- 0.995 phi' + 0.005 phi, and the policy network update is delayed to enhance stability.

[0021] The application also provides a mobile manipulator system for executing the control method.

[0022] Beneficial technical effects: dynamic priority allocation is combined with an improved repulsive potential field model, so that the system quickly responds when dynamic obstacles approach, the obstacle avoidance success rate is improved, and it is especially suitable for sudden scenes of high-speed moving obstacles; the joint reachability index is used to monitor the manipulator posture in real time, the joint optimization priority is dynamically improved, and loss of control caused by singular configuration is avoided; the quaternion attitude tracking and dynamic weight constraint of the space-time constraint model predictive control are used to generate a continuous and smooth trajectory, so that the trajectory jitter amplitude is reduced and the method is suitable for high-precision grabbing and assembly tasks; the improved double-delay deep deterministic policy gradient algorithm globally optimizes joint torque and chassis speed, and the energy consumption item in the reward function is combined, so that the overall energy consumption of the system is reduced, the continuous operation time of the mobile manipulator is prolonged, and a global optimal balance between obstacle avoidance sensitivity, posture accuracy and energy consumption is achieved; the visual Transformer and the dynamic potential field model work cooperatively, so that the trajectory prediction error of dynamic obstacles is reduced and the system response delay in the whole cycle is shortened; through dynamic weight constraint and contact force threshold limitation, the system can still stably operate in complex scenes such as narrow space and multiple obstacle coupling motion. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure Figure 1 is a flowchart of a dynamic obstacle avoidance control method of a mobile manipulator according to the application;

[0024] Figure Figure 2 is a dynamic obstacle avoidance schematic diagram of a mobile manipulator according to the application. DETAILED DESCRIPTION

[0025] As Figure 1 shown, the application discloses a dynamic obstacle avoidance control method for a mobile manipulator, comprising the following steps:

[0026] Dynamic priority distribution: according to the environmental threat and the motion state of the manipulator, the priority weight of the chassis and the manipulator is adjusted in real time to realize the balance between obstacle avoidance and motion planning.

[0027] Through a depth camera and sensors, environmental perception data and joint states are obtained, and reachability indicators Ri(q) and a dynamic weight matrix W(t) are calculated. The dynamic weight matrix W(t) is used as the weight matrix of the subsequent control objective function, and the reachability indicator Ri(q) value is used as the input of the parameter optimization module.

[0028] wherein the dynamic weight matrix W(t) is:

[0029]

[0030] wherein ζ is a forgetting factor, ζ ∈ [0.05, 0.2], used to control the decay rate of historical data, which can be adjusted according to experience or optimized in real time by the parameter optimization module, σ base , σ arm are the chassis movement priority and the manipulator joint movement priority respectively, which can be calculated by fuzzy logic or adjusted by experience, σ base , σ arm ∈ (0, 1), R base is the chassis motion reachability matrix, representing the speed limit matrix of the chassis in the current position, which can be diag(v x,max , v y,max , ω max ), R i (q) is the manipulator joint reachability indicator, R i (q) = det(JJ T ) + h, h = 10 -6 , used to prevent singularity, J is the Jacobian matrix.

[0031] When it is detected that the distance d of the front obstacle is less than the safety threshold d0, the chassis movement priority is set as σ base ← σ base + 0.7 × (1 - d / d0), the closer the obstacle, the more the chassis obstacle avoidance action is prioritized over the manipulator motion; when the joint approaches a singular configuration, i.e. R i (q) tends to 0, the manipulator joint movement priority is set as σ arm ← σ arm+0.5x(1-||Ri(q)||), so as to avoid the loss of control of the mechanical arm due to singular configuration. By assigning dynamic priorities, the success rate of obstacle avoidance can be improved, the motion flexibility of the mobile arm is monitored in real time, and the risk of singular configuration is reduced.

[0032] Dynamic obstacle prediction and repulsive potential field planning: by fusing visual perception and motion prediction, a dynamic repulsive force field is generated to guide obstacle avoidance, thereby improving the accuracy of dynamic obstacle trajectory prediction.

[0033] wherein the improved potential field function U rep is:

[0034] wherein η is the potential field strength coefficient, the initial value of η can be 2.5, which can be adjusted through a parameter optimization module or experience setting; λ is the velocity attenuation factor, λ can be 0.1; v obs is the relative velocity prediction vector of the obstacle, which is output by the visual Transformer prediction, d is the distance from the current obstacle, d0 is the safety threshold. When the obstacle is stationary, i.e. v obs =0, it is converted into a traditional potential field; when the obstacle approaches at high speed, the short-term repulsive force is enhanced through the exponential term.

[0035] wherein the visual Transformer prediction is specifically:

[0036] (1) Input the environmental point cloud data. The point cloud can be divided into 16x16x16 sub-regions according to the spatial grid, and the geometric center coordinates, point cloud density, average color, and normal vector direction features of each sub-region are extracted.

[0037] (2) The encoding layer calculates the motion correlation degree of the obstacle through the multi-head attention mechanism:

[0038]

[0039] wherein Q, K, and V represent query, key, and value matrices respectively, the query matrix Q focuses on the motion correlation requirements of the current block, if a moving obstacle is detected in the block, Q will strengthen its time dimension features, the key matrix K describes the spatio-temporal features of other blocks, and the correlation weight between other blocks and the current block is calculated, the value matrix V is the actual feature value transmitted, such as the velocity change amount, d k is the feature dimension, and the softmax function is used to convert the attention score into a probability distribution, , the high probability value indicates that the block has a significant influence on the current motion prediction, for example, a fast-approaching obstacle, and the low probability value is used to filter irrelevant noise, such as a distant static object.

[0040] (3) Output the predicted obstacle motion vector.

[0041] By obstacle motion correlation calculation, the motion trend of the obstacle, such as lateral movement or accelerated approach, can be recognized, static obstacles and dynamic obstacles can be distinguished, and the motion coupling relationship between multiple obstacles can be determined.

[0042] Space-time constraint model predictive control: under the premise of meeting the set constraints, a continuous and smooth motion trajectory planning is realized through model predictive control. The objective function of the model predictive control is:

[0043] Wherein, Q des is the expected and actual attitude quaternion, τ is the joint torque, W act is the torque weight matrix, W τ =diag(1 / τ τ max 2 ), τ max is the maximum joint torque, θ i is the joint angle, θ safe is the joint safety threshold, k is the steepness coefficient, u is the control input vector, which includes joint torque and chassis speed, t0,t f are the start and end times of the prediction time domain, respectively, α, β, γ are the kinematics item weight, the dynamics item weight, and the ergonomics item weight, respectively, which are used to adjust the balance of attitude tracking accuracy and obstacle avoidance sensitivity, suppress joint torque mutation, and avoid joint exceeding the safety angle range, and can be set to 0.6, 0.3, and 0.1, respectively.

[0044] The constraint condition of the model predictive control is:

[0045]

[0046] The constraint conditions include quaternion differential constraint, maximum torque constraint, contact force constraint, and dynamic priority constraint. By embedding the dynamic weight matrix W(t) into the constraint condition, the motion bandwidth of the high-priority task is ensured, and the Euler angle singularity problem is avoided by the quaternion differential constraint.

[0047] Parameter collaborative optimization: by using the improved double-delay deep deterministic policy gradient algorithm to dynamically adjust the parameters of each module, the global performance optimization is realized. Specifically:

[0048] (1) State-action space definition

[0049] The state s includes joint state, dynamic weight, reachability index, and obstacle velocity prediction,

[0050] The action a = [ζ, η, α, β, γ] is simultaneously optimized for dynamic priority, potential field strength, and model predictive weight coefficient.

[0051] ​(2) Reward function design

[0052]

[0053] where Δt collision is the collision time interval, set to ∞ or a fixed value such as 10s when there is no collision, E consumption is the energy consumption calculated by motor current integration, θ tool is the angle between the tool coordinate system and the target direction, σ τ is the standard deviation of joint torque, used to reflect the load balance, and the weights A, B, C, D of each item can be 0.4, 0.2, 0.3, and 0.1, respectively.

[0054] (3) Network update strategy, which is specifically:

[0055] 3.1 Double Critic inhibits overestimation: where Q θi′ is the target Critic network, providing stable Q value estimation, preventing policy shock caused by overfitting of a single network, π(s') is the target Actor network, generating suboptimal action exploration, and the delay update strategy network improves stability, j is the exploration noise, which enhances environmental exploration in the early stage of training to avoid falling into local optimum.

[0056] 3.2 Delayed policy update: perform Actor update once every 2 Critic updates;

[0057] 3.3 Target network soft update is φ'←0.995φ'+0.005φ.

[0058] Through the improved double-delay deep deterministic policy gradient algorithm, the output parameter compensation amount is output, and the model predictive control weight parameters α, β, γ, dynamic priority parameter ζ, and potential field strength η are adjusted to affect the final instruction, realizing dynamic response adjustment. For example, when an emergency obstacle avoidance is detected, the algorithm increases η, resulting in an increase in the repulsive force of the potential field, and the v base generated by the model predictive control increases.

[0059] Control instruction synthesis: generate joint angular velocity and chassis moving speed instructions.

[0060] For example Figure 2As shown, a series of obstacles (triangles in the figure) are set on the planned motion trajectory (dashed line in the figure) of the mobile manipulator, the actual motion trajectory (solid line in the figure) of the mobile manipulator can be observed, and it can be known that the control method has good control effect. The application directly embeds the dynamic priority weight into the objective function of the model predictive control, realizes the decision-planning closed loop, modifies the potential field and motion constraint in real time through the obstacle speed prediction, realizes the global optimization through the improved double-delay deep deterministic policy gradient algorithm, and breaks through the local optimal limit.

[0061] The application further provides a clamping control method of a mobile manipulator, which comprises the following steps:

[0062] Contact force sensing and dynamic coordination, in particular:

[0063] Real-time acquisition of contact force F contact =[f x ,f y ,f z ,τ x ,τ y ,τ z ] T , chassis linear velocity v base and manipulator joint torque τ arm ;

[0064] Dynamic coordination calculates motion coupling factor ω c =k1*(v base ×τ arm )+k2*||ΔF||, wherein k1 is a motion coupling coefficient, the initial value can be 0.8, k2 is a force field sensitive coefficient, the initial value can be 1.2, and ΔF is a contact force gradient field, specifically , wherein R b w is a rotation matrix from the chassis to the world coordinate system, is a tensor product;

[0065] Output motion coupling factor ω c and contact force gradient modulus ||ΔF||, wherein the motion coupling factor ω c is used to limit the chassis velocity, and the contact force gradient modulus ||ΔF|| is used for gain adjustment of the sliding mode controller.

[0066] Sliding mode control:

[0067] Real-time acquisition of expected trajectory error E, , that is, joint angle and velocity error, and motion coupling factor ω c , contact force gradient modulus ||ΔF||.

[0068] The time-varying sliding mode surface is designed as: Where γ is the dynamic integral gain, when the error is large, γ can be increased to accelerate the system convergence to the sliding surface. When approaching steady state, the gain is decreased to avoid overshoot, balancing response speed and stability. By dynamically adjusting the integral action, uncertainties such as sudden load changes and external disturbances can be effectively dealt with, reducing the risk of oscillation or instability that may be caused by traditional fixed-gain integrals. The gain matrix K(t) satisfies: Where R(x) is the environmental curvature tensor, a tensor describing the geometric characteristics of the robot's motion environment. It is determined by the curvature of the chassis trajectory and the configuration of the robotic arm. In unstructured environments (such as uneven ground), the curvature tensor reflects the degree of curvature of the motion path. , where κ i Let K(t) be the principal curvature of the trajectory. By adjusting the sliding mode gain matrix K(t), the control gain is reduced to avoid overshoot when the path curvature is larger. The energy constraint matrix Q(x) is determined by the conservation of kinetic energy and is a symmetric positive definite matrix derived from the robot's kinetic energy conservation equation. It is used to constrain the range of variation of the sliding mode gain K(t) to prevent instability caused by sudden energy changes.

[0069] The control law is:

[0070]

[0071] Among them, the pseudo-inverse J of the Jacobian matrix c + J is the Jacobian matrix of the robotic arm end effector. c The pseudo-inverse is calculated as =J c T (J c J c T ) -1 The adaptive gain ρ = ρ0 + α||ΔF||, where α is the reinforcement learning optimization parameter, and the dynamic coupling constraint matrix Λi = diag(ω c *λ i M is the inertia matrix, C is the Coriolis force estimation matrix, and G is the gravity compensation term; δ is the sliding mode boundary layer thickness. A smaller value results in higher accuracy in approximating the ideal sliding surface, but may lead to chattering. A larger value results in a smoother control signal, but introduces steady-state error. Therefore, δ can be set to 0.1||ΔF||+0.05*ω c To balance accuracy and smoothness, further dynamic adjustments can be made through parameter optimization; σ represents the sliding surface.

[0072] The gradient parameter optimization of the dual-delay deep deterministic strategy is as follows:

[0073] 1. State-Motion Space: Data acquisition includes q, joint angular velocity, and chassis attitude angle θ. base Chassis speed vbase , F contact , and set the state as , action a t =[Δk1,Δk2,Δρ,Δδ], i.e. parameter increments

[0074] 2. Reward function design:

[0075]

[0076] where σ F is the force fluctuation rate, σ F =std(F contact ), i.e. the standard deviation of the contact force, θ align is the alignment angle of the tool and the target, specifically the angle between the end tool coordinate system and the target direction, energy consumption , Vibration amp is the vibration amplitude, which is the maximum vibration amplitude of the robot end in frequency domain analysis, usually taking the peak value in the 0-100Hz frequency band, and the weights A, B, C, D of each item can be 0.4, 0.2, 0.3, 0.1 respectively.

[0077] 3. Network update strategy:

[0078] (1) Double Critic inhibits overestimation: Q = min(Q θ1′ , Q θ2′ ), where Q θ1′ , Q θ2′ is the target Critic network, a neural network used to calculate the target Q value, and the minimum value is taken by the double Critic network to inhibit Q value overestimation.

[0079] (2) Delayed Actor update: φ' <- 0.995φ' + 0.005φ;

[0080] (3) Perform 1 Actor update after every 2 Critic updates.

[0081] Output joint torque τ, chassis speed command v base '= v base ·sat(ωc), limit the speed through the saturation function sat. sat can be: .

[0082] By ω c , the chassis speed is real-time constrained, and ||ΔF|| adjusts the sliding mode control gain, forming a two-way coupling. In the sliding mode control, the double delay deep deterministic strategy dynamically adjusts δ, suppresses chattering while maintaining robustness. The double delay deep deterministic strategy realizes multi-objective optimization through the composite index of the reward function, ensuring the overall optimization of the system.

[0083] The scheme deeply integrates dynamic coordination, sliding mode control and reinforcement learning, realizes a closed loop from force perception to motion control to parameter optimization, and significantly improves the stability and precision of the mobile manipulator in the clamping operation.

[0084] In several embodiments provided by the present application, it should be understood that the disclosed method can be implemented in other ways. For example, the above-described embodiments of the application are merely illustrative, for example, the division of method modules and steps is only a logical function division, and actual implementation can have another division manner. The modules and steps described as separate components or separate steps can be or can not be physically separated, the components displayed as modules can be or can not be physical modules, and can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0085] In addition, the functional modules and steps in each embodiment of the present application can be integrated in one processing module or method, or each module or step can exist physically, or two or more modules or steps can be integrated in one module or method. The integrated module can be realized in the form of hardware or in the form of hardware plus software functional module.

[0086] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the essential characteristics of the present application.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A dynamic obstacle avoidance control method for a mobile robotic arm, characterized in that, Includes the following steps: Based on environmental threats and the robotic arm's motion state, a dynamic priority allocation mechanism is used to adjust the dynamic weight matrix of chassis movement and robotic arm joints in real time. The dynamic weight matrix is ​​as follows: , where σ base σ arm These are the chassis movement priority and the robotic arm joint movement priority, respectively. base Let R be the chassis motion reachability matrix. i (q) represents the reachability index of the robotic arm joints; The spatiotemporal features of dynamic obstacles are extracted using a visual Transformer to predict their trajectory, and an improved repulsive potential field model is combined to generate obstacle avoidance guidance force. Spatiotemporal constrained model predictive control is employed, embedding the objective function and constraints with a dynamic weight matrix to generate a smooth motion trajectory. The objective function of the spatiotemporal constrained model predictive control is: Q des Q act These are the expected and actual attitude quaternions, respectively, where τ is the joint torque and W is the expected attitude quaternion. τ W is the torque weighting matrix. τ =diag(1 / τ max 2 ), τ max For the maximum joint torque, θ i For the joint angle, θ safe The joint safety threshold is given by k, the steepness coefficient is given by u, and the control input vector includes joint torque and chassis speed. t0, t f The start and end times in the prediction time domain are denoted as α, β, and γ, respectively, and the weights of the kinematic, dynamic, and ergonomic terms are denoted as α, β, and γ, respectively. The constraints of the spatiotemporal constrained model predictive control include quaternion differential constraints, maximum torque constraints, contact force constraints, and dynamic priority constraints, specifically as follows: ; The dynamic priority parameter ζ, the repulsive potential field strength coefficient η, and the kinematic weights α, β, γ of the spatiotemporal constraint model predictive control are optimized using a dual-delay deep deterministic strategy gradient algorithm. Generate joint angular velocity and chassis movement speed commands.

2. The method according to claim 1, characterized in that, The priority coefficient adjustment rule is as follows: when the obstacle distance is less than the safety threshold, it is adjusted according to σ. base ←σ base Update with +0.7×(1-d / d0); when the joint approaches a singular configuration, adjust by σ. arm ←σ arm Update with +0.5×(1-||Ri(q)||), where d is the current distance to the obstacle and d0 is the safety threshold.

3. The method according to claim 1, characterized in that, The improved repulsive potential model is as follows: repulsive potential function Where λ is the velocity decay factor, v obs d is the relative velocity prediction vector of the obstacle, d is the current distance to the obstacle, and d0 is the safety threshold.

4. The method according to claim 1, characterized in that, The spatiotemporal feature extraction of the visual Transformer specifically includes: The environmental point cloud is divided into 16×16×16 grid regions. The geometric center coordinates, point cloud density, and normal vector direction features of each sub-region are extracted. The motion correlation between blocks is calculated through a multi-head attention mechanism, and the relative velocity prediction vector of obstacles is output.

5. The method according to claim 1, characterized in that, The dual-delay deep deterministic policy gradient algorithm specifically includes: defining a state space s containing joint states, dynamic weights, reachability indices, and obstacle relative velocity prediction vectors. Action a = [ζ, η, α, β, γ], reward function designed as follows Where, Δt collision E is the collision time interval. consumption The energy consumption is calculated by integrating the motor current, θ tool σ is the angle between the tool coordinate system and the target direction. τ Let A represent the standard deviation of joint torque, and let A, B, C, and D represent the weights of each component.

6. The method according to claim 1, characterized in that, The dual-delay deep deterministic policy gradient algorithm employs a dual Critic network to suppress overestimation. The target network achieves smooth parameter transition through soft updates φ′←0.995φ′+0.005φ, and delays the policy network update to enhance stability.

7. A mobile robotic arm system, characterized in that, Used to perform the method according to any one of claims 1-6.

Images