Methods for training models, obstacle avoidance control methods and devices for rock drilling robotic arms

Abstract

The method for training the model includes: constructing a simplified 3D model of the rock drilling robot based on its mechanical structure; constructing a collision avoidance action model based on a dual-delay depth deterministic policy gradient, and setting the state space and reward function of the collision avoidance action model according to the simplified 3D model; controlling an agent to move in the state space to train the collision avoidance action model, adjusting the position of obstacles during training to generate multiple training data points for backsight experience playback; constructing a full-body collision avoidance potential field model of the robot based on an artificial potential field using a cylindrical envelope, and establishing a real-time data channel between the collision avoidance action model and the full-body collision avoidance potential field model to obtain the obstacle avoidance control model. This invention provides a method for training a model, a method and device for obstacle avoidance control of a rock drilling robot, effectively solving the collision avoidance problem of the rock drilling robot, and enabling faster model training.

Description

Technical Field

[0001] This invention relates to the field of rock drilling robot control technology, specifically a method for training models, a method and device for obstacle avoidance control of rock drilling robots. Background Technology

[0002] Rock drilling equipment is commonly used in tunnel construction, significantly improving efficiency. However, rock drilling robotic arms need to avoid obstacles in complex environments to operate smoothly. Therefore, the most significant challenge for automated rock drilling robotic arms is collision avoidance.

[0003] To achieve collision avoidance, extensive research has been conducted on existing technologies, with Artificial Potential Fields (APFs) being a significant direction. To address the collision avoidance problem of end effectors, APF techniques convert the distance between the robotic arm's end effector and obstacles into a repulsive force. However, methods that avoid local minima using virtual obstacles require first entering the local minimum location, and then, after confirming that location as a local minimum, setting up virtual obstacles to move the robotic arm away from its current position. This method is inefficient for real-time motion planning. Generally, almost no APF-based method can provide real-time path planning to avoid local minima. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for training a model, a method and device for controlling obstacle avoidance of a rock drilling robot arm, which effectively solves the collision avoidance problem of the rock drilling robot arm and allows for faster model training.

[0005] To achieve the above objectives, the specific solution adopted by the present invention is as follows: a method for training a model, comprising:

[0006] A simplified 3D model of the rock drilling robot arm is constructed based on its mechanical structure. The simplified 3D model includes at least two joints.

[0007] A collision avoidance action model based on a dual-delay deep deterministic policy gradient is constructed, and the state space and reward function of the collision avoidance action model are set according to a simplified 3D model. Two obstacles are set in the state space.

[0008] The control agent moves in the state space to train the collision avoidance action model. During the training process, the position of the obstacle is adjusted to generate multiple training data at once for back-look experience playback. The method of adjusting the position of the obstacle is obstacle transfer.

[0009] Based on the artificial potential field, a full-body collision avoidance potential field model of the robotic arm is constructed using a cylindrical envelope, and a real-time data channel is established between the collision avoidance action model and the full-body collision avoidance potential field model to obtain the obstacle avoidance control model.

[0010] As a further optimization of the above training model method, the simplified 3D model includes a cylindrical body, and the motion joints of the rock drilling robot arm and the end of the drill arm are located in the body.

[0011] As a further optimization of the above training model method: the state space of the collision avoidance action model is: s t ={x,y,x goal ,y goal ,x obs1 ,y obs1 ,x obs2 ,y obs2}∈S;

[0012] Where (x,y) represents the current position coordinates of the simplified 3D model, (x... goal ,y goal (x) represents the target location coordinates. obs1 ,y obs1 (x) represents the coordinates of the first obstacle point. obs2 ,y obs2 () represents the coordinates of the second obstacle point.

[0013] As a further optimization of the above training model method: the reward function of the collision avoidance action model is: r t (a t ,s t ) = r step +r collision +r reach ;

[0014] r step =-(|Δx|+|Δy|)k step ;

[0015]

[0016] Among them, a t The action parameters for the agent are a, and there is a t = {Δx,Δy}∈A, (Δx,Δy) is the action range of the agent, A is the action space of the agent, r step For the step size reward function, k step r is the reward coefficient. collision For the collision reward function, C collision r is the reward value for a collision. reach To reach the reward function, C reach To reach the reward value, C thrTo reach the range of the judgment.

[0017] As a further optimization of the above training model method, specific methods for training the collision avoidance action model by controlling the agent to move in the state space include:

[0018] The control agent moves to the target position within the action space to complete one training session, and the movement process data OL of multiple training sessions is recorded to form a data pool PL;

[0019] The movement process data OL is decomposed into multiple movement segment data OS one by one, and all movement segment data OS decomposed from a movement process data OL are analyzed. If the agent collides with an obstacle after the movement segment data OS, the position of the obstacle is adjusted multiple times to generate a set of multiple data OSP.

[0020] The reward value of the OSP for the movement segment data set after recalculating the position of the obstacles;

[0021] A new training iteration is performed based on the OSP dataset of new moving segments after adjusting the positions of the obstacles;

[0022] Training ends when the target value of the collision avoidance action model meets the preset conditions.

[0023] As a further optimization of the above training model method: when adjusting the position of the obstacle, the adjustment is made within a preset dynamic adjustment range.

[0024] As a further optimization of the above training model method: the whole-body collision avoidance potential field model of the robotic arm based on the artificial potential field includes attractive force parameters and repulsive force parameters;

[0025] The attraction parameter is set as follows: Among them, M att K is the attractiveness scaling factor vector. ratt For K att The reciprocal of the corresponding element, Q goal Q is the target angle vector of the joint. cur The current angle vector of the joint;

[0026] The repulsion parameters include the nearest point repulsion sub-parameter and the joint space repulsion sub-parameter;

[0027] The nearest point repulsion parameter is set as follows:

[0028]

[0029] Among them, F rep R is the repulsive force vector, where r is the distance between the obstacle and the nearest point. field The radius of influence of the repulsive field is... K is the direction vector from the obstacle to the nearest point in the simplified 3D model. f The proportionality coefficient representing the repulsive force in the operating space, g is the weighted average value, and represents the distance between the joint position and the target position;

[0030] The joint space repulsion parameter is set to: M rep =pinv(J nm )ΔPe K rep ;

[0031] Among them, M rep The pinv() function represents the pseudo-reciprocal of the matrix, where K is the joint repulsive force (torque) vector. rep It is a vector representing the repulsive force scale factor.

[0032] The obstacle avoidance control method for a rock drilling robotic arm, based on the aforementioned obstacle avoidance control model, includes the following steps:

[0033] Determine the current joint value and target joint value of the joint points in the simplified 3D model;

[0034] The target joint value, current joint value, and obstacle coordinates are input into the collision avoidance action model to obtain a temporary target position. Based on the real-time data channel, the temporary target position is input into the whole-body collision avoidance potential field model to obtain guidance parameters. The guidance parameters are then used to guide the rock drilling robot arm's movements.

[0035] As a further optimization of the obstacle avoidance control method of the rock drilling robot arm: the attraction force parameter in the whole-body collision avoidance potential field model is used to guide the joint movement. During the guidance process, the attraction force torque value is adjusted based on the distance between the joint and the temporary target position, and the rate of change of the attraction force torque value is constrained.

[0036] The obstacle avoidance control device for the rock drilling robotic arm includes:

[0037] The data acquisition module is used to acquire the target joint value, the current joint value, and the obstacle coordinates;

[0038] The data processing module is used to run the collision avoidance action model and the whole-body collision avoidance potential field model;

[0039] The control module is used to control the movements of the rock drilling robotic arm.

[0040] Beneficial effects: This invention enhances the training and optimization process of the collision avoidance model based on the back-view experience playback technology, which can accelerate the training of the collision avoidance model and improve its performance, ensuring that the collision avoidance model can effectively solve the collision avoidance problem of the rock drilling robot arm; This invention uses a collision avoidance model based on an artificial potential field to guide the rock drilling robot arm's movements, effectively solving the collision avoidance problem of the rock drilling robot arm, and can effectively improve construction efficiency in actual construction. Attached Figure Description

[0041] Figure 1 This is a flowchart of the model training method in this invention;

[0042] Figure 2 This is a flowchart of the obstacle avoidance control method for the rock drilling robotic arm in this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, the present invention first provides a method for training a model, including SA1 to SA4.

[0045] SA1. Construct a simplified 3D model of the rock drilling robot arm based on its mechanical structure. The simplified 3D model includes at least two joints.

[0046] SA2. Construct a collision avoidance action model based on a dual-delay, deep deterministic policy gradient, and set the state space and reward function of the collision avoidance action model according to a simplified 3D model. The state space includes two obstacles. The state space of the collision avoidance action model is as follows:

[0047] s t ={x,y,x goal ,y goal ,x obs1 ,y obs1 ,x obs2 ,y obs2}∈S;

[0048] Where (x,y) represents the current position coordinates of the simplified 3D model, (x... goal ,y goal (x) represents the target location coordinates. obs1 ,y obs1 (x) represents the coordinates of the first obstacle point. obs2 ,y obs2 () represents the coordinates of the second obstacle point.

[0049] On the other hand, based on the state space, the action space of the collision avoidance action model is set as a. t = {Δx, Δy}∈A, where a t Use Δx as the step size in the x-coordinate and Δy as the step size in the y-coordinate.

[0050] The reward function for the collision avoidance model is:

[0051] r t (a t ,s t ) = r step +r collision +r reach ;

[0052] r step =-(|Δx|+|Δy|)k step ;

[0053]

[0054] Among them, a t The action parameters for the agent are a, and there is a t = {Δx,Δy}∈A, (Δx,Δy) is the action range of the agent, A is the action space of the agent, r step For the step size reward function, k step r is the reward coefficient. collision For the collision reward function, C collision r is the reward value for a collision. reach To reach the reward function, C reach To reach the reward value, C thr To reach the range of the judgment.

[0055] SA3 controls the agent's movement in the state space to train the collision avoidance model. During training, the positions of obstacles are adjusted, generating multiple training data points for replay. The method for adjusting obstacle positions is obstacle transfer. Without adding obstacles, the agent gains more collision-free experiences by randomly transferring obstacle positions, thereby continuously learning and improving its collision avoidance capabilities. Replaying the replay shows that even when the agent fails to achieve the goal, its actions are considered as experience and contain valuable information. This replay speeds up model training.

[0056] Specific methods for training collision avoidance action models by controlling the agent's movement in the state space include SA31 to SA35.

[0057] SA31: Control the agent to move to the target position within the action space to complete one training session, and record the movement process data OL of multiple training sessions to form a data pool PL.

[0058] SA32. Decompose the movement process data OL into multiple movement segment data OS one by one, and analyze all the movement segment data OS decomposed from a movement process data OL. If the agent collides with an obstacle after the movement segment data OS, adjust the position of the obstacle multiple times to generate a set of multiple data OSP.

[0059] SA33, recalculate the reward value of the OSP for the moving segment data set after adjusting the position of obstacles.

[0060] SA34. A new training iteration is performed using the OSP dataset with the new movement segments after adjusting the obstacle positions. When adjusting the obstacle positions, adjustments are made within a preset dynamic adjustment range.

[0061] SA35. Training ends when the target value of the collision avoidance action model meets the preset conditions.

[0062] Furthermore, after SA33 recalculates the reward value, the change in the reward value can be determined, i.e., the difference between the reward value before and after the update can be identified. If the updated reward value is better, the dynamic range of obstacle movement can be optimized based on the moving fragment data OS. Specifically, if the agent's movement range determined by the moving fragment data OS is small, the dynamic range can be reduced, causing the obstacle to move randomly within a smaller dynamic range. During the agent's subsequent movements, the distance between the agent and the obstacle will also be reduced, thereby training the agent's accurate collision avoidance ability.

[0063] When determining the dynamic range, the coverage area of ​​the dynamic range can be determined based on the direction of the moving segment data (OS). This allows the agent to enter the dynamic range during its next movement, thereby improving its ability to accurately avoid collisions within the dynamic range. Specifically, based on the directions of the most recent moving segment data (OS) and the agent's current position, several basic ranges are determined. All these basic ranges are then connected to form the dynamic range used for obstacle transfer. The basic ranges can be defined as circles, with adjacent basic ranges connected by a channel whose width is no greater than the radius of the basic range. When the centers of all the basic ranges enclose a specific area, this specific area is merged with all the basic ranges, and then with the channel, forming a complete dynamic range.

[0064] SA4. Based on the artificial potential field, a full-body collision avoidance potential field model of the robotic arm is constructed using a cylindrical envelope. A real-time data channel is established between the collision avoidance action model and the full-body collision avoidance potential field model to obtain the obstacle avoidance control model. Specifically, the simplified 3D model includes a cylindrical main body, with the rock drilling robotic arm's motion joints and drill arm end caps located within the main body.

[0065] The whole-body collision avoidance potential field model of the robotic arm based on the artificial potential field includes attractive force parameters and repulsive force parameters;

[0066] The attraction parameter is set as follows: Among them, M att K is the attractiveness scaling factor vector. ratt For K att The reciprocal of the corresponding element, Q goal Q is the target angle vector of the joint. cur This is the current angle vector of the joint.

[0067] The repulsion parameters include the nearest point repulsion sub-parameter and the joint space repulsion sub-parameter;

[0068] The nearest point repulsion parameter is set as follows:

[0069]

[0070] Among them, F rep R is the repulsive force vector, where r is the distance between the obstacle and the nearest point. field The radius of influence of the repulsive field is... K is the direction vector from the obstacle to the nearest point in the simplified 3D model. f The proportionality coefficient representing the repulsive force in the operating space, g is the weighted average value, and represents the distance between the joint position and the target position;

[0071] The joint space repulsion parameter is set to: M rep =pinv(J nm )ΔPe K rep ;

[0072] Among them, M rep The pinv() function represents the pseudo-reciprocal of the matrix, where K is the joint repulsive force (torque) vector. rep It is a vector representing the repulsive force scale factor.

[0073] like Figure 2 As shown, the present invention further provides an obstacle avoidance control method for a rock drilling robot arm, which, based on the above-mentioned obstacle avoidance control model, includes SB1 to SB3.

[0074] SB1. Determine the current joint value and target joint value of the joint points in the simplified 3D model.

[0075] SB2. Input the target joint value, current joint value, and obstacle coordinates into the collision avoidance action model to obtain a temporary target position. Then, based on the real-time data channel, input the temporary target position into the whole-body collision avoidance potential field model to obtain guidance parameters. Use the attraction force parameters in the whole-body collision avoidance potential field model to guide the joint movements. During guidance, adjust the attraction force torque value based on the distance between the joint and the temporary target position, and constrain the rate of change of the attraction force torque value.

[0076] Specifically, the artificial potential field method ensures that collisions are avoided during planning. While the goal unreachability problem is addressed when calculating repulsive forces, this method also suffers from local minima. Therefore, the previously generated action model is needed to guide the robot and improve global planning capabilities. After obtaining the target joint values ​​through inverse kinematics, the target values ​​of the first two joints, the simplified joint space obstacle points, and the current joint values ​​of the robot arm are input into MoledAction to obtain the next positions of the two joints. These positions are then set as temporary target positions, and the robot arm's movement is guided by generating joint attraction forces. Since MoledAction does not use precise obstacle avoidance but only provides directional guidance, fewer training steps and shorter guidance intervals are used during training. Because the guidance speed may be too slow when close to the temporary target position, the maximum attraction force can be used when the final target position is far away to accelerate the process.

[0077]

[0078] in This represents the gravitational torque value of the first joint. C represents the attraction torque value of the second joint. mattm This represents the maximum value of the attractive torque.

[0079] During the guidance process, a sudden change in the temporary target location may cause a sudden change in the direction of the attractive torque, resulting in an impact. Therefore, to avoid this situation, a limit on the rate of change of the attractive torque is added:

[0080]

[0081] in Represents the value of the gravitational torque at the previous moment, C ali and C alr It is a constant used to adjust the function.

[0082] Finally, based on the preset step size coefficient L ste Multiplying this by the resultant force M acting on the agent yields the joint change ΔQ.

[0083] SB3. Use guidance parameters to guide the rock drilling robot arm's movements.

[0084] The present invention also provides an obstacle avoidance control device for a rock drilling robotic arm, comprising a data acquisition module, a data processing module, and a control module.

[0085] The data acquisition module is used to acquire the target joint value, the current joint value, and the obstacle coordinates;

[0086] The data processing module is used to run the collision avoidance action model and the whole-body collision avoidance potential field model;

[0087] The control module is used to control the movements of the rock drilling robotic arm.

[0088] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a model, characterized in that, include: A simplified 3D model of the rock drilling robot arm is constructed based on its mechanical structure. The simplified 3D model includes at least two joints. A collision avoidance action model based on a dual-delay deep deterministic policy gradient is constructed, and the state space and reward function of the collision avoidance action model are set according to a simplified 3D model. Two obstacles are set in the state space. The control agent moves in the state space to train the collision avoidance action model. During the training process, the position of the obstacle is adjusted to generate multiple training data at once for back-look experience playback. The method of adjusting the position of the obstacle is obstacle transfer. Based on the artificial potential field, a full-body collision avoidance potential field model of the robotic arm is constructed using a cylindrical envelope, and a real-time data channel is established between the collision avoidance action model and the full-body collision avoidance potential field model to obtain the obstacle avoidance control model.

2. The method for training a model as described in claim 1, characterized in that, The simplified 3D model includes a cylindrical main body, with the rock drilling robot arm's motion joints and the drill arm's end located within the main body.

3. The method for training a model as described in claim 1, characterized in that, The state space of the collision avoidance action model is: ； in, To simplify the current position coordinates of the 3D model, The target location coordinates, The coordinates of the first obstacle point are... These are the coordinates of the second obstacle point.

4. The method for training a model as described in claim 1, characterized in that, The reward function for the collision avoidance model is: in, The action parameters for the agent, and have , Let A be the range of actions of the agent, and let A be the action space of the agent. For step size reward function, As the reward coefficient, For the collision reward function, The reward value for a collision. To reach the reward function, To reach the reward value, To reach the range of the judgment.

5. The method for training a model as described in claim 1, characterized in that, Specific methods for training a collision avoidance action model by controlling the agent's movement in the state space include: The control agent moves to the target position within the action space to complete one training session, and the movement process data OL of multiple training sessions is recorded to form a data pool PL; The movement process data OL is decomposed into multiple movement segment data OS one by one, and all movement segment data OS decomposed from a movement process data OL are analyzed. If the agent collides with an obstacle after the movement segment data OS, the position of the obstacle is adjusted multiple times to generate a set of multiple data OSP. The reward value of the OSP for the movement segment data set after recalculating the position of the obstacles; A new training iteration is performed based on the OSP dataset of new moving segments after adjusting the positions of the obstacles; Training ends when the target value of the collision avoidance action model meets the preset conditions.

6. The method for training a model as described in claim 5, characterized in that, When adjusting the position of an obstacle, adjustments are made within a preset dynamic adjustment range.

7. A rock drilling robotic arm obstacle avoidance control method, characterized in that, Based on the obstacle avoidance control model as described in any one of claims 1-6, the method includes the following steps: Determine the current joint value and target joint value of the joint points in the simplified 3D model; The target joint value, current joint value, and obstacle coordinates are input into the collision avoidance action model to obtain a temporary target position. Based on the real-time data channel, the temporary target position is input into the whole-body collision avoidance potential field model to obtain guidance parameters. Guide parameters are used to guide the rock drilling robot arm's movements.

8. The obstacle avoidance control method for a rock drilling robotic arm as described in claim 7, characterized in that, The attraction parameters in the whole-body collision avoidance potential field model are used to guide the joint movements. During the guidance process, the attraction torque value is adjusted based on the distance between the joint and the temporary target position, and the rate of change of the attraction torque value is constrained.

9. A rock drilling robot arm obstacle avoidance control device, used to implement the rock drilling robot arm obstacle avoidance control method as described in any one of claims 7-8, characterized in that, The device includes: The data acquisition module is used to acquire the target joint value, the current joint value, and the obstacle coordinates; The data processing module is used to run the collision avoidance action model and the whole-body collision avoidance potential field model; The control module is used to control the movements of the rock drilling robotic arm.

Images