Model training method and rock drilling mechanical arm obstacle avoidance control method and device

Abstract

The model training method comprises the following steps: constructing a simplified three-dimensional model of the rock drilling mechanical arm according to a mechanical structure of the rock drilling mechanical arm; constructing a collision avoidance action model based on a double-delay depth deterministic strategy gradient, and setting a state space and a reward function of the collision avoidance action model according to the simplified three-dimensional model; controlling the agent to move in the state space to train the collision avoidance action model, and adjusting the position of an obstacle in the training process to generate a plurality of pieces of training data at one time to perform rearview experience playback; and based on the artificial potential field, a whole-body collision avoidance potential field model of the mechanical arm is constructed through cylinder envelope, a real-time data channel of the collision avoidance action model and the whole-body collision avoidance potential field model is established, and an obstacle avoidance control model is obtained. According to the model training method and the rock drilling mechanical arm obstacle avoidance control method and device, the collision avoidance problem of the rock drilling mechanical arm is effectively solved, and the training speed of the model is higher.

Description

Technical Field

[0001] The invention relates to the technical field of rock drilling robot arm control, and in particular to a method for training a model, and a rock drilling robot arm obstacle avoidance control method and device. Background Art

[0002] Rock drilling equipment is a common equipment in tunnel construction, which can greatly improve the efficiency of tunnel construction. During the operation, the rock drilling robot arm needs to avoid obstacles in the complex environment on site to smoothly carry out the construction. Therefore, the main problem faced by the rock drilling robot arm in the process of automated construction is collision avoidance.

[0003] In order to achieve collision avoidance, a lot of research has been done in the existing technology, among which artificial potential field (APF) is an important direction. In order to solve the collision avoidance problem of the end effector, the artificial potential field technology converts the distance between the end effector of the robot arm and the obstacle into a repulsive force. However, the method of avoiding local minima through virtual obstacles requires first entering the local minimum position. After determining that the position is the local minimum, a virtual obstacle is set to make the robot arm leave the current position. This method is very inefficient for real-time motion planning. In general, there is almost no method based on artificial potential field that can provide real-time path planning to avoid the local minimum position. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a method for training a model, a method and a device for controlling obstacle avoidance of a rock drilling robot arm, which effectively solve the collision avoidance problem of the rock drilling robot arm and increase the training speed of the model.

[0005] In order to achieve the above object, the specific scheme adopted by the present invention is: a method for training a model, comprising: Constructing a simplified three-dimensional model of the rock drilling robot arm according to the mechanical structure of the rock drilling robot arm, wherein the simplified three-dimensional model includes at least two joint points; Construct a collision avoidance action model based on double-delayed deep deterministic policy gradient, and set the state space and reward function of the collision avoidance action model according to the simplified three-dimensional model, with two obstacles set in the state space; The control agent moves in the state space to train the collision avoidance action model. During the training process, the position of the obstacle is adjusted to generate multiple training data at a time for rear-view experience playback. The method of adjusting the position of the obstacle is obstacle transfer. Based on the artificial potential field, a full-body collision avoidance potential field model of the robot arm is constructed using a cylindrical envelope, and a real-time data channel between the collision avoidance action model and the full-body collision avoidance potential field model is established to obtain an obstacle avoidance control model.

[0006] As a further optimization of the above-mentioned method for training the model: the simplified three-dimensional model includes a cylindrical main body, and the action joints of the rock drilling robot arm and the end of the drill arm are both located in the main body.

[0007] As a further optimization of the above training model method: the state space of the collision avoidance action model is: s t ={x,y,x goal ,y goal ,x obs1 ,y obs1 ,x obs2 ,y obs2}∈S; Among them, (x, y) is the current position coordinate of the simplified 3D model, (x goal ,y goal ) is the target position coordinate, (x obs1 ,y obs1 ) is the coordinate of the first obstacle point, (x obs2 ,y obs2 ) are the coordinates of the second obstacle point.

[0008] As a further optimization of the above training model method: the reward function of the collision avoidance action model is: t (a t ,s t )=r step +r collision +r reach ; r step =-(|Δx|+|Δy|)k step ; Among them, a t is the action parameter of the agent, and there is a t ={Δx,Δy}∈A, (Δx,Δy) is the action amplitude of the agent, A is the action space of the agent, r step is the step size reward function, k step is the reward coefficient, r collision is the collision reward function, C collision is the reward value for collision, r reach To reach the reward function, C reach To reach the reward value, C thr To reach the determined range.

[0009] As a further optimization of the above-mentioned method for training the model: the specific method for controlling the agent to move in the state space to train the collision avoidance action model includes: The control agent moves to the target position in the action space to complete a training, and the movement process data OL of multiple trainings is recorded to form a data pool PL; Decompose the movement process data OL into multiple movement segment data OS one by one, and analyze all the movement segment data OS decomposed from one movement process data OL. If the agent collides with an obstacle after the movement segment data OS, adjust the position of the obstacle multiple times to generate a set OSP of multiple data; Recalculate the reward value of the moving fragment data set OSP after adjusting the position of the obstacle; A new training is performed based on the new moving segment data set OSP after adjusting the position of the obstacle; The training ends when the target value of the collision avoidance action model meets the preset conditions.

[0010] As a further optimization of the above-mentioned training model method: when adjusting the position of the obstacle, the adjustment is performed within a preset dynamic adjustment range.

[0011] As a further optimization of the above training model method: the whole body collision avoidance potential field model of the manipulator constructed based on the artificial potential field includes attraction parameters and repulsion parameters; The attraction parameters are set to: Among them, M att is the attraction scaling factor vector, K ratt K att The reciprocal of the corresponding element in Q goal is the target angle vector of the joint point, Q cur is the current angle vector of the joint point; The repulsion parameters include closest point repulsion sub-parameters and joint space repulsion sub-parameters; The closest point repulsion subparameters are set as: Among them, F rep is the repulsive force vector, r is the distance between the obstacle and the nearest point, R field is the influence radius of the repulsive field, is the direction vector from the obstacle to the nearest point in the simplified 3D model, K f represents the proportional coefficient of the repulsive force in the operation space, g is the weighted average, and represents the distance between the joint position and the target position; The joint space repulsion sub-parameters are set to: M rep =pinv(J nm )ΔPe K rep ; Among them, M rep is the joint repulsion (torque) vector, the pinv() function represents the pseudo-inverse of the matrix, K rep is the vector of repulsive force scale factors.

[0012] The rock drilling robot arm obstacle avoidance control method is based on the above obstacle avoidance control model, and the method comprises the following steps: Determining current joint values ​​and target joint values ​​of joint points in the simplified three-dimensional model; The target joint values, current joint values ​​and obstacle coordinates are input into the collision avoidance action model to obtain a temporary target position, and the temporary target position is input into the whole body collision avoidance potential field model based on the real-time data channel to obtain guidance parameters; and the guidance parameters are used to guide the action of the rock drilling robot arm.

[0013] As a further optimization of the above-mentioned rock drilling robot obstacle avoidance control method: the attraction parameters in the whole-body collision avoidance potential field model are used to guide the joint point movements, and during the guidance process, the attraction torque value is adjusted based on the distance between the joint point and the temporary target position, and the change rate of the attraction torque value is constrained.

[0014] The rock drilling robot arm obstacle avoidance control device comprises: A data acquisition module, used to acquire the target joint value, the current joint value and the obstacle coordinates; A data processing module, used for running the collision avoidance action model and the whole body collision avoidance potential field model; The control module is used to control the movement of the rock drilling robot arm.

[0015] Beneficial effects: The present invention strengthens the training optimization process of the collision avoidance model based on the rear-view experience playback technology, can accelerate the training of the collision avoidance model, and improve the effect of the collision avoidance model, ensuring that the collision avoidance model can effectively solve the collision avoidance problem of the rock drilling robot arm; the present invention adopts a collision avoidance model based on an artificial potential field to guide the movement of the rock drilling robot arm, effectively solving the collision avoidance problem of the rock drilling robot arm, and in the actual construction process, can effectively improve the construction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the training model method in the present invention; Figure 2 It is a flow chart of the obstacle avoidance control method of the rock drilling robot arm in the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the present invention first provides a method for training a model, including SA1 to SA4.

[0019] SA1. Construct a simplified three-dimensional model of the rock drilling robot arm according to the mechanical structure of the rock drilling robot arm, wherein the simplified three-dimensional model includes at least two joint points.

[0020] SA2. Construct a collision avoidance action model based on double-delayed deep deterministic policy gradient, and set the state space and reward function of the collision avoidance action model according to the simplified three-dimensional model. There are two obstacles in the state space. The state space of the collision avoidance action model is: s t ={x,y,x goal ,y goal ,x obs1 ,y obs1 ,x obs2 ,y obs2}∈S; Among them, (x, y) is the current position coordinate of the simplified 3D model, (x goal ,y goal ) is the target position coordinate, (x obs1 ,y obs1 ) is the coordinate of the first obstacle point, (x obs2 ,y obs2 ) are the coordinates of the second obstacle point.

[0021] On the other hand, based on the state space, the action space of the collision avoidance action model is set to a t ={Δx,Δy}∈A, where a t Use Δx as the step on the x-coordinate and Δy as the step on the y-coordinate.

[0022] The reward function of the collision avoidance action model is: r t (a t ,s t )=r step +r collision +r reach ; r step =-(|Δx|+|Δy|)k step ; Among them, a t is the action parameter of the agent, and there is a t ={Δx,Δy}∈A, (Δx,Δy) is the action amplitude of the agent, A is the action space of the agent, r step is the step size reward function, k step is the reward coefficient, r collision is the collision reward function, C collision is the reward value for collision, rreach To reach the reward function, C reach To reach the reward value, C thr To reach the determined range.

[0023] SA3. Control the agent to move in the state space to train the collision avoidance action model. During the training process, the position of the obstacle is adjusted to generate multiple training data at a time for rear-view experience playback. The method of adjusting the position of the obstacle is obstacle transfer. Without adding obstacles, the position of the obstacle is randomly transferred to enable the agent to obtain more collision-free experience, thereby continuously learning and improving the collision avoidance ability. When the agent does not achieve the goal, the rear-view experience playback also regards its actions as experience and believes that it contains valuable information. Through rear-view experience playback, the speed of model training can be improved.

[0024] The specific method of controlling the agent to move in the state space to train the collision avoidance action model includes SA31 to SA35.

[0025] SA31, control the agent to move to the target position in the action space to complete a training, and record the movement process data OL of multiple trainings to form a data pool PL.

[0026] SA32. Decompose the movement process data OL into multiple movement fragment data OS one by one, and analyze all the movement fragment data OS decomposed from one movement process data OL. If the agent collides with an obstacle after the movement fragment data OS, adjust the position of the obstacle multiple times to generate a collection OSP of multiple data.

[0027] SA33. Recalculate the reward value of the moving segment data set OSP after adjusting the position of the obstacle.

[0028] SA34: Perform a new training based on the new moving segment data set OSP after adjusting the position of the obstacle. When adjusting the position of the obstacle, the adjustment is performed within a preset dynamic adjustment range.

[0029] SA35. When the target value of the collision avoidance action model meets the preset conditions, the training ends.

[0030] Furthermore, after SA33 recalculates the reward value, the change of the reward value can be determined, that is, the difference between the reward values ​​before and after the update can be determined. If the updated reward value is better, the dynamic range of the obstacle movement can be optimized based on the movement segment data OS. Specifically, if the movement range of the agent determined by the movement segment data OS is small, the dynamic range can be reduced so that the obstacle moves randomly within a smaller dynamic range. In the subsequent movement process of the agent, the distance between it and the obstacle will also be reduced, thereby training the agent's ability to accurately avoid collisions.

[0031] When determining the dynamic range, the coverage area of ​​the dynamic range can be determined based on the direction of the moving fragment data OS, so that the agent can enter the dynamic range during the next movement, thereby enabling the agent to improve its ability to accurately avoid collisions within the dynamic range. Specifically, based on the directions of the most recent several moving fragment data OS, on the basis of the agent's current position, several basic ranges are determined, and then all basic ranges are connected to form a dynamic range for transferring obstacles. The basic range can be determined as a circle, and two adjacent basic ranges are connected by a channel, and the width of the channel is not greater than the radius of the basic range. When the center of all basic ranges can enclose a specific area, the specific area is merged with all basic ranges, and then merged with the channel to form a complete dynamic range.

[0032] SA4. Based on the artificial potential field, a full-body collision avoidance potential field model of the manipulator is constructed by using a cylindrical envelope, and a real-time data channel between the collision avoidance action model and the full-body collision avoidance potential field model is established to obtain an obstacle avoidance control model. Specifically, the simplified three-dimensional model includes a cylindrical main body, and the action joints of the rock drilling manipulator and the end of the drill arm are both located in the main body.

[0033] The full-body collision avoidance potential field model of the manipulator constructed based on the artificial potential field includes attraction parameters and repulsion parameters; The attraction parameters are set to: Among them, M att is the attraction scaling factor vector, K ratt K att The reciprocal of the corresponding element in Q goal is the target angle vector of the joint point, Q cur The current angle vector of the joint point.

[0034] The repulsion parameters include closest point repulsion sub-parameters and joint space repulsion sub-parameters; The closest point repulsion subparameters are set as: Among them, F rep is the repulsive force vector, r is the distance between the obstacle and the nearest point, R field is the influence radius of the repulsive field, is the direction vector from the obstacle to the nearest point in the simplified 3D model, K f represents the proportional coefficient of the repulsive force in the operation space, g is the weighted average, and represents the distance between the joint position and the target position; The joint space repulsion sub-parameters are set to: M rep =pinv(J nm )ΔPe K rep ; Among them, M repis the joint repulsion (torque) vector, the pinv() function represents the pseudo-inverse of the matrix, K rep is the vector of repulsive force scale factors.

[0035] like Figure 2 As shown, the present invention further provides an obstacle avoidance control method for a rock drilling robot arm. Based on the above obstacle avoidance control model, the method includes SB1 to SB3.

[0036] SB1. Determine the current joint values ​​and target joint values ​​of the joint points in the simplified three-dimensional model.

[0037] SB2. Input the target joint value, current joint value and obstacle coordinates into the collision avoidance action model to obtain a temporary target position, and input the temporary target position into the whole body collision avoidance potential field model based on the real-time data channel to obtain a guidance parameter. Use the attraction parameters in the whole body collision avoidance potential field model to guide the joint point action, adjust the attraction torque value based on the distance between the joint point and the temporary target position during the guidance process, and constrain the change rate of the attraction torque value.

[0038] Specifically, the artificial potential field method ensures that no collisions occur during planning. When calculating the repulsive force, the unreachable target problem is solved, but this method also has a local minimum problem, so it is necessary to use the previously generated action model to guide the machine and improve the global planning ability. After obtaining the target joint value through inverse calculation, the target values ​​of the first two joints and the obstacle points in the simplified joint space and the current joint values ​​of the robot are input into MoledAction to obtain the next position of the two joints, and set it as the temporary target position to guide the movement of the robot arm by generating joint attraction. Since MoledAction does not use precise obstacle avoidance and only performs directional guidance, the number of training steps used during training is relatively small and the guidance interval is also relatively small. Since the guidance speed may be too slow due to the proximity to the temporary target position, in order to speed up, the maximum attraction can be used when it is far away from the final target position: in Represents the attractive torque value of the first joint, Represents the attractive torque value of the second joint, C mattm Represents the maximum value of the attractive torque.

[0039] During the guidance process, a sudden change in the temporary target position may cause a sudden change in the direction of the attraction torque, resulting in impact. Therefore, in order to avoid this, a limit on the rate of change of the attraction torque is added: in represents the value of the attractive moment at the previous moment, Cali and C alr is a constant used to adjust the function.

[0040] Finally, based on the preset step size coefficient L ste Multiplying it by the resultant force M acting on the agent gives the joint change ΔQ.

[0041] SB3. Use guidance parameters to guide the movement of the rock drilling robot arm.

[0042] The present invention also provides an obstacle avoidance control device for a rock drilling robot arm, comprising a data acquisition module, a data processing module and a control module.

[0043] A data acquisition module, used to acquire the target joint value, the current joint value and the obstacle coordinates;

[0044] A data processing module, used for running the collision avoidance action model and the whole body collision avoidance potential field model;

[0045] The control module is used to control the movement of the rock drilling robot arm.

[0046] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a model, characterized in that: include: Constructing a simplified three-dimensional model of the rock drilling robot arm according to the mechanical structure of the rock drilling robot arm, wherein the simplified three-dimensional model includes at least two joint points; Construct a collision avoidance action model based on double-delayed deep deterministic policy gradient, and set the state space and reward function of the collision avoidance action model according to the simplified three-dimensional model, with two obstacles set in the state space; The control agent moves in the state space to train the collision avoidance action model. During the training process, the position of the obstacle is adjusted to generate multiple training data at a time for rear-view experience playback. The method of adjusting the position of the obstacle is obstacle transfer. Based on the artificial potential field, a full-body collision avoidance potential field model of the robot arm is constructed using a cylindrical envelope, and a real-time data channel between the collision avoidance action model and the full-body collision avoidance potential field model is established to obtain an obstacle avoidance control model.

2. The method for training a model as claimed in claim 1, characterized in that: The simplified three-dimensional model includes a cylindrical main body, and the action joints of the rock drilling robot arm and the end of the drill arm are located in the main body.

3. The method for training a model as claimed in claim 1, characterized in that: The state space of the collision avoidance action model is: s t ={x,y,x goal ,and goal ,x obs1 ,and obs1 ,x obs2 ,and obs2 }∈S; Among them, (x, y) is the current position coordinate of the simplified 3D model, (x goal ,y goal ) is the target position coordinate, (x obs1 ,y obs1 ) is the coordinate of the first obstacle point, (x obs2 ,y obs2 ) are the coordinates of the second obstacle point.

4. The method for training a model as claimed in claim 1, characterized in that: The reward function of the collision avoidance action model is: r t (a t ,s t )=r step +r collision +r reach ; r step =-(|Δx|+|Δy|)k step ; Among them, a t is the action parameter of the agent, and has a t ={Δx,Δy}∈A, (Δx,Δy) is the action amplitude of the agent, A is the action space of the agent, r step is the step size reward function, k step is the reward coefficient, r collision is the collision reward function, C collision is the reward value for collision, r reach To reach the reward function, C reach To reach the reward value, C thr To reach the determined range.

5. The method for training a model as claimed in claim 1, characterized in that: The specific methods for controlling the agent to move in the state space to train the collision avoidance action model include: The control agent moves to the target position in the action space to complete a training, and the movement process data OL of multiple trainings is recorded to form a data pool PL; Decompose the movement process data OL into multiple movement segment data OS one by one, and analyze all the movement segment data OS decomposed from one movement process data OL. If the agent collides with an obstacle after the movement segment data OS, adjust the position of the obstacle multiple times to generate a set OSP of multiple data; Recalculate the reward value of the moving fragment data set OSP after adjusting the position of the obstacle; A new training is performed based on the new moving segment data set OSP after adjusting the position of the obstacle; The training ends when the target value of the collision avoidance action model meets the preset conditions.

6. The method for training a model as claimed in claim 5, characterized in that: When adjusting the position of an obstacle, make adjustments within the preset dynamic adjustment range.

7. The method for training a model as claimed in claim 1, characterized in that: The full-body collision avoidance potential field model of the manipulator constructed based on the artificial potential field includes attraction parameters and repulsion parameters; The attraction parameters are set to: Among them, M att is the attraction scaling factor vector, K ratt K att The reciprocal of the corresponding element in Q goal is the target angle vector of the joint point, Q cur is the current angle vector of the joint point; The repulsion parameters include closest point repulsion sub-parameters and joint space repulsion sub-parameters; The closest point repulsion subparameters are set as: Among them, F rep is the repulsive force vector, r is the distance between the obstacle and the nearest point, R field is the influence radius of the repulsive field, is the direction vector from the obstacle to the nearest point in the simplified 3D model, K f represents the proportional coefficient of the repulsive force in the operation space, g is the weighted average, and represents the distance between the joint position and the target position; The joint space repulsion sub-parameters are set to: M rep =pinv(J nm )ΔPe K rep ; Among them, M rep is the joint repulsion (torque) vector, the pinv() function represents the pseudo-inverse of the matrix, K rep is the vector of repulsive force scale factors.

8. A rock drilling robot arm obstacle avoidance control method, characterized in that: Based on the obstacle avoidance control model as described in any one of claims 1 to 7, the method comprises the following steps: Determining current joint values ​​and target joint values ​​of joint points in the simplified three-dimensional model; Inputting the target joint value, the current joint value and the obstacle coordinates into the collision avoidance action model to obtain a temporary target position, and inputting the temporary target position into the whole body collision avoidance potential field model based on the real-time data channel to obtain a guidance parameter; Use guidance parameters to guide the movement of the rock drilling robot arm.

9. The rock drilling robot arm obstacle avoidance control method according to claim 8, characterized in that: The attraction parameters in the whole-body collision avoidance potential field model are used to guide the joint point movements. During the guidance process, the attraction torque value is adjusted based on the distance between the joint point and the temporary target position, and the change rate of the attraction torque value is constrained.

10. The rock drilling robot arm obstacle avoidance control device is characterized in that: include: A data acquisition module, used to acquire the target joint value, the current joint value and the obstacle coordinates; A data processing module, used for running the collision avoidance action model and the whole body collision avoidance potential field model; The control module is used to control the movement of the rock drilling robot arm.

Images