Control method and system for the end position of a continuum robot arm and robot arm system

Abstract

The present invention discloses a control method, system and robot arm system for the end position of a continuum robot arm, including training a neural network model for the inverse kinematics of a continuum robot arm; obtaining the driving variation required for control through the pre-trained neural network, and controlling the end position of the continuum robot arm to move to the desired position according to the driving variation output by the neural network; measuring the actual position of the end of the continuum robot arm in real time through a position sensor, and calculating the end position control error; if the end position control error is greater than a set threshold, predicting the real-time Jacobian matrix variation and driving compensation through online training of the neural network, etc. The present invention achieves the purpose of updating the Jacobian matrix through the output value of the online training neural network, and calculates the driving compensation of the current continuum robot through the updated Jacobian matrix, thereby realizing high-precision control of the continuum robot.

Description

Technical Field

[0001] The present invention relates to the field of robotics, and more specifically, to a method and system for controlling the end position of a continuum robotic arm, and a robotic arm system. Background Art

[0002] The continuum robot is a new type of robot based on the bionics of biological organs such as octopus tentacles and elephant trunks. It has no rigid links and discrete joints, so it has excellent flexibility and flexibility. It has very broad application prospects in minimally invasive interventional surgery, detection and rescue in confined environments, space operations, etc., and has gradually become a new research hotspot. However, its flexibility and super-redundancy also bring great challenges to the modeling and control of the continuum robot.

[0003] Unlike rigid-body robots with predictable motion, continuum robots have infinite degrees of freedom, but they are easily deformed when subjected to external loads. Therefore, the control task of continuum robots is more challenging and difficult than that of traditional manipulators. Nowadays, precise position control of continuum robots has become one of the most important research topics of continuum robots.

[0004] At present, the main method of continuum control is closed-loop feedback control with model. Researchers have discovered new inverse kinematics models for continuum robots, which can capture bending and bending capabilities to a certain extent. This type of inverse kinematics model is the constant curvature model. And the control method that relies on this specific inverse kinematics model is model-based control. The feedback control method based on the inverse kinematics model of the continuum robot has obvious disadvantages. When the continuum robot works in a constrained environment or is disturbed by external forces, its Jacobian matrix based on the inverse kinematics model is incorrect. Therefore, in an unknown environment, we cannot effectively model the configuration and Jacobian matrix of the continuum robot.

[0005] Therefore, developing a precise control method and system for the end position of a continuum robot arm is a technical problem that needs to be solved urgently. Summary of the invention

[0006] Due to the above defects in the prior art, the present invention provides a method, system and robot arm system for controlling the end position of a continuum robot arm, which adopts an online training neural network scheme to update the Jacobian matrix through the output value of the neural network. The drive compensation amount of the current continuum robot is calculated through the updated Jacobian matrix, thereby realizing high-precision control of the continuum robot.

[0007] To achieve the above object, on the one hand, the present invention provides a method for controlling the position of a continuum robot arm end, comprising the following steps:

[0008] Step S1, training a neural network model of inverse kinematics of a continuum robot arm;

[0009] Step S2, inputting a desired position of a given control, and obtaining a driving variation required for control through the pre-trained neural network; controlling the end position of the continuum robot arm to move toward the desired position according to the driving variation output by the neural network;

[0010] It is characterized in that after the steps S1 and S2 are performed, the following steps are also included:

[0011] Step S3, measuring the actual position of the end of the continuum robot arm in real time through a position sensor, thereby calculating the end position control error;

[0012] Step S4, judging whether the end position control error is greater than a set threshold; if so, it means that the control accuracy is insufficient, and then proceeding to the next step; otherwise, the control progress meets the control requirements, and the next target position can be input for control;

[0013] Step S5, predicting the real-time Jacobian matrix change through online training of the neural network, and then calculating the drive compensation through the calculated Jacobian matrix and the position control error at the current moment, and driving the end position of the robot to change according to the drive compensation, so that the control accuracy meets the control requirements.

[0014] This method combines pre-trained neural networks, real-time end position measurement and online neural network training to obtain the drive compensation of the end position of the continuum robot arm in a timely and accurate manner. Online neural network training does not require training sets and test sets. The input and output are both derived from the measurement values ​​of the position sensor at the previous moment and the current moment, and the predicted label value is the set position control target value.

[0015] Furthermore, the driving motor rotates a corresponding number of times to drive the two driving ropes in the continuum robot arm to undergo linear displacement; as the linear displacement of the driving ropes changes, the end of the continuum robot arm moves; and the driving motor performs operations according to the acquired driving change quantity instruction.

[0016] Furthermore, in step S3, a magnetic sensor is provided at the front end of the continuum robot arm, and the actual position of the end of the continuum robot arm is measured in real time through the interaction between the magnetic sensor and the external magnetic field generator.

[0017] Furthermore, in step S5, the online training neural network is a three-layer structure of an input layer, a hidden layer and an output layer: the input layer is provided with 4 neurons, the hidden layer is provided with 12 neurons, and the output layer is provided with 4 neurons; the neural network using the BP algorithm obtains the Jacobian matrix change and the drive compensation amount.

[0018] Furthermore, the loss function L selected by the online training neural network is: Where P(k) is the actual position of the end of the continuum robot at time k, P d is the desired control position, [P(k)-P d ] T is the length vector of the position control error at time k.

[0019] Furthermore, by solving the partial derivative of the loss function L with respect to the parameters of the online training neural network, the change of the parameters at each optimization is calculated, and the parameters are optimized, thereby cyclically training the parameters, updating the new Jacobian matrix, and obtaining the required drive compensation, and finally making the position control error of the end of the continuum robot arm less than the set value.

[0020] On the other hand, the present invention provides a control system for the end position of a continuum robot arm, characterized by comprising: a pre-trained neural network module configured to pre-train and obtain an inverse kinematics model of the continuum robot arm to obtain a required driving variation of a target position of the end of the continuum robot arm;

[0021] A position sensor perception module is configured to obtain an actual position of an end of the continuum robot arm;

[0022] The online training neural network module is configured to train in real time and obtain the driving compensation between the real-time displacement of the end of the continuum robot arm and the inverse kinematics model;

[0023] The driving module is configured to drive the continuum robot arm to move according to the driving change amount obtained by the pre-trained neural network module and the driving compensation amount obtained by the online trained neural network module.

[0024] On the other hand, the present invention provides an electronic device, characterized in that it includes a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the control method of the end position of the continuum robot arm is completed.

[0025] On the last aspect, the present invention provides a robotic arm system, comprising a base, a continuum robotic arm body with one end fixedly connected to the base, a driving rope inside the continuum robotic arm, a driving motor of the driving rope and a position sensor at the end of the continuum robotic arm, characterized in that it also includes a control system for the position of the end of the above-mentioned continuum robotic arm.

[0026] Compared with the prior art, the above invention has the following advantages or beneficial effects:

[0027] The present invention combines the real-time measurement of the end position of the robot arm and the online training of the neural network to obtain the driving compensation of the end position of the continuum robot arm. It is a method based on model-free online training of the neural network and then updating the Jacobian matrix to achieve closed-loop control of the continuum robot arm. This method overcomes the control problem of the continuum robot in a restricted environment, and also further improves the accuracy of the end position control of such robots, improves the control accuracy, and provides a better control strategy for the continuum robot in the control field. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention and its features, appearance and advantages will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings. The same reference numerals indicate the same parts in all the drawings. The drawings are not drawn to scale, and emphasis is placed on illustrating the subject matter of the present invention.

[0029] Figure 1 A top view of the structure of a continuum robot arm in one embodiment of the present invention;

[0030] Figure 2 A side view of an experimental arrangement according to an embodiment of the present invention;

[0031] Figure 3 A flowchart of the steps of a method for controlling the position of a continuum robot arm end in one embodiment of the present invention;

[0032] Figure 4 The meaning of the position control error of the end of the continuum robot arm in one embodiment of the present invention is described;

[0033] Figure 5 This is a control flow chart of a method for controlling the position of a continuum robot arm end in one embodiment of the present invention;

[0034] Among them, 1. continuum robot arm body; 2. first driving rope; 3. second driving rope; 4. base; 5. magnetic sensor; 6. magnetic field generator. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0036] Example 1

[0037] See also Figure 1The basic structure of a continuum robot arm includes a continuum robot arm body 1 connected to a base 4 and a driving rope. As a preferred embodiment, it is assumed that there are two driving ropes, namely, a first driving rope 2 and a second driving rope 3. The driving principle is to bend the continuum robot arm body 1 by changing the length of the two driving ropes in the continuum robot arm body. Since there are only two driving ropes, the driving space here refers to the length vector of the driving rope passing through the continuum robot, namely, Q = [q1, q2] T , where Q is the drive length vector, q1 is the length of the first drive rope (cm), and q2 is the length of the second drive rope (cm). It can be seen that the continuum robot arm has only two degrees of freedom when moving. Here, the task space refers to the two-dimensional position of the end of the continuum robot arm, and the origin of this two-dimensional position coordinate is located at the bottom center of the continuum robot arm.

[0038] See also Figure 2 and Figure 3 Based on the above model, a method for controlling the end position of a continuum robot arm includes the following steps:

[0039] Step S1, training a neural network model of inverse kinematics of a continuum robot arm;

[0040] Step S2, inputting a desired position of a given control, and obtaining the driving variation required for control through the pre-trained neural network; and controlling the end position of the continuum robot arm to move toward the desired position according to the driving variation output by the neural network.

[0041] Input the expected position of the given control, that is, assign a value to the set expected position variable; use the pre-trained neural network as the inverse kinematics model of the continuum robot arm of this embodiment. The expected position variable is converted into the driving change required for control through the pre-trained neural network. The neural network training process is as follows: First, each position task space (P1, P2, P3...P n ) and its corresponding drive space (Q1, Q2, Q3....Q n ), taking the task space data as the training input and the drive space as the target output. Use the BP algorithm to make regression predictions on the data until the test error is lower than the set value. After the neural network parameters are trained, they can be approximated as a mapping function Q = Φ (P), where P is the desired control position and Q is the predicted value of the drive amount corresponding to the desired position.

[0042] In step S2 of this embodiment, the driving change ΔQ output by the neural network in the previous step is first obtained, and then used as a control signal input, and then the driving motor changes accordingly, and the driving motor rotates a corresponding number of circles to drive the driving rope to move linearly. Finally, as the length of the driving rope changes, the end position of the continuous robot arm moves to the desired position.

[0043] Step S3, the actual position of the end of the continuum robot arm is measured in real time by the position sensor, so as to calculate the end position control error. The position sensor can be selected as a magnetic sensor 5 arranged at the front end of the continuum robot arm, and the actual position of the end of the continuum robot arm is measured in real time through the interaction between the magnetic sensor 5 and the external magnetic field generator 6. The magnetic field generator 6 can be a NDIAuroraV3 series medical magnetic field generator, and the magnetic sensor 5 can be a magnetic positioning coil matched with the magnetic field generator. It can be understood that the position sensor can also be an in vivo positioning sensor of other types or models to realize the positioning of the end position of the continuum robot arm.

[0044] See also Figure 4 , P d is the desired control position, ΔP(k) refers to the control error value at time k, ΔP r (k) refers to the actual displacement of the end of the robot arm at time k. The actual position of the end of the continuum robot arm is measured by the position sensor, and the position information is uploaded in real time and the actual end position control error ΔP(k) is calculated, where k refers to the current timeline.

[0045] Step S4, determine whether the end position control error ΔP(k) is greater than the set threshold L; if so, it means that the control accuracy is insufficient and drive compensation control is required, then proceed to the next step; otherwise, the control progress meets the control requirements and the next target position can be input for control.

[0046] Step S5, predicting the real-time Jacobian matrix change through online training of the neural network, and then calculating the drive compensation through the calculated Jacobian matrix and the position control error at the current moment, and driving the end position of the robot to change according to the drive compensation, so that the control accuracy meets the control requirements.

[0047] See also Figure 5 In order to obtain the driving compensation, the method predicts the change of the Jacobian matrix at that time by online training of the neural network, and then obtains the driving compensation Δq by the calculated Jacobian matrix and the position control error ΔP(k) at the current moment c , Δq c The calculation formula is as follows:

[0048] ΔP(k)=P(k)-P d

[0049] ΔP r (k) = P(k+1) - P(k)

[0050] J(k)=J(k-1)+ΔJ(k)

[0051] Δq c (k)=J(k)ΔP(k)

[0052] Where J(k) is the Jacobian matrix value predicted by the neural network at time k, P d is the desired control position, ΔP(k) refers to the control error value at time k, ΔP r (k) refers to the actual displacement of the end of the robot arm at time k.

[0053] Before predicting the change of Jacobian matrix through neural network, it is necessary to initialize the Jacobian matrix to obtain the initialized Jacobian matrix J0.

[0054] The Jacobian matrix is ​​initialized as follows:

[0055] Before the experimental control, the continuum robot arm is started and a small task space movement amount P = [Δx, Δz] is given. T , thus obtaining the change in the corresponding driving space Q = [Δq1, Δq2] T .

[0056] Through the above-mentioned task space movement and its corresponding driving space change, their mapping relationship is calculated, and the mapping matrix is ​​used as the initialized Jacobian matrix J0 as follows.

[0057] Q=J0P

[0058] The online real-time training neural network training process is as follows:

[0059] The neural network structure for online training is 3 layers, the input layer has 4 neurons, and the input value is P1(k)-P d (1) , P2(k)-P d (2) , P1(k)-P1(k-1), P2(k)-P2(k-1). The hidden layer has 12 neurons. The output layer has 4 neurons, and the output values ​​are ΔJ 11 ,ΔJ 12 ,ΔJ 21 ,ΔJ 22 .

[0060] in

[0061] The activation function of the hidden layer is: g[·] = [exp(x)-exp(-x)] / [exp(x)+exp(-x)]

[0062] The activation function of the output layer is: h[·] = [exp(x)] / [exp(x) + exp(-x)]

[0063] The selected loss function L is: The desired control position P d For label.

[0064] From the neural network BP algorithm, we can get:

[0065] The hidden layer output is:

[0066] Among them: x1, x2, x3, x4, w ji (1) and b j (1) are the weights and biases of the hidden layer.

[0067] The output of the output layer is:

[0068] Where: w rj (2) and b r (2) are the weights and biases of the output layer.

[0069] Jacobian matrix change:

[0070]

[0071] The drive compensation amount is:

[0072] in,

[0073] The driving quantity at time k is:

[0074] Drive compensation amount Δq c (k) acts on the continuum robot arm. At this time, the end effector position is P(k+1), and its change is ΔP r (k). And let Online neural network training boundary conditions: L(k)<L or number of training epoches>=n, where L is the error threshold and n is the set number of training times.

[0075] Next, in order to reduce the error, it is necessary to optimize the neural network parameters, namely w ji(1) ,b j (1) ,w ri (2) ,b r (2) ,. So that the loss function L is less than the critical value, the loss function L:

[0076] Find the partial derivative of the loss function L with respect to these parameters, and then calculate the change in parameters for each optimization.

[0077] The parameter optimization formula of the online neural network is as follows:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084]

[0085]

[0086]

[0087] Among them: when r = 1,

[0088] When r = 2,

[0089] When r = 3,

[0090] When r = 4,

[0091] in

[0092] in

[0093] In this way, the above parameter optimization formula is used to cyclically train the neural network parameters and update the new Jacobian matrix to obtain the required drive compensation, which ultimately changes the position of the end of the continuum robot arm until the position control error is less than the set value L.

[0094] Example 2

[0095] A control system for the end position of a continuum robot arm, comprising: a pre-trained neural network module, configured to pre-train and obtain an inverse kinematics model of the continuum robot arm, and obtain a required driving variation of a target position of the end of the continuum robot arm;

[0096] A position sensor perception module is configured to obtain an actual position of an end of the continuum robot arm;

[0097] The online training neural network module is configured to train in real time and obtain the driving compensation between the real-time displacement of the end of the continuum robot arm and the inverse kinematics model;

[0098] The driving module is configured to drive the continuum robot arm to move according to the driving change amount obtained by the pre-trained neural network module and the driving compensation amount obtained by the online trained neural network module.

[0099] The control system for the end position of the continuum robot arm can implement the control method described in Example 1.

[0100] Example 3

[0101] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the method for controlling the end position of a continuum robot arm described in Example 1 is completed.

[0102] Example 4

[0103] See also Figure 2 A robotic arm system comprises a base 4, a continuum robotic arm body 1 with one end fixedly connected to the base, a driving rope inside the continuum robotic arm (only the first driving rope 2 is shown), a driving motor of the driving rope (not shown) and a magnetic sensor 5 at the end of the continuum robotic arm, and also includes the control system for the end position of the continuum robotic arm described in Example 1.

[0104] In summary, the present invention provides a control method, system and robot arm system for the end position of a continuum robot arm, including training a neural network model of inverse kinematics of a continuum robot arm; obtaining the driving variation required for control through the pre-trained neural network, and controlling the end position of the continuum robot arm to move toward the desired position according to the driving variation output by the neural network; measuring the actual position of the end of the continuum robot arm in real time through a position sensor, and calculating the end position control error; if the end position control error is greater than a set threshold, predicting the real-time Jacobian matrix variation and driving compensation through online training of the neural network, etc. The present invention achieves the purpose of updating the Jacobian matrix through the output value of the online training neural network, and calculates the driving compensation of the current continuum robot through the updated Jacobian matrix, thereby realizing high-precision control of the continuum robot.

[0105] Those skilled in the art should understand that those skilled in the art can implement variations by combining the prior art and the above embodiments, which will not be described in detail here. Such variations do not affect the essential content of the present invention, and will not be described in detail here.

[0106] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

[0107] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. A method for controlling the end position of a continuum robot arm, characterized in that: The following steps are involved: Step S1, training a neural network model of inverse kinematics of a continuum robot arm; Step S2, inputting a desired position of a given control, and obtaining a driving variation required for control through the pre-trained neural network model; controlling the end position of the continuum robot arm to move toward the desired position according to the driving variation output by the neural network model; Step S3, measuring the actual position of the end of the continuum robot arm in real time through a position sensor, thereby calculating the end position control error; Step S4, determining whether the end position control error is greater than a set threshold; If yes, it means the control accuracy is insufficient, and then go to the next step; otherwise, the control progress meets the control requirements, and the next target position is input for control; Step S5, predicting the real-time Jacobian matrix change through online training neural network, and then calculating the drive compensation through the calculated Jacobian matrix and the position control error at the current moment, and driving the end position of the robot to change according to the drive compensation, so that the control accuracy meets the control requirements; the online training neural network is a three-layer structure of input layer, hidden layer and output layer: the input layer is provided with 4 neurons, the hidden layer is provided with 12 neurons, and the output layer is provided with 4 neurons; the neural network using BP algorithm obtains the Jacobian matrix change and the drive compensation; the loss function selected by the online training neural network is: , where P(k) is the actual position of the end of the continuum robot at time k, P d is the desired control position, is the length vector of the position control error at time k; by solving the partial derivative of the loss function L with respect to the parameters of the online training neural network, the change of the parameters at each optimization is calculated, and the parameters are optimized, so as to cyclically train the parameters, update the new Jacobian matrix, and obtain the required drive compensation, so as to finally make the position control error of the end of the continuum robot arm less than the set value.

2. A method for controlling the end position of a continuum robot arm according to claim 1, characterized in that: The driving motor rotates a corresponding number of times to drive the two driving ropes in the continuum mechanical arm to move linearly; As the linear displacement of the driving rope changes, the end of the continuum robot arm moves; the driving motor performs an operation according to the acquired driving change quantity instruction.

3. The method for controlling the end position of a continuum robot arm according to claim 1, characterized in that: In the step S3, the position sensor is provided at the end of the continuum robot arm; the position sensor is a magnetic sensor, and the actual position of the end of the continuum robot arm is measured in real time through the interaction between the magnetic sensor and the external magnetic field generator.

4. A control system for the end position of a continuum robot arm, characterized in that: A control method for implementing any one of claims 1 to 3, comprising: A pre-trained neural network module is configured to pre-train and obtain an inverse kinematics model of a continuum robot arm to obtain a required drive change amount of a target position of an end of the continuum robot arm; A position sensor perception module is configured to obtain an actual position of an end of the continuum robot arm; The online training neural network module is configured to train in real time and obtain the driving compensation between the real-time displacement of the end of the continuum robot arm and the inverse kinematics model; The driving module is configured to drive the continuum robot arm to move according to the driving change amount obtained by the pre-trained neural network module and the driving compensation amount obtained by the online trained neural network module.

5. An electronic device, characterized in that: The invention comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the control method according to any one of claims 1 to 3 is completed.

6. A robot system, comprising a base, a continuum robot with one end fixedly connected to the base, a driving rope in the continuum robot, a driving motor of the driving rope and a position sensor at the end of the continuum robot, characterized in that: Also includes the control system as claimed in claim 4.

Images