Flexible cable driven continuum joint shape estimation method
Through the multi-channel common attention recurrent neural network model, shape estimation is performed using the force information and angle information of the continuum joints, which solves the problem of low estimation accuracy in the prior art and improves the control accuracy and adaptability of the surgical robot.
Patent Information
- Application Number
- CN202410970773.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-19
AI Technical Summary
The prior art is difficult to accurately estimate the shape of the soft cord-driven continuum joint, especially when colliding with human tissue during surgery, resulting in a decrease in the accuracy of model estimation and affecting the control stability of the surgical robot.
Using a data-driven method based on multi-channel common attention recurrent neural network, the deep neural network model is constructed, and the shape estimation is used to use the force information, torque information and theoretical angle information of the continuum joint to perform shape estimation, and the prediction and regression task of X and Y coordinates is independently performed. , improve estimation accuracy.
It achieves higher estimation performance, can accurately estimate the shape of continuum joints, and improves the adaptability and control accuracy of surgical robots in complex environments.
Smart Images

Figure CN118940429A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of robot kinematics, and in particular relates to a method for estimating the joint shape of a flexible cable-driven continuum. Background Art
[0002] Inspired by biological structures such as elephant trunks and octopus tentacles, various continuum robots have been designed and applied in the field of minimally invasive surgery in recent years. Unlike traditional rigid-link robots, continuum robots have superior flexibility and adaptability, and can adjust the shape of their joints to reach the surgical target through narrow and tortuous paths and provide sufficient force at the distal end for surgical operations. During surgery, it is very important to accurately estimate the shape of continuum joints. However, due to the complex deformation of continuum joints, it is difficult to establish an accurate kinematic model to estimate the shape of continuum joints. In addition, the collision of continuum joints with surrounding human tissues further reduces the accuracy of model estimation.
[0003] At present, most studies on shape estimation of continuum joints use traditional kinematic methods for modeling. However, the modeling process based on kinematic methods is very complicated and requires a tedious parameter identification process; and due to the complex nonlinear characteristics of the cable-driven continuum joint motion, it is difficult to obtain an accurate shape estimation model. The difference is that the data-driven method learns complex mapping relationships from the input data, which can obtain a more accurate model and improve modeling efficiency. Therefore, the present invention proposes a data-driven method based on a multi-channel joint attention recurrent neural network to estimate the shape of continuum joints. Summary of the invention
[0004] Due to the complex deformation characteristics of the continuum joints and the unknown shape changes caused by the inevitable collision with human tissue during surgery, it is difficult to accurately and stably control the end effector of the continuum surgical robot. Therefore, it is necessary to accurately estimate the shape changes of the continuum joints to alleviate this problem. In view of the fact that most of the existing studies use traditional kinematic methods to estimate the shape of continuum joints, this paper proposes a data-driven continuum joint shape estimation method.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0006] A method for estimating the shape of a cable-driven continuum joint is firstly established to establish a continuum joint motion dataset of a cable-driven surgical robot, and divide it into a training set, a validation set and a test set according to the load weight at the end of the continuum joint. Then, the sample feature sequence is constructed using three force information, three torque information and a theoretical angle information input through the dSPACE real-time control system in the three-dimensional space of the end of the continuum joint link in the XYZ direction. At the same time, the coordinate position information of the marker point on the continuum joint is used as the label sequence of the sample. Further, these feature sequences are overlapped and divided according to the time length T = 50 time points (0.5s) to construct training samples, validation samples and test samples. In addition, the method independently performs two prediction regression tasks according to the X coordinate position and Y coordinate position of the marker point. In the regression task of predicting P (X or Y coordinate), the label of each sample is selected as the X (or Y) coordinate position of the marker point corresponding to the last time point among T time points. The method comprises the following steps:
[0007] Step 1: The training set samples can be expressed as Where n represents the number of samples in the training set. The corresponding sample labels are expressed as Among them, y 1 ,y 2 ,…,y n represents the X-coordinate value or Y-coordinate value of the marker point, depending on the prediction object of the current regression task. Then, each feature in the sample is preprocessed by normalizing the maximum and minimum values, and the processed training set samples are represented as
[0008] Step 2: Reshape the sample features processed in step 1 and split the features of each sample into three streams of length T. The three flows can be regarded as two-dimensional matrices with dimensions of T×3, T×3, and T×1, respectively, corresponding to three force information, three torque information, and one theoretical angle information. The sth flow can be expressed as
[0009] Step 3: Construct a multi-channel joint attention recurrent neural network model to estimate the shape of the continuum joints. Then, the training sample set after shape reshaping in step 2 is sent to the constructed deep neural network along three different channels, and the corresponding sample labels Υ are used to train the deep neural network f(·) to obtain the continuum joint shape estimator
[0010] Step 4: For any sample x in the test set, firstly go through the maximum and minimum value normalization in step 1 and the shape reshaping in step 2, and the three reshaped flows are and The three streams are then fed into the trained continuum articulated shape estimator along three different channels In this way, each sample in the test set is estimated to obtain P (X or Y coordinate) of the marked point on the continuum joint of the corresponding test sample.
[0011] Furthermore, the deep neural network model in step 3 is a multi-channel joint attention recurrent neural network, and the network structure is:
[0012] The first to third layers are three input layers, which receive three streams of sample features after sequence reshaping, with feature dimensions of T×3, T×3, and T×1 respectively. The fourth to sixth layers are three parallel one-dimensional convolutional layers, with the number of filters in each one-dimensional convolutional layer set to F and the convolution kernel size set to K. s , the step size is 1, the padding method is same padding, and the activation function is tanh function. The seventh layer is the joint attention layer, the input of which is the output of the first and second one-dimensional convolutional layers, and the adjustable mapping dimension is set to m. The eighth layer is the connection layer, which connects the output of the joint attention layer and the output of the third one-dimensional convolutional layer. The ninth layer is the shape reshaping layer, and the reshaping scale is set to d. The tenth to twelfth layers are recurrent neural network layers, with 128, 64, and 32 nodes respectively. The thirteenth and fourteenth layers are fully connected layers, with 8 and 1 nodes respectively. In addition, the network uses the adaptive moment estimation (Adam) optimizer to minimize the mean square error (MSE) loss.
[0013] Furthermore, in the deep neural network f(·) constructed in step 3, the implementation process of the three one-dimensional convolutional layers from the fourth to the sixth layers is:
[0014] For samples after maximum and minimum value processing The sth stream Input it into the corresponding one-dimensional convolution channel, and the output of the s-th one-dimensional convolution layer is
[0015]
[0016] Where F represents the number of filters in the convolutional layer. The output of the jth filter is expressed as
[0017]
[0018] Among them, tanh(·) represents the tanh activation function. represents the jth trainable filter, where K s Represents the convolution kernel size, C h Represents the number of channels of the filter. Indicates bias.
[0019] Furthermore, in the deep neural network f(·) constructed in step 3, the implementation process of the seventh common attention layer is:
[0020] The outputs of the first and second 1D convolutional layers are and is input to a co-attention layer using parallel co-attention to learn the interaction between the two input sequences. First, according to and The similarity matrix is calculated as
[0021]
[0022] in, Represents the weight matrix. Then, using the similarity matrix C, we obtain two attention maps as
[0023]
[0024] in, represents the weight matrix. m represents the adjustable mapping dimension. Next, Perform linear mapping operations to obtain the attention weights:
[0025]
[0026] in, Represents the mapping matrix. Finally, according to a v and a q , and the attention matrices are
[0027]
[0028] in, represents a matrix of all 1s, It stands for the element-wise product operator. It is worth noting that the weighted input features are not summed during the calculation process, so that the output of the attention layer still has temporal nature.
[0029] Furthermore, in the deep neural network f(·) constructed in step 3, the implementation process of the connection layer of the eighth and ninth layers and the shape reshaping layer is:
[0030] First, the output of the joint attention layer and The output of the third 1D convolutional layer Connect, represented by Then, O c Reshape Here, the integer d ≥ 1 represents the reshaping scale.
[0031] Furthermore, the implementation process of the 10th to 12th recurrent neural network layers and the 13th and 14th fully connected layers is as follows:
[0032] set up The tth time step of Therefore, for time t, the output of the third recurrent neural network layer can be obtained as
[0033]
[0034] in, Represent the mapping process of the first, second, and third recurrent neural network layers respectively. 1 、n 2 、n 3 They represent the output dimensions of the three recurrent neural network layers at time t. Therefore, the overall output of the third recurrent layer can be expressed as
[0035]
[0036] Then, using the last time step As the input of the first fully connected layer, and the second fully connected layer is used for position prediction. Finally, the prediction P (X or Y coordinate) of sample x is obtained as
[0037]
[0038] in, and Represent the weight matrix and bias of the first and second fully connected layers respectively, is the number of nodes in the first fully connected layer.
[0039] Beneficial effects: As shown in the figure, a flexible cable-driven continuum joint shape estimation method provided by the present invention pre-processes the training samples represented by the change information (force information and torque information) of the continuum joint force and the theoretical angle information, and reshapes the processed sample features. These training samples after shape reshaping are then input into a specially constructed deep neural network, and the optimal deep neural network continuum joint shape estimator is solved in combination with the labels corresponding to the training samples. In the test phase, the test samples represented by the change information (force information and torque information) of the continuum joint force and the theoretical angle information are input into the optimal continuum joint shape estimator to estimate the coordinate position of the marked points on the continuum joint of the corresponding sample.
[0040] Most of the existing studies on shape estimation of flexible cable-driven continuum joints focus on modeling joint shape changes using traditional kinematic methods. However, due to the flexible deformation ability of continuum joints, the kinematic modeling method usually requires a tedious parameter identification process, and it is difficult to parse an accurate shape estimation model. In addition, during actual surgery, the collision between the continuum joint and human tissue produces unknown shape changes, which further reduces the accuracy of continuum joint shape estimation and leads to insufficient adaptability to complex surgical environments.
[0041] Therefore, the present invention adopts a data-driven continuum joint shape estimation method in a flexible cable-driven continuum joint shape estimation method. Experimental results show that compared with the traditional regression method, the method has better estimation performance and can accurately estimate the shape of the continuum joint. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 The present invention is a flow chart of a method for estimating the joint shape of a flexible cable-driven continuum.
[0043] Figure 2 The X coordinate of the marker point 1 in a set of training data with a load weight of 5g changes over time.
[0044] Figure 3 The Y coordinate of the marker point 1 in a set of training data with a load weight of 5g changes over time.
[0045] Figure 4 The theoretical angle of the system input changes over time in a set of training data with a load weight of 5g. DETAILED DESCRIPTION
[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.
[0047] like Figure 1As shown, the present invention first performs minimum and maximum normalization preprocessing on the training samples represented by the change information (force information and torque information) of the continuum joint force and the theoretical angle information, and reshapes the processed sample features, and then inputs them into the constructed deep neural network estimator to predict the coordinate positions of the marker points on the continuum joints. In the training stage, firstly, the training samples represented by the change information (force information and torque information) of the continuum joint force and the theoretical angle information are preprocessed by minimum and maximum normalization, and reshapes the processed sample features, and further combines the corresponding training sample labels to solve the optimal continuum joint shape estimator; in the testing stage, the obtained optimal continuum joint shape estimator is used in combination with the test samples represented by the change information (force information and torque information) of the continuum joint force and the theoretical angle information to predict the coordinate positions of the marker points on the continuum joints corresponding to each test sample.
[0048] The method of the present invention is compared with the traditional regression method in terms of mean square error (MSE), mean absolute error (MAE) and Pearson correlation coefficient (PCC) by experimental methods.
[0049] The experiment uses the cable-driven continuum joint motion dataset of Hefei University of Technology to verify the effectiveness of the method of the example of the present invention.
[0050] In order to simulate the force of the continuum joint in the actual surgical environment, the data in this dataset are collected under the no-load and loaded states of the continuum joint, and contain a total of 12 sets of data. Among them, 2 sets of data were collected under the no-load state; 2 sets of data were collected under the load weight of 5g; 4 sets of data were collected under the load weight of 10g; and 4 sets of data were collected under the load weight of 15g.
[0051] In order to carry out specific research on the shape estimation problem of flexible continuum joints, the present invention uses a 3-DOF continuum surgical robot constructed by Hefei University of Technology as an experimental prototype. The prototype contains three degrees of freedom, namely, the rotational degree of freedom, the mobile degree of freedom and the bending degree of freedom of the continuum joint. The experimental prototype is mainly composed of a flexible cable drive module, a rotation and movement module and a single-degree-of-freedom continuum joint. The flexible cable drive module is equipped with three linear motors and several pulleys and Bowden cables. The three motors drive the movement of the three flexible cables respectively, two of which are responsible for providing the power required for the bending movement of the continuum joint, and the other one provides the power required for the opening and closing movement of the end clamp. The pulley and Bowden cable are mainly responsible for guiding the two flexible cables that drive the bending of the continuum joint. One end of the flexible cable is fixed on the linear motor, and the other end is fixed to the end of the continuum joint after being guided. In addition, multiple sensors are installed on the experimental prototype. The magnetic grating on the magnetic ring and the grating on the mobile module are used to detect the rotation angle and linear movement distance of the overall prototype respectively, and the six-dimensional force sensor is used to detect the change of the end force of the continuum. The continuum joint is made of Niti alloy with an elastic modulus of 65Gpa and a Poisson's ratio of 0.3, which makes it have good flexibility. Considering the limitations of the surgical space, the outer diameter of the joint is set to 10mm and the length is set to 80mm. In addition, four marking points are placed on the continuum joint, of which the bottom marking point is the fixed point. The coordinate positions of the first three marking points are collected for subsequent shape estimation research. The dSPACE system sends control commands to each motor to control the movement of the entire prototype in three directions, and collects data information detected by the sensor in real time as the data basis for experimental research.
[0052] The linear motor uses the LCA-32 linear motor produced by SMAC in the United States, with an effective stroke of 12mm, a continuous force of 30N, a resolution of 5μm, and is equipped with an LCC-10 driver. The DC servo motor uses the ECG1636 servo motor of Weichuan Company to drive the rotation of the overall prototype, with a rated torque of 100.2mNm and a rated speed of 684.2rpm. The driver uses the ACJ-005-09 driver. The effective stroke of the linear module is 200mm, the lead is 4mm, and the Panasonic MHMF AC servo motor is used to drive the overall prototype for linear motion, with a rated torque of 640mNm and a rated speed of 3000rpm. The driver uses the Panasonic A6 driver. The magnetic grating uses the MSR500 model of IBB. The grating uses the Renishaw grating model RH100X30D50A with a resolution of 1μm. The six-dimensional force sensor uses the Kunwei KWR36A sensor with a force measurement range of 30N and a torque range of 1.5Nm, which is used to detect the change of the end force of the continuum. The tension sensor uses the MSQ-1 model with a range of 50N and a resolution of 0.1N to detect the tension of the driving cable. The dSPACE real-time control system communicates with the linear motor, DC servo motor and AC servo motor through the RS232 communication module, and communicates with the six-dimensional force sensor through RS485.
[0053] The sampling time of each set of data is 100s, the sampling rate is 50Hz, and a total of 5000 time points are included. In addition, each set of data includes 16 sequence data information, and the specific information is: three force information (N) and three torque information (N·M) detected by the six-dimensional force sensor (6 sequences); the X coordinate (mm) and Y coordinate (mm) of the three marking points (6 sequences); the actual bending angle (deg) of the continuum joint (1 sequence); the tension of the two flexible cables (N) (2 sequences); the theoretical input angle (deg) (1 sequence). The present invention selects to use the three force information, three torque information and a theoretical angle information input through the dSPACE real-time control system in the XYZ direction of the end of the continuum joint link in the three-dimensional space to construct sample features, and selects the X (or Y) coordinate information of the marking point 1 as the label of the corresponding sample. Among them, the theoretical angle information is first written by the host computer through MATLAB / Simulink to write a control program, which is then imported into dSPACE for execution after compilation. Then dSPACE sends the control instructions to each motor driver through the RS232 communication module, and converts the control instructions into the drive input information of the six-dimensional force sensor through the RS485 communication module. Then the motor driver further converts the control instructions into the drive input information of the LCA-32 linear motor, ECG1636 DC servo motor and MHMF AC servo motor.
[0054] Taking into account the contact between continuum joints and human tissues in practical applications, 2 sets of data under no-load state and 2 sets of data under load weight of 5g are used as training sets; 4 sets of data under load weight of 10g are used as validation sets; 4 sets of data under load weight of 15g are used as test sets. In addition, the number of time points for each sample is set to 50, and 50×7-dimensional sample features can be obtained. For the overlap between samples, the overlap of samples in the training set and validation set is set to 40, while the overlap of samples in the test set is set to 49 to predict P (X or Y coordinate) corresponding to each time point in the test set. Therefore, in each regression task, the number of samples in the training set, validation set, and test set are 1984, 1984, and 19804, respectively. The change of the X (or Y) coordinate of the marker point 1 in the 1 set of training set data under the load weight of 5g over time is shown as follows Figure 2 , Figure 3 As shown in the figure, the system signal changes with time as Figure 4 shown.
[0055] It should be noted that for the regression task of predicting coordinate X and the regression task of predicting coordinate Y, although the proposed network structure is the same, the specific parameter settings are not completely consistent.
[0056] In the regression task of predicting coordinate X, the specific parameters in the network structure are set as follows: the number of filters in each 1DCNN layer is set to 64, the convolution kernel size is set to 3, the step size is set to 1, the padding method is same padding, and the activation function is selected as the tanh function. In addition, the mapping dimension of the common attention layer is set to 32. The reshaping scale d is set to 5, and the reshaped sample feature dimension can be obtained to be 10×960, which is then sent to the recurrent neural network layer for processing. In the three stacked recurrent neural network layers, the number of nodes in the first to third layers is set to 128, 64, and 32 respectively. Finally, the number of nodes for the two stacked fully connected layers is set to 8 and 1 respectively.
[0057] In the regression task of predicting coordinate Y, the specific parameters in the network structure are set as follows: the number of filters for each 1DCNN layer is set to 128, the convolution kernel size is set to 3, the step size is set to 1, the padding method is set to same padding, and the activation function is set to the tanh function. In addition, the mapping dimension of the common attention layer is set to 64, and the reshaping scale d is set to 5. Then, for the three stacked recurrent neural network layers, the number of nodes is set to 128, 64, and 32, respectively. Finally, for the two stacked fully connected layers, the number of nodes is set to 8 and 1, respectively.
[0058] However, the same hyperparameter tuning strategy and optimal model preservation strategy were adopted for both tasks. Specifically, the maximum number of training iterations was set to 50, the batch size range was {8, 16, 32, 64, 128}, and the initial learning rate range was set to {0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001} to record the best results. In addition, the adaptive moment estimation (Adam) optimizer and mean square error (MSE) were selected as the optimizer and loss function of the network. For the preservation strategy of the optimal model, within the hyperparameter range and 50 iterations, the MSE indicator of the network model after each iteration was calculated using the validation set, and the optimal model corresponding to the optimal MSE result was saved. The optimal model was then applied to the test set to evaluate the prediction performance of the model.
[0059] In the experiment, in order to demonstrate the effect of the method of the present invention, the proposed method is compared with other methods suitable for regression tasks. The compared methods include: SVR (Support Vector Regression), DEN (Dense Network), LSTM (Long Short-Term Memory) / GRU (Gated Recurrent Unit) / Bi-LSTM (Bi-directional Long Short-Term Memory) / Bi-GRU (Bi-directional Gated Recurrent Unit), Multi-Channel LSTM (Multi-Channel LSTM, MC LSTM), Multi-Channel GRU (Multi-Channel GRU, MC GRU), Multi-Channel Bi-LSTM (Multi-Channel Bi-LSTM, MC Bi-LSTM), Multi-Channel Bi-GRU (Multi-Channel Bi-GRU, MC Bi-GRU), Multi-Channel Self-Attention LSTM (Multi-Channel Self-Attention LSTM, MC-SA LSTM), Multi-Channel Self-Attention GRU (Multi-Channel Self-Attention GRU, MC-SA GRU), Multi-Channel Self-Attention Bi-LSTM (Multi-Channel The method proposed in the present invention is Multi-Channel Co-Attention GRU (MC-CAGRU).
[0060] In the experiments, for LSTM, GRU, Bi-LSTM and Bi-GRU experiments, the constructed networks all use three stacked recurrent neural network layers and two stacked fully connected layers, and the number of nodes in each layer is set to be consistent with the proposed method. For MCLSTM, MC GRU, MCBi-LSTM and MC Bi-GRU experiments, the constructed networks use the same multi-channel module as the proposed method. In addition, the number of filters and the range of convolution kernel size in the convolution layer are set to {32, 64, 128} and {3, 5} respectively for parameter tuning. For MC-SA LSTM, MC-SA GRU, MC-SA Bi-LSTM and MC-SA Bi-GRU experiments, the constructed networks first merge the outputs of multiple channels and then send them to a self-attention layer. Finally, for MC-CALSTM, MC-CABi-LSTM and MC-CABi-GRU experiments, the constructed networks introduce a common attention layer between the multi-channel module and the recurrent neural network module, and adopt the same parameter setting and tuning strategy as the proposed method.
[0061] Table 1
[0062]
[0063]
[0064] Tables 1 and 2 show the MSE, MAE, and PCC results of the proposed method and the comparative method on the validation set and test set in the task of predicting the coordinate X position and Y position, respectively. As can be seen from Table 1, the proposed method shows better performance on both the validation set and the test set, indicating that the proposed method can effectively predict the coordinate X position of the marker point 1. Then, it can also be seen from Table 1 that for the network model based on the multi-channel structure, in most cases, the network model with the self-attention layer added shows worse prediction performance than the model without the self-attention layer, which indicates that the use of the self-attention layer in the network model may reduce the accuracy of the model in predicting the coordinate X position. In addition, it can be seen from Table 2 that on the validation set, the MC-SA GRU method obtained worse experimental results than the MC GRU method, which to some extent indicates that adding the self-attention layer to the network does not necessarily improve the accuracy of the model in predicting the coordinate Y position. However, combining the results in Table 1 and Table 2, it can be inferred that for the network model based on the multi-channel structure, the model with the common attention layer added has better prediction performance than the model without the common attention layer. This shows that the use of the common attention layer in the network model can effectively learn the correlation between multiple sequences, thereby improving the accuracy of the model's prediction of the coordinates X and Y positions.
[0065] Table 2
[0066]
[0067]
[0068] Next, in order to observe in more detail the performance of the position prediction task using the common attention layer and the self-attention layer in the model, Tables 3 and 4 respectively show the experimental results of the MC-SA GRU method and the MC-CA GRU method in the tasks of predicting the coordinate X position and the coordinate Y position when the attention layer uses different mapping dimensions (m∈{32,64,128}). It should be noted that for each mapping dimension, the data shown in the table are the best results obtained within the range of all hyperparameters in the convolutional layer.
[0069] As can be seen from Tables 3 and 4, under the same mapping dimension, the experimental results of the MC-CA GRU method on the validation set and the test set are better than those of the MC-SA GRU method, indicating that applying the common attention layer to the network model can more effectively improve the performance of the network model in the task of predicting coordinate positions. Moreover, the advantage of the common attention layer in the task of predicting the coordinate X position is more obvious.
[0070] Table 3
[0071]
[0072] Table 4
[0073]
[0074]
[0075] Finally, in order to comprehensively observe the performance of the network model in the task of predicting coordinate positions, according to the experimental results in Tables 1 and 2, a network model containing a self-attention layer and a common attention layer was selected. At the same time, based on the multi-channel module of the proposed method and combined with the encoder of the Transformer, a multi-channel Transformer (MC Transformer) model was constructed. Then, based on the X and Y positions of the coordinates predicted by these models, the average Euclidean distance (AED) and average Manhattan distance (AMD) between the true coordinate position and the predicted coordinate position of the samples in the validation set and the test set were calculated according to the following formulas.
[0076]
[0077] Among them, (xi ,y i )and They represent the actual coordinate position and predicted coordinate position of the marked point corresponding to the i-th sample, and n represents the number of samples.
[0078] Table 5
[0079]
[0080] Table 5 shows the experimental results of the network model containing the self-attention layer and the common attention layer and the MC Transformer model on the validation set and the test set. The results show that compared with other methods, the method containing the common attention layer has better performance in the validation set and the test set, further indicating that the use of the common attention layer can effectively improve the performance of the model in the task of predicting coordinate positions.
Claims
1. A method for estimating the joint shape of a cable-driven continuum. It is characterized in that The method comprises: firstly establishing a continuous body joint motion data set of a flexible cable-driven surgical robot, and dividing it into a training set, a validation set and a test set according to the load weight at the end of the continuous body joint. Then, three force information, three torque information and a theoretical angle information input through a dSPACE real-time control system in the three-dimensional space of the end of the continuous body joint link are used to construct a sample feature sequence. At the same time, the coordinate position information of the marker point on the continuous body joint is used as the label sequence of the sample. Further, these feature sequences are overlapped and divided according to the time length T=50 time points (0.5s) to construct training samples, validation samples and test samples. In addition, the method independently performs two prediction regression tasks according to the X coordinate position and Y coordinate position of the marker point. In the regression task of predicting P (P represents the X or Y coordinate), the label of each sample is selected as the X (or Y) coordinate position of the marker point corresponding to the last time point among the T time points.
2. A method for estimating joint shape of a flexible cable-driven continuum according to claim 1, It is characterized in that The specific steps of the method include: Step 1: The training set samples are represented as Where n represents the number of samples in the training set. The corresponding sample labels are expressed as Among them, y 1 ,y 2 ,…,y n represents the X-coordinate value or Y-coordinate value of the corresponding marked point, depending on the prediction object of the current regression task. Then, each feature in the sample is preprocessed by normalizing the maximum and minimum values, and the processed training set samples are represented as Step 2: Reshape the sample features processed in step 1 and split the features of each sample into three streams of length T. The three flows can be regarded as two-dimensional matrices with dimensions of T×3, T×3, and T×1, respectively, corresponding to three force information, three torque information, and one theoretical angle information. The sth flow is expressed as Step 3: Construct a multi-channel joint attention recurrent neural network model to estimate the shape of the continuum joints. Then, the training sample set after shape reshaping in step 2 is sent to the constructed neural network along three different channels, and the corresponding sample labels Υ are used to train the deep neural network f(·) to obtain the continuum joint shape estimator Step 4: For any sample x in the test set, firstly go through the maximum and minimum value normalization in step 1 and the shape reshaping in step 2, and the three reshaped flows are and The three streams are then fed into the trained continuum joint shape estimator along three different channels In this way, each sample in the test set is estimated to obtain P (X or Y coordinate) of the marked point on the continuum joint of the corresponding test sample.
3. A method for estimating joint shape of a flexible cable-driven continuum according to claim 2, It is characterized in that The neural network model in step 3 is a multi-channel joint attention recurrent neural network with the following structure: The first to third layers are three parallel input layers, which receive three streams of sample features after sequence reshaping: force information, torque information, and theoretical angle information, with feature dimensions of T×3, T×3, and T×1 respectively. The fourth to sixth layers are three parallel one-dimensional convolutional layers, corresponding to the above three streams in turn; the number of filters in each one-dimensional convolutional layer is set to F, and the convolution kernel size is set to K s , the step size is 1, the padding method is same padding, and the activation function is tanh function. The seventh layer is the common attention layer, the input of which is the output of the first and second one-dimensional convolutional layers, and the adjustable mapping dimension is set to m. The eighth layer is the connection layer, which connects the output of the common attention layer and the output of the third one-dimensional convolutional layer. The ninth layer is the reshape layer, and the reshape scale is set to d. The tenth to twelfth layers are recurrent neural network layers, with 128, 64 and 32 nodes respectively. The thirteenth and fourteenth layers are fully connected layers, with 8 and 1 nodes respectively. In addition, the network uses the adaptive moment estimation (Adam) optimizer to minimize the mean square error (MSE) loss.
4. A method for estimating the joint shape of a flexible cable-driven continuum according to claim 2, It is characterized in that In the deep neural network f(·) constructed in step 3, the implementation process of the three one-dimensional convolutional layers from the fourth to the sixth layer is: For samples after maximum and minimum processing The sth stream Input it into the corresponding one-dimensional convolution channel, and the output of the s-th one-dimensional convolution layer is Where F represents the number of filters in the convolutional layer. The output of the jth filter is expressed as Among them, tanh(·) represents the tanh activation function. represents the jth trainable filter, where K s Represents the convolution kernel size, C h Represents the number of channels of the filter. Indicates bias.
5. The method for estimating the joint shape of a flexible cable-driven continuum according to claim 2, It is characterized in that In the deep neural network f(·) constructed in step 3, the implementation process of the seventh common attention layer is: The outputs of the first and second 1D convolutional layers and is input to a co-attention layer using parallel co-attention to learn the interaction between the two input sequences. First, according to and The similarity matrix is calculated as in, Represents the weight matrix. Then, using the similarity matrix C, we obtain two attention maps as in, represents the weight matrix. m represents the adjustable mapping dimension. Next, Perform linear mapping operations to obtain the attention weights: in, Represents the mapping matrix. Finally, according to a v and a q , and the attention matrices are in, represents a matrix of all 1s, Represents the element-wise product operator. The weighted input features are not summed during the calculation process, so that the output of the attention layer still has temporal nature.
6. A method for estimating the joint shape of a cable-driven continuum according to claim 2, It is characterized in that In the deep neural network f(·) constructed in step 3, the implementation process of the eighth and ninth connection layers and the reshape layer is as follows: First, the output of the common attention layer is and The output of the third 1D convolutional layer Connect, represented by Then, O c Reshape Here, the integer d ≥ 1 represents the reshaping scale.
7. The method for estimating the joint shape of a flexible cable-driven continuum according to claim 2, It is characterized in that In the deep neural network f(·) constructed in step 3, the implementation process of the 10th to 12th recurrent neural network layers and the 13th and 14th fully connected layers is as follows: set up The tth time step of Therefore, for time t, the output of the third recurrent neural network layer can be obtained as in, Represent the mapping process of the first, second, and third recurrent neural network layers respectively. 1 、n 2 、n 3 They represent the output dimensions of the three recurrent neural network layers at time t. Therefore, the overall output of the third recurrent layer is expressed as Then, using the last time step As the input of the first fully connected layer, and the second fully connected layer is used for position prediction. Finally, the prediction P (X or Y coordinate) of sample x is obtained as in, and Represent the weight matrix and bias of the first and second fully connected layers respectively, is the number of nodes in the first fully connected layer.
Citation Information
Patent Citations
Human motion action analysis method based on time series regression prediction
CN110956139A
Lower limb motion intention prediction method based on attention mechanism
CN112906673A
Motion estimation method for end effector of flexible cable-driven minimally invasive surgical instrument
CN116453673A
Prediction method for stall and surging of axial-flow compressor based on deep autoregressive network
US20240133391A1
Hand joint angle estimation method
WO2024103371A1
