Mechanism and data model dual-drive robot surgical operation force prediction method and device

Abstract

A method and device for predicting surgical operation force of a dual-driven robot driven by a mechanism and a data model, which relates to the field of medical robot technology. The method comprises: establishing a mechanism prediction model and a double-layer time series data prediction model for surgical operation force respectively; training the mechanism prediction model and the double-layer time series data prediction model, and setting corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model; using the trained mechanism prediction model and the double-layer time series data prediction model to predict surgical operation force respectively, and obtaining the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result by inputting the operation information in the real-time surgical process into the trained model; and combining the weight coefficients to perform weighted calculation on the above two surgical operation force prediction results to obtain the real-time prediction result of the surgical operation force. The prediction result of the present invention is accurate, and the doctor's on-the-spot perception is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a method and device for predicting surgical operating force of a dual-drive robot using a mechanism and a data model. Background Art

[0002] With the development of medical robot technology, the rise of robot-assisted minimally invasive surgery has changed the implementation method of traditional minimally invasive surgery, improved the flexibility and quality of surgical operations, and expanded the categories of surgical procedures. During robot-assisted minimally invasive surgery, the doctor controls the remote slave robot by operating the master hand of the main control console to complete the surgical operation. The master-slave remote operation mode of robot-assisted minimally invasive surgery allows doctors to stay away from the surgical site and realize telemedicine. However, since the current robot system used for robot-assisted minimally invasive surgery does not have a force sensing function, doctors cannot perceive the changes in surgical operating force during the operation and can only rely on visual images of the lesion area to judge the degree of surgical action. This makes it impossible for doctors to control the surgical operating force, resulting in the operating force being too small to complete the surgical action, or the operating force being too large to cause damage to the biological soft tissue structure.

[0003] At present, some existing solutions attempt to study force feedback from indirect variables that can reflect force information, and use methods based on electric current, position error, and external strain gauges to extract the force between robotic surgical instruments and biological tissue structures. However, obtaining surgical operation force information through indirect variables is easily affected by interference signals and cannot truly reflect the dynamic characteristics of robotic surgical interaction. During the operation, surgical instruments interact with biological soft tissues. Factors such as human errors, puncture speed, and soft tissue deformation will affect the accuracy of the operation. Due to the heterogeneity and deformation characteristics of soft tissues, the force size and direction of surgical instruments must be accurate, otherwise serious complications will occur. It can be seen that a surgical operation force prediction solution that can effectively adapt to robot-assisted minimally invasive surgery scenarios and take into account the optimization of solution accuracy is a key technical problem that needs to be solved to enable doctors to perceive the surgical operation force during surgery.

[0004] Among them, Kim et al. embedded capacitive force sensors into the two tips of the gripper. The reading of each unit was converted into two normal forces and two shear forces, and the three-degree-of-freedom tensile force and single grasping force were calculated through geometric relationships. However, when only one side of the gripper contacts the tissue, the sensor can only provide force information of two degrees of freedom, and the sanitation problem of the sensor during miniaturization and embedded packaging is an important issue that needs to be solved urgently. "U.Kim, D.-H.Lee, WJYoon, B.Hannaford and HRChoi,"Force Sensor Integrated Surgical ForcepsforMinimallyInvasive Robotic Surgery,"in IEEE Transactions on Robotics,vol.31,no.5,pp.1214-1224,Oct.2015,doi:10.1109 / TRO.2015.2473515.".

[0005] Dalvand et al. designed and developed a force-sensing stainless steel sleeve. Through its built-in strain gauge, it can measure the force applied to the instrument from soft tissue, avoiding the use of sensors at the tip of the instrument. However, for actual use in surgery, the sterilization of the force-sensing sleeve and the optimal selection of materials require further research. "Moradi Dalvand, Mohsen." An actuated force feedback-enabled laparoscopic instrument for robotic-assisted surgery Actuated force feedback-enabled laparoscopic instrument". The international journal of medical robotics + computer assisted surgery (1478-5951), 10 (1), p. 11".

[0006] Peirs et al. developed a micro optical sensor for sensing operating force based on optical fiber. The sensor reflects the slight deformation of the surgical instrument structure when it is subjected to force by changing the angle of the optical path of the end faces of three parallel optical fibers. The force conditions in various directions are obtained by decoupling the sensor. However, optical components are easily damaged and require precise interfaces with other system components. Smaller fiber bends can cause signal attenuation and compensation distortion, while larger fiber bends can cause damage to the fiber core. "J. Peirs, J. Clijnen, D. Reynaerts, HV Brussel, P. Herijgers, B. Corteville, S. Boone, A microoptical force sensor for force feedback during minimally invasive robotic surgery (SPEC. ISS), Sens. Actuators.: APhys. Vol. 115 (No. 2–3) (2004) 447–455".

[0007] Since the surgical environment has strict requirements on the size, biocompatibility, sterilizability, etc. of force sensors, the method of obtaining surgical operating force through indirect variables is difficult to apply to actual surgery. Summary of the invention

[0008] In order to solve the problems existing in the prior art, the present invention provides a method and device for predicting the surgical operation force of a dual-drive robot based on a mechanism and a data model. The present invention has accurate prediction results and improves the on-site perception of doctors.

[0009] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0010] The present invention provides a method for predicting the surgical operation force of a dual-drive robot based on a mechanism and a data model, which mainly includes the following steps:

[0011] Step S1: Establish a mechanism prediction model for surgical operation force by combining biological soft tissue mechanical properties and robot kinematics;

[0012] Step S2: establishing a double-layer time series data prediction model for surgical operation force based on in vitro biological soft tissue surgical operation experimental data;

[0013] Step S3: training the mechanism prediction model and the double-layer time series data prediction model respectively, obtaining the prediction results of the mechanism prediction model and the double-layer time series data prediction model, setting corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model, and obtaining the trained mechanism prediction model and the double-layer time series data prediction model;

[0014] Step S4: using the trained mechanism prediction model to predict the surgical operation force, by inputting the operation information during the real-time surgery into the trained mechanism prediction model to obtain the prediction result of the mechanism-driven surgical operation force;

[0015] Step S5: using the trained double-layer time series data prediction model to predict surgical operation force, by inputting the operation information during the real-time operation into the trained double-layer time series data prediction model to obtain the data model driven surgical operation force prediction result;

[0016] Step S6: Based on the real-time surgical scene and the operation information during the real-time surgery, the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result are weightedly calculated in combination with the weight coefficient to obtain the real-time prediction result of the surgical operation force.

[0017] Furthermore, the calculation formula of the mechanism prediction model is:

[0018]

[0019] ∈=g(v,l)

[0020] Where FN1 represents the prediction result of the mechanism prediction model, M R represents the elastic modulus of biological soft tissue, ∈ represents the deformation of biological soft tissue, η represents the viscosity coefficient of biological soft tissue, v represents the moving speed of the end of the surgical instrument, and l represents the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue; ∈=g(v, l) means that the deformation of biological soft tissue is represented by the moving speed of the end of the surgical instrument and the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue.

[0021] Furthermore, the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue is obtained by calculating using the position information of the end of the surgical instrument; the calculation formula of the position information of the end of the surgical instrument is:

[0022]

[0023] where q 1 , q2 and q3 represent the three joints of the surgical robot; P:x, P:y and P:z represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics; represents the kinematics matrix of the three joints of the surgical robot, (1, 4) represents the matrix 1 row and 4 columns, (2, 4) represents the matrix 2 rows and 4 columns, (2, 4) represents the matrix 2 rows and 4 columns;

[0024] At the i-th moment, the relative displacement l between the surgical instrument and the biological soft tissue i The calculation formula is:

[0025] l i =((P:x i -P:x i-1 ) 2 +(P:y i -P:y i-1 ) 2 +(P:z i -P:z i-1 ) 2 ) 0.5

[0026] Where P:x i 、P:y i and P:z i They represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-th moment; P: x i-1 、P:y i-1 and P:Z i-1 They respectively represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-1th moment.

[0027] Furthermore, the calculation formula for the moving speed of the end of the surgical instrument is:

[0028]

[0029]

[0030]

[0031] in Represents the three joints q of the surgical robot 1 ,q 2 and q 3 The velocity matrix, The Jacobian matrix representing the movement speed of the surgical robot joint and the movement speed of the end of the surgical instrument, v is divided into the linear velocity in the three directions of x, y, and z, which is a 3×1 column vector; ω is divided into the angular velocity in the three directions of x, y, and z, which is a 3×1 column vector; Represents the linear velocity Jacobian matrix, which is a 3×3 matrix; Represents the angular velocity Jacobian matrix, which is a 3×3 matrix.

[0032] Furthermore, the ex vivo biological soft tissue surgery operation experimental data is obtained by conducting an ex vivo biological soft tissue surgery operation experiment using a surgical robot platform; the ex vivo biological soft tissue surgery operation experimental data includes motion data of the surgical instrument end and surgical operation force data during the surgical operation, and the motion data includes position change data and speed change data after the surgical instrument end contacts the biological soft tissue.

[0033] Furthermore, the specific establishment process of the two-layer time series data prediction model is as follows:

[0034] S201: Data collection;

[0035] Collect and organize time series data for training and testing, perform data preprocessing on the time series data, and divide the data set into training set and test set;

[0036] S202: data serialization;

[0037] Convert the preprocessed time series data into a sequence format acceptable to the model, and use the sliding window method to create input sequences and corresponding target sequences;

[0038] S203: Establish a time series data prediction model with a double-layer long short-term neural network LSTM structure;

[0039] Import the Pytorch deep learning framework, define the model architecture, build a time series data prediction model with a two-layer long short-term neural network LSTM, add an appropriate number of neurons, activation functions, and other layers to control the complexity of the model and prevent overfitting; compile the model, select the appropriate loss function, optimizer, and evaluation indicators;

[0040] S204: adjusting model parameters;

[0041] Adjust the parameters of each layer to obtain the best model;

[0042] S205: training model;

[0043] Train the model using the training set, monitor the performance of the model on the validation set, and monitor the training loss and validation loss during training;

[0044] S206: Evaluation model;

[0045] Use the test set to evaluate the trained model and analyze the evaluation indicators;

[0046] S207: Adjustment and optimization:

[0047] Adjust the model architecture and hyperparameters based on the evaluation results, and repeat the training and evaluation process until performance requirements are met;

[0048] S208: predict future values;

[0049] Use the trained model to predict future values;

[0050] S209: Save and deploy;

[0051] Save the trained model to a file for later use.

[0052] Furthermore, the specific training process of the two-layer time series data prediction model is as follows:

[0053] (1) The motion data and time information in the in vitro biological soft tissue surgery operation experimental data are used as input data, and the surgical operation force data is used as label data;

[0054] (2) The input motion data and surgical operation force data are normalized, and the calculation formula is as follows:

[0055]

[0056]

[0057] in Represents surgical instrument motion data x (v，l) The normalized value, x i(v，l) represents the motion data of the surgical instrument at the i-th moment; represents the normalized value of the surgical operation force data yF, y iF represents the surgical instrument motion data x measured at the i-th moment i(v，l) The corresponding surgical operation force data; min() represents the minimum value function, and max() represents the maximum value function;

[0058] (3) The processed input data and label data are input into the two-layer time series data prediction model for training.

[0059] The present invention provides a mechanism and data model dual-drive robot surgical operation force prediction device, which is used to implement the mechanism and data model dual-drive robot surgical operation force prediction method, and the device includes: a model building module, a training module, a solution module and an output module;

[0060] The model building module is used to respectively build a mechanism prediction model for surgical operation force and a double-layer time series data prediction model;

[0061] The training module is used to obtain parameters related to different surgical scenarios and experimental data parameters of in vitro biological soft tissue surgical operations, respectively train the mechanism prediction model and the double-layer time series data prediction model, and set corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model, so as to obtain the trained mechanism prediction model and the double-layer time series data prediction model;

[0062] The solution module is used to obtain the operation information during the real-time operation and input it into the trained mechanism prediction model and the double-layer time series data prediction model for prediction, so as to obtain the prediction results of the mechanism-driven surgical operation force and the data model-driven surgical operation force;

[0063] The output module is used to perform step-by-step weighted calculation on the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result to obtain a real-time prediction result of the surgical operation force.

[0064] The present invention provides a mechanism and data model dual-drive robot surgical operation force prediction device, comprising: a memory and a processor; the memory stores executable instructions, and the processor is configured to execute the executable instructions in the memory to implement the steps of the mechanism and data model dual-drive robot surgical operation force prediction method described in any one of claims 1 to 8.

[0065] The beneficial effects of the present invention are:

[0066] The present invention provides a method and device for predicting robot surgical operation force driven by both mechanism and data models, which breaks through the traditional method of studying force feedback through indirect variables. It innovatively starts from the mechanical properties of biological soft tissue, and establishes a mechanism prediction model for surgical operation force by studying the mechanical properties of biological soft tissue and the interactive information of robot surgery. It not only considers the influence of the viscoelastic properties of biological soft tissue on the surgical operation force during surgery, but also considers the influence of the surgical robot operation variables on the deformation of biological soft tissue and the surgical operation force during real-time surgery. Further, by acquiring in vitro biological soft tissue surgical operation experimental data, a double-layer time series data prediction model for predicting surgical operation force based on robot surgical operation data is established, and the characteristics of the mechanism prediction model and the double-layer time series data prediction model are fully considered. The two models are selected through weight coefficients, and finally a method for predicting robot surgical operation force driven by both mechanism and data models is realized, which solves the problem that doctors cannot perceive the surgical operation force during real-time surgery of the surgical robot, and improves the accuracy and precision of surgical operation force prediction in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1A flow chart of a method for predicting the operating force of a dual-drive robot surgery based on a mechanism and data model provided by the present invention;

[0068] Figure 2 A structural block diagram of a dual-drive robot surgical operation force prediction device according to a mechanism and data model of the present invention;

[0069] Figure 3 This is a structural block diagram of a dual-drive robot surgical operation force prediction device based on a mechanism and data model of the present invention. DETAILED DESCRIPTION

[0070] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0071] In a first aspect, the present invention provides a method for predicting the surgical operating force of a dual-drive robot using a mechanism and a data model.

[0072] like Figure 1 As shown, a method for predicting the surgical operation force of a dual-drive robot based on a mechanism and data model of the present invention has a specific operation process as follows:

[0073] Step S1: Establish a mechanism prediction model for surgical operation force by combining biological soft tissue mechanical properties and robot kinematics;

[0074] The mechanism prediction model of surgical operation force established by the present invention is a model between biological soft tissue deformation and surgical operation force established based on the mechanical properties of biological soft tissue and robot kinematics. After the biological soft tissue comes into contact with the surgical instrument, the biological soft tissue is deformed under the action of the surgical instrument movement, thereby generating an interaction force between the two. The so-called surgical instrument movement refers to the movement of the end of the surgical instrument driven by the joint movement of the surgical robot.

[0075] Among them, the biological soft tissue mechanical properties refer to the fact that biological soft tissue is a viscoelastic body, which has both viscosity and elasticity, and has two characteristic parameters: elastic modulus and viscosity coefficient. Since biological soft tissue will undergo irregular deformation during surgery, which is not easy to measure, the present invention associates the deformation information of biological soft tissue with the motion information of surgical instruments, and integrates the motion information of surgical instruments into the viscoelastic biomechanical model to establish a mechanism prediction model that is more suitable for use by surgical robots. The specific calculation formula of the established mechanism prediction model for surgical operating force is as follows:

[0076]

[0077] ∈=g(v,l)

[0078] Among them, FN1 represents the prediction result of the mechanism prediction model, M Rrepresents the elastic modulus of biological soft tissue, ∈ represents the deformation of biological soft tissue, η represents the viscosity coefficient of biological soft tissue, v represents the moving speed of the end of the surgical instrument, and l represents the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue. Among them, ∈=g(v, l) represents that the deformation of biological soft tissue is represented by the moving speed of the end of the surgical instrument and the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue.

[0079] The motion information of the surgical instrument can be obtained by calculating the robot kinematics and solving the Jacobian matrix. In a specific embodiment, the specific calculation formula for the position information of the end of the surgical instrument and the moving speed of the end of the surgical instrument is as follows:

[0080]

[0081]

[0082]

[0083]

[0084]

[0085] Among them, q 1 , q2 and q3 represent the three joints of the surgical robot; P:x, P:y and P:z represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics; represents the kinematics matrix of the three joints of the surgical robot, (1, 4) represents the matrix 1 row and 4 columns, (2, 4) represents the matrix 2 rows and 4 columns, (2, 4) represents the matrix 2 rows and 4 columns; Represents the three joints q of the surgical robot 1 ,q 2 and q 3 The velocity matrix, The Jacobian matrix represents the movement speed of the surgical robot joint and the movement speed of the end of the surgical instrument; v is divided into the linear velocity in the three directions of x, y, and z, which is a 3×1 column vector; ω is divided into the angular velocity in the three directions of x, y, and z, which is a 3×1 column vector; Represents the linear velocity Jacobian matrix, which is a 3×3 matrix; Represents the angular velocity Jacobian matrix, which is a 3×3 matrix.

[0086] In a specific embodiment, the relative displacement l between the surgical instrument and the biological soft tissue at the i-th moment i It can be calculated by the following formula:

[0087] l i =((P:x i -P:x i-1 ) 2 +(P:y i -P:y i-1 ) 2 +(P:z i -P:z i-1 ) 2 ) 0.5

[0088] Where P:x i 、P:y i and P:z i They represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-th moment; P: x i-1 、P:y i-1 and P:z i-1 They respectively represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-1th moment.

[0089] The present invention establishes a correlation model between the deformation information of biological soft tissue and the motion information of surgical instruments, and then establishes a mechanism prediction model for surgical operating force. It not only takes into account the viscoelastic properties of biological soft tissue, but also takes into account the influence of the surgical robot on the surgical operating force, thereby improving the accuracy of the mechanism prediction model.

[0090] Step S2: using a surgical robot platform to perform an in vitro biological soft tissue surgical operation experiment, thereby obtaining in vitro biological soft tissue surgical operation experimental data, and establishing a double-layer time series data prediction model (DTSP) for surgical operation force based on the in vitro biological soft tissue surgical operation experimental data;

[0091] Among them, the in vitro biological soft tissue surgery operation experimental data mainly includes the motion data of the surgical instrument end under different doctors' operation styles and the surgical operation force data during the surgical operation. The so-called motion data mainly includes the position change data and speed change data after the surgical instrument end contacts the biological soft tissue. The motion data can be directly obtained by reading through the surgical robot system, and the surgical operation force data is obtained by reading through the six-dimensional sensor installed on the surgical instrument. The sampling frequency of the six-dimensional sensor is preferably 1000Hz, but not limited to this.

[0092] Among them, the different operating styles of doctors refer to the fact that some doctors are skilled in operation and relatively fast in speed; some doctors are steady in operation and relatively slow in speed.

[0093] In the present invention, the in vitro biological soft tissue surgical operation experiment is carried out on a surgical robot platform. A statistically significant three-dimensional tissue surgical operation experiment is designed for the in vitro biological soft tissue, and the surgical instrument motion data and surgical operation force data during the experiment are collected. In order to more comprehensively consider different surgical scenarios and doctor operation styles, multiple experts in professional fields can be hired to perform surgical operations, which not only increases the breadth of data, but also takes into account the impact of different skill levels on the surgical process. This diversity of data samples can significantly improve the adaptability of the model in different situations, improve the robustness of the model, and provide strong technical support for the subsequent realization of surgical robot force feedback.

[0094] In a specific implementation, the two-layer time series data prediction model for surgical operation force is obtained by optimizing the long short-term neural network LSTM, and the specific establishment process is as follows:

[0095] S201: Data collection;

[0096] Collect and organize time series data for training and testing, perform data preprocessing on the time series data, including cleaning outliers, filling missing values, normalization or standardization, etc., and then divide the data set into training set and test set;

[0097] S202: data serialization;

[0098] Convert the preprocessed time series data into a sequence format acceptable to the model, and use the sliding window method to create input sequences and corresponding target sequences;

[0099] S203: Establish a time series data prediction model with a double-layer long short-term neural network LSTM structure;

[0100] Import the Pytorch deep learning framework, define the model architecture, build a time series data prediction model with a two-layer long short-term neural network LSTM, add an appropriate number of neurons, activation functions, and other layers (such as Dropout layer) to control the complexity of the model and prevent overfitting; compile the model, select the appropriate loss function, optimizer, and evaluation indicators;

[0101] S204: adjusting model parameters;

[0102] For each layer, the corresponding parameters can be adjusted through experiments and validation set performance to obtain the best model;

[0103] For example, adjust dropout or randomly discard the proportion of neurons during training to prevent overfitting; or adjust the hyperparameters of the optimizer such as learning rate and batch size;

[0104] S205: training model;

[0105] Use the training set to train the model, monitor the performance of the model on the validation set, as well as the training loss and validation loss during training; if the model is overfitting, consider adding a Dropout layer or reducing the number of neurons;

[0106] S206: Evaluation model;

[0107] Use the test set to evaluate the trained model and analyze evaluation metrics such as root mean square error (RMSE);

[0108] S207: Adjustment and optimization:

[0109] Adjust the model architecture and hyperparameters based on the evaluation results, and repeat the training and evaluation process until performance requirements are met;

[0110] S208: predict future values;

[0111] Use the trained model to predict future values;

[0112] S209: Save and deploy;

[0113] Save the trained model to a file for future use and, if needed, deploy the model in production for real-time predictions.

[0114] In the process of establishing a two-layer time series data prediction model, the present invention takes into account the characteristics of the data set and the performance of the validation set, through cleverly designed network layers and targeted parameter adjustments, to achieve a more comprehensive and flexible capture of sequence features on multiple time scales, significantly improve the prediction accuracy of the two-layer time series data prediction model, and make the model show excellent versatility and robustness.

[0115] Step S3: According to different surgical scenarios and doctors' operating styles, the mechanism prediction model and the two-layer time series data prediction model are trained respectively to obtain the prediction results of the mechanism prediction model and the two-layer time series data prediction model, and the corresponding weight coefficients are set according to the prediction results of the mechanism prediction model and the two-layer time series data prediction model to obtain the trained mechanism prediction model and the two-layer time series data prediction model.

[0116] In a specific implementation, the specific training process of the two-layer time series data prediction model is as follows:

[0117] (1) The motion data and time information in the in vitro biological soft tissue surgery operation experimental data are used as input data, and the surgical operation force data are used as label data;

[0118] (2) The input motion data and surgical operation force data are normalized, and the specific calculation formula is as follows:

[0119]

[0120]

[0121] in Represents surgical instrument motion data x (v，l) The normalized value, x i(v，l) represents the motion data of the surgical instrument at the i-th moment; Indicates surgical operation force data y F The normalized value, y iF represents the surgical instrument motion data x measured at the i-th moment i(v，l) The corresponding surgical operation force data; min() represents the minimum value function, and max() represents the maximum value function;

[0122] (3) The processed input data and label data are input into the two-layer time series data prediction model for training.

[0123] Step S4: using the trained mechanism prediction model to predict the surgical operation force, by inputting the operation information obtained during the real-time surgery into the trained mechanism prediction model to obtain the mechanism-driven surgical operation force prediction result FN1;

[0124] Step S5: using the trained double-layer time series data prediction model to predict the surgical operation force, by inputting the operation information obtained during the real-time surgery into the trained double-layer time series data prediction model to obtain the data model driven surgical operation force prediction result FN2;

[0125] Step S6: Based on the real-time surgical scene and the operation information obtained during the real-time surgery, the mechanism-driven surgical operation force prediction result FN1 and the data model-driven surgical operation force prediction result FN2 are weightedly calculated in combination with the weight coefficient to obtain the real-time prediction result FN of the surgical operation force, and the surgical operation force is continuously predicted in a stage-by-stage rolling manner, and the surgical operation force prediction result is displayed in real time on the surgical robot control interface.

[0126] The above-mentioned stage-by-stage rolling refers to the cyclic and continuous prediction of the surgical operating force during the operation. Therefore, the prediction of the surgical operating force is a continuous process rather than just a one-time prediction.

[0127] In a second aspect, the present invention provides a mechanism and data model dual-drive robot surgical operation force prediction device, which is mainly used to implement a mechanism and data model dual-drive robot surgical operation force prediction method provided in the first aspect.

[0128] like Figure 2 As shown, a mechanism and data model dual-drive robot surgical operation force prediction device of the present invention mainly includes: a model building module 210, a training module 220, a solution module 230 and an output module 240.

[0129] The specific functions of each module are as follows:

[0130] A model building module 210, which is mainly used to respectively build a mechanism prediction model for surgical operation force and a double-layer time series data prediction model;

[0131] The training module 220 is mainly used to obtain parameters related to different surgical scenarios and experimental data parameters of in vitro biological soft tissue surgical operations, train the mechanism prediction model and the double-layer time series data prediction model respectively, set corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model respectively, and obtain the trained mechanism prediction model and the double-layer time series data prediction model;

[0132] The solution module 230 is mainly used to obtain the operation information during the real-time operation and input it into the trained mechanism prediction model and the double-layer time series data prediction model for prediction, so as to obtain the prediction results of the mechanism-driven surgical operation force and the data model-driven surgical operation force;

[0133] The output module 240 is mainly used to perform step-by-step weighted calculation on the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result to obtain a real-time prediction result of the surgical operation force.

[0134] In a third aspect, the present invention provides a mechanism and data model dual-drive robot surgical operation force prediction device, which is mainly used to run a mechanism and data model dual-drive robot surgical operation force prediction method provided in the first aspect.

[0135] like Figure 2 As shown, a mechanism and data model dual-drive robot surgical operation force prediction device of the present invention mainly includes: a processor 310, a memory 320, an input device 330 and an output device 340; wherein the number of the processor 310 can be one or more, Figure 3 Only one processor 310 is used as an example; the processor 310, the memory 320, the input device 330 and the output device 340 are all connected via a bus or other means. Figure 3In the example, the bus connection is taken as an example; the memory 320, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules; the processor 310 can execute various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 320, that is, to realize a mechanism and data model dual-drive robot surgical operation force prediction method provided in the first aspect; the input device 330 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the device; the output device 340 may include display devices such as display screens.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. Mechanism and data model The dual-drive robot surgical operation force prediction method is characterized by: The following steps are involved: Step S1: Establish a mechanism prediction model for surgical operation force by combining biological soft tissue mechanical properties and robot kinematics; The calculation formula of the mechanism prediction model is: ∈=g(v,l); Where FN1 represents the prediction result of the mechanism prediction model, M R represents the elastic modulus of biological soft tissue, ∈ represents the deformation of biological soft tissue, η represents the viscosity coefficient of biological soft tissue, v represents the moving speed of the end of the surgical instrument, and l represents the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue; ∈=g(v,l) represents that the deformation of biological soft tissue is represented by the moving speed of the end of the surgical instrument and the relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue; Step S2: establishing a double-layer time series data prediction model for surgical operation force based on in vitro biological soft tissue surgical operation experimental data; The specific establishment process of the two-layer time series data prediction model is as follows: S201: Data collection; Collect and organize time series data for training and testing, perform data preprocessing on the time series data, and divide the data set into training set and test set; S202: data serialization; Convert the preprocessed time series data into a sequence format acceptable to the model, and use the sliding window method to create input sequences and corresponding target sequences; S203: Establish a time series data prediction model with a double-layer long short-term neural network LSTM structure; Import the Pytorch deep learning framework, define the model architecture, build a time series data prediction model with a two-layer long short-term neural network LSTM, add an appropriate number of neurons, activation functions, and other layers to control the complexity of the model and prevent overfitting; compile the model, select the appropriate loss function, optimizer, and evaluation indicators; S204: adjusting model parameters; Adjust the parameters of each layer to obtain the best model; S205: training model; Train the model using the training set, monitor the performance of the model on the validation set, and monitor the training loss and validation loss during training; S206: Evaluation model; Use the test set to evaluate the trained model and analyze the evaluation indicators; S207: Adjustment and optimization: Adjust the model architecture and hyperparameters based on the evaluation results, and repeat the training and evaluation process until performance requirements are met; S208: predict future values; Use the trained model to predict future values; S209: Save and deploy; Save the trained model to a file for future use; Step S3: training the mechanism prediction model and the double-layer time series data prediction model respectively, obtaining the prediction results of the mechanism prediction model and the double-layer time series data prediction model, setting corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model, and obtaining the trained mechanism prediction model and the double-layer time series data prediction model; Step S4: using the trained mechanism prediction model to predict the surgical operation force, by inputting the operation information during the real-time surgery into the trained mechanism prediction model to obtain the prediction result of the mechanism-driven surgical operation force; Step S5: using the trained double-layer time series data prediction model to predict surgical operation force, by inputting the operation information during the real-time operation into the trained double-layer time series data prediction model to obtain the data model driven surgical operation force prediction result; Step S6: Based on the real-time surgical scene and the operation information during the real-time surgery, the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result are weightedly calculated in combination with the weight coefficient to obtain the real-time prediction result of the surgical operation force.

2. The method for predicting the surgical operation force of a dual-drive robot based on the mechanism and data model according to claim 1 is characterized in that: The relative displacement between the surgical instrument and the biological soft tissue after the surgical instrument contacts the biological soft tissue is obtained by calculation using the position information of the end of the surgical instrument; the calculation formula of the position information of the end of the surgical instrument is: where q 1、 q2 and q3 represent the three joints of the surgical robot; P:x, P:y, and P:z represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics; represents the kinematics matrix of the three joints of the surgical robot, (1,4) represents the matrix 1 row and 4 columns, (2,4) represents the matrix 2 rows and 4 columns, (2,4) represents the matrix 2 rows and 4 columns; At the i-th moment, the relative displacement l between the surgical instrument and the biological soft tissue i The calculation formula is: l i =((P:x i -P:x i-1 ) 2 +(P:y i -P:y i-1 ) 2 +(P:z i -P:z i-1 ) 2 ) 0.5 ; Where P:x i 、P:y i and P:z i They represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-th moment; P:x i-1 、P:y i-1 and P:z i-1 They respectively represent the position coordinates of the end of the surgical instrument calculated by solving the robot kinematics at the i-1th moment.

3. The method for predicting the surgical operation force of a dual-drive robot based on the mechanism and data model according to claim 1 is characterized in that: The calculation formula for the moving speed of the end of the surgical instrument is: in Represents the three joints q of the surgical robot 1、 The velocity matrix of q2 and q3, The Jacobian matrix representing the movement speed of the surgical robot joint and the movement speed of the end of the surgical instrument, v is divided into the linear velocity in the three directions of x, y, and z, which is a 3×1 column vector; ω is divided into the angular velocity in the three directions of x, y, and z, which is a 3×1 column vector; Represents the linear velocity Jacobian matrix, which is a 3×3 matrix; Represents the angular velocity Jacobian matrix, which is a 3×3 matrix.

4. The method for predicting the surgical operation force of a dual-drive robot based on the mechanism and data model according to claim 1 is characterized in that: The in vitro biological soft tissue surgery experiment data is obtained by performing an in vitro biological soft tissue surgery experiment using a surgical robot platform; the in vitro biological soft tissue surgery experiment data includes motion data of the surgical instrument end and surgical operation force data during the surgical operation, and the motion data includes position change data and speed change data after the surgical instrument end contacts the biological soft tissue.

5. The method for predicting the surgical operation force of a dual-drive robot based on the mechanism and data model according to claim 1 is characterized in that: The specific training process of the two-layer time series data prediction model is as follows: (1) The motion data and time information in the in vitro biological soft tissue surgery operation experimental data are used as input data, and the surgical operation force data is used as label data; (2) The input motion data and surgical operation force data are normalized, and the calculation formula is as follows: in Represents surgical instrument motion data x (v,l) The normalized value, x i(v,l) represents the motion data of the surgical instrument at the i-th moment; Indicates surgical operation force data y F The normalized value, y iF represents the surgical instrument motion data x measured at the i-th moment i(v,l) The corresponding surgical operation force data; min() represents the minimum value function, and max() represents the maximum value function; (3) The processed input data and label data are input into the two-layer time series data prediction model for training.

6. Mechanism and data model of dual-drive robot surgical operation force prediction device, characterized in that: A method for predicting the surgical operating force of a dual-drive robot using a mechanism and a data model as described in any one of claims 1 to 5, the device comprising: a model building module, a training module, a solution module, and an output module; The model building module is used to respectively build a mechanism prediction model for surgical operation force and a double-layer time series data prediction model; The training module is used to obtain parameters related to different surgical scenarios and experimental data parameters of in vitro biological soft tissue surgical operations, respectively train the mechanism prediction model and the double-layer time series data prediction model, and set corresponding weight coefficients according to the prediction results of the mechanism prediction model and the double-layer time series data prediction model, so as to obtain the trained mechanism prediction model and the double-layer time series data prediction model; The solution module is used to obtain the operation information during the real-time operation and input it into the trained mechanism prediction model and the double-layer time series data prediction model for prediction, so as to obtain the prediction results of the mechanism-driven surgical operation force and the data model-driven surgical operation force; The output module is used to perform step-by-step weighted calculation on the mechanism-driven surgical operation force prediction result and the data model-driven surgical operation force prediction result to obtain a real-time prediction result of the surgical operation force.

7. Mechanism and data model of dual-drive robot surgical operation force prediction device, characterized in that, include: Memory and processor; The memory stores executable instructions, and the processor is configured to execute the executable instructions in the memory to implement the steps of the mechanism and data model dual-drive robot surgical operation force prediction method described in any one of claims 1 to 5.

Images