Hip-knee integrated surgery robot and registration method thereof

Abstract

The present invention discloses a hip-knee integrated surgical robot and its alignment method, comprising a surgical instrument mounted at the end of a multi-degree-of-freedom robotic arm for controlling the motion path and speed of the surgical instrument; a medical image fusion module for receiving and processing image data from medical imaging equipment to construct a three-dimensional model of the hip joint and knee joint; an image alignment module for aligning the three-dimensional model with the real-time image during surgery; an optical tracking module for setting marking points, which are set on the surgical instrument. The optical tracking module obtains the position and posture of the surgical instrument in three-dimensional space in real time by tracking the marking points. The hip-knee integrated surgical robot system of the present invention is applied to hip and knee replacement or repair surgery to solve the problems of insufficient surgical accuracy, complicated and cumbersome operation, and long time consumption in current technology, and realizes the integration of hip and knee dual joint surgery, thereby significantly improving the accuracy, safety and efficiency of the surgery.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical devices, and relates to a hip-knee integrated surgery robot and a registration method thereof. BACKGROUND

[0002] At present, although hip joint and knee joint surgeries have robot assistance, most of the existing surgery robots independently serve hip joint or knee joint surgeries, and lack integrated solutions. Meanwhile, the existing surgery registration methods may have problems such as insufficient accuracy, complex operation, and long time consumption. Therefore, there is an urgent need for a hip-knee integrated surgery robot capable of simultaneously serving hip-knee joint surgeries, realizing accurate registration and efficient operation. SUMMARY

[0003] The application is provided to overcome at least one deficiency of the prior art, and provides a hip-knee integrated surgery robot and a registration method thereof.

[0004] In order to achieve the above-mentioned purpose, the application adopts the following technical scheme: a hip-knee integrated surgery robot, comprising

[0005] A multi-degree-of-freedom mechanical arm is installed at the end of the surgery instrument, used to control the movement path and speed of the surgery instrument;

[0006] A medical image fusion module is used to receive and process image data from a medical image device, and construct a three-dimensional model of the hip joint and the knee joint;

[0007] An image registration module is used for registration of the three-dimensional model and the intraoperative real-time image;

[0008] An optical tracking module is provided with a marker point, the marker point is arranged on the surgery instrument, and the optical tracking module acquires the position and posture of the surgery instrument in the three-dimensional space in real time by tracking the marker point;

[0009] A magnetic navigation module is provided with a magnetic marker, the magnetic marker is arranged on the surgery instrument, and the magnetic navigation module acquires the position and posture of the surgery instrument in the three-dimensional space in real time through the magnetic marker implanted in the body;

[0010] A force torque sensor and a visual sensor are used to monitor the force and position information when the surgery instrument contacts the bone in real time;

[0011] A dynamic path optimization module is used to dynamically optimize the movement path of the surgery instrument according to the preoperative path planning and the real-time feedback information of the force torque sensor and the visual sensor during the operation;

[0012] An adaptive control assembly is internally provided with an adaptive control algorithm, used to adjust the pushing speed, force and direction of the surgery instrument according to the path optimization data of the dynamic path optimization module;

[0013] The surgical operation system receives data of the dynamic path optimization module and the adaptive control component, and includes an operation interface through which one-stop operation is performed.

[0014] Further, the multi-degree-of-freedom mechanical arm comprises a telescopic structure.

[0015] Further, the surgical instrument is a hip joint or knee joint dedicated surgical tool module.

[0016] A registration method of a hip-knee integrated surgical robot, comprising the following steps:

[0017] Step S1: The multi-modal image data of the patient is loaded into the medical image fusion module before surgery, the multi-modal image data is fused and processed by the medical image fusion module, a three-dimensional model of the hip joint and the knee joint is established, and a surgical path is planned according to the three-dimensional model;

[0018] Step S2: The actual anatomic structure images of the hip joint and the knee joint are collected, the image registration module is built-in image registration algorithm, and the three-dimensional model is registered with real-time images through the image registration algorithm;

[0019] Step S3: The optical tracking module and the magnetic navigation module are installed and verified to ensure that the position of the surgical instrument can be tracked and corrected in real time during surgery;

[0020] Step S4: The hip joint and the knee joint are preliminarily positioned before surgery, the image registration module combines the three-dimensional model planned before surgery with real-time image feedback information, and an optimized registration algorithm is used to dynamically calibrate the hip joint and the knee joint, and correct the actual position of the surgical instrument;

[0021] Step S5: Step S5; the dynamic path optimization module dynamically optimizes the path of the surgical instrument according to the real-time feedback information of the force torque sensor and the visual sensor, the adaptive control component obtains adjustment instructions according to the path optimization data, the multi-degree-of-freedom mechanical arm intelligently adjusts and updates the path of the surgical instrument according to the adjustment instructions, and updates the pushing speed, force and direction of the surgical instrument.

[0022] Further, the multi-modal image data is collected by a CT, MRI device.

[0023] Further, the method for registering the three-dimensional model with real-time images in step S2 based on the gray value registration algorithm is:

[0024] The registration formula is established as argmax_TMI (P1(x), P2(Tx));

[0025] Wherein, P1(x) and P2(Tx) represent the pixel gray distribution of the image data of two different modalities or different time points before and after registration respectively, x is the known registration transformation, Tx is the registration transformation to be solved, and argmax_T represents finding the transformation parameter that maximizes the registration algorithm.

[0026] Further, the image registration algorithm adopts one of a gray value-based registration algorithm and a feature point-based registration algorithm.

[0027] Further, the method for solving the joint angle of the robot arm in step S4 to make the tip of the surgical instrument be located at the preset initial position is that:

[0028] Joint angle = inverse kinematics (initial position, initial orientation).

[0029] Further, the method for correcting the actual position of the surgical instrument in step S4 is that:

[0030] Surgical instrument actual position = planning model (preset position, real-time feedback information).

[0031] In summary, the present application has the following advantages:

[0032] 1) The hip-knee integrated surgical robot system of the present application is applied to hip joint and knee joint replacement or repair surgery, solves the problems of insufficient surgical precision, complex operation, long time consumption and the like in the current technology, realizes the integration of hip-knee double joint surgery, and significantly improves the precision, safety and efficiency of the surgery.

[0033] 2) The hip-knee integrated surgical robot system of the present application improves the positioning reliability by fusing multi-modal images through a medical image fusion module, combines an optical tracking module and a magnetic navigation module, reduces the error of a single system, double-verify and real-time correct the position and posture of the surgical instrument in the patient's body, improves the registration precision, and intelligently adjusts the pushing speed, force and direction of the surgical instrument through adaptive control components, avoids tissue damage, and solves the complex registration and operation problems in hip-knee joint surgery, greatly improves the precision, safety and efficiency of the surgery, and opens up a new road for the development of orthopedic surgical robot technology. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a schematic diagram of a hip-knee integrated surgical robot of the present application.

[0035] Figure 2 It is a registration method flow chart of a hip-knee integrated surgical robot of the present application. DETAILED DESCRIPTION

[0036] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0037] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0038] All directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, horizontal, vertical...) are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0039] Due to installation errors and other reasons, the parallel relationship referred to in the embodiments of the present invention may actually be an approximately parallel relationship, and the perpendicular relationship may actually be an approximately perpendicular relationship.

[0040] Example 1:

[0041] like Figure 1 As shown, a hip-knee integrated surgical robot includes.

[0042] The multi-degree-of-freedom robotic arm includes a telescopic structure, which enables the surgical robot to adapt to different surgical needs of hip and knee joints. Surgical instruments are installed at the end of the multi-degree-of-freedom robotic arm. The multi-degree-of-freedom robotic arm drives the surgical instruments to move in three-dimensional space and controls their movement path and speed.

[0043] The surgical instrument is a hip joint-specific or knee joint-specific surgical tool module, which is selectively installed according to the type of surgery.

[0044] Medical image fusion module, used to receive and process image data from medical imaging equipment and construct three-dimensional models of hip and knee joints;

[0045] Image registration module, used for registration of 3D models with real-time intraoperative images;

[0046] An optical tracking module is configured to acquire the position and direction of the surgical instrument in three-dimensional space in real time, the optical tracking module adopts a triangulation principle for positioning, the optical tracking module is provided with a marker point, the marker point is arranged on the surgical instrument and moves simultaneously with the surgical instrument, and the optical tracking module acquires the position and posture of the surgical instrument in three-dimensional space in real time by tracking the marker point;

[0047] A magnetic navigation module is configured to acquire the position and direction of the surgical instrument in three-dimensional space in real time, the magnetic navigation module adopts magnetic field gradient information to calculate the position, the magnetic navigation module is provided with a magnetic marker, the magnetic marker is arranged on the surgical instrument and moves simultaneously with the surgical instrument, and the magnetic navigation module acquires the position and posture of the surgical instrument in three-dimensional space in real time by the magnetic marker implanted in the body;

[0048] The medical image fusion module, the image registration module, the optical tracking module and the magnetic navigation module constitute an integrated registration system, the optical tracking module and the magnetic navigation module are combined, the two are complementary, the positioning reliability is improved, the single system error is reduced, the position and posture of the surgical instrument in the patient's body are verified and corrected in real time, and the registration accuracy is improved.

[0049] A force torque sensor and a visual sensor are configured to monitor the force and position information when the surgical instrument contacts the bone in real time;

[0050] A dynamic path optimization module is configured to dynamically optimize the motion path of the surgical instrument according to the preoperative path planning and the real-time feedback information of the force torque sensor and the visual sensor;

[0051] An adaptive control assembly is configured to intelligently adjust the pushing speed, force and direction of the surgical instrument according to the path optimization data of the dynamic path optimization module to avoid tissue damage;

[0052] A surgical operation system is configured to receive the data of the dynamic path optimization module and the adaptive control assembly, integrate the functions of surgical planning, navigation, monitoring and control, and the like, and include an operation interface, so that a doctor can perform one-stop operation through the intuitive and easy-to-use operation interface, greatly simplifying the surgical process and improving the surgical efficiency.

[0053] As shown in Figure 2 A registration method of a hip-knee integrated surgical robot, comprising the following steps:

[0054] Step S1: The multi-modal image data of the patient is loaded into the medical image fusion module before surgery, the medical image fusion module performs fusion processing on the multi-modal image data, establishes a three-dimensional model of the hip joint and the knee joint, and plans a surgical path according to the three-dimensional model;

[0055] The multi-modal image data of the patient is obtained by a CT, MRI or the like;

[0056] Step S2; Collect the actual anatomic image of the hip and knee joint, the image registration module has an image registration algorithm, which registers the three-dimensional model with the real-time image through the image registration algorithm;

[0057] The image registration algorithm adopts one of the gray value-based registration algorithm (such as Mutual Information, MI) or the feature point-based registration algorithm (such as Iterative Closest Point, ICP);

[0058] Step S3: Install and verify the optical tracking module and the magnetic navigation module to ensure that the position of the surgical instrument can be tracked and corrected in real time during the operation;

[0059] Step S4: Preoperative preliminary positioning of the hip and knee joint, the image registration module combines the three-dimensional model planned before the operation with the real-time image feedback information, and uses an optimized registration algorithm to dynamically calibrate the hip and knee joint and correct the actual position of the surgical instrument;

[0060] This embodiment uses inverse kinematics to solve the joint angle of the robot arm to ensure that the tip of the surgical instrument is located at the preset initial position;

[0061] Joint angle = inverse kinematics (initial position, initial orientation);

[0062] Correct the actual position of the surgical instrument:

[0063] Surgical instrument actual position = planning model (preset position, real-time feedback information);

[0064] Step S5; The intraoperative dynamic path optimization module dynamically optimizes the path of the surgical instrument according to the real-time feedback information of the force torque sensor and the visual sensor, the adaptive control component obtains adjustment instructions according to the path optimization data, and the multi-degree-of-freedom mechanical arm intelligently adjusts and updates the path of the surgical instrument according to the adjustment instructions, updates the advancing speed, force and direction of the surgical instrument.

[0065] The adjustment instruction is obtained as follows: adjustment instruction = intelligent control (force error, visual feedback information);

[0066] Update the path: updated path = update planning path (original planning path adjustment instruction);

[0067] The method of registering the three-dimensional model with the real-time image based on the gray value-based registration algorithm in step S2 is as follows:

[0068] Establish the registration formula: argmax_TMI (P1(x), P2(Tx));

[0069] Wherein, P1(x) and P2(Tx) represent the pixel gray distribution of the image data of two different modalities or different time points before and after registration, x is the known registration transformation, Tx is the registration transformation to be solved, and argmax_T represents finding the transformation parameter that maximizes the registration algorithm.

[0070] The step S1 of three-dimensional model construction and path planning is:

[0071] Step S11: data preprocessing;

[0072] The multi-modal images are standardized, the average gray value is subtracted or the histogram equalization method is applied to enhance the image contrast, the two-dimensional slice images are stacked to form a three-dimensional body data, and each voxel corresponds to a gray value.

[0073] Step S12: three-dimensional convolutional neural network (3D CNN) modeling;

[0074] A network architecture similar to 3D U-Net is used, which includes an encoder and a decoder, and a skip connection mechanism.

[0075] Features are extracted through a series of 3D convolutional layers, activation functions (such as ReLU) and maximum pooling layers, spatial dimensions are gradually down-sampled, and the number of channels is increased. [\text{Encoder:}x^{(l+1)}=f(W^{(l)}*x^{(l)}+b^{(l)})] Wherein, (x^{(l)}) represents the input of the (l) layer, (W^{(l)}) is the convolution kernel weight matrix, (*) represents the three-dimensional convolution operation, (b^{(l)}) is the bias term, and (f) is the activation function.

[0076] Step S13: voxel-level segmentation;

[0077] The spatial dimensions are up-sampled through deconvolution operation, and the high-level semantic features of the encoder stage are transmitted to the corresponding decoder layer through skip connection, so as to realize accurate segmentation and prediction of different anatomical structures of the hip and knee joint.

[0078] [\text{Decoder:}y^{(l)}=u(f(W^{(l)}*x^{(l)}+b^{(l)})+s(x^{(l+1)}))] Wherein, (u) represents up-sampling operation, and (s) represents skip connection operation.

[0079] Step S14: loss function and optimization;

[0080] Use loss functions such as binary cross entropy or Di ce coefficient to evaluate the gap between the model's predicted segmentation results and the true label, and optimize the network weights through backpropagation. [\text{Loss}=-\frac{1}{N}\sum_{i=1}^{N}[y_i\l og(p_i)+(1-y_i)\l og(1-p_i)]] or [\text{D i ceLoss}=1-\frac{2\sum_{i=1}^{N}y_i p_i}{\sum_{i=1}^{N}y_i^2+\sum_{i=1}^{N}p_i^2};

[0081] Step S15: construct and output the three-dimensional model;

[0082] The output is a 3D voxel grid with a probability value for each voxel belonging to a specific anatomical structure. By setting a threshold on the probability value, the 3D bounding box and internal structure of the hip and knee joint can be obtained.

[0083] Step S16: converting the three-dimensional model into a graphic structure;

[0084] The segmented three-dimensional model is converted into a directed or undirected graph G = (V, E), where the vertex set V represents the key points or areas in the three-dimensional model, and the edge set E represents the connectivity between points.

[0085] Step S17: Path planning:

[0086] Apply a path planning algorithm, such as Dijkstra's algorithm or A* search algorithm, to find the optimal path from the starting point to the destination in this graph;

[0087] Using Dijkstra's algorithm: Initialize the distances of all nodes to infinity, and set the distance of the starting node to 0. [dist[s]=0;\fora llv\i nV-{s},dist[v]=\i nfty]· Repeat the following process until the target node is found or all nodes have been visited: Select the node u with the smallest distance from all nodes that have not been visited and have a known shortest path length. Update the distances of all unvisited neighbors v of u: [dist[v]=min(dist[v],dist[u]+w(u,v))]· Further introduce the heuristic function h(n), combining the actual distance g(n) and the estimated cost to reach the target: [f(n)=g(n)+h(n)] Select the next node according to the f(n) value to expand the search tree;

[0088] Step S18: surgical sequence decision simulation;

[0089] The surgical operation sequence is simulated by using a recurrent neural network (RNN) or a long short-term memory network (LSTM), considering the sequential dependency between surgical steps.

[0090] The LSTM unit operation is as follows:

[0091] The hidden state update is as follows:

[0092] [i_t=\s igma(W_{ix}x_t+W_{i h}h_{t-1}+b_i)][f_t=\s i gma(W_{fx}x_t+W_{fh}h_{t-1}+b_f)][\t i l de{C}_t=\tanh(W_{cx}x_t+W_{ch}h_{t-1}+b_c)][C_t=f_t\odotC_{t-1}+i_t\odot\t i l de{C}_t][o_t=\s i gma(W_{ox}x_t+W_{oh}h_{t-1}+b_o)][h_t=o_t\odot\tanh(C_t)],In the surgical sequence decision, the input of each time step t may include current position information, surgical instrument state and environmental feedback, etc. The next operation category and its parameters are predicted through the LSTM model. Finally, based on the accurate anatomical structure information of the three-dimensional model and the above path planning and sequence decision method, the system can generate a preliminary path diagram and operation strategy for hip and knee joint surgery, laying a foundation for subsequent dynamic optimization.

[0093] The step S5 of dynamically optimizing the path of the surgical instrument is specifically as follows:

[0094] Step S51: During the operation, the real-time collected data such as the position, posture and force feedback of the surgical instrument are input into the pre-trained model, and the model updates the actual state of the surgical instrument in real time to provide the latest environmental state information for the reinforcement learning algorithm;

[0095] Step S52: dynamically optimizing the path of the surgical instrument:

[0096] According to the Q-learning, Deep Q-Network (DQN) or Proximal Policy Optimization (PPO) algorithm, the next optimal action (i.e., adjusting the moving path or operation mode of the instrument) is selected;

[0097] A long-term reward function that comprehensively evaluates the safety, accuracy and efficiency of the operation is designed, for example

[0098] reward = safety_coeff * safety_score + accuracy_coeff * accuracy_score + efficiency_coeff * efficiency_score, where each coefficient is adjusted according to actual conditions;

[0099] "observe-action-reward" cycle: the intelligent agent continuously learns and updates the strategy by interacting with the environment (observing the state, taking action, and obtaining reward) to quickly adjust the surgical path to minimize the risk of tissue damage when encountering unexpected situations such as changes in bone hardness or variations in blood vessels.

[0100] The system continuously accumulates experience data during each operation, including indicators such as the success or failure of the operation, operation time, and degree of tissue damage, and uses the accumulated practical experience data to train the deep learning and reinforcement learning models online or offline after the operation. The model parameters are updated through optimization algorithms such as gradient descent, so that the surgical path planning and operation strategy are more in line with actual needs and individual differences after multiple iterations, continuously improving the accuracy and safety of the operation.

[0101] Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

Claims

1. A hip-knee integrated surgical robot, characterized by: include A surgical instrument is installed at the end of the multi-degree-of-freedom robotic arm to control the movement path and speed of the surgical instrument; Medical image fusion module, used to receive and process image data from medical imaging equipment and construct three-dimensional models of hip and knee joints; Image registration module, used for registration of 3D models with real-time intraoperative images; An optical tracking module, which sets markers on the surgical instrument and obtains the position and posture of the surgical instrument in three-dimensional space in real time by tracking the markers; Force-torque sensors and vision sensors are used to monitor the force and position information of surgical instruments in real time when they come into contact with bones; Dynamic path optimization module, used to dynamically optimize the motion path of surgical instruments based on preoperative path planning and real-time feedback from force-torque sensors and vision sensors during surgery; Adaptive control component, with built-in adaptive control algorithm, used to adjust the propulsion speed, force and direction of surgical instruments based on the path optimization data of the dynamic path optimization module; The surgical operating system receives data from the dynamic path optimization module and the adaptive control component. The surgical operating system includes an operation interface, through which one-stop operation is performed.

2. The hip-knee integrated surgical robot according to claim 1, characterized in that: The multi-degree-of-freedom robotic arm includes a telescopic structure.

3. The hip-knee integrated surgical robot according to claim 1, characterized in that: The surgical instrument is a hip joint-specific or knee joint-specific surgical tool module.

4. The hip-knee integrated surgical robot according to claim 1, characterized in that: It also includes a magnetic navigation module, which is equipped with magnetic markers. The magnetic markers are set on the surgical instrument. The magnetic navigation module obtains the position and posture of the surgical instrument in three-dimensional space in real time through the magnetic markers implanted in the body.

Images