Hip joint surgery robot based on three-dimensional scanning and image processing and its rapid and accurate registration method

Abstract

The present invention discloses a hip joint surgical robot based on three-dimensional scanning and image processing and a fast and accurate alignment method thereof, comprising a three-dimensional scanner, which is integratedly installed at the end of the hip joint surgical robot and is used to perform three-dimensional contour scanning of the inside of the acetabulum to generate three-dimensional image data; a central processing unit, which is tightly coupled with the three-dimensional scanner and is used to receive and analyze and process the three-dimensional image data in real time; an image processing module, which is configured to pre-process the three-dimensional image data obtained by the scan, identify and extract the bone structure, separate the bone structure contour in the acetabulum, and obtain bone structure information; the present invention can accurately distinguish between orthopedic flesh and soft tissue through the image processing module, eliminate the influence of soft tissue deformation on alignment accuracy, and achieve more accurate surgical instrument positioning and implant placement. The technical means of the present invention can not only improve the accuracy of surgical operations, but also reduce operation time, reduce the workload of doctors, and improve surgical safety.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical devices, and relates to a hip joint surgery robot based on three-dimensional scanning and image processing and a rapid and accurate registration method thereof. BACKGROUND

[0002] In the positioning and implant placement of a traditional hip joint surgery robot, two stages of coarse registration and fine registration are usually required, and a probe is often used to contact soft tissue to mark multiple points during the fine registration process. This method is time-consuming and inefficient, and the registration accuracy is limited due to the dependence on a large number of soft tissue points, which is easily affected by factors such as soft tissue deformation, and is not conducive to the accurate implementation of the surgery. SUMMARY

[0003] The application is provided to overcome at least one deficiency of the prior art, and provides a hip joint surgery robot based on three-dimensional scanning and image processing and a rapid and accurate registration method thereof.

[0004] To achieve the above-mentioned purpose, the application adopts the following technical solution: a hip joint surgery robot based on three-dimensional scanning and image processing, comprising

[0005] a three-dimensional scanner, which is integrated and installed at the end of the hip joint surgery robot, and is used to perform three-dimensional contour scanning on the inside of the acetabular fossa to generate three-dimensional image data;

[0006] a central processing unit, which is tightly coupled with the three-dimensional scanner, and is used to receive and analyze the three-dimensional image data in real time;

[0007] an image processing module, which is configured to pre-process the three-dimensional image data obtained by scanning, identify and extract the bony structure, separate the bony structure contour in the acetabular fossa, and obtain bony structure information;

[0008] a three-dimensional model matching and registration module, which is used to extract key feature points from the bony structure information and obtain the spatial correspondence between the bony structure information and a pre-constructed three-dimensional model and registration parameters.

[0009] Further, the three-dimensional scanner is a multi-beam laser or a multi-view camera system.

[0010] Further, the key feature points contain position information and have invariance in different postures and scales.

[0011] Further, the key feature point extraction algorithm uses one of scale invariant feature transform (SIFT), fast robust feature (SURF), or faster rotation invariant local binary pattern (ORB).

[0012] Further, the mechanical arm of the hip joint surgery robot is installed with a surgical instrument, and the hip joint surgery robot adjusts the position and posture of the mechanical arm in real time according to the registration parameters.

[0013] The fast and accurate registration method of the hip joint surgery robot comprises the following steps:

[0014] Step S1: A three-dimensional model of the acetabulum is established according to two-dimensional image data of the acetabulum of a patient before surgery;

[0015] Step S2: A three-dimensional image data of the acetabulum of the patient is obtained by scanning the acetabulum using a three-dimensional scanner before surgery;

[0016] Step S3: The image processing module performs bone structure recognition and extraction on the three-dimensional image data to obtain bone structure information;

[0017] Step S4: The three-dimensional model matching and registration module matches the extracted bone structure information with the set three-dimensional model to obtain accurate registration parameters;

[0018] Step S5: The hip joint surgery robot adjusts the position and posture of the mechanical arm in real time according to the registration parameters, so that the surgical robot performs accurate surgical instrument positioning and implant placement operation.

[0019] Further, the method for matching the bone structure information with the set three-dimensional model in step S4 is:

[0020] Step S41: Set a bone structure information scanning point cloud data set P, P={p_1, p_2,. p_i, p_n}; i=1, 2...n, i is the serial number of the bone structure information scanning point, and n is the total number of the bone structure information scanning points;

[0021] A three-dimensional model point cloud data set M, M={m_1, m_2,. m_j, m_x}, j=1.2...x, j is the serial number of the three-dimensional model point, and x is the total number of the three-dimensional model points;

[0022] Step S42: For each iteration, find the nearest point m_{ij} in set M for each point p_i in set P;

[0023] Step S43: Calculate the minimum value of all corresponding points (p_i, m_{ij});

[0024] Perform rigid transformation on the corresponding points (p_i, m_{ij}) to obtain ||Rp_i+t-m_{ij}||^2 (1), R is a rotation matrix, t is a translation vector, and ||·|| is the Euclidean distance. Transform (1) into a minimum loss function:

[0025] [E(R, t) = \sum_{i=1}^{n}||Rp_i+t-m_{ij}||^2];

[0026] In the above formula, E is a minimized loss function;

[0027] Step S44: update the transformation matrix R and t, solve the equation set of step S43 by gradient descent method or other nonlinear optimization strategy;

[0028] Step S45: apply the updated transformation T=[R|t] to the bone structure information scanning points, and judge whether the stop condition is reached, if the stop condition is not reached, return to step S42 for iteration until convergence, if the stop condition is reached, determine T as the best transformation matrix between the bone structure information scanning points and the model.

[0029] In summary, the present application has the following advantages:

[0030] 1) The present application directly obtains real-time bone structure information through a three-dimensional scanner, which enables the surgical robot to have direct scanning registration function, thereby eliminating the need for traditional surgical robot rough registration and soft tissue point marking steps, greatly improving the speed and efficiency of surgical registration; at the same time, the three-dimensional model matching and registration module has high-precision identification and registration characteristics, which can effectively overcome the influence of soft tissue deformation, and ensure the accuracy of surgical instrument positioning and implant placement; through the above technical scheme, the surgical efficiency and safety are effectively improved, the surgical preparation time is reduced and the surgical operation accuracy is improved, the workload of the doctor is reduced, and the surgical risk and complication rate are reduced. The surgical robot can directly achieve accurate registration by scanning the inside of the acetabular cavity, without the cumbersome manual probe point operation, which significantly shortens the surgical preparation time.

[0031] 2) The image processing module of the present application can accurately distinguish between orthopedic blood and soft tissue, eliminate the influence of soft tissue deformation on registration accuracy, and achieve more accurate surgical instrument positioning and implant placement.

[0032] 3) The technical means of the present application not only can improve the accuracy of surgical operation, but also can reduce the operation time, reduce the workload of the doctor, and improve the safety of the operation. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The present application is a modular diagram of a hip joint surgical robot. DETAILED DESCRIPTION

[0034] Following make the present application's implementation mode through specific concrete example, the person skilled in the art can easily understand the other advantages and efficacy of the present application from the disclosure of this specification.The present application can also be implemented or applied through another different specific implementation, and the details in the specification can be variously modified or changed based on different views and applications without departing from the spirit of the present application.It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0035] It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and only the components related to the present application are shown in the diagrams, not drawn according to the number, shape and size of the components when actually implemented, the actual implementation of each component can be a random change in type, number and proportion, and the component layout type can be more complex.

[0036] All directional indications (such as up, down, left, right, front, back, transverse, longitudinal,...) in the embodiments of the present application are only used to explain the relative positional relationship, motion condition, etc. between the components in a certain specific posture, and if the specific posture changes, the directional indications will also change accordingly.

[0037] Due to installation errors and other reasons, the parallel relationship referred to in the embodiments of the present application may actually be an approximate parallel relationship, and the vertical relationship may actually be an approximate vertical relationship.

[0038] Example one:

[0039] As shown in Figure 1 The hip joint surgery robot based on three-dimensional scanning and image processing includes

[0040] The three-dimensional scanner is integratedly installed at the end of the hip joint surgery robot, the three-dimensional scanner is a multi-beam laser or a multi-view camera system, the three-dimensional scanner has ultra-high resolution and sensitivity, can perform full-range, dead-angle-free three-dimensional contour scanning inside the acetabular cavity, generate high-resolution, high-quality three-dimensional image data, and the scanning process of the three-dimensional scanner can be completed within a few seconds, ensuring the timeliness and accuracy of the data; compared with the rough registration and cumbersome soft tissue point marking link required by the traditional surgery robot, the hip joint surgery robot adopts a direct scanning mode of the three-dimensional scanner, completely eliminates the step of manual intervention, and greatly improves the speed and efficiency of the surgery registration.

[0041] The central processing unit is tightly coupled with the three-dimensional scanner, and is used for receiving and analyzing the three-dimensional image data in real time; in the embodiment, the three-dimensional image data is transmitted to the central processing unit for analysis in real time by the three-dimensional scanner, which significantly shortens the preoperative preparation time and gains valuable surgery window period for the clinician.

[0042] The central processor includes an image processing module configured to preprocess the three-dimensional image data obtained by scanning, identify and extract the bony structure, accurately separate the bony structure contour in the acetabular fossa using deep learning, edge detection and other technologies, exclude the influence of soft tissues, and obtain bony structure information;

[0043] The embodiment preprocesses the three-dimensional image data by the image processing module, effectively excludes the interference of soft tissue factors such as muscles, fats and ligaments, and ensures the accuracy and integrity of the extracted bony structure contour.

[0044] The three-dimensional model matching and registration module has a built-in key feature point extraction algorithm, such as scale invariant feature transform (SIFT), fast robust feature (SURF) or faster rotation invariant local binary pattern (ORB) algorithm, which extracts unique and repeatedly identifiable key feature points from the bony structure information. Through the Iterative Closest Point (ICP) or other similar optimization algorithm, the spatial correspondence between the bony structure information and the pre-constructed three-dimensional model is obtained, the best rigid transformation is determined, the accurate registration in three-dimensional space is realized, and the accurate registration parameters are obtained.

[0045] The key feature points contain position information and have invariance in different postures and scales, providing reliable anchor points for subsequent registration.

[0046] The mechanical arm of the hip joint surgical robot is installed with a surgical instrument, and the hip joint surgical robot adjusts the position and posture of the mechanical arm in real time according to the registration parameters, ensures the accurate positioning of the surgical instrument, and effectively reduces the surgical risk and complications.

[0047] The fast and accurate registration method of the hip joint surgical robot comprises the following steps:

[0048] Step S1: A three-dimensional model of the acetabular fossa is established according to the two-dimensional image data of the acetabular fossa of the patient before surgery;

[0049] Step S2: A three-dimensional scanner is used to scan the acetabular fossa of the patient to obtain three-dimensional image data before surgery;

[0050] Step S3: The image processing module identifies and extracts the bony structure to obtain bony structure information;

[0051] Step S4: The three-dimensional model matching and registration module matches the extracted bony structure information with the set three-dimensional model to obtain accurate registration parameters;

[0052] The three-dimensional model matching and registration module identifies the key feature points of the bony structure information, and uses the ICP algorithm to gradually approach the optimal transformation matrix through the iteration process, and accurately registers the bony structure information and the three-dimensional model.

[0053] Step S5: The intraoperative hip surgery robot adjusts the position and attitude of the mechanical arm in real time according to the registration parameters, so that the surgical robot can accurately position the surgical instrument and place the implant, thereby effectively reducing the risk and complications of surgery.

[0054] The steps of step S3 include:

[0055] Step S31: Preprocess the three-dimensional image data to remove random noise in the image and improve image quality and structural information clarity.

[0056] Filtering the three-dimensional image: For each pixel (I(x, y)) in the three-dimensional image data, the new value (I'(x, y)) after filtering can be calculated by the following formula: [I'(x, y)=\frac{1}{2\pi\sigma^2}\sum_{m=-\infty}^{\infty}\sum_{n=-\infty}^{\infty}I(x+m,y+n)e^{-\frac{m^2+n^2}{2\sigma^2}}] where (\sigma) is the standard deviation of the Gaussian kernel, (m) and (n) are the offsets relative to the current pixel point;

[0057] Edge enhancement of the filtered three-dimensional image: Use edge enhancement algorithm to calculate the second derivative of the image to highlight the edges.

[0058] The two-dimensional discrete Laplace operator is: [\nabla^2f=\frac{\partial^2f}{\partialx^2}+\frac{\partial^2f}{\partialy^2}] The corresponding 3x3 template is: [\begin{bmatrix}0&1&0\1&-4&1\0&1&0\end{bmatrix}] Apply this template to the image, which can enhance the edge profile.

[0059] These filtering and enhancement techniques may need to adjust parameters flexibly according to the specific image quality and requirements, for example, the selection of the standard deviation (\sigma) of the Gaussian filter needs to be determined according to the noise level, the window size of the median filter affects the denoising effect and edge protection ability, and the edge enhancement effect of the Laplace operator needs to consider the needs of subsequent processing to avoid the appearance of false edges caused by excessive enhancement.

[0060] In summary, the three-dimensional image data preprocessing stage lays a solid foundation for subsequent bone structure recognition, feature point extraction and three-dimensional model matching through fine filtering techniques and edge enhancement strategies, ensuring the efficiency and accuracy of the entire image processing process.

[0061] Step 32: Bone structure identification and extraction: Accurately identify and separate the bone structure contour of the acetabular socket from the three-dimensional image data, providing clear and accurate structural information for subsequent key feature point extraction and three-dimensional model matching and registration.

[0062] Step 321: Threshold segmentation;

[0063] Calculate the gray histogram: Calculate the gray or intensity histogram of the preprocessed three-dimensional image data to identify the main gray distribution interval of bone tissue and other soft tissues.

[0064] Step 322: Determine threshold;

[0065] Use automatic threshold method (such as Otsu method) or manually set fixed threshold to segment the image into two parts, where the area above the threshold is considered as bone tissue area. [T=\argmin_{t}\left[w_1(t)\cdot\sigma_1^2(t)+w_2(t)\cdot\sigma_2^2(t)\right]] Where (T) is the optimal threshold, (w_1(t)) and (w_2(t)) are the proportion of pixels below and above (t), respectively, and (\sigma_1^2(t)) and (\sigma_2^2(t)) are the variances of the two parts.

[0066] Step 323: Erosion;

[0067] Use a structure element (such as a small sphere or disc) to perform erosion on the segmented bone tissue area to remove isolated small particle noise. [E=(A\ominusB)] Where (E) is the eroded image, (A) is the original image, (\ominus) represents the erosion operation, and (B) is the structure element.

[0068] Step 324: Dilation;

[0069] Perform dilation on the eroded image to restore the structure boundary that may be lost due to erosion and smooth the boundary. [D=(A\oplusB)] Where (D) is the dilated image and (\oplus) represents the dilation operation.

[0070] Step 325: Accurate bone structure area:

[0071] Use region growing method: Select appropriate seed points (usually stable points inside the bone structure), expand according to certain growth criteria (such as based on gray or texture similarity), until encountering obvious boundaries or reaching preset conditions, forming a complete bone structure area.

[0072] The watershed method treats the image as terrain and grayscale values ​​as height. By simulating water flow, the image is segmented into multiple basins, each corresponding to a connected region. For bony structure recognition, known bony seed points are marked to guide the watershed algorithm for segmentation, precisely defining the bony region.

[0073] The method for matching the bone structure information in step S4 with the set three-dimensional model is:

[0074] Step S41: Set a bone structure information scanning point cloud data set P, where P = {p_1, p_2, ..., p_i, p_n}; i = 1, 2, ..., n, where i is the sequence number of the bone structure information scanning point and n is the total number of the bone structure information scanning points;

[0075] 3D model point cloud data set M, M = {m_1, m_2, m_j, m_x}, j = 1.2...x, j is the sequence number of the 3D model point, x is the total number of 3D model points;

[0076] Step S42: For each iteration, find the point m_{ij} in the set M to which each point p_i in the set P is closest;

[0077] Step S43: Calculate the minimum value of all corresponding points (p_i, m_{ij});

[0078] Perform rigid body transformation on the corresponding points (p_i, m_{ij}) to obtain ||Rp_i+t-m_{ij}||^2 (1), where R is the rotation matrix, t is the translation vector, and ||·|| is the Euclidean distance. (1) is converted into the minimization loss function:

[0079] [E(R, t)=\sum_{i=1}^{n}||Rp_i+t-m_{ij}||^2];

[0080] In the above formula, E is the minimization loss function.

[0081] Step S44: Update the transformation matrices R and t, and solve the equations of step S43 by gradient descent or other nonlinear optimization strategies;

[0082] Step S45: Apply the updated transformation T=[R|t] to transform the bone structure information scanning points, and determine whether the stopping condition is met. If the stopping condition is not met, return to step S42 to continue iterating until convergence. If the stopping condition is met, determine that T is the optimal transformation matrix between the bone structure information scanning points and the dimensional model.

[0083] This embodiment uses a continuous iterative optimization process to find the optimal transformation matrix between the bone structure information scanning points and the 3D model, thereby achieving accurate registration of the two in three-dimensional space.

[0084] The stop condition can adopt an error threshold, an iteration number limit, etc., and the embodiment is not limited thereto.

[0085] Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

Claims

1. Hip joint surgery robot based on 3D scanning and image processing, characterized by: include A 3D scanner, which is integrated and installed at the end of the hip joint surgical robot and is used to perform a 3D contour scan of the interior of the acetabulum and generate 3D image data; A central processing unit, which is tightly coupled with the 3D scanner and is used to receive and analyze and process 3D image data in real time; An image processing module is configured to pre-process the scanned three-dimensional image data, identify and extract the bone structure, separate the bone structure contour in the acetabulum, and obtain bone structure information; The three-dimensional model matching and registration module is used to extract key feature points from the bone structure information and obtain the spatial correspondence between the bone structure information and the pre-built three-dimensional model as well as the registration parameters.

2. The hip joint surgery robot based on 3D scanning and image processing according to claim 1, characterized in that: The three-dimensional scanner is a multi-beam laser or multi-view camera system.

3. The hip joint surgery robot based on 3D scanning and image processing according to claim 1, characterized in that: The key feature points contain position information and remain invariant under different postures and scales.

4. The hip joint surgery robot based on 3D scanning and image processing according to claim 1, characterized in that: The key feature point extraction algorithm adopts one of Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF) or the faster Rotation Invariant Local Binary Pattern (ORB).

5. The hip joint surgery robot based on 3D scanning and image processing according to claim 1, characterized in that: The hip joint surgical robot has a robotic arm equipped with surgical instruments, and the hip joint surgical robot adjusts the position and posture of the robotic arm in real time according to the registration parameters.

Images