Interventional puncture positioning method and system combining MRI system and depth camera

Abstract

The present invention discloses an interventional puncture positioning method and system combining an MRI system and a depth camera, comprising: obtaining first MRI data of a patient's surgical area, and obtaining a reconstructed MRI image based on the first MRI data; obtaining first RGBD data of a puncture needle in real time, and obtaining the real-time relative position of the front end of the puncture needle in the reconstructed MRI image based on the first RGBD data. The present invention realizes real-time navigation and intraoperative guidance functions, so that doctors or surgical robots can obtain accurate patient anatomical structure information at any time during the operation, and can more accurately locate and operate surgical tools.

Description

Technical Field

[0001] The present invention relates to the field of magnetic resonance imaging technology, and in particular to an interventional puncture positioning method and system combining an MRI system and a depth camera. Background Art

[0002] As the demand and requirements for real-time navigation interventional surgery systems gradually expand, precise navigation and interventional operations as well as high-definition, real-time display of surgical scenes are crucial to improving surgical outcomes and reducing the risk of complications.

[0003] The existing technologies on the market currently use CT, ultrasound or single MRI (Magnetic Resonance Imaging) technology to obtain real-time images of the patient's body, and only perform interventional surgery based on the patient's body images. However, CT scanning will expose patients to high doses of radiation, and the long surgical process will cause great harm to the patient's body. In addition, CT has poor soft tissue imaging, which limits its application in many surgeries. Ultrasonic imaging depth is poor and the imaging quality is poor, which is not conducive to clear and accurate interventional surgery. Single MRI scanning is also limited to some surgical situations, and has poor real-time performance and poor navigation effect. In addition, the existing technologies mostly use two-dimensional ordinary cameras to capture surgical scenes, and the display effect is single.

[0004] Therefore, there is a need for an interventional puncture positioning system and method that can solve the problems of poor real-time effect, poor imaging effect and low accuracy in existing navigation interventional surgery. Summary of the invention

[0005] In order to solve the technical problems existing in the background technology, the present invention proposes an interventional puncture positioning method and system combining an MRI system and a depth camera.

[0006] The present invention proposes an interventional puncture positioning method combining an MRI system and a depth camera, comprising:

[0007] Acquire first MRI data of the surgical area of ​​the patient at a preset time interval, and obtain a reconstructed MRI image based on the first MRI data;

[0008] The first RGBD data of the puncture needle is acquired in real time, and the real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data.

[0009] Preferably, obtaining the real-time relative position of the front end of the puncture needle in the reconstructed MRI image according to the first RGBD data specifically includes:

[0010] Obtaining first real-time data of the tail end of the puncture needle according to the first RGBD data;

[0011] Obtaining second real-time data of the front end of the puncture needle according to the first real-time data of the rear end of the puncture needle and the pre-stored structural data of the puncture needle;

[0012] The puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle, so as to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0013] Preferably, the puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, specifically including:

[0014] A model of the puncture needle is simulated according to the pre-stored structural data of the puncture needle, and the model of the puncture needle is superimposed on the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0015] Preferably, superimposing the model of the puncture needle onto the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image specifically includes:

[0016] According to the reconstructed MRI image, the preliminary position of the front end of the puncture needle in the reconstructed MRI image is obtained;

[0017] The puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0018] Preferably, the model of the puncture needle is a three-dimensional model.

[0019] Preferably, the first MRI data is specifically undersampled MRI data.

[0020] Preferably, obtaining a reconstructed MRI image according to the first MRI data specifically includes:

[0021] Deep learning reconstruction is performed on the first MRI data as under-sampled MRI data to obtain a deep learning reconstructed MRI image.

[0022] Preferably, the real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data, and then the method further includes:

[0023] The real-time relative position of the front end of the puncture needle in the reconstructed MRI image is displayed.

[0024] Preferably, acquiring the first RGBD data of the puncture needle in real time specifically includes:

[0025] Acquire the first RGBD data of the patient's surgical area and puncture needle in real time.

[0026] Preferably, the first RGBD data of the patient's surgical area and the puncture needle is acquired in real time, and then the method further includes:

[0027] Performing three-dimensional reconstruction on the first RGBD data to obtain a three-dimensional point cloud model;

[0028] Simulate real-time surgical scenes based on the 3D point cloud model.

[0029] Preferably, a real-time surgical scene is simulated according to the three-dimensional point cloud model, and then the method further includes:

[0030] The real-time surgical scene is displayed.

[0031] Preferably, acquiring first MRI data of a surgical area of ​​a patient and obtaining a reconstructed MRI image according to the first MRI data further includes:

[0032] Acquire second MRI data of a magnetic resonance scanning area corresponding to a surgical area of ​​the patient and second RGBD data of a puncture needle installation area and a magnetic resonance scanning area corresponding to the puncture needle;

[0033] The second MRI data and the second RGBD data are registered.

[0034] Preferably, the second MRI data and the second RGBD data are registered, and then the process further comprises:

[0035] Acquire positioning MRI data of a magnetic resonance scanning area corresponding to a surgical area of ​​the patient;

[0036] Reconstructing the positioning MRI data to obtain a reconstructed positioning MRI image;

[0037] Determine whether the reconstructed positioning MRI image meets the preset imaging quality;

[0038] If not, the parameters of the magnetic resonance scan are adjusted;

[0039] If so, an interventional puncture procedure is performed.

[0040] The present invention also proposes an interventional puncture positioning system combining an MRI system and a depth camera, comprising:

[0041] A magnetic resonance scanning module, used for acquiring first MRI data of a surgical area of ​​a patient at a preset time interval;

[0042] An imaging module, configured to obtain a reconstructed MRI image based on the first MRI data;

[0043] A depth camera module, used to obtain the first RGBD data of the puncture needle in real time;

[0044] The positioning module is used to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image according to the first RGBD data.

[0045] Preferably, the positioning module includes:

[0046] A first processing submodule is used to obtain first real-time data of the tail end of the puncture needle according to the first RGBD data, and to obtain second real-time data of the front end of the puncture needle according to the first real-time data of the tail end of the puncture needle and pre-stored structural data of the puncture needle;

[0047] The second processing submodule is used to simulate the puncture needle and superimpose it into the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle, so as to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0048] Preferably, the second processing submodule is used to simulate and superimpose the puncture needle into the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, specifically including:

[0049] The second processing submodule is also used to simulate a model of the puncture needle according to the pre-stored structural data of the puncture needle, and superimpose the model of the puncture needle on the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0050] Preferably, superimposing the model of the puncture needle onto the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image specifically includes:

[0051] According to the reconstructed MRI image, the preliminary position of the front end of the puncture needle in the reconstructed MRI image is obtained;

[0052] The puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0053] Preferably, the model of the puncture needle is a three-dimensional model.

[0054] Preferably, the positioning module further comprises a display submodule, and the display submodule is used to display the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0055] Preferably, the first MRI data is specifically undersampled MRI data.

[0056] Preferably, the imaging module is used to obtain a reconstructed MRI image according to the first MRI data, and specifically includes:

[0057] The imaging module is used to perform deep learning reconstruction on the first MRI data as under-sampled MRI data to obtain a deep learning reconstructed MRI image.

[0058] Preferably, the depth camera module is used to obtain the first RGBD data of the puncture needle in real time, specifically including:

[0059] The depth camera module is used to obtain the first RGBD data of the patient's surgical area and puncture needle in real time.

[0060] Preferably, the positioning module further includes a three-dimensional reconstruction submodule, and the three-dimensional reconstruction submodule is used to perform three-dimensional reconstruction on the first RGBD data to obtain a three-dimensional point cloud model;

[0061] The second processing submodule is also used to simulate a real-time surgical scene based on the three-dimensional point cloud model.

[0062] Preferably, the display submodule is also used to display the real-time surgical scene.

[0063] Preferably, the magnetic resonance scanning module is further used to acquire second MRI data of a magnetic resonance scanning area corresponding to the surgical area of ​​the patient;

[0064] The depth camera module is also used to obtain second RGBD data of the puncture needle installation area and the magnetic resonance scanning area corresponding to the puncture needle;

[0065] It also includes a registration module, which is used to register the second MRI data and the second RGBD data.

[0066] Preferably, the magnetic resonance scanning module is further used to obtain positioning MRI data of the magnetic resonance scanning area corresponding to the surgical area of ​​the patient after the registration module registers the second MRI data and the second RGBD data;

[0067] The imaging module is also used to reconstruct the positioning MRI data to obtain a reconstructed positioning MRI image;

[0068] It also includes an imaging quality judgment module, which is used to judge whether the reconstructed positioning MRI image meets the preset imaging quality;

[0069] If not, adjusting the parameters of the magnetic resonance scanning module;

[0070] If so, an interventional puncture procedure is performed.

[0071] Preferably, the magnetic resonance scanning module includes a magnetic resonance imager, which includes a magnetic resonance bed and an open magnetic resonance coil for wrapping the patient's surgical area, and the open magnetic resonance coil is placed on the magnetic resonance bed; the depth camera module includes a depth camera installed on the shell of the magnetic resonance imager; the display submodule includes a display, a navigation handle bracket installed on the open magnetic resonance coil, and a navigation handle installed on the navigation handle bracket, and the navigation handle is used to install the puncture needle.

[0072] During the interventional puncture process, by acquiring the first RGBD data of the puncture needle and the first MRI data of the patient's surgical area, and obtaining the real-time relative position information of the lesion in the patient's surgical area and the front end of the puncture needle based on the first RGBD data and the first MRI data, it is convenient for subsequent doctors or surgical robots to perform interventional puncture based on the real-time relative position information.

[0073] Since MRI has no radiation, it is friendly to the patient's body and suitable for long-term interventional surgery. Its multi-parameter imaging is conducive to the analysis and surgery of different types of lesion areas, and the depth camera can accurately locate the second position information of the front end of the puncture needle in the patient's surgical area. In the present invention, the proposed interventional puncture positioning method and system combining the MRI system and the depth camera obtain the real-time relative position information of the lesion and the front end of the puncture needle in the patient's surgical area according to the first RGBD data and the first MRI data, so as to facilitate real-time navigation and guidance functions during the operation, so that the doctor or surgical robot can obtain accurate patient anatomical structure information at any time during the operation, and can more accurately locate and operate surgical tools, which is conducive to others to guide the operation and make timely adjustments, thereby improving the efficiency and success rate of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a schematic diagram of a flow chart of puncture needle positioning by combining an MRI system and a depth camera in one embodiment of the present invention.

[0075] Figure 2 It is a schematic diagram of the structure of an interventional puncture positioning system combining an MRI system and a depth camera in one embodiment of the present invention.

[0076] Figure 3 This is a schematic structural diagram of an open magnetic resonance coil and a navigation handle bracket in one embodiment of the present invention.

[0077] Figure 4 A schematic diagram of a usage scenario of a depth camera module in an embodiment of the present invention (the open magnetic resonance coil and the navigation handle bracket are not shown). DETAILED DESCRIPTION

[0078] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0079] First of all, you need to know that MRI stands for magnetic resonance imaging. Among them, a depth camera refers to a camera that uses depth sensing technology to capture three-dimensional data of an object or scene. The depth camera can measure the distance from the camera to each point in the scene, so that a three-dimensional model and reconstruction can be generated. The RGBD data (R: red, G: green, B: blue, D: depth) is generated by the RGB camera and depth sensor of the depth camera. Among them, the RGB image provides the color information of the scene, and the depth image provides the distance information from each pixel in the scene to the camera, that is, the depth information. The patient's surgical area refers to the area where the patient will undergo interventional puncture surgery, including the lesion and the area around the lesion.

[0080] Reference Figure 1 The present invention also proposes an interventional puncture positioning method combining an MRI system and a depth camera, comprising:

[0081] Acquire first MRI data of the surgical area of ​​the patient at a preset time interval, and obtain a reconstructed MRI image based on the first MRI data;

[0082] The first RGBD data of the puncture needle is acquired in real time, and the real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data.

[0083] During the interventional puncture process, the present invention obtains the first MRI data of the patient's surgical area at a preset interval, and obtains a reconstructed MRI image based on the first MRI data; and obtains the first RGBD data of the puncture needle in real time, and obtains the real-time relative position of the front end of the puncture needle in the reconstructed MRI image based on the first RGBD data.

[0084] Since MRI has no radiation, it is friendly to the patient's body and suitable for long-term interventional surgery. Its multi-parameter imaging is conducive to the analysis and surgery of different types of lesion areas, and the depth camera 3 can accurately locate the second position information of the front end of the puncture needle in the patient's surgical area. Therefore, the present invention is based on the combination of the MRI system and the depth camera 3, and can obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, so that the doctor or the surgical robot can obtain accurate patient anatomical structure information at any time during the operation, which is convenient for the doctor or the surgical robot to more accurately locate and operate the surgical tools during the continued puncture process, realizing real-time intraoperative navigation and intraoperative guidance functions, which is conducive to others to guide the operation and make timely adjustments, thereby improving the efficiency and success rate of the operation.

[0085] In order to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, in this embodiment, the real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data, specifically including:

[0086] Obtaining first real-time data of the tail end of the puncture needle according to the first RGBD data;

[0087] Obtaining second real-time data of the front end of the puncture needle according to the first real-time data of the rear end of the puncture needle and the pre-stored structural data of the puncture needle;

[0088] The puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle, so as to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0089] Among them, in this embodiment, the puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, specifically including: simulating a model of the puncture needle according to the pre-stored structural data of the puncture needle, and superimposing the model of the puncture needle on the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0090] The method of superimposing the model of the puncture needle onto the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image specifically includes:

[0091] According to the reconstructed MRI image, the preliminary position of the front end of the puncture needle in the reconstructed MRI image is obtained;

[0092] The puncture needle is simulated and superimposed on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0093] Since the lesion location and the preliminary position of the front end of the puncture needle can be determined based on the reconstructed MRI image, this embodiment can obtain the preliminary position of the front end of the puncture needle in the reconstructed MRI image based on the reconstructed MRI image, preliminarily position the puncture needle, and then simulate the puncture needle and superimpose it on the reconstructed MRI image based on the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image, so as to accurately obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0094] Among them, the model of the puncture needle in this embodiment is a three-dimensional model, which can be superimposed on the reconstructed MRI image, and the position and orientation of the front end of the puncture needle inside the patient's surgical area can be observed from multiple angles and dimensions, which is beneficial for the doctor to continue the subsequent puncture.

[0095] In another embodiment, the model of the puncture needle is a two-dimensional model.

[0096] When performing magnetic resonance scanning, if full sampling is used to acquire images, that is, the MRI system is allowed to sequentially acquire each pixel information of a specified section by row and column, it will result in a long acquisition time. In order to solve this problem, in this embodiment, a magnetic resonance undersampling scan is performed on the surgical area of ​​the patient during surgery to obtain undersampling MRI data, and the undersampling MRI data is used as the first MRI data, that is, the first MRI data in this embodiment is specifically undersampling MRI data.

[0097] Although the magnetic resonance undersampling scanning technology speeds up the sampling speed, the undersampling MRI data obtained by the magnetic resonance undersampling scanning is an aliasing artifact image. In order to be able to convert the undersampling aliasing artifact image into a fully sampled reconstructed image, in this embodiment, the reconstructed MRI image is obtained according to the first MRI data, specifically including: performing deep learning reconstruction on the first MRI data as the undersampling MRI data to obtain a deep learning reconstructed MRI image.

[0098] In this embodiment, the real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data, and then it also includes: displaying the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, so that the doctor can perform subsequent puncture according to the displayed real-time relative position of the front end of the puncture needle in the reconstructed MRI image during the puncture process, thereby improving the accuracy of the puncture.

[0099] In this embodiment, real-time acquisition of the first RGBD data of the puncture needle specifically includes: real-time acquisition of the first RGBD data of the patient's surgical area and the puncture needle. This embodiment is configured in this way to facilitate subsequent simulation of real-time surgical scenes based on the first RGBD data, and to obtain real-time relative position information from multiple dimensions and angles, which is conducive to doctors performing interventional punctures more accurately.

[0100] In this embodiment, first RGBD data of the patient's surgical area and puncture needle are acquired in real time, and then the following steps are included: three-dimensionally reconstructing the first RGBD data to obtain a three-dimensional point cloud model; and simulating a real-time surgical scene based on the three-dimensional point cloud model.

[0101] This embodiment is configured so that real-time relative position information can be obtained from multiple angles and dimensions, so that the doctor can observe the puncture needle during the interventional puncture process from multiple dimensions and angles, which is helpful for the doctor to perform interventional puncture more accurately.

[0102] In order to facilitate the remote end to clearly understand the surgical scene, in this embodiment, the real-time surgical scene is simulated based on the three-dimensional point cloud model, and then the real-time surgical scene is displayed.

[0103] In order to improve the accuracy of real-time relative position information during interventional puncture, alignment is required before interventional puncture. Therefore, in this embodiment, first MRI data of the patient's surgical area is obtained, and a reconstructed MRI image is obtained based on the first MRI data. Prior to this, it also includes: obtaining second MRI data of the magnetic resonance scanning area corresponding to the patient's surgical area and second RGBD data of the puncture needle installation area and the magnetic resonance scanning area corresponding to the puncture needle; and aligning the second MRI data and the second RGBD data.

[0104] However, in practical applications, the imaging quality of the reconstructed MRI image can easily affect the doctor's interpretation of the image and diagnosis of the lesion. In order to solve this problem, the second MRI data and the second RGBD data are registered, and then the following steps are included: obtaining the positioning MRI data of the magnetic resonance scanning area corresponding to the patient's surgical area; reconstructing the positioning MRI data to obtain a reconstructed positioning MRI image; judging whether the reconstructed positioning MRI image meets the preset imaging quality; if not, adjusting the parameters of the magnetic resonance scanning; if so, performing an interventional puncture surgery.

[0105] In other embodiments, intelligent mechanical mechanisms such as robots and robotic arms may be used to perform interventional puncture based on the navigation results.

[0106] It should be noted that before performing the registration, it is also necessary to prepare the interventional puncture tool and manually calibrate the position of the depth camera 3 used to obtain the first RGBD data and the second RGBD data so that it can capture the position of the patient's surgical area and identify the characteristic points (coordinate points) of the magnetic resonance scanner; and to position the patient and calibrate the magnetic field center frequency of the open magnetic resonance imager 1 to ensure that the scanning position encoding center is the magnetic field center.

[0107] like Figure 2-4 As shown, the present invention also proposes an interventional puncture positioning system combining an MRI system and a depth camera, comprising:

[0108] A magnetic resonance scanning module, used for acquiring first MRI data of a surgical area of ​​a patient at a preset time interval;

[0109] An imaging module, configured to obtain a reconstructed MRI image based on the first MRI data;

[0110] A depth camera module, used to obtain the first RGBD data of the puncture needle in real time;

[0111] The positioning module is used to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image according to the first RGBD data.

[0112] Since the depth camera 3 can take pictures at high speed and in real time, while the magnetic resonance scanning module can only scan at preset intervals and cannot perform magnetic resonance scanning in real time, the present invention obtains the first MRI data of the patient's surgical area at preset time intervals through the magnetic resonance scanning module, and obtains the reconstructed MRI image according to the first MRI data through the imaging module, and simultaneously obtains the first RGBD data of the puncture needle in real time through the depth camera module; and obtains the real-time relative position of the front end of the puncture needle in the reconstructed MRI image according to the first RGBD data through the positioning module, that is, reconstructs the real-time image of the front end of the puncture needle in the patient's body, realizes real-time navigation and intraoperative guidance functions, so that the doctor can obtain accurate patient anatomical structure information at any time during the operation, can more accurately locate and operate surgical tools, which is conducive to others to guide the operation and make timely adjustments, thereby improving the efficiency and success rate of the operation.

[0113] In order to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, the positioning module in this embodiment includes:

[0114] A first processing submodule is used to obtain first real-time data of the tail end of the puncture needle according to the first RGBD data, and to obtain second real-time data of the front end of the puncture needle according to the first real-time data of the tail end of the puncture needle and pre-stored structural data of the puncture needle;

[0115] The second processing submodule is used to simulate the puncture needle and superimpose it into the reconstructed MRI image according to the second real-time data and the pre-stored structural data of the puncture needle, so as to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0116] In this embodiment, the second processing submodule is used to simulate the puncture needle according to the second real-time data and the pre-stored structural data of the puncture needle and superimpose it into the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, specifically including: the second processing submodule is used to simulate the model of the puncture needle according to the pre-stored structural data of the puncture needle, and superimpose the model of the puncture needle into the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0117] Specifically, the model of the puncture needle in this embodiment is a three-dimensional model, which can be superimposed on the reconstructed MRI image, so as to facilitate the subsequent acquisition of the real-time relative position of the front end of the puncture needle in a three-dimensional state in the reconstructed MRI image from multiple angles and dimensions, thereby facilitating subsequent puncture.

[0118] It should be noted that since the reconstructed MRI image corresponds to the image of the patient's surgical area, what is shown in the reconstructed MRI image may only be the part of the puncture needle puncturing the patient's body. Similarly, the model simulated and superimposed on the reconstructed MRI image may only be the front end of the puncture needle.

[0119] Since the lesion location and the preliminary position of the front end of the puncture needle can be determined based on the reconstructed MRI image, in order to improve the accuracy of the obtained real-time relative position of the front end of the puncture needle in the reconstructed MRI image, in this embodiment, the model of the puncture needle is superimposed on the reconstructed MRI image according to the second real-time data to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image, which specifically includes: obtaining the preliminary position of the front end of the puncture needle in the reconstructed MRI image according to the reconstructed MRI image; simulating the puncture needle and superimposing it on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0120] During the doctor's manual puncture, in order to facilitate the doctor to clearly and intuitively understand the real-time relative position of the patient's surgical area and the front end of the puncture needle during the puncture process, so that the doctor can more accurately position and operate the surgical tools later, in this embodiment, the positioning module also includes a display submodule, which is used to display the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

[0121] In this embodiment, the first MRI data is specifically under-sampled MRI data.

[0122] The magnetic resonance scanning module in this embodiment performs a magnetic resonance under-sampling scan on the patient's surgical area to obtain under-sampling MRI data, and uses the under-sampling MRI data as the first MRI data, thereby solving the problem of long acquisition time caused by full sampling and effectively speeding up the sampling speed.

[0123] In this embodiment, the imaging module is used to obtain a reconstructed MRI image based on the first MRI data, and specifically includes: an imaging module, used to perform deep learning reconstruction on the first MRI data as undersampled MRI data to obtain a deep learning reconstructed MRI image.

[0124] This embodiment performs deep learning reconstruction on under-sampled MRI data, and can convert under-sampled aliasing artifact images into fully sampled reconstructed images, so as to obtain high-quality three-dimensional reconstructed images.

[0125] In this embodiment, the depth camera module is used to obtain the first RGBD data of the puncture needle in real time, specifically including: the depth camera module is used to obtain the first RGBD data of the patient's surgical area and the puncture needle in real time.

[0126] In this embodiment, the depth camera module is used to obtain the first RGBD data of the patient's surgical area and the puncture needle. Subsequently, the first RGBD data can be reconstructed in three dimensions through the three-dimensional reconstruction sub-module to obtain a three-dimensional point cloud model. The real-time surgical scene can be simulated according to the three-dimensional point cloud model, which facilitates the acquisition of real-time relative position information from multiple angles and dimensions, thereby performing interventional puncture more accurately.

[0127] In order to obtain real-time relative position information from multiple angles and dimensions, so that doctors can observe the puncture needle during the interventional puncture process from multiple dimensions and angles, which is conducive to doctors performing interventional puncture more accurately, in this embodiment, the positioning module also includes a three-dimensional reconstruction submodule, which is used to perform three-dimensional reconstruction on the first RGBD data to obtain a three-dimensional point cloud model; the second processing submodule is also used to simulate a real-time surgical scene based on the three-dimensional point cloud model.

[0128] In order to facilitate doctors during surgery or at remote ends to understand the surgical scene in a timely and intuitive manner, in this embodiment, the display submodule is also used to display the real-time surgical scene.

[0129] In this embodiment, the magnetic resonance scanning module is further used to obtain second MRI data of a magnetic resonance scanning area corresponding to the surgical area of ​​the patient;

[0130] The depth camera module is also used to obtain second RGBD data of the puncture needle installation area and the magnetic resonance scanning area corresponding to the puncture needle;

[0131] It also includes a registration module, which is used to register the second MRI data and the second RGBD data.

[0132] This embodiment can register the second MRI data and the second RGBD data before interventional puncture, which can improve the accuracy of real-time relative position information in the subsequent interventional puncture process.

[0133] In this embodiment, the magnetic resonance scanning module is further used to obtain positioning MRI data of the magnetic resonance scanning area corresponding to the patient's surgical area after the registration module registers the second MRI data and the second RGBD data;

[0134] The imaging module is also used to reconstruct the positioning MRI data to obtain a reconstructed positioning MRI image;

[0135] It also includes an imaging quality judgment module, which is used to judge whether the reconstructed positioning MRI image meets the preset imaging quality;

[0136] If not, adjust the parameters of the magnetic resonance scanning module; if so, perform interventional puncture surgery.

[0137] This configuration of the present embodiment can ensure that the reconstructed positioning MRI image meets the preset imaging quality, thereby preventing the imaging quality from affecting the doctor's interpretation of the image and diagnosis of the lesion.

[0138] In one specific embodiment, the magnetic resonance scanning module includes a magnetic resonance imager 2, which includes a magnetic resonance bed 6 and an open magnetic resonance coil 4 for wrapping the patient's surgical area, and the open magnetic resonance coil 4 is placed on the magnetic resonance bed 6; the depth camera module includes a depth camera 3 installed on the shell of the magnetic resonance imager 2; the display submodule includes a display 1, a navigation handle bracket 5 installed on the open magnetic resonance coil 4, and a navigation handle installed on the navigation handle bracket 5, and the navigation handle is used to install the puncture needle.

[0139] Specifically, a universal joint with a locking mechanism is connected between the navigation handles to facilitate a small universal rotation of the navigation handle to adjust the orientation of the puncture needle. The navigation handle bracket 5 can be installed on the open magnetic resonance coil 4 by means of a clamp, a bolt, or a screw hole.

[0140] The navigation handle is a hollow cylinder, the hollow part of which can hold the puncture needle and allow the puncture needle to move along its axial direction; coordinate marking points are arranged on the navigation handle, and patterns are arranged on the coordinate marking points, and the patterns are used as feature points for the depth camera 3 to identify.

[0141] Among them, the open magnetic resonance coil 4 in this embodiment is a cylindrical frame structure, the navigation handle bracket 5 is detachably installed on the open magnetic resonance coil 4, and the installation position of the navigation handle bracket 5 on the open magnetic resonance coil 4 is adjustable so that the navigation handle bracket 5 can be installed at any position of the open magnetic resonance coil 4.

[0142] It should be noted that the open magnetic resonance coil 4 in this embodiment can be used in joints, abdomen, head and other parts of the human body.

[0143] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An interventional puncture positioning method combining an MRI system and a depth camera, characterized in that: include: Acquire first MRI data of the surgical area of ​​the patient at a preset time interval, and obtain a reconstructed MRI image based on the first MRI data; Acquire the first RGBD data of the puncture needle in real time; Obtaining first real-time data of the tail end of the puncture needle according to the first RGBD data; Obtaining second real-time data of the front end of the puncture needle according to the first real-time data of the rear end of the puncture needle and the pre-stored structural data of the puncture needle; Simulating a model of the puncture needle according to the pre-stored structural data of the puncture needle; According to the reconstructed MRI image, the preliminary position of the front end of the puncture needle in the reconstructed MRI image is obtained; The model of the puncture needle is superimposed on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

2. The interventional puncture positioning method combining an MRI system and a depth camera according to claim 1, characterized in that: The first MRI data is specifically under-sampled MRI data; The step of obtaining a reconstructed MRI image according to the first MRI data specifically includes: Deep learning reconstruction is performed on the first MRI data as under-sampled MRI data to obtain a deep learning reconstructed MRI image.

3. The interventional puncture navigation method combining an MRI system and a depth camera according to claim 1, characterized in that: The real-time relative position of the front end of the puncture needle in the reconstructed MRI image is obtained according to the first RGBD data, and then the following steps are further included: The real-time relative position of the front end of the puncture needle in the reconstructed MRI image is displayed.

4. The interventional puncture positioning method combining an MRI system and a depth camera according to claim 3, characterized in that: Real-time acquisition of the first RGBD data of the puncture needle includes: Acquire the first RGBD data of the patient's surgical area and puncture needle in real time; Among them, the first RGBD data of the patient's surgical area and puncture needle is obtained in real time, and then it also includes: Performing three-dimensional reconstruction on the first RGBD data to obtain a three-dimensional point cloud model; Simulate real-time surgical scenes based on 3D point cloud models; The real-time surgical scene is displayed.

5. The interventional puncture positioning method combining an MRI system and a depth camera according to claim 1, characterized in that: Acquiring first MRI data of a surgical area of ​​a patient, and obtaining a reconstructed MRI image according to the first MRI data, wherein the process also includes: Acquire second MRI data of a magnetic resonance scanning area corresponding to a surgical area of ​​the patient and second RGBD data of a puncture needle installation area and a magnetic resonance scanning area corresponding to the puncture needle; The second MRI data and the second RGBD data are registered.

6. The interventional puncture positioning method combining an MRI system and a depth camera according to claim 5, characterized in that: The second MRI data and the second RGBD data are registered, and then: Acquire positioning MRI data of a magnetic resonance imaging region corresponding to a surgical region of the patient; Reconstructing the positioning MRI data to obtain a reconstructed positioning MRI image; Determine whether the reconstructed positioning MRI image meets the preset imaging quality; If not, the parameters of the magnetic resonance scan are adjusted; If so, an interventional puncture procedure is performed.

7. An interventional puncture positioning system combining an MRI system and a depth camera, characterized in that: include: A magnetic resonance scanning module, used for acquiring first MRI data of a surgical area of ​​a patient at a preset time interval; An imaging module, configured to obtain a reconstructed MRI image based on the first MRI data; A depth camera module, used to obtain the first RGBD data of the puncture needle in real time; A positioning module, used to obtain a real-time relative position of the front end of the puncture needle in the reconstructed MRI image according to the first RGBD data; Among them, the positioning module includes: A first processing submodule is used to obtain first real-time data of the tail end of the puncture needle according to the first RGBD data, and to obtain second real-time data of the front end of the puncture needle according to the first real-time data of the tail end of the puncture needle and pre-stored structural data of the puncture needle; The second processing submodule is used to simulate a model of the puncture needle according to the pre-stored structural data of the puncture needle; obtain a preliminary position of the front end of the puncture needle in the reconstructed MRI image according to the reconstructed MRI image; and superimpose the model of the puncture needle on the reconstructed MRI image according to the second real-time data and the preliminary position of the front end of the puncture needle in the reconstructed MRI image to obtain the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

8. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 7, characterized in that: The model of the puncture needle is a three-dimensional model.

9. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 7, characterized in that: The first MRI data is specifically under-sampled MRI data, and the imaging module is used to perform deep learning reconstruction on the first MRI data as the under-sampled MRI data to obtain a deep learning reconstructed MRI image.

10. The interventional puncture navigation system combining an MRI system and a depth camera according to claim 7, characterized in that: The navigation module also includes a display submodule, which is used to display the real-time relative position of the front end of the puncture needle in the reconstructed MRI image.

11. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 10, characterized in that: The depth camera module is used to obtain the first RGBD data of the puncture needle in real time, specifically including: A depth camera module, used to obtain the first RGBD data of the patient's surgical area and puncture needle in real time; The positioning module also includes a three-dimensional reconstruction submodule, which is used to perform three-dimensional reconstruction on the first RGBD data to obtain a three-dimensional point cloud model; The second processing submodule is also used to simulate a real-time surgical scene based on the three-dimensional point cloud model; The display submodule is also used to display the real-time surgical scene.

12. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 11, characterized in that: The magnetic resonance scanning module includes a magnetic resonance imager, which includes a magnetic resonance bed and an open magnetic resonance coil for wrapping the patient's surgical area, and the open magnetic resonance coil is placed on the magnetic resonance bed; the depth camera module includes a depth camera installed on the housing of the magnetic resonance imager; The display submodule includes a display, a navigation handle bracket installed on the open magnetic resonance coil, and a navigation handle installed on the navigation handle bracket, wherein the navigation handle is used to install the puncture needle.

13. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 7, characterized in that: The magnetic resonance scanning module is also used to obtain second MRI data of a magnetic resonance scanning area corresponding to the surgical area of ​​the patient; The depth camera module is also used to obtain second RGBD data of the puncture needle installation area and the magnetic resonance scanning area corresponding to the puncture needle; It also includes a registration module, which is used to register the second MRI data and the second RGBD data.

14. The interventional puncture positioning system combining an MRI system and a depth camera according to claim 13, characterized in that: The magnetic resonance scanning module is also used to obtain positioning MRI data of the magnetic resonance scanning area corresponding to the patient's surgical area after the registration module registers the second MRI data and the second RGBD data; The imaging module is also used to reconstruct the positioning MRI data to obtain a reconstructed positioning MRI image; It also includes an imaging quality judgment module, which is used to judge whether the reconstructed positioning MRI image meets the preset imaging quality; If not, adjusting the parameters of the magnetic resonance scanning module; If so, an interventional puncture procedure is performed.

Images