Interventional equipment positioning method and device, electronic equipment and storage medium

Abstract

The invention discloses an intervention equipment positioning method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first medical image of the in-vivo vasculature of a target patient, and generating a first vasculature model based on the first medical image, continuous multi-frame second medical images in the blood vessel are obtained through intervention equipment arranged in the blood vessel of the vasculature in the body of the target patient, a second vasculature model is generated based on the second medical images, registration processing is conducted on the first vasculature model and the second vasculature model, and a target registration matrix corresponding to the target patient is obtained; and acquiring a third medical image in the vasculature blood vessel in the target patient in real time through the interventional device, and mapping the third medical image to the first vasculature model based on the target registration matrix so as to display the position of the interventional device on the first vasculature model in real time. By adopting the method and the device, the purpose of accurately positioning the interventional equipment in the vasculature can be achieved without a lead plate operation room.

Description

Technical Field

[0001] The present invention relates to the field of medical assistance technologies, and particularly to a positioning method, device, electronic device, and storage medium for an interventional device. Background Art

[0002] Human aortic dissection is a disease that seriously endangers life safety. Once discovered, immediate treatment is required without delay. Some Stanford A dissections and all Stanford B dissections can be treated by minimally invasive interventional stent implantation. Another part of Stanford A dissections requires surgical thoracotomy for artificial blood vessel replacement followed by interventional surgery. In addition, the vast majority of aortic aneurysms and penetrating ulcers can also be effectively treated by interventional surgery, which has the advantages of small trauma and fast recovery. CTA (Computed Tomography Angiography) and MRI (Magnetic Resonance Imaging) are medical examination methods that can clearly display the human vascular system and are commonly used for preoperative examinations of vascular system diseases. However, both can only show the overall outline of the vascular system and cannot obtain specific lesion information within the vascular system. In order to obtain specific lesion information within the vascular system, intravascular ultrasound and OCT (Optical Coherence Tomography) examinations are needed. Among them, the two modalities (CTA and MRI, intravascular ultrasound and OCT) complement each other in terms of advantages and disadvantages but are independent of each other. Currently, they are only used for the judgment of the disease condition and cannot accurately locate the lesion site and guide the operation during the operation by combining with each other.

[0003] In the related art, in order to accurately locate the lesion site and perform interventional surgery, it is usually necessary to perform the surgery in a lead plate operating room that can prevent the penetration of rays, with the doctor wearing heavy lead clothing. During the surgery, the vascular stent is sent into the patient's vascular system with the assistance of a delivery system, and it is necessary to continuously inject contrast agent into the patient to display the position status of the blood vessels and the interventional devices implanted in the human body. At the same time, during and after the release of the vascular stent, multiple fluoroscopies are required to determine the position of the vascular stent and the release effect. However, multiple X-ray fluoroscopies are undoubtedly very unfavorable for the patients and the operating medical staff, and the toxicity and sensitization of the contrast agent to the human kidneys also increase the iatrogenic risk of the patients. In addition, the lead plate operating rooms are expensive and limited in number. Queuing up to use the lead plate operating rooms for interventional surgery makes many medical staff and patients wait until late at night, which is undoubtedly a great waste of human resources. At the same time, it also brings uncertain factors to the treatment of diseases. Sometimes, patients have anxiety during the waiting process, which may lead to the occurrence of malignant cardiovascular events. Further, since surgical operations mainly intervene in the lesion vascular regions near the heart, while interventional surgery treats the lesion vascular regions far from the heart. Therefore, if the timing of the completion of the surgical operation does not match the available time of the lead plate operating room, there will be a situation where the general anesthesia patient waits for the lead plate operating room while maintaining the anesthesia state, which brings a heavy burden to the patients, medical workers and the medical system.

[0004] Therefore, there is an urgent need for a new method that can accurately locate interventional devices in the vascular system without the need for a lead plate operating room. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a positioning method, device, electronic device and storage medium for interventional devices to solve the technical problems existing in the above-mentioned background technology.

[0006] In a first aspect, the embodiments of the present invention provide a positioning method for interventional devices, including:

[0007] Obtain a first medical image of the vascular system in the target patient's body, and generate a corresponding first vascular system model based on the first medical image, where the first medical image includes an angiography image and a magnetic resonance image;

[0008] Through an interventional device disposed in the blood vessels of the vascular system in the target patient's body, obtain multiple consecutive frames of second medical images in the blood vessels, and generate a three-dimensionally reconstructed second vascular system model based on the multiple consecutive frames of the second medical images, where the second medical images include intravascular ultrasound images and optical coherence tomography images;

[0009] Perform registration processing on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient;

[0010] The third medical image within the vasculature of the target patient's body is acquired in real time through the intervention device, and based on the target registration matrix, the third medical image is mapped onto the first vasculature model to display the position of the intervention device in real time on the first vasculature model.

[0011] In some embodiments, before the step of acquiring the first medical image of the vasculature of the target patient, the method further includes:

[0012] Acquiring first initial images of the vasculature of different patients, where the first initial images include angiography images and magnetic resonance images;

[0013] Performing rendering processing and segmentation processing on the first initial images to obtain multiple first initial three-dimensional vasculature models corresponding to different patients;

[0014] Using the first initial images as inputs and the first initial three-dimensional vasculature models corresponding to the first initial images as outputs, training the deep learning neural network model to be trained until convergence to obtain a trained first generation model;

[0015] The step of generating the corresponding first vasculature model based on the first medical image includes:

[0016] Invoking the trained first generation model to perform generation processing on the first medical image to generate a first vasculature model corresponding to the first medical image.

[0017] In some embodiments, before the step of acquiring consecutive multiple frames of second medical images within the vasculature by an intervention device disposed within the vasculature of the target patient, the method further includes:

[0018] In the manner of using an intervention device to enter the human vasculature, acquiring consecutive multiple frames of second initial images within the vasculature of different patients, where the second initial images include intravascular ultrasound images and optical coherence tomography images;

[0019] Based on the first position information of the intervention device in each frame of the second initial images and the second position information of the intervention device relative to the blood vessel inner wall, determining the three-dimensional coordinates of the vasculature in each frame of the second initial images;

[0020] Based on the three-dimensional coordinates of the vasculature in consecutive multiple frames of the second initial images, performing three-dimensional reconstruction on the vasculature of different patients to obtain second three-dimensional human vasculature models corresponding to different patients;

[0021] Using the consecutive multiple frames of the second initial images corresponding to different patients as inputs, and using the second initial three-dimensional vascular system models corresponding to the consecutive multiple frames of the second initial images as outputs, training the deep learning neural network model to be trained until convergence to obtain a trained second generation model;

[0022] The step of generating a three-dimensionally reconstructed second vascular system model based on the consecutive multiple frames of the second medical images includes:

[0023] Invoking the trained second generation model to perform generation processing on the consecutive multiple frames of the second medical images to generate a second vascular system model corresponding to the consecutive multiple frames of the second medical images.

[0024] In some embodiments, the registration process includes a rigid registration process. The step of performing a registration process on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient includes:

[0025] Performing rotation and translation operations on the second vascular system model to spatially align the second vascular system model with the first vascular system model;

[0026] Determining a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0027] In some embodiments, the registration process further includes a non-rigid registration process. The step of performing a registration process on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient includes:

[0028] Performing elastic transformation and thin plate spline operations on the second vascular system model to spatially align the second vascular system model with the first vascular system model;

[0029] Determining a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0030] In some embodiments, the interventional device includes a probe, a vascular stent, and a delivery rod. The probe is disposed at an end of the delivery rod, and the vascular stent is disposed around and closely attached to an outer sidewall of the delivery rod. The method further includes:

[0031] When the position of the probe displayed in real time on the first vascular system model is the target position, displaying a prompt message indicating arrival at the target position;

[0032] In response to a vascular stent release instruction, controlling the vascular stent to be released from the outer sidewall of the delivery rod and perform an expansion operation.

[0033] In some embodiments, before the step of, in response to a blood vessel stent release instruction, controlling the blood vessel stent to be released from the outer sidewall of the delivery rod and performing a dilation operation, the method further includes:

[0034] Obtaining an intravascular image captured by the probe;

[0035] When the position of the probe in the intravascular image is not the target position and a blood vessel stent release instruction is detected, stop executing the blood vessel stent release operation and display a prompt message indicating a position error.

[0036] In a second aspect, an embodiment of the present invention provides a positioning device for an interventional device, including:

[0037] A first acquisition module, configured to acquire a first medical image of a vascular system in a target patient and generate a corresponding first vascular system model based on the first medical image, where the first medical image includes an angiography image and a magnetic resonance image;

[0038] A second acquisition module, configured to, through an interventional device disposed in the blood vessels of the vascular system in the target patient, acquire a plurality of consecutive second medical images in the blood vessels and generate a three-dimensionally reconstructed second vascular system model based on the plurality of consecutive second medical images, where the second medical image includes an intravascular ultrasound image and an optical coherence tomography image;

[0039] A registration module, configured to perform a registration process on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient;

[0040] A positioning module, configured to, through the interventional device, acquire a third medical image in the blood vessels of the vascular system in the target patient in real time and map the third medical image onto the first vascular system model based on the target registration matrix to display the position of the interventional device on the first vascular system model in real time.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the positioning method of the interventional device described in any one of the above are implemented.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the positioning method of the interventional device described in any one of the above are implemented.

[0043] An embodiment of the present invention provides a positioning method, device, electronic device and storage medium for an interventional device. The method obtains a first medical image of the vascular system in a target patient, generates a corresponding first vascular system model based on the first medical image, and at the same time, obtains consecutive multiple frames of second medical images in the blood vessels of the vascular system in the target patient through the interventional device disposed in the blood vessels of the vascular system, and generates a second vascular system model after three-dimensional reconstruction based on the consecutive multiple frames of second medical images. Thus, by performing registration processing on the first vascular system model and the second vascular system model, a target registration matrix corresponding to the target patient can be obtained. Furthermore, when the interventional device obtains a third medical image in the blood vessels of the vascular system in the target patient in real time, based on the target registration matrix, the third medical image can be mapped onto the first vascular system model to display the position of the interventional device in real time on the first vascular system model, achieving the purpose of accurately positioning the interventional device in the vascular system without the need for a lead plate operating room. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 FIG. 6 is a schematic flowchart of a positioning method for an interventional device provided by an embodiment of the present invention;

[0045] Figure 2 FIG. 10 is a schematic flowchart of a training method for a first generation model provided by an embodiment of the present invention;

[0046] Figure 3 FIG. 14 is a schematic flowchart of a training method for a second generation model provided by an embodiment of the present invention;

[0047] Figure 4 FIG. 18 is a schematic diagram of a registration effect provided by an embodiment of the present invention;

[0048] Figure 5 FIG. 22 is a schematic diagram of an application scenario for obtaining a third medical image provided by an embodiment of the present invention;

[0049] Figure 6a FIG. 26 is a schematic diagram of a human aortic dissection provided by an embodiment of the present invention;

[0050] Figure 6b FIG. 30 is a schematic diagram of a vascular stent after expansion in a blood vessel provided by an embodiment of the present invention;

[0051] Figure 7 FIG. 34 is a schematic structural diagram of a positioning device for an interventional device provided by an embodiment of the present invention;

[0052] Figure 8 FIG. 38 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

[0053] Figure 9 FIG. 42 is another schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0056] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0057] In the related art, in order to accurately locate the lesion site and perform interventional surgery, it is usually necessary to perform the surgery in a lead plate operating room that can prevent radiation penetration, with the doctor wearing a heavy lead suit. During the operation, the vascular stent is delivered into the patient's vascular system with the assistance of a delivery system, and the patient needs to be continuously injected with contrast agents to display the position and status of the blood vessels and the interventional devices implanted in the human body. At the same time, the vascular stent requires multiple fluoroscopy during and after the release process to determine the position and release effect of the vascular stent. Multiple X-ray fluoroscopy is undoubtedly very unfavorable for patients and operating medical staff, and the toxicity and sensitization of contrast agents to human kidneys also increase the iatrogenic risk of patients. In addition, the lead plate operating room is expensive and limited in number. Queuing to use the lead plate operating room for interventional surgery makes many medical staff and patients wait until late at night, which is undoubtedly a huge waste of human resources. It also brings uncertainty to the treatment of the disease. Sometimes patients become anxious during the waiting process, which leads to the occurrence of malignant cardiovascular events. Furthermore, since surgical operations mainly intervene in the diseased vascular area at the proximal end, interventional surgery treats the diseased vascular area at the distal end. Therefore, if the timing of completing the surgical operation and the time for the lead plate operating room to be vacated do not match, patients under general anesthesia will have to maintain anesthesia while waiting for the lead plate operating room, which will impose a heavy burden on patients, medical workers and the medical system.

[0058] Therefore, there is an urgent need for a new method to accurately position interventional devices within the vascular system without the need for a lead-plate operating room.

[0059] To solve the technical problems existing in the related art, an embodiment of the present invention provides a positioning method for an interventional device. Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a positioning method for an interventional device provided by an embodiment of the present invention. The method includes steps 101 to 104;

[0060] Step 101, obtain a first medical image of the vascular system in a target patient, and generate a corresponding first vascular system model based on the first medical image.

[0061] In this embodiment, the first medical image provided in this embodiment may include an angiography image and a magnetic resonance image. Among them, angiography refers to spiral CT scanning after intravenous injection of a contrast agent. When performing three-dimensional reconstruction, structures such as skin, muscle, and bone that do not need to be displayed are removed, and only three-dimensional vascular structures and visceral structures are displayed. Magnetic resonance uses the principle of nuclear magnetic resonance (NMR). Based on the different attenuation of the released energy in different structural environments within a substance, by applying an external gradient magnetic field to detect the emitted electromagnetic waves, the position and type of the atomic nuclei that make up this object can be known, and thus the internal structure of the object can be mapped. Therefore, the first medical image provided in this embodiment refers to a 3D image that can display the three-dimensional vascular structure of the vascular system in a target patient.

[0062] After obtaining the angiography image and magnetic resonance image of the vascular system in the target patient, since the angiography image and magnetic resonance image not only show the three-dimensional vascular structure but also the image content of the visceral structure, therefore, in this embodiment, it is also necessary to preprocess the angiography image and magnetic resonance image to obtain a vascular structure image that only belongs to the human vascular system, so as to generate a corresponding three-dimensional first vascular system model.

[0063] Specifically, the preprocessing provided in this embodiment may include rendering processing, denoising and smoothing processing, contrast enhancement processing, standardization and normalization, and image segmentation processing, so as to accurately remove the vascular structures that do not belong to the human vascular system and generate a three-dimensional first vascular system model.

[0064] In some embodiments, in order to improve the generation efficiency of the three-dimensional first vascular system model, before generating the first vascular system model, this embodiment may also pre-train a first generation model for automatically generating a corresponding first vascular system model according to the input first medical image. Specifically, please refer to Figure 2 , Figure 2 which is a schematic flowchart of a training method for the first generation model provided by an embodiment of the present invention. As Figure 2As shown, the training method of the first generation model provided in this embodiment includes steps 201 to 203;

[0065] Step 201, obtain the first initial images of the vascular systems in different patients.

[0066] Among them, the first initial images provided in this embodiment include angiography images and magnetic resonance images. In order to complete the training of the model, this embodiment needs to obtain a large number of first initial images of the vascular systems in different patients.

[0067] Step 202, perform rendering processing and segmentation processing on the first initial images to obtain multiple first initial three-dimensional vascular system models corresponding to different patients.

[0068] In this embodiment, since angiography images and magnetic resonance images, that is, the first initial images, not only show the three-dimensional vascular structures but also show the image contents of visceral structures, this embodiment needs to perform rendering processing and segmentation processing on the first initial images to remove the vascular structures that do not belong to the human vascular system and generate three-dimensional first initial three-dimensional vascular system models.

[0069] Specifically, before performing the segmentation processing operation on the first initial images, this embodiment can also perform operations such as denoising and smoothing processing, contrast enhancement processing, standardization and normalization, etc., which can improve the segmentation accuracy and segmentation efficiency of the segmentation processing, so as to accurately remove the vascular structures that do not belong to the human vascular system and generate three-dimensional first initial three-dimensional vascular system models.

[0070] Step 203, use the first initial images as the input and the first initial three-dimensional vascular system models corresponding to the first initial images as the output, and train the deep learning neural network model to be trained until convergence to obtain the trained first generation model.

[0071] In this embodiment, this embodiment mainly uses the first initial images and the corresponding first initial three-dimensional vascular system models as training data, and constructs a training data set and a validation data set according to different ratios. Among them, the ratio of the training data in the training data set and the validation data set can be preset, such as ratios of 8:2 or 9:1, etc. In addition, the deep learning neural network model provided in this embodiment can be any convolutional neural network model that can implement image segmentation functions, and no specific limitation is made here.

[0072] In this way, by training and validating the deep learning neural network model to be trained through the constructed training data set and validation data set, a first generation model that can generate corresponding three-dimensional human vascular system models according to the first initial images can be established.

[0073] Specifically, in one case, the way to train the deep learning neural network model to be trained until convergence can be that the number of training times reaches a preset number of training times, that is: input the first initial image into the deep learning neural network model to be trained for generation processing to obtain a generation result; use a preset loss function to calculate the loss value of the first initial three-dimensional vascular system model corresponding to the input first initial image relative to the generation result, and fine-tune the model parameters of the deep learning neural network model to be trained according to the loss value; continuously use the training data set to train the deep learning neural network model to be trained and fine-tune the model parameters until the calculated number of training times reaches the preset number of training times, such as 50,000 times or 100,000 times, then the model convergence can be determined.

[0074] In another case, the way to train the deep learning neural network model to be trained until convergence can also be that the accuracy rate of the generation result of the model reaches a preset accuracy rate threshold, that is: input the first initial image into the deep learning neural network model to be trained for generation processing to obtain a generation result; use a preset loss function to calculate the loss value of the first initial three-dimensional vascular system model corresponding to the input first initial image relative to the generation result, and fine-tune the model parameters of the deep learning neural network model to be trained according to the loss value; continuously use the training data set to train the deep learning neural network model to be trained and fine-tune the model parameters until the accuracy rate of the generation result generated by the deep learning neural network model to be trained reaches the preset accuracy rate threshold, such as 95% or 98%, then the model convergence can be determined.

[0075] Only two of the model convergence methods are described above, and other methods for determining model convergence can also be used, which are not specifically limited here.

[0076] After obtaining the trained first generation model, the step of generating the corresponding first vascular system model based on the first medical image provided in this embodiment can specifically be: call the trained first generation model to perform generation processing on the first medical image to generate a first vascular system model corresponding to the first medical image. In this way, the generation efficiency of generating the first vascular system model can be effectively improved.

[0077] Step 102, through an intervention device arranged in the blood vessels of the vascular system in the target patient, obtain consecutive multiple frames of second medical images in the blood vessels, and generate a three-dimensionally reconstructed second vascular system model based on the consecutive multiple frames of the second medical images.

[0078] In this embodiment, the second medical image provided in this embodiment may include an intravascular ultrasound image and an optical coherence tomography image. Among them, intravascular ultrasound (IVUS) refers to a technique that uses catheter technology to send a micro ultrasound probe into the blood vessel lumen to display the cross-sectional image of the blood vessel, thereby providing in-vivo images of the blood vessel lumen. Optical Coherence Tomography (OCT) is a non-contact and non-invasive optical imaging technique that uses the principle of light interference to perform high-resolution tomographic imaging of biological tissues. Therefore, the second medical image provided in this embodiment also refers to a 3D image that can display the three-dimensional vascular structure of the vascular system in the target patient's body.

[0079] It should be noted that, compared with the first medical image, the second medical image needs to send a micro probe into the blood vessel lumen to obtain, while the first medical image can be obtained without sending a micro probe into the blood vessel lumen; and precisely because the second medical image can only be obtained by sending a micro probe into the blood vessel lumen, therefore, the second medical image may also contain the position information of the micro probe and the relative position information between the micro probe and the inner wall of the blood vessel lumen, so that the three-dimensional coordinates of the vascular system can be determined based on these position information, which is convenient for three-dimensional reconstruction based on the three-dimensional coordinates to generate the second vascular system model corresponding to the second medical image. Further, by using a micro probe to obtain ultrasound images or OCT images in the blood vessel lumen, it can not only avoid the renal toxicity of the contrast agent and the increased iatrogenic risk to the patient, but also avoid the radiation hazards of X-ray fluoroscopy to the patient and medical staff, thus effectively improving the safety of interventional surgery.

[0080] Specifically, the second vascular system model provided in this embodiment can be obtained by three-dimensional reconstruction based on all the three-dimensional coordinates of the vascular system, or by three-dimensional reconstruction based on the three-dimensional coordinates of several key points of the vascular system. Which specific key point three-dimensional coordinates are selected for three-dimensional reconstruction can be set according to actual application requirements, and no specific limitation is made here.

[0081] Exemplarily, in this embodiment, the micro probe can be introduced into the human vascular system through the femoral artery, and then intravascular ultrasound or OCT examination is performed to obtain ultrasound or OCT images. Then, the three-dimensional coordinates of the four vertices in each frame of ultrasound sequence image or each frame of OCT sequence image are extracted by the micro probe, and the three-dimensional model data is reconstructed through a preset three-dimensional reconstruction algorithm. It should be noted that the ultrasound or OCT sequence image is a group of ultrasound or OCT sequence images collected by a continuous micro probe. An electromagnetic sensor can be provided on the micro probe, and the electromagnetic sensor can be connected to an electromagnetic positioning system for obtaining the three-dimensional coordinates of the vertices of the ultrasound sequence image or OCT sequence image.

[0082] In some embodiments, also for the purpose of improving the generation efficiency of the three-dimensional second vascular system model, before generating the second vascular system model, this embodiment can also pre-train a second generation model for automatically generating the corresponding second vascular system model according to the input second medical image. Specifically, please refer to Figure 3 , Figure 3 which is a schematic flowchart of a training method of the second generation model provided by an embodiment of the present invention. As Figure 3 shown, the training method of the second generation model provided by this embodiment includes steps 301 to 304;

[0083] Step 301: Obtain a continuous multi-frame second initial image inside the blood vessels of the vascular system in different patients by means of an interventional device entering the human vascular system.

[0084] Among them, the second initial image provided by this embodiment includes intravascular ultrasound images and optical coherence tomography images. The interventional device provided by this embodiment can be a microprobe. Specifically, the microprobe can be sent into the blood vessel lumen through catheter technology to perform intravascular ultrasound or OCT examinations through the microprobe in the blood vessel lumen, so as to obtain ultrasound or OCT images, that is, the second initial images.

[0085] Step 302: Determine the three-dimensional coordinates of the vascular system in each frame of the second initial image according to the first position information of the interventional device in each frame of the second initial image and the second position information of the interventional device relative to the blood vessel inner wall.

[0086] In this embodiment, since the interventional device provided by this embodiment is a microprobe, therefore, in each frame of the second initial image collected by the interventional device, the relative second position information between the inner wall of the blood vessel lumen and the microprobe in each frame of the second initial image can be determined. Then, the starting position of the interventional device entering the blood vessel lumen can be used as the origin to construct a three-dimensional coordinate system. Thus, according to the first position information of the interventional device in each frame of the second initial image (which can be calculated according to the moving distance of the interventional device or the relative distance from the origin), and the relative second position information between the inner wall of the blood vessel lumen and the microprobe in each frame of the second initial image, the three-dimensional coordinates of the vascular system in different second initial images can be calculated.

[0087] Step 303: Based on the three-dimensional coordinates of the vascular system in the continuous multi-frame second initial images, perform three-dimensional reconstruction on the vascular systems in different patients to obtain the corresponding second three-dimensional human vascular system models of different patients.

[0088] In this embodiment, after obtaining the three-dimensional coordinates of the vascular system in the second initial image of each frame, the three-dimensional coordinates of the vascular system in the target patient's body can be determined. Then, a three-dimensional reconstruction algorithm is used to perform three-dimensional reconstruction processing on these three-dimensional coordinates to obtain the corresponding second three-dimensional human vascular system model.

[0089] Step 304: Use the consecutive multiple frames of the second initial images corresponding to different patients as the input, and use the corresponding second initial three-dimensional vascular system models of the consecutive multiple frames of the second initial images as the output to train the deep learning neural network model to be trained until convergence, obtaining the trained second generation model.

[0090] In this embodiment, this embodiment mainly uses the consecutive multiple frames of the second initial images and the corresponding second initial three-dimensional vascular system models as training data, and constructs a training data set and a validation data set according to different ratios. Among them, the ratio of the proportion of the training data in the training data set and the validation data set can be preset, such as ratios of 8:2 or 9:1, etc. In addition, the deep learning neural network model provided in this embodiment can be any convolutional neural network model that can implement the three-dimensional reconstruction function, and no specific limitation is made here.

[0091] In this way, by using the constructed training data set and validation data set to train and validate the deep learning neural network model to be trained, a second generation model that can generate the corresponding three-dimensional human vascular system model according to the consecutive multiple frames of the second initial images can be established.

[0092] Specifically, in one case, the method of training the deep learning neural network model to be trained until convergence can be that the number of training times reaches a preset number of training times, that is: input the consecutive multiple frames of the second initial images into the deep learning neural network model to be trained for generation processing to obtain a generation result; use a preset loss function to calculate the loss value of the corresponding second initial three-dimensional vascular system model of the input consecutive multiple frames of the second initial images relative to the generation result, and fine-tune the model parameters of the deep learning neural network model to be trained according to the loss value; continuously use the training data set to train the deep learning neural network model to be trained and fine-tune the model parameters until the calculated number of training times reaches the preset number of training times, such as 50,000 times or 100,000 times, then the model convergence can be determined.

[0093] In another case, the way to train the deep learning neural network model to be trained until convergence can also be that the accuracy of the generated result of the model reaches a preset accuracy threshold, that is: inputting a continuous plurality of frames of the second initial image into the deep learning neural network model to be trained for generation processing to obtain a generated result; using a preset loss function to calculate the loss value of the second initial three-dimensional vascular system model corresponding to the input continuous plurality of frames of the second initial image relative to the generated result, and fine-tuning the model parameters of the deep learning neural network model to be trained according to the loss value; continuously using the training data set to train the deep learning neural network model to be trained and fine-tuning the model parameters until the accuracy of the generated result generated by the deep learning neural network model to be trained reaches a preset accuracy threshold, such as 95% or 98%, then the model convergence can be determined.

[0094] Only two ways of the model convergence method are recorded above. Other methods for determining model convergence can also be used, and no specific limitation is made here.

[0095] After obtaining the trained second generation model, the steps of generating the three-dimensional reconstructed second vascular system model based on the continuous plurality of frames of the second medical image provided in this embodiment can specifically be: calling the trained second generation model to perform generation processing on the continuous plurality of frames of the second medical image to generate a second vascular system model corresponding to the continuous plurality of frames of the second medical image. In this way, the generation efficiency of generating the second vascular system model can be effectively improved.

[0096] Among them, the deep learning neural network model to be trained used in the process of training the first generation model and the second generation model in this embodiment can be a 3D U-Net model, or other neural network models capable of generating 3D models, and no specific limitation is made here.

[0097] It should be noted that steps 101 to the embodiment including step 102 provided in this embodiment are steps to be executed before the interventional surgery, and step 103 and the embodiments after step 103 are steps to be executed during the interventional surgery.

[0098] At this point, after obtaining the trained first generation model and the second generation model, the first medical image (angiography image or magnetic resonance image) of the target patient's internal vascular system can be input into the trained first generation model in the application scenario, and the second medical image (intravascular ultrasound image or optical coherence tomography image) of the target patient's internal vascular system can be input into the trained second generation model, so that the accurate first vascular system model and the second vascular system model can be quickly obtained, which is convenient for performing subsequent registration processing, and during the interventional surgery, the position of the interventional device can be mapped to the first vascular system model in real time, providing the clinician with navigation information of the interventional device in the target patient's vascular system, so that the clinician can accurately locate the lesion site and guide the interventional surgery, thereby breaking the rigid condition barrier that the current use of interventional equipment must be carried out under the protection of lead plates, so that interventional surgery can be performed in ordinary operating rooms without the need for fluoroscopy, and there is no need to queue up for the extremely small and expensive lead plate operating rooms, which reduces the burden on patients, medical workers and the medical system and improves the efficiency of interventional surgery.

[0099] Step 103: performing registration processing on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient.

[0100] In order to display the navigation information of the interventional device in the blood vessel in real time on the first vascular system model, this embodiment needs to accurately fuse the second vascular system model with the first vascular system model so that the positioning information displayed on the second vascular system model can be equivalent to that displayed on the first vascular system model. Specifically, this embodiment can perform registration processing on the first vascular system model and the second vascular system model so that the first vascular system model and the second vascular system model are integrated, thereby accurately mapping the position of any key point on the second vascular system model to the first vascular system model.

[0101] The registration process provided in this embodiment may include steps such as feature extraction, feature matching, transformation model estimation, image resampling and optimization, and is specifically implemented as follows:

[0102] Step 1: Feature extraction, the purpose is to extract key points or regions that can be used for registration from the image or point cloud. Common feature extraction methods include but are not limited to corner detection, edge detection, region detection, etc. By performing feature extraction on the first vascular system model and the second vascular system model respectively, corresponding feature points or feature descriptors are obtained.

[0103] Step 2: Feature matching. The purpose is to match the feature points or feature descriptors in the model to be registered and the target model, and find the corresponding point pairs. Common methods include but are not limited to nearest neighbor matching, RANDSAC (Random Sample Consensus), etc. By performing feature matching on the two sets of feature points or feature descriptors obtained in Step 1, a set of matching point pairs is obtained.

[0104] Step 3: Transformation model estimation. The purpose is to estimate the spatial transformation model required to align the model to be registered to the target model based on the matching point pairs. Common transformation models include but are not limited to rigid transformation, affine transformation, and non-rigid transformation. Common estimation methods include but are not limited to the least squares method, iterative optimization, etc. By performing transformation model estimation on the matching point pairs obtained in Step 2, the transformation model required to align the second vasculature model to the first vasculature model can be output.

[0105] Step 4: Image resampling. The purpose is to resample the model to be registered according to the estimated transformation model to generate an aligned model. Common interpolation methods include but are not limited to nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. By using the transformation model output in Step 3, the second vasculature model is resampled to output a new model aligned to the first vasculature model.

[0106] Step 5: Optimization. The purpose is to further improve the registration accuracy by iteratively optimizing the transformation model. Common optimization methods include but are not limited to the gradient descent method, genetic algorithm, similarity metric algorithm, etc. Similarity metrics include but are not limited to the following methods:

[0107] Mean Squared Error (MSE): Applicable to strongly similar images, where I ref , I float are the intensities of the model to be registered and the target model respectively, and (x i , y i ) and (x i ′, y i ′) are the corresponding point coordinates.

[0108] Mutual Information (MI): Applicable to multimodal image registration, where p(a, b) is the joint probability, and p ref (a), p float (b) are the marginal probability distributions.

[0109] Through the optimization method, a highly accurate transformation model is obtained, and this transformation model is the registration matrix for spatially aligning the second vasculature model to the first vasculature model. The specific registration process is as Figure 4 shown, Figure 4It is a schematic diagram of the registration effect provided by an embodiment of the present invention. Among them, the model to be registered is the second vascular system model, the target model is the first vascular system model, and the result after registration is to convert the model to be registered into the spatial coordinate system of the target model, which is the target model.

[0110] In one implementation, the registration process provided in this embodiment may include a rigid registration process. Specifically, the step of registering the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient provided in this embodiment may specifically be: performing rotation and translation operations on the second vascular system model to align the second vascular system model with the first vascular system model in space; determining a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0111] Among them, the rigid transformation can be defined as: p′ = R * p + t, where p is the point coordinate in the second vascular system model, p′ is the transformed point coordinate, R is the rotation matrix, and t is the translation vector.

[0112] In this way, the rigid registration process can achieve spatial alignment between different modality images, thereby ensuring the accuracy of mapping the position of any key point on the second vascular system model to the first vascular system model, and further providing accurate navigation and positioning information for clinicians for the interventional device within the vascular system.

[0113] In another implementation, the registration process provided in this embodiment may further include a non-rigid registration process. The step of registering the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient may specifically be: performing elastic transformation and thin plate spline operations on the second vascular system model to align the second vascular system model with the first vascular system model in space; determining a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0114] Among them, the non-rigid transformation is divided into B-spline transformation and thin plate spline (TPS). The B-spline transformation formula is:

[0115]

[0116] Among them, ci,j,k is the control point displacement, and β i (u), β j (v), β k (w) are B-spline basis functions.

[0117] The thin plate spline formula is:

[0118]

[0119] Among them, c i is a control point, w i is a weight, and Φ(r) is a radial basis function.

[0120] In this way, the use of non-rigid registration processing can solve the problem of low registration accuracy caused by tracheal movement, deformation, etc., thereby further ensuring the accuracy of mapping the position of any key point on the second vascular system model to the first vascular system model, and effectively improving the accuracy of providing navigation and positioning information of the intervention device in the first vascular system model for clinicians.

[0121] In addition, the registration processing provided in this embodiment may further include affine transformation registration processing. Specifically, the affine transformation provided in this embodiment is a three-dimensional affine transformation, and its formula is:

[0122]

[0123] Among them, a ij (i, j ∈ (1~3)) are the elements of the linear transformation matrix, and t x , t y , t z is the translation amount.

[0124] Step 104, the third medical image in the vascular system of the target patient is obtained in real time through the intervention device, and based on the target registration matrix, the third medical image is mapped onto the first vascular system model to display the position of the intervention device on the first vascular system model in real time.

[0125] Among them, both the third medical image and the second medical image provided in this embodiment are intravascular ultrasound images and optical coherence tomography images of multiple consecutive frames in the blood vessel collected through an intravascular intervention device. Specifically, the second medical image is mainly collected through the intervention device before the interventional surgery; the third medical image is mainly collected in real time through the intervention device during the access surgery.

[0126] After obtaining the target registration matrix through the registration process, the positioning information of any point in the third medical image obtained in real time can be mapped onto the first vascular system model by means of the target registration matrix. Specifically, in this embodiment, the positioning information of the intervention device in the third medical image is mainly mapped onto the first vascular system model, so that the accurate positioning information of the intervention device can be displayed in real time on the first vascular system model, providing navigation information of the intervention device in the blood vessels of the target patient for clinicians, and thus facilitating clinicians to accurately locate the lesion site and guide the intervention operation. In this way, the rigid condition barrier that the current use of intervention devices must be carried out under the protection of lead plates is broken, enabling the intervention operation to be carried out in an ordinary operating room without the need for fluoroscopy, without having to queue for the extremely few and costly lead plate operating rooms, reducing the burden on patients, medical workers and the medical system, and improving the efficiency of the intervention operation.

[0127] Specifically, the process of obtaining the third medical image in this embodiment can be as Figure 5 shown. Figure 5 FIG. is a schematic diagram of an application scenario for obtaining the third medical image provided by an embodiment of the present invention. Specifically, in this embodiment, a positioning device can be assembled outside the patient. The positioning device is connected to a sensor through a connecting line. The sensor is fixed on the probe of the intervention device. The third medical image is obtained through the probe to obtain the position of the probe of the intervention device in real time in the coordinate system of the positioning device: p′ = M1·M2·p, where p is the coordinate position of the probe of the intervention device in the image, M1 is the transformation matrix from the image to the sensor coordinate system, and M2 is the transformation matrix from the sensor coordinate system to the positioning device coordinate system. The coordinates of the third medical image in the coordinate system of the positioning device can be obtained through the above transformation.

[0128] In some embodiments, in an actual application scenario, this embodiment can not only display the navigation and positioning information of the interventional device in the blood vessel in real time, but also control the interventional device to achieve different functions. Specifically, the interventional device provided in this embodiment may include a probe, a vascular stent, and a delivery rod. The probe is disposed at the end of the delivery rod, and the vascular stent is disposed around and closely attached to the outer wall of the delivery rod. Among them, the probe provided in this embodiment may be a microprobe that can be delivered into the blood vessel lumen; the vascular stent may be a cylindrical and hollow mesh cover with an elastic deformation function (expandable and contractible), and the mesh cover of the vascular stent has the characteristic of isolating the passage of blood, so as to ensure that blood can only pass through the hollow position of the mesh cover. In addition, the length of the vascular stent can be custom-set according to the length of the diseased area in the blood vessel; the delivery rod may be a microcatheter that can be delivered into the blood vessel lumen. The delivery rod is designed with a hollow structure. A data transmission line connected to the probe may be disposed in the hollow structure of the delivery rod. One end of the data transmission line is connected to the probe, and the other end may be connected to a terminal device such as a PC. The PC has a display device for displaying a first vascular system model and for real-time displaying the position of the probe on the first vascular system model. At the same time, a micro motor may also be disposed in the hollow structure of the delivery rod. The micro motor is used to control the bending angle of the end of the delivery rod, so as to achieve the delivery of the end of the delivery rod in different angles / directions and avoid the delivery rod touching and damaging the inner wall of the blood vessel of the vascular system.

[0129] Specifically, the positioning method of the interventional device provided in this embodiment may further include: when the position of the probe displayed in real time on the first vascular system model is the target position, displaying a prompt message for reaching the target position; in response to a vascular stent release instruction, controlling the vascular stent to be released from the outer wall of the delivery rod and performing an expansion operation.

[0130] Among them, the target position may be the proximal position of the intravascular lesion area. The length of the vascular stent may be set by pre-calculating the distance between the proximal position and the distal position of the intravascular lesion area. For example, scale lines may be set on the delivery rod, and then the position reached by the probe of the interventional device may be recorded when the second medical image of the blood vessel is collected by the interventional device before the interventional operation (when the second medical image of the blood vessel is collected by the interventional device, the interventional device only includes the probe and the delivery rod, and at this time, no vascular stent is set on the delivery rod). Specifically: when the probe reaches the distal end of the intravascular lesion area, record the scale D1 corresponding to the delivery rod; when the probe reaches the proximal end of the intravascular lesion area, record the scale D2 corresponding to the delivery rod. Then, the length of D2 - D1 can be determined as the blood vessel length of the intravascular lesion area. At this time, a vascular stent with a corresponding length (D2 - D1) can be selected and set on the delivery rod. Finally, during the interventional operation, the delivery rod provided with the vascular stent and the probe is delivered into the blood vessel cavity of the human vascular system of the target patient.

[0131] It should be noted that the proximal position and the distal position of the intravascular lesion area can be determined by collecting the second medical image of the blood vessel through the interventional device before the interventional operation, that is, step 102. Specifically, please refer to Figure 6a , Figure 6a which is a schematic diagram of a human aortic dissection provided by an embodiment of the present invention. As Figure 6a shown, point A in the figure is the proximal position of the intravascular lesion area, point B is the distal position of the intravascular lesion area, and the length AB between points A and B is the length of the intravascular lesion area and also the shortest length of the vascular stent, that is, the length between the blood vessels needs to be at least greater than the length of AB.

[0132] When the position of the probe shown in real time on the first medical image is the target position, a vascular stent release instruction can be used to control the vascular stent to be released from the outer wall of the delivery rod and perform an expansion operation. After being released, the vascular stent performs an expansion operation and expands closely against the inner wall of the blood vessel, so that the blood vessel wall of the blood vessel lesion area with a human aortic dissection becomes in a blocked state. The final effect is as Figure 6b shown, Figure 6b which is a schematic diagram of the vascular stent after expansion in the blood vessel provided by an embodiment of the present invention.

[0133] In some other embodiments, in order to improve the accuracy of positioning of the interventional device, before the step of, in response to a blood vessel stent release instruction, controlling the blood vessel stent to be released from the outer sidewall of the delivery rod and performing a dilation operation, the positioning method of the interventional device provided in this embodiment may further include: acquiring an intravascular image captured by the probe; when the position of the probe in the intravascular image is not the target position and a blood vessel stent release instruction is detected, stopping the execution of the blood vessel stent release operation and displaying a prompt message indicating a position error.

[0134] In this way, in the case of double verification (verification of the navigation and positioning information of the probe in the blood vessel in the first medical image displayed in real time, and verification of the intravascular imaging image), the accuracy of positioning of the interventional device in the blood vessel can be effectively guaranteed.

[0135] As an optional embodiment, in order to further improve the accuracy of positioning of the interventional device, in this embodiment, in the case of the above-mentioned double verification, the scale information recorded by the scale lines provided on the delivery rod can be further combined to further verify the accuracy of positioning of the interventional device, so as to achieve the purpose of triple verification. Specifically, in this embodiment, the distance D3 between the proximal end of the blood vessel stent and the probe can be determined in advance. When the interventional device loaded with the blood vessel stent enters the vascular system of the target patient again, the positioning information of the probe of the interventional device is displayed on the first vascular system model in real time. The operator can, through the probe positioning information displayed in real time outside the body, judge the position of the blood vessel stent in the human vascular system in real time; then, when the probe of the interventional device reaches the proximal end of the intravascular lesion area, at this time, the specific position of the interventional device can be judged through the scale D2 on the delivery rod, and the value of D2 + D3 can be used as the distance reference for the blood vessel stent in the blood vessel to reach the proximal end of the intravascular lesion area, so as to judge the position of the blood vessel stent in the human vascular system; in addition, the intravascular ultrasound image or OCT image collected in real time by the probe of the interventional device can be combined for joint reference to verify the position of the blood vessel stent in the human vascular system, ensuring that the blood vessel stent can accurately reach and completely cover the intravascular lesion area.

[0136] When it is determined that the blood vessel stent reaches and completely covers the intravascular lesion area, a blood vessel stent release instruction can be used to control the blood vessel stent to be released from the outer sidewall of the delivery rod and perform a dilation operation. After the blood vessel stent is released, the delivery rod can be controlled to slowly retract, and during the retraction process, the intravascular ultrasound image or OCT image can be collected in real time by the probe to observe and detect the placement position and wall attachment situation of the blood vessel stent, effectively ensuring the accuracy of the interventional operation performed in an ordinary operating room and avoiding exposing the patient and medical staff to a radiation environment.

[0137] In summary, by adopting the positioning method of the interventional device provided in the embodiment of the present invention, the method includes obtaining a first medical image of the vascular system in a target patient's body, generating a first vascular system model based on the first medical image, obtaining consecutive multiple frames of second medical images in the blood vessel through an interventional device disposed in the blood vessel of the vascular system in the target patient's body, generating a second vascular system model based on the consecutive multiple frames of the second medical images, performing a registration process on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient, obtaining a third medical image in the blood vessel of the vascular system in the target patient's body in real time through the interventional device, and mapping the third medical image onto the first vascular system model based on the target registration matrix to display the position of the interventional device on the first vascular system model in real time, so as to achieve the purpose of accurately positioning the interventional device in the vascular system without the need for a lead plate operating room, providing navigation information of the interventional device in the blood vessel of the target patient for clinicians, thereby facilitating clinicians to accurately locate the lesion site and guiding the interventional operation, breaking the current hard condition barrier that the use of interventional devices must be carried out under the protection of lead plates, enabling interventional operations to be carried out in an ordinary operating room without the need for fluoroscopy by a fluoroscope, eliminating the need to queue for the extremely few and costly lead plate operating rooms, reducing the burden on patients, medical workers, and the medical system, and improving the efficiency of interventional operations.

[0138] According to the method described in the above embodiment, this embodiment will further describe from the perspective of the positioning device of the interventional device. The positioning device of the interventional device can be specifically implemented as an independent entity, or can be integrated in an electronic device, such as a terminal. The terminal can include a mobile phone, a tablet computer, etc.

[0139] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of a positioning device of an interventional device provided in an embodiment of the present invention. As Figure 7 shown, the positioning device 500 of the interventional device provided in an embodiment of the present invention includes: a first acquisition module 501, a second acquisition module 502, a registration module 503, and a positioning module 504;

[0140] Among them, the first acquisition module 501 is configured to obtain a first medical image of the vascular system in a target patient's body, and generate a corresponding first vascular system model based on the first medical image. The first medical image includes an angiography image and a magnetic resonance image.

[0141] The second acquisition module 502 is configured to obtain consecutive multiple frames of second medical images in the blood vessel through an interventional device disposed in the blood vessel of the vascular system in the target patient's body, and generate a three-dimensionally reconstructed second vascular system model based on the consecutive multiple frames of the second medical images. The second medical image includes an intravascular ultrasound image and an optical coherence tomography image.

[0142] A registration module 503 is configured to perform a registration process on the first vasculature model and the second vasculature model to obtain a target registration matrix corresponding to the target patient.

[0143] A positioning module 504 is configured to obtain, in real time through the interventional device, a third medical image within the vasculature of the target patient, and map the third medical image onto the first vasculature model based on the target registration matrix, so as to display the position of the interventional device on the first vasculature model in real time.

[0144] In some embodiments, the positioning device 500 of the interventional device provided in this embodiment may further include a first training module and a second training module;

[0145] The first training module is configured to obtain first initial images of the vasculature in different patients, where the first initial images include angiography images and magnetic resonance images; perform rendering processing and segmentation processing on the first initial images to obtain multiple first initial three-dimensional vasculature models corresponding to different patients; use the first initial images as inputs and the first initial three-dimensional vasculature models corresponding to the first initial images as outputs to train a deep learning neural network model to be trained until convergence, so as to obtain a trained first generation model.

[0146] At this time, the first acquisition module 501 provided in this embodiment may further be configured to call the trained first generation model to perform a generation process on the first medical image to generate a first vasculature model corresponding to the first medical image.

[0147] The second training module is configured to obtain, by means of the interventional device entering the human vasculature, consecutive multiple frames of second initial images within the vasculature of different patients, where the second initial images include intravascular ultrasound images and optical coherence tomography images; determine the three-dimensional coordinates of the vasculature in each frame of the second initial images according to the first position information of the interventional device in each frame of the second initial images and the second position information of the interventional device relative to the vascular inner wall; perform three-dimensional reconstruction on the vasculature in different patients based on the three-dimensional coordinates of the vasculature in consecutive multiple frames of the second initial images to obtain second three-dimensional human vasculature models corresponding to different patients; use the consecutive multiple frames of the second initial images corresponding to different patients as inputs and the second initial three-dimensional vasculature models corresponding to the consecutive multiple frames of the second initial images as outputs to train a deep learning neural network model to be trained until convergence, so as to obtain a trained second generation model.

[0148] At this time, the second acquisition module 502 provided in this embodiment can also be used to call the trained second generation model to perform generation processing on the consecutive multi-frame second medical images, and generate a second vascular system model corresponding to the consecutive multi-frame second medical images.

[0149] In some embodiments, the registration processing provided in this embodiment includes rigid registration processing. The registration module 503 is specifically configured to: perform rotation and translation operations on the second vascular system model to align the second vascular system model with the first vascular system model in space; determine a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0150] In other embodiments, the registration processing provided in this embodiment further includes non-rigid registration processing. The registration module 503 is specifically configured to: perform elastic transformation and thin plate spline operations on the second vascular system model to align the second vascular system model with the first vascular system model in space; determine a target registration matrix corresponding to the target patient according to the second vascular system model before and after spatial alignment.

[0151] As an optional embodiment, the intervention device provided in this embodiment may include a probe, a vascular stent, and a delivery rod. The probe is disposed at the end of the delivery rod, and the vascular stent is disposed around and closely attached to the outer side wall of the delivery rod. The positioning device 500 of the intervention device provided in this embodiment may further include: a first prompting module, a control module, a third acquisition module, and a second prompting module;

[0152] Among them, the first prompting module is configured to display a prompt message indicating that the target position has been reached when the position of the probe displayed in real time on the first vascular system model is the target position.

[0153] The control module is configured to, in response to a vascular stent release instruction, control the vascular stent to be released from the outer side wall of the delivery rod and perform a dilation operation.

[0154] Optionally, the interventional device provided in this embodiment further includes a stent fixing structure. A concave fixing groove is provided at a position on the delivery rod at a preset distance from the end. The stent fixing structure is configured to be disposed in the fixing groove, or to fix the vascular stent in the fixing groove of the delivery rod, and to make one end of the vascular stent abut against the side wall of the fixing groove away from the end. The stent fixing structure has a cylindrical hollow structure, and the inner diameter of the stent fixing structure is greater than the outer diameter of the delivery rod. Specifically, the control module provided in this embodiment is further configured to, in response to a vascular stent release instruction, control the stent fixing structure to move in a direction away from the end, so as to release the vascular stent starting from a position close to the end, and cause the vascular stent to start expanding after being released from the fixation of the stent fixing structure.

[0155] A third acquisition module, configured to acquire the intravascular image captured by the probe.

[0156] A second prompt module, configured to, when the position of the probe in the intravascular image is not the target position and a vascular stent release instruction is detected, stop executing the vascular stent release operation and display a prompt message indicating a position error.

[0157] In specific implementation, each of the above modules and / or units may be implemented as an independent entity, or may be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of each of the above modules and / or units, reference may be made to the foregoing method embodiments. For the specific beneficial effects that can be achieved, reference may also be made to the beneficial effects in the foregoing method embodiments, which will not be elaborated herein.

[0158] In addition, please refer to Figure 8 , Figure 8 which is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. The electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. As Figure 8 shown, the electronic device 600 includes a processor 601 and a memory 602. Among them, the processor 601 is electrically connected to the memory 602.

[0159] The processor 601 is the control center of the electronic device 600. It connects various parts of the entire electronic device through various interfaces and lines. By running or loading the application programs stored in the memory 602, and calling the data stored in the memory 602, it executes various functions of the electronic device 600 and processes data, thereby monitoring the entire electronic device 600.

[0160] In this embodiment, the processor 601 in the electronic device 600 will load the instructions corresponding to the processes of one or more application programs into the memory 602 according to the following steps, and the processor 601 will run the application programs stored in the memory 602, so as to implement any step in the positioning method of the intervention device provided in the above embodiment.

[0161] The electronic device 600 can implement the steps in any embodiment of the positioning method of the intervention device provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that any positioning method of the intervention device provided in the embodiments of the present invention can achieve. For details, please refer to the previous embodiments and will not be elaborated here.

[0162] Please refer to Figure 9 , Figure 9 which is another structural schematic diagram of the electronic device provided in the embodiments of the present invention. As Figure 9 shown, Figure 9 it shows the specific structural block diagram of the electronic device provided in the embodiments of the present invention. The electronic device 700 can be used to implement the positioning method of the intervention device provided in the above embodiment. The electronic device 700 can be a mobile terminal such as a smart phone or a notebook computer.

[0163] The RF circuit 710 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. The RF circuit 710 may include various existing circuit elements for performing these functions. For example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, a memory, and so on. The RF circuit 710 can communicate with various networks such as the Internet, an enterprise intranet, a wireless network or communicate with other devices through a wireless network. The above-mentioned wireless network may include a cellular phone network, a wireless local area network or a metropolitan area network. The above-mentioned wireless network can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, and may even include those protocols that have not been developed yet.

[0164] The memory 720 can be used to store software programs and modules, such as the program instructions / modules corresponding to the positioning method of the intervention device in the above embodiments. The processor 780 executes various functional applications and the positioning of the intervention device by running the software programs and modules stored in the memory 720.

[0165] The memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 720 may further include a memory remotely located with respect to the processor 780, and these remote memories may be connected to the electronic device 700 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0166] The input unit 730 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. The touch-sensitive surface 731, also known as a touch display screen or a touchpad, can collect touch operations of a user thereon or nearby (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch-sensitive surface 731), and drive corresponding connection devices according to a preset program. Optionally, the touch-sensitive surface 731 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 780, and can receive and execute commands sent by the processor 780. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch-sensitive surface 731. In addition to the touch-sensitive surface 731, the input unit 730 may further include other input devices 732. Specifically, the other input devices 732 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0167] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device 700. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 740 may include a display panel 741. Optionally, the display panel 741 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 731 can cover the display panel 741. When the touch-sensitive surface 731 detects a touch operation on or near it, it is transmitted to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides a corresponding visual output on the display panel 741 according to the type of touch event. Although in the figure, the touch-sensitive surface 731 and the display panel 741 are implemented as two independent components to realize the input and output functions, in some embodiments, the touch-sensitive surface 731 and the display panel 741 can be integrated to realize the input and output functions.

[0168] The electronic device 700 may further include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor can generate an interruption when the flip cover is closed or opened. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used in applications for identifying the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as a pedometer, tapping), etc. As for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor that the electronic device 700 can also be configured with, they will not be elaborated here.

[0169] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the electronic device 700. The audio circuit 760 can transmit the electrical signal converted from the received audio data to the speaker 761, and the speaker 761 converts it into a sound signal for output. On the other hand, the microphone 762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 760 and then converted into audio data. After the audio data is output to the processor 780 for processing, it is sent through the RF circuit 710 to, for example, another terminal, or the audio data is output to the memory 720 for further processing. The audio circuit 760 may also include an earphone jack to provide communication between the peripheral earphone and the electronic device 700.

[0170] The electronic device 700 can help users receive requests, send information, etc. through a transmission module 770 (such as a Wi-Fi module), which provides users with wireless broadband Internet access. Although the transmission module 770 is shown in the figure, it can be understood that it does not belong to the essential components of the electronic device 700 and can be omitted completely within the scope of not changing the essence of the invention according to needs.

[0171] The processor 780 is the control center of the electronic device 700, connecting various parts of the entire mobile phone through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 720, and calling data stored in the memory 720, it executes various functions of the electronic device 700 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 780 may include one or more processing cores; in some embodiments, the processor 780 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 780 either.

[0172] The electronic device 700 further includes a power supply 790 (such as a battery) for powering each component. In some embodiments, the power supply can be logically connected to the processor 780 through a power management system, thereby realizing functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 790 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0173] Although not shown, the electronic device 700 further includes a camera (such as a front camera and a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal further includes a memory, and one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors to implement any step in the positioning method of the intervention device provided in the above embodiment.

[0174] During specific implementation, the above-mentioned modules can be implemented as independent entities, or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of the above-mentioned modules, reference can be made to the method embodiments described above, which will not be elaborated here.

[0175] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions or by controlling relevant hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium in which multiple instructions are stored. When these instructions are executed by a processor, any step in the positioning method of the intervention device provided in the above embodiments can be realized.

[0176] Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0177] Since the instructions stored in the storage medium can execute the steps in any embodiment of the positioning method of the intervention device provided in the embodiments of the present invention, the beneficial effects that can be achieved by any positioning method of the intervention device provided in the embodiments of the present invention can be realized. For details, see the previous embodiments and will not be repeated here.

[0178] The above has introduced in detail a positioning method, device, electronic device and storage medium of an intervention device provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation of the present application. Moreover, for those of ordinary skill in the technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A positioning method for an interventional device, characterized in that Including: Obtaining a first medical image of the vascular system in a target patient, and generating a corresponding first vascular system model based on the first medical image, where the first medical image includes an angiography image and a magnetic resonance image; Obtaining, through an intervention device disposed in the blood vessels of the vascular system in the target patient, a plurality of consecutive frames of second medical images within the blood vessels, and generating a second vascular system model after three-dimensional reconstruction based on the plurality of consecutive frames of the second medical images, where the second medical images include intravascular ultrasound images and optical coherence tomography images; Performing a registration process on the first vascular system model and the second vascular system model to obtain a target registration matrix corresponding to the target patient; Real-time obtaining, through the intervention device, a third medical image within the blood vessels of the vascular system in the target patient, and mapping the third medical image onto the first vascular system model based on the target registration matrix to display the position of the intervention device in real time on the first vascular system model.

2. The positioning method of the interventional device according to claim 1, characterized in that, Before the step of obtaining the first medical image of the vascular system in the target patient, the method further includes: Obtaining first initial images of the vascular systems in different patients, where the first initial images include angiography images and magnetic resonance images; Performing a rendering process and a segmentation process on the first initial images to obtain a plurality of first initial three-dimensional vascular system models corresponding to different patients; Using the first initial images as inputs and the first initial three-dimensional vascular system models corresponding to the first initial images as outputs to train a deep learning neural network model to be trained until convergence to obtain a trained first generation model; The step of generating a corresponding first vascular system model based on the first medical image includes: Invoking the trained first generation model to perform a generation process on the first medical image to generate a first vascular system model corresponding to the first medical image.

3. The positioning method of the interventional device according to claim 1, characterized in that Before the step of obtaining, through an intervention device disposed in the blood vessels of the vascular system in the target patient, a plurality of consecutive frames of second medical images within the blood vessels, the method further includes: Obtaining, in a manner of using an intervention device to enter the human vascular system, a plurality of consecutive frames of second initial images within the blood vessels of the vascular systems in different patients, where the second initial images include intravascular ultrasound images and optical coherence tomography images; Determining the three-dimensional coordinates of the vascular system in each frame of the second initial images according to the first position information of the intervention device in each frame of the second initial images and the second position information of the intervention device relative to the blood vessel inner wall; Performing three-dimensional reconstruction on the vascular systems in different patients based on the three-dimensional coordinates of the vascular system in the plurality of consecutive frames of the second initial images to obtain second three-dimensional human vascular system models corresponding to different patients; Using the plurality of consecutive frames of the second initial images corresponding to different patients as inputs and the second initial three-dimensional vascular system models corresponding to the plurality of consecutive frames of the second initial images as outputs to train a deep learning neural network model to be trained until convergence to obtain a trained second generation model; The step of generating the three-dimensionally reconstructed second vasculature system model based on consecutive multiple frames of the second medical image includes: Invoking the trained second generation model to perform generation processing on consecutive multiple frames of the second medical image to generate a second vasculature system model corresponding to the consecutive multiple frames of the second medical image.

4. The positioning method of the interventional device according to claim 1, wherein, The registration processing includes rigid registration processing. The step of performing registration processing on the first vasculature system model and the second vasculature system model to obtain a target registration matrix corresponding to the target patient includes: Performing rotation and translation operations on the second vasculature system model to spatially align the second vasculature system model with the first vasculature system model; Determining a target registration matrix corresponding to the target patient according to the second vasculature system model before and after spatial alignment.

5. The positioning method of the interventional device according to claim 1, characterized in that, The registration processing further includes non-rigid registration processing. The step of performing registration processing on the first vasculature system model and the second vasculature system model to obtain a target registration matrix corresponding to the target patient includes: Performing elastic transformation and thin plate spline operations on the second vasculature system model to spatially align the second vasculature system model with the first vasculature system model; Determining a target registration matrix corresponding to the target patient according to the second vasculature system model before and after spatial alignment.

6. The positioning method of the interventional device according to claim 1, characterized in that, The interventional device includes a probe, a vascular stent, and a delivery rod. The probe is disposed at an end of the delivery rod, and the vascular stent is disposed around and closely attached to an outer sidewall of the delivery rod. The method further includes: When the position of the probe displayed in real time on the first vasculature system model is the target position, displaying a prompt message indicating reaching the target position; In response to a vascular stent release instruction, controlling the vascular stent to be released from the outer sidewall of the delivery rod and performing an expansion operation.

7. The positioning method of the interventional device according to claim 6, wherein, Before the step of, in response to a vascular stent release instruction, controlling the vascular stent to be released from the outer sidewall of the delivery rod and performing an expansion operation, the method further includes: Obtaining an intravascular image captured by the probe; When the position of the probe in the intravascular image is not the target position and a vascular stent release instruction is detected, stopping the execution of the vascular stent release operation and displaying a prompt message indicating a position error.

8. A positioning device for an interventional device, characterized in that, Includes: A first acquisition module, configured to acquire a first medical image of a vasculature system in a target patient and generate a corresponding first vasculature system model based on the first medical image, where the first medical image includes an angiography image and a magnetic resonance image; A second acquisition module, configured to acquire consecutive multiple frames of second medical images in a blood vessel through an interventional device disposed in the blood vessel of the vasculature system in the target patient, and generate a three-dimensionally reconstructed second vasculature system model based on the consecutive multiple frames of the second medical image, where the second medical image includes an intravascular ultrasound image and an optical coherence tomography image; A registration module, configured to perform registration processing on the first vasculature system model and the second vasculature system model to obtain a target registration matrix corresponding to the target patient; A positioning module, configured to obtain in real time a third medical image within the blood vessels of the vascular system in the target patient through the interventional device, and map the third medical image onto the first vascular system model based on the target registration matrix, so as to display the position of the interventional device on the first vascular system model in real time.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

Images