Image processing-based puncture guiding method, device and equipment and storage medium

Abstract

The present application relates to the technical field of medical devices, in particular to a puncture guiding method, device and equipment based on image processing and a storage medium, the method comprising: acquiring a two-dimensional perspective image including a target tissue region in real time, the target tissue region including a portal vein region and a diaphragm region; identifying the diaphragm region in the two-dimensional perspective image to obtain first edge position information of the top of the diaphragm region; acquiring three-dimensional image information of the portal vein region and second edge position information of the top of the diaphragm region, the three-dimensional image information and the second edge position information being determined based on a three-dimensional reconstruction image including the target tissue region; superimposing the three-dimensional image information and the second edge position information into the two-dimensional perspective image to obtain a superimposed image; and displaying the superimposed image to guide puncture. The method can provide intraoperative image guidance for TIPS to improve puncture accuracy and shorten puncture time.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a puncture guidance method, device, equipment and storage medium based on image processing. Background Technology

[0002] Transjugular intrahepatic portosystemic shunt (TIPS) creates a shunt within the liver parenchyma between the hepatic vein and the portal vein, structurally reducing portal vein resistance in a minimally invasive manner. Compared to drug therapy and endoscopic treatment, TIPS can fundamentally reduce portal vein pressure and is an effective method for treating a range of portal hypertension complications, such as esophageal and gastric variceal bleeding and refractory ascites.

[0003] The procedure of puncturing the portal vein branch from the hepatic vein is one of the technical challenges of TIPS surgery. Currently, the standard procedure is based on fluoroscopic guidance using an angiography machine, with blind puncture performed according to the surgeon's experience. Because the real-time position and morphology of the vessels cannot be visualized, improper operation may cause damage to the hepatic artery and bile ducts, or even displacement of the liver capsule, leading to serious complications. Secondly, puncturing the portal vein branch from the hepatic vein is usually the most time-consuming step, exposing both the surgeon and the patient to high doses of X-rays. Furthermore, the spatial relationship between the hepatic vein and the portal vein is complex, and anatomical variations and pathological changes in cirrhosis can alter this relationship. In addition, the patient's respiratory movements during TIPS surgery can cause body displacement and positional deviations, making portal vein puncture localization difficult. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, the present invention aims to provide an image processing-based puncture guidance method, apparatus, device, and storage medium that can provide intraoperative image guidance for TIPS, thereby improving puncture accuracy and shortening puncture time.

[0005] To address the above problems, this invention provides an image processing-based puncture-guided method, comprising:

[0006] Real-time acquisition of two-dimensional perspective images of the target tissue region, including the portal vein region and the diaphragm region;

[0007] Identify the diaphragm region in the two-dimensional perspective image and obtain the position information of the first edge of the top of the diaphragm region;

[0008] Three-dimensional image information of the portal vein region and the position information of the second edge of the top of the diaphragm region are obtained, and both the three-dimensional image information and the second edge position information are determined based on a three-dimensional reconstructed image including the target tissue region.

[0009] The three-dimensional image information and the second edge position information are superimposed on the two-dimensional perspective image to obtain a superimposed image;

[0010] The overlay image is displayed to guide the puncture.

[0011] Furthermore, the method also includes:

[0012] The first edge position information and the second edge position information in the overlay image are marked respectively.

[0013] Further, the step of marking the first edge position information and the second edge position information in the overlay image includes:

[0014] When the first edge position information coincides with the second edge position information, the first edge position information and the second edge position information are marked first.

[0015] When the first edge position information and the second edge position information do not coincide, the first edge position information is marked with a second mark, and the second edge position information is marked with a third mark.

[0016] Furthermore, the method also includes:

[0017] Obtain a three-dimensional reconstructed image including the target tissue region;

[0018] The three-dimensional reconstructed image is segmented based on the first preset image segmentation algorithm to obtain the three-dimensional image information of the portal vein region;

[0019] Based on the second preset image segmentation algorithm, the diaphragm region in the three-dimensional reconstructed image is identified, and the position information of the second edge of the top of the diaphragm region is obtained.

[0020] Further, acquiring the three-dimensional reconstructed image including the target tissue region includes:

[0021] Acquire cone-beam computed tomography (CBCT) image data including the target tissue region;

[0022] A three-dimensional reconstructed image including the target tissue region is obtained based on the cone-beam computed tomography image data.

[0023] Further, acquiring the three-dimensional reconstructed image including the target tissue region includes:

[0024] Acquire computed tomography (CT) angiography or magnetic resonance angiography (MRI) images of the target tissue region.

[0025] A three-dimensional reconstructed image including the target tissue region is obtained based on the computed tomography angiography image data or the magnetic resonance angiography image data.

[0026] Furthermore, the method also includes:

[0027] The three-dimensional reconstructed image is registered to the cone-beam computed tomography (CBCT) imaging space to obtain a three-dimensional reconstructed image in the CBCT imaging space.

[0028] Another aspect of the present invention provides an image processing-based puncture guidance device, comprising:

[0029] The image acquisition module is used to acquire two-dimensional perspective images of the target tissue region in real time, including the portal vein region and the diaphragm region;

[0030] The diaphragm region recognition module is used to identify the diaphragm region in the two-dimensional perspective image and obtain the first edge position information of the top of the diaphragm region.

[0031] The information acquisition module is used to acquire three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region. The three-dimensional image information and the second edge position information are both determined based on a three-dimensional reconstructed image including the target tissue region.

[0032] An image overlay module is used to overlay the three-dimensional image information and the second edge position information onto the two-dimensional perspective image to obtain an overlay image;

[0033] An image display module is used to display the superimposed image for puncture guidance.

[0034] In another aspect, the present invention provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the method as described above.

[0035] In another aspect, the present invention provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the method described above.

[0036] Due to the above technical solution, the present invention has the following beneficial effects:

[0037] According to the puncture guidance method of the present invention, by fusing and superimposing the three-dimensional image information of the portal vein region and the edge position information of the top of the diaphragm region with the real-time acquired two-dimensional fluoroscopic image, the three-dimensional structure of the blood vessels can be displayed in the two-dimensional fluoroscopic image, thereby providing intraoperative image guidance for TIPS. This allows the timing of puncture to be determined based on the patient's real-time respiratory status during the operation, reducing the reliance on the doctor's experience to a certain extent, reducing positional deviations caused by the patient's respiratory movements during the operation, thereby improving puncture accuracy, reducing puncture attempts, and shortening puncture time. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0039] Figure 1 This is a schematic diagram of the implementation environment provided in one embodiment of the present invention;

[0040] Figure 2 This is a flowchart of an image processing-based puncture guidance method provided in one embodiment of the present invention;

[0041] Figure 3 This is a flowchart of an automatic segmentation process of the portal vein region provided in one embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the segmentation result of the portal vein region provided in one embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of an overlay image provided in one embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of an image processing-based puncture guidance device provided in one embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the structure of an electronic device provided in another embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0048] Reference manual attached Figure 1 This illustrates a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 As shown, the implementation environment may include at least one medical scanning system 110 and an image processing system 120. The image processing system 120 and each of the medical scanning systems 110 may be directly or indirectly connected via wired or wireless communication. This embodiment of the invention does not impose any limitations on this.

[0049] The medical scanning system 110 may include, but is not limited to, an X-ray angiography system. The image processing system 120 may include, but is not limited to, various servers, personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be an independent server or a server cluster or distributed system composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0050] In practical applications, the medical scanning system 110 can acquire CBCT image data including the target tissue region using cone-beam computed tomography (CBCT) technology, and acquire two-dimensional fluoroscopic images including the target tissue region in real time using X-ray angiography technology.

[0051] The image processing system 120 can acquire CBCT image data and two-dimensional fluoroscopic images obtained by the medical scanning system 110, and through the image processing-based puncture guidance method provided in this embodiment of the invention, determine and display the superimposed image after superimposing the three-dimensional image information of the portal vein region and the edge position information of the top of the diaphragm region on the two-dimensional fluoroscopic image, so as to guide the doctor to perform puncture from the hepatic vein to the portal vein and place the stent during the TIPS procedure.

[0052] In one possible embodiment, the medical scanning system 110 may further include a computed tomographic angiography (CTA) system or a magnetic resonance angiography (MRA) system.

[0053] The medical scanning system 110 can acquire CTA image data including the target tissue region using enhanced CTA technology, or acquire MRA image data including the target tissue region using enhanced MRA technology, or acquire two-dimensional fluoroscopic images including the target tissue region in real time using X-ray angiography technology.

[0054] The image processing system 120 can acquire CTA image data (or MRA image data) and two-dimensional fluoroscopic images obtained by the medical scanning system 110, and through the image processing-based puncture guidance method provided in this embodiment of the invention, determine and display the superimposed image after superimposing the three-dimensional image information of the portal vein region and the edge position information of the top of the diaphragm region on the two-dimensional fluoroscopic image, so as to guide the doctor to perform puncture from the hepatic vein to the portal vein and place the stent during the TIPS procedure.

[0055] It should be noted that, Figure 1 This is merely an example. Those skilled in the art will understand that, although... Figure 1 Only one medical scanning system 110 is shown in the figure, but this does not constitute a limitation on the embodiments of the present invention. In practical applications, more medical scanning systems 110 may be included than those shown in the figure.

[0056] Reference manual attached Figure 2This illustrates the flowchart of an image processing-based puncture guidance method provided by an embodiment of the present invention, which can be applied to... Figure 1 In the image processing system 120, specifically as follows: Figure 2 As shown, the method may include the following steps:

[0057] S210: Real-time acquisition of a two-dimensional perspective image including the target tissue region, which includes the portal vein region and the diaphragm region.

[0058] In this embodiment of the invention, during TIPS surgery, two-dimensional perspective images of the target tissue region, including the portal vein region and the diaphragm region, can be acquired in real time from patients with portal hypertension. These images are available for review by the physician during the TIPS procedure, guiding the puncture of the hepatic vein into the portal vein and the placement of the stent. The target tissue region can be the portal venous system region of the patient with portal hypertension.

[0059] The two-dimensional fluoroscopic image can be sourced from relevant data directly imported from a medical scanning system, or it can be obtained in real-time through a connection configured from other resource libraries. This embodiment of the invention does not impose specific limitations on this. For example, a two-dimensional fluoroscopic image including the target tissue region can be acquired in real-time using X-ray angiography equipment such as an angiography machine and imported into the image processing system.

[0060] S220: Identify the diaphragm region in the two-dimensional perspective image and obtain the position information of the first edge of the top of the diaphragm region.

[0061] In this embodiment of the invention, a pre-set image segmentation algorithm can be used to identify the boundary of the region including the top of the diaphragm in the two-dimensional perspective image, and finally output the first edge position information of the top of the segmented diaphragm region.

[0062] The pre-set image segmentation algorithm can be any type of image segmentation algorithm in the prior art, such as traditional image segmentation algorithms like the watershed algorithm or image segmentation algorithms based on artificial intelligence (AI) technology. This embodiment of the invention does not impose any specific limitations on this.

[0063] It should be noted that the specific process of determining the position information of the first edge at the top of the diaphragm region based on the pre-set image segmentation algorithm can refer to the existing technology, and will not be repeated here in the embodiments of the present invention.

[0064] S230: Obtain three-dimensional image information of the portal vein region and second edge position information of the top of the diaphragm region, wherein the three-dimensional image information and the second edge position information are both determined based on a three-dimensional reconstructed image including the target tissue region.

[0065] Since the relative positions of the hepatic vein and portal vein are not visible under conventional fluoroscopy, the three-dimensional structure of the portal vein can be superimposed on the two-dimensional fluoroscopic image to increase the doctor's confidence in determining the puncture location. However, the superimposed three-dimensional structure will inevitably fluctuate due to the patient's breathing. Therefore, the position of the diaphragm top can be used as an indirect respiratory gating point to track the patient's normal respiratory cycle. The doctor can then synchronize the puncture and stent placement procedures with the patient's breathing based on the diaphragm top position.

[0066] Therefore, in this embodiment of the invention, a three-dimensional reconstructed image of the target tissue region, including the portal vein region and the diaphragm region, of a patient with portal hypertension can be obtained in advance. The three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region can be obtained by segmenting the three-dimensional reconstructed image. These information can be superimposed on the real-time acquired two-dimensional fluoroscopic image during the TIPS procedure for the doctor to review, guiding the doctor to perform hepatic vein to portal vein puncture and stent placement.

[0067] Specifically, before performing step S230, the method may further include the following steps:

[0068] Obtain a three-dimensional reconstructed image including the target tissue region;

[0069] The three-dimensional reconstructed image is segmented based on the first preset image segmentation algorithm to obtain the three-dimensional image information of the portal vein region;

[0070] Based on the second preset image segmentation algorithm, the diaphragm region in the three-dimensional reconstructed image is identified, and the position information of the second edge of the top of the diaphragm region is obtained.

[0071] In an optional embodiment, acquiring a three-dimensional reconstructed image including the target tissue region may include:

[0072] Acquire cone-beam computed tomography (CBCT) image data including the target tissue region;

[0073] A three-dimensional reconstructed image including the target tissue region is obtained based on the cone-beam computed tomography image data.

[0074] Specifically, during TIPS surgery, CBCT image data, including the target tissue region, can be pre-acquired using enhanced CBCT technology from a patient with portal hypertension via equipment such as a CBCT-equipped angiography machine. A three-dimensional reconstructed image including the target tissue region is then reconstructed based on the CBCT image data and imported into the image processing system. The target tissue region can be the portal venous system region of the patient with portal hypertension.

[0075] Alternatively, during TIPS surgery, CBCT image data including the target tissue region of the patient with portal hypertension can be acquired in advance using equipment such as an angiography machine with CBCT function, and imported into the image processing system. The image processing system can then reconstruct a three-dimensional reconstructed image including the target tissue region based on the received CBCT image data.

[0076] It is understandable that existing angiography machines with CBCT capabilities can acquire both three-dimensional reconstructed images and two-dimensional fluoroscopic images, enabling intraoperative puncture guidance without the need for additional external equipment, thus saving costs for hospitals and reducing the medical burden on patients.

[0077] In another alternative embodiment, acquiring a three-dimensional reconstructed image of the target tissue region may include:

[0078] Acquire computed tomography (CT) angiography or magnetic resonance angiography (MRI) images of the target tissue region.

[0079] A three-dimensional reconstructed image including the target tissue region is obtained based on the computed tomography angiography image data or the magnetic resonance angiography image data.

[0080] Optionally, prior to TIPS surgery, CTA image data including the target tissue region can be acquired in advance using enhanced CTA technology from a patient with portal hypertension. A three-dimensional reconstructed image including the target tissue region can then be reconstructed based on the CTA image data and imported into the image processing system. The target tissue region can be the portal venous system region of the patient with portal hypertension.

[0081] Alternatively, before performing TIPS surgery, CTA image data including the target tissue region of a patient with portal hypertension can be acquired in advance using a CTA device and imported into the image processing system. The image processing system can then reconstruct a three-dimensional reconstructed image including the target tissue region based on the received CTA image data.

[0082] Alternatively, prior to TIPS surgery, MRA image data including the target tissue region of a patient with portal hypertension can be acquired in advance using an MRA device based on enhanced MRA technology, and a three-dimensional reconstructed image including the target tissue region can be obtained based on the MRA image data and imported into the image processing system.

[0083] Alternatively, before performing TIPS surgery, MRA image data including the target tissue region of the patient with portal hypertension can be acquired in advance using an MRA device and imported into the image processing system. The image processing system can then reconstruct a three-dimensional reconstructed image including the target tissue region based on the received MRA image data.

[0084] It should be noted that the specific methods for reconstructing three-dimensional images based on CBCT image data, CTA image data, or MRA image data can refer to existing technologies, and will not be elaborated here in the embodiments of the present invention.

[0085] It should be noted that the steps of acquiring CBCT image data, CTA image data, or MRA image data, and reconstructing a three-dimensional reconstructed image based on the CBCT image data, the CTA image data, or the MRA image data can be performed by an image processing system that implements the method provided in this embodiment of the invention, or by other devices, and the obtained three-dimensional reconstructed image is sent to the image processing system. This embodiment of the invention does not limit this.

[0086] In this embodiment of the invention, after obtaining the three-dimensional reconstructed image, the portal vein region can be automatically segmented based on a first preset image segmentation algorithm, and the top of the diaphragm region can be automatically identified based on a second preset image segmentation algorithm, so as to obtain the three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region.

[0087] The first preset image segmentation algorithm and the second preset image segmentation algorithm can be various image segmentation algorithms in the prior art, such as traditional image segmentation algorithms such as threshold segmentation algorithm and watershed algorithm, or image segmentation algorithms based on AI technology. The first preset image segmentation algorithm and the second preset image segmentation algorithm can be the same algorithm or different algorithms. This embodiment of the invention does not impose specific restrictions on this.

[0088] The following example illustrates the automatic segmentation of the portal vein region using a classic threshold segmentation algorithm. Please refer to the appendix of the instruction manual. Figure 3 The process of segmenting the 3D reconstructed image based on a threshold segmentation algorithm to obtain the 3D image information of the portal vein region may include the following steps:

[0089] S310: Preprocess the three-dimensional reconstructed image to obtain the corresponding grayscale image.

[0090] Specifically, a frequency domain method can be used first to process the useful information in the low-frequency part of the three-dimensional reconstructed image to eliminate noise. Then, the pixels of the noise-reduced image can be traversed, and the number of pixels belonging to each gray level can be summarized to obtain the grayscale image corresponding to the three-dimensional reconstructed image.

[0091] S320: Determine the corresponding grayscale histogram based on the grayscale image.

[0092] Specifically, a corresponding grayscale histogram can be drawn based on the grayscale image using a calculation function.

[0093] S330: Determine the target threshold based on the grayscale histogram, and segment the grayscale image based on the target threshold to obtain the segmented grayscale image.

[0094] Specifically, the optimal threshold can be selected based on the grayscale histogram to perform image segmentation and obtain the segmented grayscale image.

[0095] S340: Add marker points to the segmented grayscale image.

[0096] S350: Extract the three-dimensional image corresponding to the marker point in the three-dimensional reconstructed image to obtain the three-dimensional image information of the portal vein region.

[0097] It should be noted that other related aspects of the automatic segmentation process of the portal vein region using the threshold segmentation algorithm can be found in existing technologies, and will not be elaborated upon here in this embodiment of the invention.

[0098] For example, refer to the appendix to the specification. Figure 4 This illustrates a schematic diagram of the portal vein region segmentation result provided in an embodiment of the present invention. Figure 4 As shown, by using the threshold segmentation algorithm to segment the 3D reconstructed image shown in Figure A, we can obtain 3D image information of the portal vein region, including the portal vein and its main branches, as shown in Figure B.

[0099] In one specific embodiment, the watershed algorithm can be used to identify the boundary of the region, including the top of the diaphragm region, based on the three-dimensional reconstructed image, to obtain the second edge position information of the top of the diaphragm region.

[0100] It should be noted that other related aspects of the automatic identification process of the top of the diaphragm region using the watershed algorithm can be found in existing technologies, and will not be elaborated upon here in this embodiment of the invention.

[0101] In practical applications, the final output can be a three-dimensional image of the segmented portal vein region, and a segmented image of the top of the diaphragm region marked with the second edge position information of the top of the diaphragm region; or the identified second edge position information can be marked in the three-dimensional image of the segmented portal vein region to obtain a segmented image including the three-dimensional image of the portal vein region and the second edge position information of the top of the diaphragm region.

[0102] It should be noted that the above-described implementation methods for automatic segmentation of the portal vein region using the threshold segmentation algorithm and automatic identification of the top of the diaphragm region using the watershed algorithm are merely examples. In practical applications, other image segmentation algorithms can also be used. The specific segmentation and identification process can be referred to the existing technology. The embodiments of the present invention will not be described in detail here.

[0103] It is understandable that by automatically segmenting the portal vein region and automatically identifying the top position of the diaphragm region through image segmentation algorithms, no manual operation is required, and the accuracy of segmentation / identification can be improved, thereby further enhancing the precision of puncture and reducing the number of punctures required for successful puncture.

[0104] In some possible embodiments, after obtaining the three-dimensional reconstructed image, the portal vein region can be automatically segmented and the top of the diaphragm region can be automatically identified manually to obtain the three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region.

[0105] It should be noted that the above-mentioned steps of automatically segmenting the portal vein region and automatically identifying the top of the diaphragm region based on the three-dimensional reconstructed image can be performed by the image processing system that implements the method provided in the embodiments of the present invention, or by other devices, and the obtained three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region are sent to the image processing system. The embodiments of the present invention do not limit this.

[0106] In practical applications, when a three-dimensional reconstructed image is obtained by reconstructing a three-dimensional image based on CTA or MRA image data acquired before TIPS surgery, the three-dimensional reconstructed image obtained by reconstructing a three-dimensional reconstructed image based on CTA or MRA image data can be registered with the three-dimensional reconstructed image obtained by reconstructing a three-dimensional image based on CBCT image data during TIPS surgery. This registration places the three-dimensional reconstructed image obtained by reconstructing a three-dimensional image in the cone-beam computed tomography (CBCT) imaging space, thereby obtaining a three-dimensional reconstructed image in the CBCT imaging space. This facilitates the subsequent overlay of the three-dimensional image information of the portal vein region obtained by segmenting the three-dimensional reconstructed image obtained by reconstructing a three-dimensional image based on the CTA or MRA image data, as well as the position information of the second edge of the top of the diaphragm region, onto a two-dimensional fluoroscopic image.

[0107] In other words, when other devices are used to perform the above steps of automatic segmentation of the portal vein region and automatic identification of the top of the diaphragm region based on three-dimensional reconstructed images, and the implementation method is to reconstruct three-dimensional reconstructed images based on CTA image data or MRA image data acquired before TIPS surgery, it is also necessary to send the three-dimensional reconstructed images reconstructed based on CTA image data or MRA image data to the image processing system to complete the subsequent registration and image overlay steps.

[0108] In this embodiment of the invention, the image processing system can use the above method to determine the three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region based on the three-dimensional reconstructed image including the target tissue region. Alternatively, the three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region can be imported from other devices. This embodiment of the invention does not limit this.

[0109] S240: The three-dimensional image information and the second edge position information are superimposed onto the two-dimensional perspective image to obtain a superimposed image.

[0110] In this embodiment of the invention, the three-dimensional image information of the segmented portal vein region and the identified second edge position information of the diaphragm roof region can be superimposed with a real-time acquired two-dimensional fluoroscopic image to display the three-dimensional structure of the portal vein region and the second edge position of the diaphragm roof region at the corresponding positions in the two-dimensional fluoroscopic image. During TIPS surgery, the three-dimensional structure of the portal vein region can be used to indicate the puncture location, and the first and second edge position information of the diaphragm roof region can be used to indicate the respiratory phase, thereby guiding the puncture of the hepatic vein into the portal vein and the placement of the stent.

[0111] It should be noted that the specific process of image overlay can refer to relevant existing technologies, and will not be repeated here in the embodiments of the present invention. For example, the three-dimensional reconstructed image including the target tissue region can be registered with the two-dimensional perspective image first to determine the position of the portal vein region and the top region of the diaphragm, and then the three-dimensional image information and the second edge position information can be overlaid at the corresponding positions.

[0112] In one possible embodiment, the method may further include the following steps:

[0113] The first edge position information and the second edge position information in the overlay image are marked respectively.

[0114] In this embodiment of the invention, the first edge position information and the second edge position information in the superimposed image can be marked respectively, so as to determine the top position of the diaphragm region in the two-dimensional perspective image and the top position of the diaphragm region in the three-dimensional reconstructed image.

[0115] Specifically, marking the first edge position information and the second edge position information in the overlay image may include:

[0116] When the first edge position information coincides with the second edge position information, the first edge position information and the second edge position information are marked first.

[0117] When the first edge position information and the second edge position information do not coincide, the first edge position information is marked with a second mark, and the second edge position information is marked with a third mark.

[0118] In this embodiment of the invention, when the first edge position information and the second edge position information do not coincide, different marking methods can be used to mark the first edge position information and the second edge position information to distinguish the top position of the diaphragm region in the two-dimensional perspective image and the top position of the diaphragm region in the three-dimensional reconstructed image. That is, the second mark and the third mark are different marks.

[0119] For example, refer to the appendix to the specification. Figure 5 It illustrates a schematic diagram of an overlay image provided in an embodiment of the present invention, such as... Figure 5 As shown, the overlaid image may include a two-dimensional perspective view of the target tissue region and a three-dimensional structure of the portal vein region, as shown in Figure A and part (a) of Figure B. When the position information of the first edge at the top of the diaphragm region does not coincide with the position information of the second edge, lines of different colors can be used to mark the first position information and the second position information, with the marking effect as shown in Figure B. Figure 5As shown in Figure A. The upper line (b) can be blue and is used to mark the first edge position information of the top of the diaphragm region in the two-dimensional perspective image; the lower line (c) can be yellow and is used to mark the second edge position information of the top of the diaphragm region in the three-dimensional reconstructed image.

[0120] In this embodiment of the invention, when the first edge position information coincides with the second edge position information, a marking method different from both the second and third markings can be used to mark the first edge position information and the second edge position information, so as to indicate that the top position of the diaphragm region in the current two-dimensional perspective image is consistent with the top position of the diaphragm region in the three-dimensional reconstructed image. That is to say, the first marking is different from both the second and third markings.

[0121] For example, when the position information of the first edge at the top of the diaphragm region coincides with the position information of the second edge, a different color than the color used when they do not coincide can be used for marking, and the marking effect is as follows. Figure 5 As shown in Figure B, the line (d) marking the first edge position information and the second edge position information can be green.

[0122] It should be noted that the above-described marking method using different colored lines is only an example. In practical applications, the first mark, the second mark, and the third mark can be selected according to actual needs. Furthermore, the first mark, the second mark, and the third mark can be the same, partially the same, or different from each other. Preferably, the first mark, the second mark, and the third mark are all different. This embodiment of the invention does not impose any restrictions on this.

[0123] It is understandable that by using different marking methods when the top position of the diaphragm region in the current two-dimensional fluoroscopic image is the same as the top position of the diaphragm region in the three-dimensional reconstructed image, the surgeon can be provided with a more intuitive display and guidance, thereby further reducing the number of punctures required for successful puncture and improving puncture accuracy.

[0124] S250: Display the overlay image for puncture guidance.

[0125] In this embodiment of the invention, a two-dimensional fluoroscopic image can be acquired in real time, and three-dimensional image information of the portal vein region and the second edge position information of the diaphragm top region can be superimposed and displayed in real time. At this time, the three-dimensional image of the portal vein region in the superimposed image can provide the doctor with guidewire guidance. After the guidewire and catheter are in place, the position of the top of the diaphragm region on different marked modalities (including two-dimensional fluoroscopic images and three-dimensional reconstructed images) is used as a respiratory gating reference. When it is determined that the patient's current actual portal vein position is consistent with the position of the three-dimensional image of the portal vein region in the superimposed image, it can be used as a reference for puncture and stent placement. That is, the doctor can perform hepatic vein to portal vein puncture and stent placement in this state. Furthermore, after confirming the recovery of the portal-vena cava gradient, the three-dimensional image of the portal vein region in the superimposed image can provide an intuitive display of the positional relationship between the stent and the portal vein, guiding the real-time adjustment, confirmation, and release of the stent position.

[0126] For example, the reference method for respiratory gating is as follows: when the patient's current actual position of the top of the diaphragm region (i.e., the first position information in the two-dimensional perspective image) coincides with the reference position of the top of the diaphragm region (i.e., the second position information in the three-dimensional reconstructed image), it can be determined that the patient's current actual portal vein position is consistent with the three-dimensional image position of the portal vein region in the overlaid image, and the marker of the top position of the diaphragm region will change as follows. Figure 5 The green color shown in Figure B indicates that the timing of the puncture under 3D image guidance was good; otherwise, it might include... Figure 5 The blue and yellow markings in Figure A indicate the locations of the top of the two diaphragm regions.

[0127] In summary, the puncture guidance method according to embodiments of the present invention, by fusing and superimposing the three-dimensional image information of the portal vein region and the edge position information of the top of the diaphragm region with the real-time acquired two-dimensional fluoroscopic image, can enhance the display of the three-dimensional structure of blood vessels in the two-dimensional fluoroscopic image, thereby providing intraoperative image guidance for TIPS. This allows the timing of puncture to be determined based on the patient's real-time respiratory status during surgery, reducing reliance on the doctor's experience to a certain extent, reducing positional deviations caused by the patient's respiratory movements during surgery, thereby improving puncture accuracy, reducing puncture attempts, shortening puncture time, reducing angiography and radiation doses, and reducing the health impact on patients and surgeons.

[0128] Furthermore, the puncture guidance method according to embodiments of the present invention provides the surgeon with a more intuitive display by visualizing the portal venous system and respiratory movements. The assistance of multiple pieces of information increases the surgeon's confidence in decision-making, avoids reliance on experience alone, thereby simplifying the surgical procedure and increasing surgical safety.

[0129] Reference manual attached Figure 6This illustrates the structure of an image processing-based puncture guidance device 600 provided in one embodiment of the present invention. For example... Figure 6 As shown, the device 600 may include:

[0130] Image acquisition module 610 is used to acquire two-dimensional perspective images of a target tissue region in real time, the target tissue region including the portal vein region and the diaphragm region;

[0131] Diaphragm region recognition module 620 is used to recognize the diaphragm region in the two-dimensional perspective image and obtain the first edge position information of the top of the diaphragm region.

[0132] Information acquisition module 630 is used to acquire three-dimensional image information of the portal vein region and second edge position information of the top of the diaphragm region, wherein the three-dimensional image information and the second edge position information are both determined based on the three-dimensional reconstructed image of the target tissue region;

[0133] The image overlay module 640 is used to overlay the three-dimensional image information and the second edge position information onto the two-dimensional perspective image to obtain an overlay image;

[0134] The image display module 650 is used to display the superimposed image for puncture guidance.

[0135] Alternatively, the image acquisition module 610 can be an angiography machine; the diaphragm region recognition module 620, the information acquisition module 630, the image overlay module 640, and the image display module 650 can be integrated into an image processing device, which implements the functions of the above modules. In other words, the device 600 can include an angiography machine and an image processing device, thereby realizing the above functions.

[0136] Alternatively, the image acquisition module 610, the diaphragm region recognition module 620, the information acquisition module 630, the image overlay module 640, and the image display module 650 can be integrated into an angiography machine, and the angiography machine can realize the functions of the above modules.

[0137] In one possible embodiment, the device 600 may further include:

[0138] The marking module is used to mark the first edge position information and the second edge position information in the overlay image, respectively.

[0139] In one possible embodiment, the device 600 may further include:

[0140] The image acquisition module is used to acquire a three-dimensional reconstructed image of the target tissue region;

[0141] The first image segmentation module is used to segment the three-dimensional reconstructed image based on a first preset image segmentation algorithm to obtain the three-dimensional image information of the portal vein region;

[0142] The second image segmentation module is used to identify the diaphragm region in the three-dimensional reconstructed image based on the second preset image segmentation algorithm, and obtain the second edge position information of the top of the diaphragm region.

[0143] Alternatively, the image acquisition module, the first image segmentation module, and the second image segmentation module can also be integrated into the angiography machine, with the angiography machine implementing the functions of each module. In other words, the device 600 may include an angiography machine and an image processing device (or only an angiography machine), thereby achieving the aforementioned functions.

[0144] Alternatively, the image acquisition module, the first image segmentation module, and the second image segmentation module can be integrated into a CTA device or an MRA device, and the functions of the above modules can be implemented by the CTA device or the MRA device. That is, the device 600 may include an angiography machine, an image processing device, and a CTA / MRA device (or only an angiography machine and a CTA / MRA device), thereby realizing the above functions.

[0145] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus provided in the above embodiments and the corresponding method embodiments belong to the same concept, and the specific implementation process can be found in the corresponding method embodiments, which will not be repeated here.

[0146] One embodiment of the present invention also provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing-based puncture guidance method provided in the above method embodiments.

[0147] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0148] In one specific embodiment Figure 7 A schematic diagram of the hardware structure of an electronic device for implementing the image processing-based puncture guidance method provided in the embodiments of the present invention is shown. The electronic device can be a computer terminal, a mobile terminal, or other devices. The electronic device can also participate in or include the image processing-based puncture guidance device provided in the embodiments of the present invention. Figure 7 As shown, the electronic device 700 may include a memory 710 with one or more computer-readable storage media, a processor 720 with one or more processing cores, an input unit 730, a display unit 740, a radio frequency (RF) circuit 750, a wireless fidelity (WiFi) module 760, and a power supply 770, among other components. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on electronic device 700, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0149] The memory 710 can be used to store software programs and modules. The processor 720 performs various functional applications and data processing by running or executing the software programs and modules stored in the memory 710 and by calling data stored in the memory 710. The memory 710 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 710 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 710 may also include a memory controller to provide the processor 720 with access to the memory 710.

[0150] The processor 720 is the control center of the electronic device 700. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 710, and by calling data stored in the memory 710, it performs various functions and processes data of the electronic device 700, thereby providing overall monitoring of the electronic device 700. The processor 720 can be a central processing unit, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0151] The input unit 730 can be used to receive input numerical or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 730 may include a touch-sensitive surface 731 and other input devices 732. Specifically, the touch-sensitive surface 731 may include, but is not limited to, a touchpad or a touch screen, and other input devices 732 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0152] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of electronic devices. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 740 may include a display panel 741, which may optionally be configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0153] The RF circuit 750 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 720 for processing; additionally, it transmits uplink data to the base station. Typically, the RF circuit 750 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 750 can also communicate wirelessly with networks and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0154] WiFi is a short-range wireless transmission technology. Electronic device 700, through WiFi module 760, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 7 WiFi module 760 is shown, but it is understood that it is not a necessary component of electronic device 700 and can be omitted as needed without changing the nature of the invention.

[0155] The electronic device 700 also includes a power supply 770 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 720 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 770 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0156] It should be noted that, although not shown, the electronic device 700 may also include a Bluetooth module, etc., which will not be described in detail here.

[0157] One embodiment of the present invention also provides a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing an image processing-based puncture guidance method, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing-based puncture guidance method provided in the above-described method embodiment.

[0158] Optionally, in embodiments of the present invention, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0159] One embodiment of the present invention also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing-based puncture guidance method provided in the various optional implementations described above.

[0160] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0161] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0162] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A puncture guidance device based on image processing, characterized in that, include: The image acquisition module is used to acquire two-dimensional perspective images of the target tissue region in real time, including the portal vein region and the diaphragm region; The diaphragm region recognition module is used to identify the diaphragm region in the two-dimensional perspective image and obtain the first edge position information of the top of the diaphragm region. The information acquisition module is used to acquire three-dimensional image information of the portal vein region and the second edge position information of the top of the diaphragm region. The three-dimensional image information and the second edge position information are both determined based on a three-dimensional reconstructed image including the target tissue region. An image overlay module is used to overlay the three-dimensional image information and the second edge position information onto the two-dimensional perspective image to obtain an overlay image; An image display module is used to display the superimposed image for puncture guidance; A marking module is used to mark the first edge position information and the second edge position information in the overlay image respectively; when the first edge position information and the second edge position information coincide, the first edge position information and the second edge position information are marked first; when the first edge position information and the second edge position information do not coincide, the first edge position information is marked second and the second edge position information is marked third; the second mark and the third mark are different marks.

2. The puncture guiding device according to claim 1, characterized in that, Also includes: The image acquisition module is used to acquire a three-dimensional reconstructed image including the target tissue region; The first image segmentation module is used to segment the three-dimensional reconstructed image based on a first preset image segmentation algorithm to obtain the three-dimensional image information of the portal vein region; The second image segmentation module is used to identify the diaphragm region in the three-dimensional reconstructed image based on the second preset image segmentation algorithm, and obtain the second edge position information of the top of the diaphragm region.

3. The puncture guiding device according to claim 2, characterized in that, The image acquisition module is used to acquire cone-beam computed tomography (CBCT) image data including the target tissue region, and to reconstruct a three-dimensional reconstructed image including the target tissue region based on the CBCT image data.

4. The puncture guiding device according to claim 2, characterized in that, The image acquisition module is used to acquire computed tomography angiography image data or magnetic resonance angiography image data including the target tissue region, and to reconstruct a three-dimensional reconstructed image including the target tissue region based on the computed tomography angiography image data or the magnetic resonance angiography image data.

5. The puncture guiding device according to claim 4, characterized in that, The image acquisition module is used to register the three-dimensional reconstructed image to the cone-beam computed tomography (CBCT) imaging space to obtain a three-dimensional reconstructed image in the CBCT imaging space.

Images