Blood vessel tracking method for medical image, and surgical path planning method and system

Abstract

The invention discloses a blood vessel tracking method for a medical image, and a surgical path planning method and system. The blood vessel tracking method comprises the following steps: acquiring a contrast agent blood vessel image and a non-contrast agent blood vessel image of the same detection object; processing the contrast agent blood vessel image, and determining a background region of the contrast agent blood vessel image and a blood vessel region of the contrast agent blood vessel image; constructing a motion model according to the background area of the contrast agent blood vessel image and the blood vessel area of the contrast agent blood vessel image; a predicted vessel region is determined in the non-contrast vessel image using the motion model. According to the method, the position of the blood vessel is accurately predicted on the image where the blood vessel is not developed by utilizing the XR image sequence without any time phase information and combining the condition that the blood vessel moves along with breathing / heartbeat on the developed image.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a blood vessel tracking method of medical imaging, and a surgical path planning method and system. Background Art

[0002] During vascular interventional surgery, doctors inject contrast agents and then take X-rays to obtain the relative position of surgical instruments such as guidewires in real time within blood vessels. This allows doctors to determine the location of plaques / lesions equivalent to surgical instruments and make the next surgical decision.

[0003] However, on the one hand, the contrast agent flows quickly and is highly toxic, making it impossible to visualize the blood vessels for a long time during interventional surgery; on the other hand, due to factors such as breathing or heartbeat, the blood vessels move, making interventional surgery more difficult.

[0004] Therefore, how to accurately predict the location of blood vessels in medical images after the contrast agent disappears is an urgent problem to be solved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the defect in the prior art that the blood vessel position cannot be accurately predicted on an image in which the blood vessels are not visualized, and to provide a blood vessel tracking method for medical images, a surgical path planning method and system.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] In a first aspect, the present invention provides a blood vessel tracking method for medical images, the blood vessel tracking method comprising:

[0008] Acquire contrast agent vascular images and non-contrast agent vascular images of the same detection object;

[0009] Processing the contrast agent blood vessel image to determine a background area of ​​the contrast agent blood vessel image and a blood vessel area of ​​the contrast agent blood vessel image;

[0010] Constructing a motion model according to the background region of the contrast agent vascular image and the vascular region of the contrast agent vascular image; the motion model reflects a motion relationship matrix of the vascular region of the contrast agent vascular image and / or the background region between different frames;

[0011] Using the motion model, a predicted vessel region is determined in the non-contrast agent vessel image.

[0012] Preferably, the contrast agent vascular image includes a single-frame standard enhanced image and multiple frames of temporally continuous non-standard enhanced images, and the step of constructing a motion model based on the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image includes:

[0013] Processing the standard enhanced image to obtain a first blood vessel region and a first background region, and processing the non-standard enhanced image to obtain a second blood vessel region and a second background region;

[0014] calculating a first blood vessel motion between a pixel point corresponding to the second blood vessel region and a pixel point corresponding to the first blood vessel region;

[0015] Calculating a first background motion between a pixel point corresponding to the first background area and a pixel point corresponding to the second background area;

[0016] The motion model is constructed according to the first blood vessel motion and the first background motion.

[0017] Preferably, the step of determining the predicted blood vessel region in the non-contrast agent blood vessel image using the motion model comprises:

[0018] calculating a second background motion between the non-contrast agent blood vessel image and the standard enhanced image;

[0019] Calculating a second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion and the motion model;

[0020] The position of the predicted blood vessel region is determined according to the second blood vessel motion.

[0021] In a second aspect, the present invention provides a blood vessel tracking system for medical images, the blood vessel tracking system comprising:

[0022] An acquisition module, used for acquiring a contrast agent blood vessel image and a non-contrast agent blood vessel image of the same detection object;

[0023] A region delineation module, used for processing the contrast agent blood vessel image to determine a background region of the contrast agent blood vessel image and a blood vessel region of the contrast agent blood vessel image;

[0024] A construction module, used to construct a motion model according to the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image; the motion model reflects the motion relationship matrix of the vascular area of ​​the contrast agent vascular image and / or the background area between different frames;

[0025] A determination module is used to determine a predicted blood vessel region in the non-contrast agent blood vessel image by using the motion model.

[0026] Preferably, the contrast agent vascular image includes a single-frame standard enhanced image and multiple-frame temporally continuous non-standard enhanced images, and the construction module includes:

[0027] a processing unit, configured to process the standard enhanced image to obtain a first blood vessel region and a first background region, and to process the non-standard enhanced image to obtain a second blood vessel region and a second background region;

[0028] A first calculation unit, configured to calculate a first blood vessel motion between a pixel point corresponding to the second blood vessel region and a pixel point corresponding to the first blood vessel region;

[0029] A second calculation unit, used for calculating a first background motion between a pixel point corresponding to the first background area and a pixel point corresponding to the second background area;

[0030] A construction unit is used to construct the motion model according to the first blood vessel motion and the first background motion.

[0031] Preferably, the determining module comprises:

[0032] a third calculation unit, configured to calculate a second background motion between the non-contrast agent blood vessel image and the standard enhanced image;

[0033] a fourth calculation unit, configured to calculate a second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion and the motion model;

[0034] A determination unit is used to determine the position of the predicted blood vessel region according to the second blood vessel motion.

[0035] In a third aspect, the present invention provides a surgical path planning method, the planning method comprising:

[0036] Using the blood vessel tracking method of medical imaging as described in the first aspect, predicting the position information of the predicted blood vessel area in the non-contrast agent blood vessel image;

[0037] Acquiring a relative position of a surgical instrument in the predicted blood vessel region based on the position information;

[0038] The planned path of the vascular intervention surgery is obtained according to the relative position and the preoperative path.

[0039] In a fourth aspect, the present invention provides a surgical path planning system, the planning system comprising:

[0040] A prediction module, used to predict the position information of the predicted blood vessel area in the non-contrast agent blood vessel image by using the blood vessel tracking system of the medical image as described in the second aspect;

[0041] A position acquisition module, used for acquiring the relative position of the surgical instrument in the predicted blood vessel area based on the position information;

[0042] A path generation module is used to obtain a planned path for the vascular interventional surgery based on the relative position and the preoperative path.

[0043] In a fifth aspect, the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the blood vessel tracking method for medical images as described in the first aspect, or implements the surgical path planning method as described in the third aspect.

[0044] In a sixth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the blood vessel tracking method of medical imaging as described in the first aspect, or implements the surgical path planning method as described in the third aspect.

[0045] The positive and progressive effects of the present invention are: obtaining contrast agent vascular images and non-contrast agent vascular images of the same detection object, processing the contrast agent vascular images, determining the background area of ​​the contrast agent vascular images and the vascular area of ​​the contrast agent vascular images; constructing a motion model based on the background area of ​​the contrast agent vascular images and the vascular area of ​​the contrast agent vascular images; using the motion model, determining the predicted vascular area in the non-contrast agent vascular images. The present invention integrates multi-frame vascular information to obtain rich vascular; using XR image sequences without any phase information, combined with the movement of blood vessels with breathing / heartbeat on the developed images, accurately predicts the vascular position in real time on the images where the blood vessels are not developed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of the blood vessel tracking method of medical images according to the first embodiment of the present invention.

[0047] Figure 2 4 is a flow chart of step S3 of the blood vessel tracking method for medical images according to embodiment 1 of the present invention.

[0048] Figure 3 4 is a flow chart of step S4 of the blood vessel tracking method for medical images according to embodiment 1 of the present invention.

[0049] Figure 4 This is a diagram illustrating the principle of the blood vessel tracking method for medical images according to Embodiment 1 of the present invention.

[0050] Figure 5 This is a first application example diagram of the blood vessel tracking method for medical images according to Embodiment 1 of the present invention.

[0051] Figure 6 This is a schematic diagram of a second application of the blood vessel tracking method for medical images according to the first embodiment of the present invention.

[0052] Figure 7 This is a schematic diagram of modules of a blood vessel tracking system for medical images according to Embodiment 2 of the present invention.

[0053] Figure 8 This is a flowchart of a surgical path planning method according to Embodiment 3 of the present invention.

[0054] Fig. 9 This is a module diagram of a surgical path planning system according to Example 4 of the present invention.

[0055] Fig.10 This is a schematic diagram of the structure of an electronic device for implementing a blood vessel tracking method for medical images according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0057] In vascular interventional surgery, doctors usually take X-ray images after injecting contrast agents into patients to obtain the relative position of surgical instruments such as guidewires in blood vessels in real time, thereby determining the position of plaques or lesions in blood vessels relative to surgical instruments to determine the next decision. Generally, contrast agents flow quickly and are toxic in themselves. The amount of contrast agents that can be taken is limited by regulations, making it impossible to visualize blood vessels for a long time during surgery. At the same time, physiological movement causes blood vessel movement, further increasing the difficulty of surgery. This application proposes a medical imaging blood vessel tracking method to address the problem that blood vessels cannot be visualized for a long time due to fast contrast agent flow. The method includes:

[0058] First, a non-contrast agent blood vessel image of the subject is acquired.

[0059] In one embodiment, the non-contrast agent vascular image is a vascular image obtained by scanning and imaging after contrast agent is injected into the detection object and the contrast agent disappears; or, the non-contrast agent vascular image is a vascular image obtained by scanning and imaging without injecting contrast agent into the detection object.

[0060] Secondly, a motion model is obtained, where the motion model can reflect the motion relationship matrix of the blood vessel region and / or the background region, where the motion relationship matrix can characterize the real-time motion of the blood vessel caused by the motion of breathing and / or heartbeat.

[0061] In one embodiment, the motion model can be obtained by training a neural network with paired historical contrast agent vascular images and historical non-contrast agent vascular images, which are acquired when contrast agent is injected and after contrast agent disappears in the same test.

[0062] In one embodiment, the motion model may be obtained by processing a contrast agent blood vessel image of the detection object, wherein the contrast agent blood vessel image and the non-contrast agent blood vessel image of the detection object are both obtained during the current detection.

[0063] In one embodiment, the motion model can be obtained by processing the historical contrast agent vascular images of the detection object. Specifically, the historical contrast agent vascular images are obtained in the historical period before the current detection of the detection object, and the non-contrast agent vascular images of the detection object are obtained in the current detection, and the historical period can be 3 months, 6 months, 9 months or a longer time interval from the current detection. The historical contrast agent vascular images can first be registered with the non-contrast agent vascular images of the detection object to meet the consistency of scanning in different time periods, and the intermediate historical contrast agent vascular images can be obtained, and the motion model is obtained by processing the intermediate historical contrast agent vascular images.

[0064] Next, the non-contrast agent vascular image is input into the motion model to determine the vascular area, and the vascular area is displayed in the non-contrast agent vascular image. That is, the vascular position is displayed on the vascular non-development image to obtain the same or similar effect as the contrast agent vascular image.

[0065] Example 1

[0066] This embodiment provides a blood vessel tracking method for medical images, such as Figure 1 As shown, by way of example and not limitation, the method may include the following steps:

[0067] S1. Acquire a contrast-mediated vascular image and a non-contrast-mediated vascular image of the same detection object.

[0068] S2. Segment the contrast agent blood vessel image to determine a background region of the contrast agent blood vessel image and a blood vessel region of the contrast agent blood vessel image.

[0069] S3. Construct a motion model based on the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image; the motion model reflects the motion relationship matrix of the vascular area and / or the background area of ​​the contrast agent vascular image between different frames.

[0070] S4. Determine the predicted vascular region in the non-contrast agent vascular image using the motion model.

[0071] With respect to the above step S1, medical images can be obtained from a medical imaging system such as a hospital's clinical information system or a radiology department information management system. The medical images can be obtained by detecting the same test object in the past historical period, or can be obtained by detecting the same test object in the current period. Multiple frames of continuous contrast agent vascular images and multiple frames of continuous non-contrast agent vascular images can be screened out from the medical images detected in the current period, or multiple frames of continuous contrast agent vascular images can be screened out from the medical images detected in the past historical period.

[0072] The contrast agent vascular image and the non-contrast agent vascular image are two images of a certain part of the same patient, both of which contain vascular areas. The contrast agent vascular image is a medical image in the development stage, and the non-contrast agent vascular image is a medical image in the non-development stage. It can be understood that the two images can be any 2D image data (such as Extended Reality, XR images for short) of the human head blood vessels, neck artery blood vessels, pulmonary artery blood vessels, pulmonary vein blood vessels, group artery blood vessels, upper limb artery blood vessels and lower limb artery blood vessels. In this embodiment, the contrast agent vascular image and the non-contrast agent vascular image are DSA images taken by a digital subtraction angiography (Digital subtraction angiography) device. The contrast agent vascular image and the non-contrast agent vascular image are obtained by taking pictures during the current treatment process, and the contrast agent vascular image is taken after the contrast agent is injected into the test object, and the non-contrast agent vascular image is taken after the contrast agent completely disappears or weakens and the blood vessels cannot be enhanced.

[0073] For the above step S2, the contrast agent vascular image can be processed by a manual delineation process of an operator, or by an automatic detection process through a neural network, or by automatic determination based on a segmentation method such as a threshold algorithm, an edge detection algorithm, a clustering algorithm, a deformable model algorithm, a region segmentation algorithm, etc. Taking the region segmentation algorithm as an example, the vascular segmentation algorithm is used to segment the vascular and non-vascular from the contrast agent vascular image. For example, for an m×n contrast agent vascular image The vascular template can be obtained through the vascular segmentation algorithm Thus, according to the blood vessel template M iH The background area of ​​the contrast agent blood vessel image and the blood vessel area of ​​the contrast agent blood vessel image are determined. Alternatively, the contrast agent blood vessel image may be input into a blood vessel segmentation model obtained by pre-training according to training samples of blood vessel classification information to obtain a blood vessel segmentation result, which may be a result of distinguishing the background area of ​​the contrast agent blood vessel image from the blood vessel area of ​​the contrast agent blood vessel image.

[0074] For the above step S3, a single frame of standard enhanced image is selected from the contrast agent vascular image, and the motion relationship matrix of the background area and the motion relationship matrix of the vascular area of ​​the contrast agent vascular image between any two different frames are obtained. The background motion is determined according to the motion relationship matrix of the background area, and the vascular motion is determined according to the motion relationship matrix of the vascular area. Further, a motion model is constructed based on the relationship model determined by combining vascular motion, background motion and experience.

[0075] With respect to the above step S4, the motion of the background area between the standard enhanced image and the non-contrast agent vascular image in which the blood vessels are no longer visualized is calculated using an optical flow algorithm. The motion of the vascular area between the standard enhanced image and the non-contrast agent vascular image is inferred by combining with a pre-established motion model. The specific position of the predicted vascular area in the non-contrast agent image is determined based on the motion of the vascular area.

[0076] In an alternative embodiment, Figure 2 As shown, the contrast agent blood vessel image includes a single-frame standard enhanced image and multiple-frame temporally continuous non-standard enhanced images. Step S3 includes:

[0077] S31 . Process the standard enhanced image to obtain a first blood vessel region and a first background region, and process the non-standard enhanced image to obtain a second blood vessel region and a second background region.

[0078] S32, calculating the first blood vessel motion between the pixel points corresponding to the second blood vessel region and the pixel points corresponding to the first blood vessel region.

[0079] S33, calculating the first background motion between the pixel points corresponding to the first background area and the pixel points corresponding to the second background area.

[0080] S34, constructing a motion model according to the first blood vessel motion and the second background motion.

[0081] For the above step S31, the standard enhanced image can be input into a pre-trained vascular segmentation model, and the vascular segmentation model obtains the vascular space step information corresponding to the standard enhanced image. The non-standard enhanced image can be input into a pre-trained vascular segmentation model, and the vascular segmentation model obtains the vascular space step information corresponding to the non-standard enhanced image, thereby enhancing the accuracy of the vascular segmentation result with the help of the vascular space step information. It can be understood that the vascular segmentation model is obtained by training according to the training samples with the classification information of the blood vessels of interest.

[0082] A region segmentation algorithm can also be used to segment the standard enhanced image and the non-standard enhanced image, thereby dividing the first blood vessel region, the second blood vessel region, and the first background region and the second background region of the non-blood vessel portion. Specifically, a blood vessel segmentation algorithm such as the Otsu method, the region growing method, the Sato filter method, and the watershed algorithm is used to segment the blood vessels from the standard enhanced image of 2D XA (X-ray Angiography) to obtain the first blood vessel template. M i1The pixel value includes 0 or 1, 0 means that the pixel position in the standard enhanced image is the first background area, and 1 means that the pixel position in the standard enhanced image is the first blood vessel area. It can be understood that the blood vessel segmentation algorithm based on deep learning can also be used to segment it, which improves the segmentation accuracy of the blood vessel segmentation process and improves the accuracy of the subsequent motion model construction. By analogy, the same blood vessel segmentation algorithm is used to segment the blood vessels from the non-standard enhanced image of 2D XA to obtain the second blood vessel template M i2 The pixel value also includes 0 or 1, 0 indicates that the position of the pixel point in the non-standard enhanced image is the second background area, and 1 indicates that the position of the pixel point in the non-standard enhanced image is the second blood vessel area.

[0083] In this embodiment, the following formula is used to express the blood vessel template M iH , I iH represents the contrast agent vascular image, and θ represents the learned network model parameters.

[0084] M iH =f s (I iH ;θ)

[0085] For the above steps S32-S33, a dense optical flow algorithm or a sparse optical flow algorithm is used to calculate the change in the time domain between the pixel points corresponding to the second blood vessel region and the pixel points corresponding to the first blood vessel region, so as to obtain the motion relationship of the first blood vessel. Similarly, the dense optical flow algorithm or the sparse optical flow algorithm is used to calculate the change in the time domain between the corresponding pixel points in the second background area and the corresponding pixel points in the first background area to obtain the first background motion relationship D ij Each pixel in represents two frames of image I i and I j The movement along the x or y direction between them, k represents the number of consecutive non-standard enhanced images in a multi-frame time sequence.

[0086] In this embodiment, considering the real-time performance of the algorithm, a sparse optical flow algorithm based on the Shi-Tomasi feature pyramid LK (Lucas-Kanade) is preferably used for calculation, but no specific limitation is imposed on the optical flow algorithm.

[0087] D ij =f o (I i ,I j )

[0088] In step S34, the first blood vessel motion between the standard enhanced image and the non-standard enhanced image is calculated using the blood vessel segmentation algorithm and the optical flow algorithm. and the first background motion k represents the number of consecutive non-standard enhanced images in a multi-frame temporal sequence. The motion model parameters are estimated based on the first blood vessel motion, the first background motion, and the empirically determined relationship model g. In this embodiment, the specific form of the relationship model g is not limited.

[0089]

[0090] In an alternative embodiment, Figure 3 As shown, step S4 includes:

[0091] S41. Calculate the second background motion between the non-contrast agent vascular image and the standard enhanced image.

[0092] S42: Calculate a second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion and the motion model.

[0093] S43: Determine the position of the predicted blood vessel region according to the motion of the second blood vessel.

[0094] With respect to the above steps S41-S43, for the non-contrast agent vascular image I where the blood vessels are no longer visualized j , using the dense optical flow algorithm or the sparse optical flow algorithm to calculate the second background motion of the background area between it and the standard enhanced image. Using the second background motion and motion model parameters The second vessel motion between the non-contrast agent vessel image and the standard enhanced image is inferred.

[0095]

[0096] In an alternative embodiment, Figure 4 As shown, a multi-frame continuous vascular development image sequence and a single-frame reference frame image are obtained. The vascular development image sequence and the reference frame are images obtained after shooting the same part, both of which contain blood vessels and are in the development stage. The vascular development image sequence and the reference frame are simultaneously segmented using a vascular segmentation algorithm to obtain two parts: the vascular area and the background area. The motion between the reference frame image and each frame of the vascular development image is calculated using an optical flow algorithm, and a relationship model is constructed by learning the relationship between vascular motion and background motion. For the image of the vascular position to be predicted where the blood vessel is no longer developed, the motion between the image to be predicted and the reference frame is obtained, and the vascular motion of the frame image is predicted in combination with the vascular motion-background motion relationship, thereby predicting the vascular position on the image to be predicted.

[0097] In an alternative embodiment, Figure 5As shown in the figure, the white area represents the blood vessel area of ​​the contrast medium blood vessel image, and the other non-white area represents the background area of ​​the contrast medium blood vessel image. After segmenting the contrast medium blood vessel image sequence, the blood vessel area and background area of ​​the contrast medium blood vessel image are extracted, and the motion model is established using the blood vessel area and background area. Figure 6 As shown in the figure, the white area indicates the predicted vascular area. By calculating the background motion between the non-contrast agent vascular image and the standard enhanced image, and then using the pre-built motion model, the vascular motion is predicted based on the background motion, and the vascular motion in each frame of the non-contrast agent vascular image after the contrast agent disappears is tracked in real time, thereby determining the specific location of the predicted vascular area.

[0098] In this embodiment, a blood vessel tracking method for medical imaging is provided, which obtains contrast agent blood vessel images and non-contrast agent blood vessel images of the same detection object; processes the contrast agent blood vessel images to determine the background area of ​​the contrast agent blood vessel images and the blood vessel area of ​​the contrast agent blood vessel images; constructs a motion model based on the background area of ​​the contrast agent blood vessel images and the blood vessel area of ​​the contrast agent blood vessel images; and uses the motion model to determine the predicted blood vessel area in the non-contrast agent blood vessel images. The present invention uses XR image sequences without any phase information, combined with the situation that the blood vessels move with breathing / heartbeat in the developed images, to accurately predict the blood vessel position in the images where the blood vessels are not developed.

[0099] Example 2

[0100] This embodiment provides a blood vessel tracking system for medical images, such as Figure 7 As shown, by way of example and not limitation, the blood vessel tracking system includes: an acquisition module 210 , a region delineation module 220 , a construction module 230 and a determination module 240 .

[0101] The acquisition module 210 is used to acquire a contrast agent vascular image and a non-contrast agent vascular image of the same detection object.

[0102] The region delineation module 220 is used to process the contrast agent blood vessel image to determine the background region of the contrast agent blood vessel image and the blood vessel region of the contrast agent blood vessel image.

[0103] The construction module 230 is used to determine a motion model according to the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image; the motion model reflects the motion relationship matrix of the vascular area and / or background area of ​​the contrast agent vascular image between different frames.

[0104] The determination module 240 is used to determine the predicted blood vessel region in the non-contrast agent blood vessel image by using the motion model.

[0105] The acquisition module 210 can acquire medical images from a medical imaging system such as a hospital's clinical information system or a radiology department information management system. The medical images can be acquired by detecting the same detection object in the past historical period, or can be acquired by detecting the same detection object in the current period. Multiple frames of continuous contrast agent vascular images and multiple frames of continuous non-contrast agent vascular images can be screened out from the medical images acquired by detecting the current period, or multiple frames of continuous contrast agent vascular images can be screened out from the medical images acquired by detecting the past historical period.

[0106] The contrast agent vascular image and the non-contrast agent vascular image are two images of a certain part of the same patient, both of which contain vascular areas. The contrast agent vascular image is a medical image in the development stage, and the non-contrast agent vascular image is a medical image in the non-development stage. It can be understood that the two images can be any 2D image data (XR images) of the human head blood vessels, neck artery blood vessels, pulmonary artery blood vessels, pulmonary vein blood vessels, group artery blood vessels, upper limb artery blood vessels and lower limb artery blood vessels. In this embodiment, the contrast agent vascular image and the non-contrast agent vascular image are DSA images taken by a digital subtraction angiography (Digital subtraction angiography) device. The contrast agent vascular image and the non-contrast agent vascular image are obtained by taking pictures during the current treatment process, and the contrast agent vascular image is taken after the contrast agent is injected into the test object, and the non-contrast agent vascular image is taken after the contrast agent completely disappears or weakens and the blood vessels cannot be enhanced.

[0107] The region delineation module 220 processes the contrast agent vascular image by manual delineation by an operator, or by automatic detection through a neural network, or by automatic determination based on a segmentation method such as a threshold algorithm, an edge detection algorithm, a clustering algorithm, a deformable model algorithm, or a region segmentation algorithm. Taking the region segmentation algorithm as an example, the vascular segmentation algorithm is used to segment the vascular and non-vascular from the contrast agent vascular image. For example, for an m×n contrast agent vascular image The vascular template can be obtained through the vascular segmentation algorithm Thus, according to the blood vessel template M iH The background area of ​​the contrast agent blood vessel image and the blood vessel area of ​​the contrast agent blood vessel image are determined. Alternatively, the contrast agent blood vessel image may be input into a blood vessel segmentation model obtained by pre-training according to training samples of blood vessel classification information to obtain a blood vessel segmentation result, which may be a result of distinguishing the background area of ​​the contrast agent blood vessel image from the blood vessel area of ​​the contrast agent blood vessel image.

[0108] A single frame of standard enhanced image is selected from the contrast agent vascular image, and a motion relationship matrix of the background region and a motion relationship matrix of the vascular region of the contrast agent vascular image between any two different frames are obtained. The background motion is determined according to the motion relationship matrix of the background region, and the vascular motion is determined according to the motion relationship matrix of the vascular region. The construction module 230 further constructs a motion model according to the vascular motion, the background motion and the relationship model determined by combining experience.

[0109] The motion of the background area between the standard enhanced image and the non-contrast agent vascular image where the blood vessels are no longer visualized is calculated using an optical flow algorithm. The motion of the vascular area between the standard enhanced image and the non-contrast agent vascular image is inferred by combining a pre-established motion model. The determination module 240 determines the specific position of the predicted vascular area in the non-contrast agent image based on the motion relationship matrix of the vascular area.

[0110] In an optional embodiment, the contrast agent vascular image includes a single-frame standard enhanced image and multiple frames of temporally continuous non-standard enhanced images, and the construction module 230 includes:

[0111] The processing unit 231 is used to process the standard enhanced image to obtain a first blood vessel region and a first background region, and to process the non-standard enhanced image to obtain a second blood vessel region and a second background region.

[0112] The first calculation unit 232 is used to calculate the first blood vessel motion between the pixel points corresponding to the second blood vessel region and the pixel points corresponding to the first blood vessel region.

[0113] The second calculating unit 233 is used to calculate the first background motion between the pixel points corresponding to the first background area and the pixel points corresponding to the second background area.

[0114] The construction unit 234 is configured to construct a motion model according to the first blood vessel motion and the first background motion.

[0115] The processing unit 231 can input the standard enhanced image into a pre-trained blood vessel segmentation model, and the blood vessel segmentation model obtains the blood vessel spatial step information corresponding to the standard enhanced image. The non-standard enhanced image can be input into the pre-trained blood vessel segmentation model, and the blood vessel segmentation model obtains the blood vessel spatial step information corresponding to the non-standard enhanced image, thereby enhancing the accuracy of the blood vessel segmentation result with the blood vessel spatial step information. It can be understood that the blood vessel segmentation model is obtained by training according to the training samples with the classification information of the blood vessels of interest.

[0116] The processing unit 231 can also use a region segmentation algorithm to segment the standard enhanced image and the non-standard enhanced image, thereby dividing the first blood vessel region, the second blood vessel region, and the first background region and the second background region of the non-blood vessel portion. Specifically, a blood vessel segmentation algorithm such as the Otsu method, the region growing method, the Sato filter method, and the watershed algorithm is used to segment the blood vessels from the standard enhanced image of 2D XA (X-ray Angiography) to obtain a first blood vessel template. M i1 The pixel value includes 0 or 1, 0 means that the pixel position in the standard enhanced image is the first background area, and 1 means that the pixel position in the standard enhanced image is the first blood vessel area. It can be understood that the blood vessel segmentation algorithm based on deep learning can also be used to segment it, which improves the segmentation accuracy of the blood vessel segmentation process and improves the accuracy of the subsequent motion model construction. By analogy, the same blood vessel segmentation algorithm is used to segment the blood vessels from the non-standard enhanced image of 2D XA to obtain the second blood vessel template M i2 The pixel value also includes 0 or 1, 0 indicates that the position of the pixel point in the non-standard enhanced image is the second background area, and 1 indicates that the position of the pixel point in the non-standard enhanced image is the second blood vessel area.

[0117] In this embodiment, the following formula is used to express the blood vessel template M iH , I iH represents the contrast agent vascular image, and θ represents the learned network model parameters.

[0118] M iH =f s (I iH ;θ)

[0119] The first calculation unit 232 uses a dense optical flow algorithm or a sparse optical flow algorithm to calculate the change in the time domain between the pixel points corresponding to the second blood vessel region and the pixel points corresponding to the first blood vessel region to obtain the motion of the first blood vessel. Similarly, the dense optical flow algorithm or the sparse optical flow algorithm is used to calculate the change in the time domain between the corresponding pixel points of the second background area and the corresponding pixel points of the first background area to obtain the first background motion D ij Each pixel in represents two frames of image I i and I j The movement along the x or y direction between them, k represents the number of consecutive non-standard enhanced images in a multi-frame time sequence.

[0120] In this embodiment, considering the real-time performance of the algorithm, a sparse optical flow algorithm based on the Shi-Tomasi feature pyramid LK (Lucas-Kanade) is preferably used for calculation, but no specific limitation is imposed on the optical flow algorithm.

[0121] D ij =f o (I i ,I j )

[0122] The first vessel motion between the standard enhanced image and the non-standard enhanced image is calculated using the vessel segmentation algorithm and the optical flow algorithm. and the first background motion k represents the number of consecutive non-standard enhanced images in a multi-frame temporal sequence. The construction unit 234 estimates the motion model parameters according to the first blood vessel motion, the first background motion and the empirically determined relationship model g. In this embodiment, the specific form of the relationship model g is not limited.

[0123]

[0124] In an optional implementation, the determination module 240 includes:

[0125] The third calculation unit 241 is used to calculate the second background motion between the non-contrast agent blood vessel image and the standard enhanced image.

[0126] The fourth calculation unit 242 is used to calculate the second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion and the motion model.

[0127] The determination unit 243 is configured to determine the position of the predicted blood vessel region according to the motion of the second blood vessel.

[0128] For non-contrast vascular images where the vessels are no longer visible I j The third calculation unit 241 calculates the second background motion of the background area between it and the standard enhanced image using a dense optical flow algorithm or a sparse optical flow algorithm. The fourth calculation unit 242 calculates the second background motion and the motion model The second vessel motion between the non-contrast agent vessel image and the standard enhanced image is inferred.

[0129]

[0130] In this embodiment, a medical imaging vascular tracking system is provided, wherein an acquisition module acquires a contrast agent vascular image and a non-contrast agent vascular image of the same detection object; a region delineation module processes the contrast agent vascular image to determine the background region of the contrast agent vascular image and the vascular region of the contrast agent vascular image; a construction module constructs a motion model based on the background region of the contrast agent vascular image and the vascular region of the contrast agent vascular image; and a determination module determines the predicted vascular region in the non-contrast agent vascular image using the motion model. The present invention utilizes an XR image sequence without any phase information, and combines the situation that the blood vessels in the developed image move with breathing / heartbeat to accurately predict the position of the blood vessels in the image where the blood vessels are not developed.

[0131] Example 3

[0132] This embodiment provides a surgical path planning method, such as Figure 8 As shown, by way of example and not limitation, the planning method includes the following steps:

[0133] Step S100: using the blood vessel tracking method for medical images as in Example 1, predicting the position information of the predicted blood vessel region in the non-contrast agent blood vessel image.

[0134] Step S200: acquiring the relative position of the surgical instrument in the predicted blood vessel region based on the position information.

[0135] Step S300: obtaining a planned path for the vascular interventional surgery according to the relative position and the preoperative path.

[0136] With respect to the above step S100, for the non-contrast agent blood vessel image, the position information of each pixel in the non-contrast agent blood vessel image is determined. The blood vessel tracking method of medical imaging in Example 1 is used to calculate the position information of the predicted blood vessel region on the non-contrast agent blood vessel image with blood vessel development, taking into account the movement of the blood vessel with factors such as breathing / heartbeat.

[0137] For the above steps S200-S300, the relative position of the surgical instrument in the blood vessel during the interventional surgery is obtained based on the predicted position information of the vascular region. Combined with the preoperative path planned before the interventional surgery, the planned path of the vascular interventional surgery is obtained. Specifically, according to the relative position in the predicted vascular region, a three-dimensional image of the blood vessel can be generated by a seed point method or a region growing algorithm, and the pathological position is determined based on the three-dimensional image of the blood vessel. The region growing algorithm refers to an iterative process of developing grouped pixels or regions into larger regions, maintaining certain similarities and connectivity during the merging process. The use of the region generation algorithm can quickly, accurately and reliably segment and identify the target in the image. In the three-dimensional space coordinate system, the puncture navigation path is performed according to the pathological position and the required surgery to generate a planned path. This method allows the vascular path to be obtained after the contrast agent disappears, effectively assisting doctors in performing vascular interventional surgery, shortening the operation time, improving the accuracy of interventional surgery, and reducing the required contrast dose.

[0138] In this embodiment, a surgical path planning method is provided, which generates a vascular path according to the predicted position of the vascular area, thereby solving the problem that the blood vessels cannot be visualized for a long time due to the fast flow of contrast agent. The position of the blood vessels can still be predicted in real time after the contrast agent is sold, effectively assisting doctors in surgery. The interventional surgery time is shortened, the accuracy of the interventional surgery is improved, and the required dose of contrast agent is reduced.

[0139] Example 4

[0140] This embodiment provides a surgical path planning system, such as Fig. 9 As shown, the planning system includes: an acquisition module 310 , a blood vessel path generation module 320 and a surgical path generation module 330 .

[0141] A prediction module 310, for predicting the position of a predicted blood vessel region in a non-contrast agent blood vessel image using the blood vessel tracking system for medical imaging in Example 2;

[0142] A position acquisition module 320, for acquiring a relative position of the surgical instrument in the predicted blood vessel region based on the position information;

[0143] The path generation module 330 is used to obtain a planned path for the vascular interventional surgery according to the relative position and the preoperative path.

[0144] For the non-contrast agent blood vessel image, the position information of each pixel in the non-contrast agent blood vessel image is determined. Using the medical image blood vessel tracking system in Example 2, the prediction module 310 calculates the position information of the predicted blood vessel area in the case where the blood vessel moves with factors such as breathing / heartbeat on the non-contrast agent blood vessel image where the blood vessel is not visualized.

[0145] The position acquisition module 320 acquires the relative position of the surgical instrument in the predicted vascular area based on the position information, and the path generation module 330 combines the preoperative path planned before the interventional surgery to obtain the planned path for the vascular interventional surgery. Specifically, according to the relative position in the predicted vascular area, a three-dimensional image of the blood vessel can be generated by a seed point method or a region growing algorithm, and the pathological position is determined based on the three-dimensional image of the blood vessel. The region growing algorithm refers to an iterative process of developing grouped pixels or regions into larger regions, maintaining certain similarities and connectivity during the merging process. The use of the region generation algorithm can quickly, accurately and reliably segment and identify the target in the image. In the three-dimensional space coordinate system, the puncture navigation path is performed according to the pathological position and the required surgery to generate a planned path. This method allows the vascular path to be obtained after the contrast agent disappears, effectively assisting doctors in performing vascular interventional surgery, shortening the operation time, improving the accuracy of interventional surgery, and reducing the required contrast dose.

[0146] In this embodiment, a surgical path planning system is provided, which generates a vascular path according to the predicted position of the vascular area, thereby solving the problem that the blood vessels cannot be visualized for a long time due to the fast flow of contrast agent. The position of the blood vessels can still be predicted in real time after the contrast agent is sold, effectively assisting doctors in surgery. The interventional surgery time is shortened, the accuracy of the interventional surgery is improved, and the required dose of contrast agent is reduced.

[0147] Example 5

[0148] Fig.10 The present invention provides a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the blood vessel tracking method of medical imaging of embodiment 1 or the surgical path planning method of embodiment 3 when executing the program. Fig.10 The electronic device 90 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0149] like Fig.10 As shown, the electronic device 90 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0150] The bus 93 includes a data bus, an address bus, and a control bus.

[0151] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0152] The memory 92 may also include a program / utility 925 having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0153] The processor 91 executes various functional applications and data processing by running the computer programs stored in the memory 92, such as the blood vessel tracking method of medical imaging in Example 1 or the surgical path planning method in Example 3 of the present invention.

[0154] The electronic device 90 may also communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 95. Furthermore, the model generating device 90 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. Fig.10 As shown, the network adapter 96 communicates with other modules of the model-generated device 90 via the bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0155] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0156] Example 6

[0157] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the medical image blood vessel tracking method of embodiment 1 or the steps in the surgical path planning method of embodiment 3 are implemented.

[0158] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0159] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the medical imaging blood vessel tracking method of Example 1 or the steps in the surgical path planning method of Example 3.

[0160] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0161] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A blood vessel tracking method for medical images, characterized in that: The blood vessel tracking method comprises: Acquire contrast agent vascular images and non-contrast agent vascular images of the same detection object; Processing the contrast agent blood vessel image to determine a background area of ​​the contrast agent blood vessel image and a blood vessel area of ​​the contrast agent blood vessel image; Constructing a motion model according to the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image; the motion model reflects the motion relationship matrix of the vascular area and / or the background area of ​​the contrast agent vascular image between different frames; Using the motion model, a predicted vessel region is determined in the non-contrast agent vessel image.

2. The blood vessel tracking method of medical imaging according to claim 1, characterized in that: The contrast agent vascular image includes a single-frame standard enhanced image and multiple frames of temporally continuous non-standard enhanced images. The step of constructing a motion model according to the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image includes: Processing the standard enhanced image to obtain a first blood vessel region and a first background region, and processing the non-standard enhanced image to obtain a second blood vessel region and a second background region; calculating a first blood vessel motion between a pixel point corresponding to the second blood vessel region and a pixel point corresponding to the first blood vessel region; Calculating a first background motion between a pixel point corresponding to the first background area and a pixel point corresponding to the second background area; The motion model is constructed according to the first blood vessel motion and the first background motion.

3. The blood vessel tracking method of medical imaging as claimed in claim 2, characterized in that: The step of determining the predicted blood vessel region in the non-contrast agent blood vessel image using the motion model comprises: calculating a second background motion between the non-contrast agent blood vessel image and the standard enhanced image; Calculating a second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion and the motion model; The position of the predicted blood vessel region is determined according to the second blood vessel motion relationship.

4. A blood vessel tracking system for medical imaging, characterized in that: The blood vessel tracking system comprises: An acquisition module, used for acquiring a contrast agent blood vessel image and a non-contrast agent blood vessel image of the same detection object; A region delineation module, used for processing the contrast agent blood vessel image to determine a background region of the contrast agent blood vessel image and a blood vessel region of the contrast agent blood vessel image; A construction module, used to construct a motion model according to the background area of ​​the contrast agent vascular image and the vascular area of ​​the contrast agent vascular image; the motion model reflects the motion relationship matrix of the vascular area of ​​the contrast agent vascular image and / or the background area between different frames; A determination module is used to determine a predicted blood vessel region in the non-contrast agent blood vessel image by using the motion model.

5. The medical imaging blood vessel tracking system according to claim 4, characterized in that: The contrast agent blood vessel image includes a single-frame standard enhanced image and multiple-frame temporally continuous non-standard enhanced images. The construction module includes: a processing unit, configured to process the standard enhanced image to obtain a first blood vessel region and a first background region, and to process the non-standard enhanced image to obtain a second blood vessel region and a second background region; A first calculation unit, configured to calculate a first blood vessel motion between a pixel point corresponding to the second blood vessel region and a pixel point corresponding to the first blood vessel region; A second calculation unit, used for calculating a first background motion between a pixel point corresponding to the first background area and a pixel point corresponding to the second background area; A construction unit is used to construct the motion model according to the first blood vessel motion relationship and the first background motion relationship.

6. The medical imaging blood vessel tracking system according to claim 5, characterized in that: The determining module comprises: a third calculation unit, configured to calculate a second background motion between the non-contrast agent vascular image and the standard enhanced image; a fourth calculation unit, configured to calculate a second blood vessel motion between the non-contrast agent blood vessel image and the standard enhanced image according to the second background motion relationship and the motion model; A determination unit is used to determine the position of the predicted blood vessel region according to the second blood vessel motion.

7. A surgical path planning method, characterized in that: The planning method includes: Using the blood vessel tracking method of medical imaging as claimed in any one of claims 1 to 3, predicting the position information of the predicted blood vessel area in the non-contrast agent blood vessel image; Acquiring a relative position of a surgical instrument in the predicted blood vessel region based on the position information; The planned path of the vascular intervention surgery is obtained according to the relative position and the preoperative path.

8. A surgical path planning system, characterized in that: The planning system comprises: A prediction module, configured to predict position information of a predicted blood vessel region in a non-contrast agent blood vessel image by using the blood vessel tracking system for medical imaging as described in any one of claims 4 to 6; A position acquisition module, used for acquiring the relative position of the surgical instrument in the predicted blood vessel area based on the position information; A path generation module is used to obtain a planned path for the vascular interventional surgery based on the relative position and the preoperative path.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the blood vessel tracking method of medical images as described in any one of claims 1 to 3, or implements the surgical path planning method as described in claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the blood vessel tracking method of medical imaging as described in any one of claims 1-3, or when executed, it implements the surgical path planning method as described in claim 7.

Images