3D vascular centerline reconstruction from 2D medical images

Through deep reinforcement learning training, intelligent artificial agents determine the 3D centerline of blood vessels in two-dimensional medical images, solving the problem of difficulty in distinguishing bifurcated blood vessels from overlapping blood vessels and limited number of two-dimensional images, achieving more accurate understanding of vascular structure and imaging Analysis results.

CN114072838BActive Publication Date: 2024-11-05SIEMENS HEALTHINEERS AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980098476.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-07-17
Publication Date
2024-11-05
Estimated Expiration
2039-07-17

AI Technical Summary

Technical Problem

The prior art is difficult to effectively distinguish bifurcated blood vessels from overlapping blood vessels in two-dimensional medical images, and performing three-dimensional centerline reconstruction is a major challenge when the number of two-dimensional medical images is limited.

Method used

Using deep reinforcement learning to train intelligent artificial agents, by observing the current state in 2D medical images and calculating the strategy value, they move autonomously to determine the 3D centerline of the blood vessel, and iteratively move using multiple actions until the target position is reached, thereby determining the 3D of the blood vessel path.

Benefits of technology

It achieves better understanding of vascular structure in overlapping branch areas, improves the results of imaging analysis tasks, can more accurately track the blood vessel center line, solves the problem of distinguishing bifurcation and overlapping blood vessels in two-dimensional images, and overcomes the problem of Reconstruction challenges with limited number of 2D images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114072838B_ABST
    Figure CN114072838B_ABST
Patent Text Reader

Abstract

A system and method for determining a 3D centerline of a blood vessel is provided. A current state observation of an artificial agent is determined based on one or more image view sets, each image view set including a 2D medical image of the blood vessel, a current position of the artificial agent in the 2D medical image, and a start position and a target position in the 2D medical image. A policy value for multiple actions for moving the artificial agent in 3D is calculated based on the current state observation using a trained machine learning model. The artificial agent moves according to a specific action based on the policy value. The steps of determining, calculating, and moving are repeated for multiple iterations to move the artificial agent along a 3D path between the start position and the target position. The 3D centerline of the blood vessel is determined as the 3D path.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates generally to 3D vessel centerline reconstruction, and more particularly to deep reinforcement learning for 3D vessel centerline reconstruction from 2D angiographic medical images. Background Art

[0002] Detection of blood vessels in medical images facilitates the diagnosis, treatment and monitoring of many vascular diseases. An important step in blood vessel detection is centerline tracking to extract a centerline representation of the blood vessel, thereby enabling specific visualization or quantitative assessment of the blood vessel. Conventional methods for centerline tracking are not well suited for two-dimensional medical images, such as, for example, x-ray angiography medical images. A conventional method for centerline tracking calculates the centerline path by minimizing costs such as betweenness or vascularity. Another conventional method for centerline tracking uses flow-based tracking based on an estimated orientation tensor. However, such conventional methods for centerline tracking are unable to distinguish between bifurcated vessels and overlapping vessels in two-dimensional medical images. In addition, three-dimensional centerline reconstruction based on two-dimensional medical images is a major challenge, especially when the number of two-dimensional medical images is limited. Summary of the invention

[0003] According to one or more embodiments, a system and method for determining a 3D centerline of a blood vessel in one or more two-dimensional (2D) medical images is provided. A current state observation of an artificial agent is determined based on one or more image view sets. Each of the one or more image view sets includes one or more 2D medical images of a blood vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images. A policy value for multiple actions for moving an artificial agent in three dimensions (3D) is calculated based on the current state observation using a trained machine learning model. The artificial agent moves according to a specific action based on the policy value. The steps of determining, calculating, and moving are repeated for multiple iterations to move the artificial agent along a 3D path between a start position and a target position. The 3D centerline of the blood vessel is determined as a 3D path. The 2D centerline of the blood vessel can be determined based on the 3D centerline of the blood vessel.

[0004] In one embodiment, the steps of determining, calculating, and moving are repeated until the current position of the artificial agent is the target position or a maximum number of iterations have been performed.

[0005] In one embodiment, the plurality of actions include discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upward direction, and a downward direction. In one embodiment, the plurality of actions include continuous movements in any direction. In one embodiment, based on a current state observation, a trained deep neural network (DNN) trained using deep reinforcement learning is used to calculate policy values ​​for the plurality of actions.

[0006] In one embodiment, the one or more 2D medical images of the blood vessels are channels of angiographic x-ray images. At least one of the one or more image view sets may include a vasculature image, an intensity image, one or more orientation images, branch overlap scores and / or keypoint detection. In one embodiment, the one or more image view sets are a single image view set, which further includes a contrast agent arrival time and a blood flow velocity or a contrast agent propagation velocity.

[0007] In one embodiment, a proximal endpoint and a plurality of corresponding distal endpoints in a vessel tree are detected, and the steps of determining a current state observation, calculation, movement, iteration, and determination of a 3D centerline of the vessel are repeated for each proximal endpoint-distal endpoint pair using the proximal endpoint as a starting position and the corresponding distal endpoint as a target position to determine the 3D centerline of the vessel tree.

[0008] These and other advantages of the present invention will become apparent to those of ordinary skill in the art by reference to the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 An exemplary 2D x-ray medical image of a patient depicting blood vessel branching is shown;

[0010] Figure 2 A method for determining a 3D centerline of a blood vessel using an artificial agent is shown;

[0011] Figure 3 A high-level framework for determining the 3D centerline of a blood vessel using an artificial agent is shown;

[0012] Figure 4 A high-level framework for detecting distal endpoints of a vascular tree is shown;

[0013] Figure 5 A method for training an artificial agent to determine the 3D centerline of a blood vessel is shown; and

[0014] Figure 6 A high-level block diagram of a computer is shown. DETAILED DESCRIPTION

[0015] The invention presented generally relates to methods and systems for three-dimensional (3D) vessel centerline reconstruction from two-dimensional (2D) medical images. Embodiments of the invention are described herein to give a visual understanding of methods for 3D vessel centerline reconstruction from 2D medical images. A digital image is typically composed of digital representations of one or more objects (or shapes). Digital representations of objects are generally described herein in terms of identifying and manipulating the objects. Such manipulations are virtual manipulations that are performed in the memory or other circuitry / hardware of a computer system. Therefore, it will be understood that embodiments of the invention can be executed within a computer system using data stored within the computer system.

[0016] Furthermore, it should be understood that although the embodiments discussed herein may be discussed with respect to vessel centerline reconstruction from medical images, the present invention is not limited thereto and may be applied to reconstruct any tubular object of interest in any type of image.

[0017] Figure 1 A 2D x-ray medical image 100 of a patient (or any other subject) depicting vascular branching is shown, including a region 102 of overlapping blood vessels. The medical image 100 may be acquired to facilitate a clinical examination of the patient, such as, for example, an angiogram. To facilitate vessel detection and other imaging analysis tasks for such clinical examinations, centerline tracking techniques may be applied to the medical image 100 to extract a centerline representation of the vascular branching.

[0018] Conventional centerline tracking techniques are unable to distinguish between bifurcations of branches and overlaps of branches at region 102. Therefore, such conventional centerline tracking techniques may erroneously interpret region 102 as a bifurcation of a vascular branch, thereby tracing an erroneous shortcut path of the branch onto the overlapping branch. Furthermore, 3D centerline reconstruction from 2D medical images is a significant challenge, especially when the number of 2D medical images is limited.

[0019] Embodiments of the present invention apply deep reinforcement learning to train an intelligent artificial agent for determining a 3D centerline and / or a 2D centerline of a blood vessel from a 2D medical image. By executing computer program instructions (code) loaded into a memory, using one or more computers (e.g., Figure 6The artificial agent is implemented using a computer 602 (of a computer 602). The artificial agent observes its environment (e.g., one or more image view sets including one or more 2D medical images, the current position of the artificial agent, and the start and target positions), and moves autonomously within the environment to determine the 3D centerline of the blood vessel based on a strategy learned using machine learning. The movement of the artificial agent is a virtual movement in 3D space relative to a reference direction. The artificial agent can be triggered on demand, or persistent and always run in the background. The input for the artificial agent (e.g., one or more image view sets) is used to determine its state. The output of the artificial agent is an "action" that moves the artificial agent to form a 3D path representing the 3D centerline of the blood vessel. The 3D centerline of the blood vessel provides a better understanding of the vascular structure in the 2D medical image, particularly in areas of overlapping branches (such as, for example, area 102). The 3D centerline of the blood vessel can be used for imaging analysis tasks with improved results.

[0020] Figure 2 A method 200 for determining a 3D centerline of a blood vessel using an artificial agent according to one or more embodiments is shown. The method 200 may be performed by any suitable computing device such as, for example, Figure 6 The reference Figure 3 Also describe Figure 2 Method 200, Figure 3 A high-level framework 300 for determining the 3D centerline of a vessel using an artificial agent according to one or more embodiments is shown. The artificial agent may be trained during a prior training or offline phase, which is described below with respect to Figure 5 Described in further detail.

[0021] At step 202, a current state observation of an artificial agent is determined based on one or more image view sets. Each of the one or more image view sets includes one or more 2D medical images of a vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images. Figure 3 The framework 300 of FIG. 300 shows a first image view set 302 and a second image view set 304 associated with different views of a blood vessel. The first image view set 302 and the second image view set 304 include one or more medical images 302-C and 304-C, heat maps 302-B and 304-B ​​indicating a start position and a target position in the one or more medical images 302-C and 304-C, respectively, and heat maps 302-A and 304-A indicating a current position of an artificial agent in the one or more medical images 302-C and 304-C. Although the framework 300 is shown with the first image view set 302 and the second image view set 304, it should be understood that the framework 300 can employ any number of image view sets.

[0022] In one embodiment, the one or more 2D medical images are channels of an x-ray angiography image. However, it should be understood that the one or more 2D medical images may be of any suitable modality, such as, for example, magnetic resonance imaging (MRI), computed tomography (CT), dynamic CT, ultrasound (US), or any other suitable domain or combination of domains. The one or more 2D medical images of each image view set may be of the same or different modalities, and may be acquired from the same patient at different times, or from different patients. The one or more 2D medical images may be received directly from an image acquisition device such as a CT scanner, an MRI scanner, an ultrasound device, a C-arm image acquisition device, or may be received by loading a previously stored medical image from a memory or storage device of a computer system or receiving a medical image in electronic transmission from another computer system.

[0023] The starting position and the target position are positions on the blood vessel between which the centerline of the blood vessel is to be tracked. The starting position and the target position can be manually defined by the user or automatically detected, for example, using a deep learning method. For example, the starting position can be detected by automatically detecting the position of the catheter tip or mouth, and the target position can be automatically detected by automatically detecting the distal end of the blood vessel. The current position of the artificial agent 306 can also be manually defined by the user or automatically detected, for example, based on the forward projection of the position of the artificial agent in 3D onto the 2D image (using the projection matrix available from the image acquisition) to automatically detect. Although the starting and target positions in the framework 300 are shown as 2D heat maps 302-B and 304-B, and the current position of the artificial agent is shown as 2D heat maps 302-A and 304-A, it should be understood that the starting and target positions and the current position can be represented in any suitable form.

[0024] In possible embodiments, one or more image view sets (e.g., image view sets 302 and / or 304) may alternatively or additionally include other features of one or more 2D medical images. For example, one or more image view sets may include a vascularity image representing a pixel-level probability that a pixel represents a vessel, an intensity image representing a pixel-level intensity value, one or more orientation images representing a pixel-level probability that a vessel in a pixel is oriented in a corresponding orientation (e.g., vertical, horizontal, a diagonal formed between the lower left corner and the upper right corner, or a diagonal formed between the upper left corner and the lower right corner), a branch overlap score representing a pixel-level fraction of pixels being overlapping branches, keypoint detection (e.g., bifurcation), etc.

[0025] At step 204, policy values ​​for a plurality of actions for moving the artificial agent in 3D are calculated based on the current state observation using the trained machine learning model. The current state observation is input into the trained machine learning model, and the policy values ​​for a plurality of actions are calculated based on the current observed state using the trained machine learning model. The policy values ​​may be action values ​​(where the artificial agent is trained using deep reinforcement learning using Q-learning), probabilities (where the artificial agent is trained using deep reinforcement learning using a policy gradient method), or any other suitable policy values. Framework 300 shows that the artificial agent 306 receives the image view sets 302 and 304 for calculating or predicting policy values ​​for a plurality of actions.

[0026] By executing computer program instructions (code) loaded into memory, using one or more computers (e.g. Figure 6 The artificial agent 306 is implemented by a computer 602. The artificial agent 306 is trained to perform one of a plurality of actions based on a policy and reward mechanism learned using machine learning, such as, for example, a deep neural network (DNN). In one embodiment, the artificial agent 306 is trained using deep reinforcement learning during a previous training or offline phase to learn a policy that maps states to actions to maximize rewards. In one embodiment, the artificial agent 306 is described below with respect to Figure 5 The described are trained during a training phase.

[0027] The policy values ​​for the plurality of actions are calculated by accumulating the rewards for the one or more 2D medical images for each image view set according to the policy. The policy value is calculated for each corresponding action based on the projected position of the artificial agent 306 (as it would move according to the corresponding action), the 2D ground truth centerline of the one or more 2D medical images, and the start and target positions. In one embodiment, if the 3D ground truth centerline is available (e.g., from CTA), the rewards can also be calculated in 3D or as a combination of both 2D and 3D rewards.

[0028] Multiple actions are used to virtually move the artificial agent 306 within a 3D space relative to a reference direction. The 3D space has a 3D coordinate system that corresponds to the 2D coordinate system of one or more 2D medical images. In one embodiment, the multiple actions are discrete movements (i.e., predetermined movements or a finite set of movements in a specific direction). For example, the multiple actions may include discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upward direction, and a downward direction relative to the reference direction. Movements in other directions (e.g., diagonal) are also contemplated. In another embodiment, for example, depending on the method of reinforcement learning being implemented, the multiple actions are continuous movements (i.e., movements in any direction relative to the reference direction within a certain range (e.g., a 360-degree range)). Multiple actions are defined during the training phase.

[0029] At step 206, the artificial agent is moved according to a specific action among the multiple actions based on the policy values ​​of the multiple actions. For example, the artificial agent may be moved for a specific action with the highest policy value (e.g., the highest action value). The movement of the artificial agent is a virtual movement in a 3D space that is accomplished in a memory or other circuitry / hardware of a computer system.

[0030] At step 208, it is determined whether a stop condition is satisfied. For example, the stop condition may be that the current position of the artificial agent 306 is the target position or that a maximum number of iterations have been performed. Any other suitable stop condition may also be employed. If the stop condition is not satisfied, the method 200 returns to step 202 and repeats the steps of determining (step 202), calculating (step 204), and moving (step 206) to incrementally move the artificial agent to form a 3D path in the 3D space between the start position and the target position. Thus, steps 202, 204, and 206 are iteratively repeated until the stop condition is satisfied. During each iteration, only the current position of the artificial agent is updated in each image view set to determine the current state observation (as shown in the framework 300), and the one or more 2D medical images, the start position, and the target position (and any other inputs) in each image view set remain unchanged. When the stop condition is satisfied, the method 200 proceeds to step 210.

[0031] At step 210, the 3D centerline of the vessel is determined as the 3D path of the artificial agent. In one embodiment, the 3D centerline of the vessel (and the 3D path of the artificial agent) is represented by a set of 3D coordinates, each 3D coordinate representing an incremental movement of the artificial agent between a starting position and a target position in 3D space.

[0032] At step 212, a 2D centerline of the vessel is determined based on the 3D centerline of the vessel. In one embodiment, the 2D centerline of the vessel is determined by projecting the 3D centerline of the vessel into a 2D space or view. The 2D centerline of the vessel is represented by a set of 2D coordinates, each 2D coordinate representing an incremental movement of the artificial agent between a starting position and a target position in 2D space.

[0033] At step 214, the 3D centerline and / or 2D centerline of the blood vessel is output. For example, the 3D centerline and / or 2D centerline of the blood vessel may be outputted by, for example, a computer system (e.g., Figure 6 The 3D centerline and / or 2D centerline of the blood vessel is displayed on a display device of a computer system (e.g., Figure 6The 3D centerline and / or 2D centerline of the blood vessel may be stored on a memory or storage device of a computer 602 or output by transmitting the 3D centerline and / or 2D centerline of the blood vessel to a remote computer system. In one embodiment, the 2D centerline is displayed on a display device overlaid on one or more 2D medical images. Although the final 3D centerline and / or 2D centerline of the blood vessel is output in method 200, it is also possible that an incremental 3D centerline and / or 2D centerline of the blood vessel (from the incremental 3D path of the artificial agent) may be output (e.g., displayed on a display of a computer system) at each iteration of steps 202-212. Such incremental results would allow, for example, a user to view the 3D and / or 2D path of the artificial agent in real time as the artificial agent moves.

[0034] In one embodiment, method 200 can be applied to extract the centerline of a complete vascular tree. A proximal endpoint and multiple distal endpoints are determined for the vascular tree. In one embodiment, the proximal endpoint and multiple distal endpoints are manually identified by a user. In another embodiment, a deep learning network such as, for example, a multi-objective deep convolutional neural network can be used to automatically detect the proximal endpoint and multiple distal endpoints of the vascular tree. For example, Figure 4 A high-level framework 400 for detecting distal endpoints of a tree vessel using a deep learning model is shown in accordance with one or more embodiments. A medical image 402 depicting a vessel tree is input to a trained deep learning model 404 (e.g., a multi-object deep convolutional neural network trained using known methods), which generates a plurality of heatmaps 406 indicating the locations of proximal endpoints and distal endpoints of the vessel tree. The method 200 may be performed for each proximal endpoint-distal endpoint pair (i.e., a pair between a proximal endpoint and each of a plurality of distal endpoints) using the proximal endpoint as a starting position and the corresponding distal endpoint as a target position to determine a 3D centerline of the complete vessel tree. In some embodiments, where the vessel tree has one or more branches (e.g., bifurcations or trifurcations), the artificial agent may continue along one of the branches, and one or more additional artificial agents may be created as actions to move along other branches.

[0035] In one embodiment, the framework 300 can be modified to determine the 3D centerline of the vessel from a single image view set (e.g., image view set 302 or 304). In particular, the framework 300 can be modified so that the single image view set includes the arrival time of the contrast agent at a specific point along the vessel (between the start location and the target location) and the blood flow velocity or the contrast agent propagation velocity at the specific point along the vessel. Thus, the artificial agent learns to move in 3D space such that its travel distance is consistent with the estimated 3D position of each specific point (e.g., determined based on the arrival time or blood flow velocity or the contrast agent propagation velocity of the contrast agent at the specific point), while ensuring that the 3D path of the artificial agent projected in 2D matches the image features of the single image view set.

[0036] In one embodiment, the arrival time of the contrast agent can be determined as the acquisition time of the frame in which the contrast agent arrives at a specific point along the blood vessel minus the acquisition time of the frame in which the contrast agent arrives at the starting position. Any other suitable method for determining the arrival time of the contrast agent can also be used.

[0037] In one embodiment, blood flow velocity and contrast agent propagation velocity may be determined based on one or more 2D medical images. For example, given a starting location and a target location on a blood vessel, the time required to propagate a contrast agent (e.g., the front of the contrast agent) from the starting location to the target location may be estimated based on frame counts. Thus, given a distance between the starting location and the target location (e.g., determined based on a 3D reconstruction of the blood vessel), the contrast agent propagation velocity may be approximated as a ratio between the distance and the time required for blood flow propagation. Alternatively, given an estimate of the blood flow velocity or contrast agent propagation velocity, the blood flow propagation time may be used to estimate the distance between two points (e.g., the starting location and the target location). In other embodiments, blood flow velocity and contrast agent propagation velocity may be determined based on direct measurements such as population averages (e.g., depending on location on the coronary artery tree and subject characteristics), invasive (e.g., intravascular Doppler), or non-invasive (e.g., Doppler or phase contrast MRI), or estimated using physiological modeling as described in U.S. Patent No. 9,918,690, entitled “Synthetic Data-Driven Hemodynamic Determination in Medical Imaging,” filed on October 7, 2015, the disclosure of which is incorporated herein by reference in its entirety. Any other suitable method for determining blood flow velocity or contrast agent propagation velocity may also be employed.

[0038] Figure 5 A method 500 for training an artificial agent to determine a 3D centerline of a blood vessel according to one or more embodiments is shown. The method 500 may be performed by any suitable computing device, such as, for example, Figure 6The method 500 is performed during an offline or training phase to train an artificial agent. For example, Figure 2 In the method 200 of , the trained artificial agent may be applied during an online or testing phase.

[0039] At step 502, training images are received. In one embodiment, the training images may include training medical images of blood vessels having ground truth centerlines of the blood vessels. The training medical images may be of any suitable modality, may be of the same modality or of different modalities, and may be acquired from the same patient or from different patients at different times. The training medical images may include actual medical images of patients acquired from a medical image acquisition device. These training medical images may be obtained by receiving images from an image acquisition device for acquiring images or by loading previously acquired images from a storage device or memory of a computer system. The ground truth centerlines of these training medical images may be acquired by manual expert annotation or by applying existing image centerline tracking methods. The training medical images may also include synthetic images. Synthetic images may be generated using any suitable method.

[0040] In one embodiment, a synthetic image is generated by synthetically generating an anatomical model of the arterial tree to provide a ground truth centerline. The anatomical model of the arterial tree can be generated according to the method described in U.S. Patent No. 9,918,690 filed on October 7, 2015, the disclosure of which is incorporated herein by reference in its entirety. A physiological model with different settings (e.g., heart rate values, contrast injection rates, etc.) is then used to simulate blood flow and contrast propagation. A synthetic coronary angiogram is generated by projecting the time-resolved results of the simulation onto one or more planes representing the sensor positions of the imaging scanner.

[0041] In another embodiment, a synthetic image is generated by forward projecting the coronary artery segmentation from a single-phase or multi-phase CTA volume onto multiple 2D views. To simulate the motion of the heart, a prior motion model can be applied to the cardiac structures (e.g., four chambers) segmented in the CTA images to deform the coronary artery segmentation.

[0042] At step 504, state inputs to the machine learning model are defined for the task of determining the 3D centerline of the vessel for which the machine learning model is being trained. The state input to the machine learning model is one or more 2D medical images of the vessel, the current position of the artificial agent in the one or more 2D medical images, and the start position and target position in the one or more 2D medical images. The state may also include other inputs such as, for example, the arrival time of the contrast agent, blood flow velocity, contrast agent propagation velocity, etc.

[0043] At step 506, a plurality of actions are defined. The plurality of actions represent a set of possible actions for moving the artificial agent. For example, the plurality of actions may include discrete movements in the forward, backward, left, right, upward, and downward directions. Other movements (e.g., diagonal) are also contemplated. In some embodiments, the plurality of actions may be continuous movements in any direction.

[0044] At step 508, a reward mechanism is defined. The reward mechanism assigns a reward to each of a plurality of actions depending on the effect of the action on the artificial agent path trajectory. A higher reward is assigned to an action that results in a more accurate vessel centerline.

[0045] At step 510, a machine learning model is trained based on the state input using the training images to predict a policy value for each of a plurality of actions. The policy value of an action represents the overall reward expected if the action is performed. The policy value can be any suitable value that defines the action space, for example, depending on the method used to train the machine learning model. For example, in deep reinforcement learning using Q-learning, the policy value is the action value. In deep reinforcement learning using policy gradient methods, the policy value is the probability over the action state given the input state. The machine learning model can be any suitable machine learning-based model, such as, for example, a deep neural network. The artificial agent is implemented by a machine learning model that takes the state as input and outputs a policy value for each of a plurality of actions.

[0046] In one embodiment, deep reinforcement learning can be used to train a deep neural network. Reinforcement learning is a type of machine learning in which a software-based artificial agent uses reward feedback to automatically learn ideal behavior in a specific context and for a specific task. In deep reinforcement learning, which combines deep neural networks with reinforcement learning, the policy learning process is formulated as a reinforcement learning problem, and the action-value function is estimated as an iterative update following the Bellman equation. In deep reinforcement learning (and reinforcement learning), the training of the artificial agent is typically unsupervised, and the artificial agent is free to evolve in its environment according to its current policy estimate.

[0047] In one embodiment, a deep neural network may be trained using deep reinforcement learning using Q-learning. In Q-learning, a deep neural network is trained to predict action values ​​for each of a plurality of actions. In another embodiment, a deep neural network may be trained using deep reinforcement learning using a policy gradient method. Using a policy gradient method, a deep neural network is trained to predict the probability of taking each of a plurality of actions. Deep reinforcement learning may be implemented using any other suitable method.

[0048] The trained machine learning model is stored, for example, in a memory or storage device of a computer system, or on a remote cloud-based computer system. The trained machine learning model can then be loaded and used to determine the 3D centerline of the blood vessel by iteratively predicting a policy value for each of a plurality of actions based on a current state and executing a selected action with an optimal policy value (e.g., a highest predicted action value).

[0049] The systems, devices, and methods described herein can be implemented using digital circuits or using one or more computers using known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. A computer may also include or be coupled to one or more mass storage devices, such as one or more magnetic disks, internal hard disks and removable disks, magneto-optical disks, optical disks, and the like.

[0050] The systems, devices and methods described herein can be implemented using computers operating in a client-server relationship. Typically, in such a system, the client computer is located remotely from the server computer and interacts via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers.

[0051] The systems, devices, and methods described herein can be implemented within a network-based cloud computing system. In such a network-based cloud computing system, a server or another processor connected to the network communicates with one or more client computers via the network. For example, a client computer can communicate with a server via a web browser application that resides and operates on the client computer. The client computer can store data on the server and access the data via the network. The client computer can transmit a request for data or a request for an online service to the server via the network. The server can perform the requested service and provide the data to (one or more) client computers. The server can also transmit data suitable for causing the client computer to perform a specific function, such as performing a calculation, displaying specified data on a screen, etc. For example, the server can transmit a request suitable for causing the client computer to perform one or more steps or functions of the methods and workflows described herein, including Figure 2 and 5 One or more steps or functions of the methods and workflows described herein - including Figure 2 and 5 One or more steps or functions of the method described herein may be performed by a server or another processor in a network-based cloud computing system. Certain steps or functions of the method and workflow described herein include Figure 2 and 5One or more steps of the method and workflow described herein may be performed by a client computer in a network-based cloud computing system. The steps or functions of the method and workflow described herein include Figure 2 and 5 One or more steps of - may be performed by a server and / or a client computer in a network-based cloud computing system in any combination.

[0052] The systems, apparatus, and methods described herein may be implemented using a computer program product tangibly embodied in an information carrier, such as in a non-transitory machine-readable storage device, for execution by a programmable processor; and the method and workflow steps described herein - including Figure 2 and 5 One or more steps or functions of a program may be implemented using one or more computer programs executable by such a processor. A computer program is a collection of computer program instructions that can be used directly or indirectly in a computer to perform a certain activity or bring about a certain result. A computer program may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0053] Figure 6 6 is a high-level block diagram of an example computer 602 that can be used to implement the systems, apparatus, and methods described herein. The computer 602 includes a processor 604 operatively coupled to a data storage device 612 and a memory 610. The processor 604 controls the overall operation of the computer 602 by executing computer program instructions that define such operations. The computer program instructions may be stored in the data storage device 612 or other computer-readable medium and loaded into the memory 610 when execution of the computer program instructions is desired. Thus, Figure 2 and 5 The methods and workflow steps or functions may be defined by computer program instructions stored in the memory 610 and / or the data storage device 612 and controlled by the processor 604 executing the computer program instructions. For example, the computer program instructions may be implemented as a computer executable code programmed by a person skilled in the art to perform Figure 2 and 5 Thus, by executing computer program instructions, processor 604 performs Figure 2 and 5The computer 602 may also include one or more network interfaces 606 for communicating with other devices via a network. The computer 602 may also include one or more input / output devices 608 (e.g., a display, keyboard, mouse, speaker, buttons, etc.) that enable a user to interact with the computer 602.

[0054] The processor 604 may include both general and special purpose microprocessors, and may be the sole processor or one of multiple processors of the computer 602. For example, the processor 604 may include one or more central processing units (CPUs). The processor 604, the data storage device 612, and / or the memory 610 may include, be supplemented by, or be incorporated into one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs).

[0055] The data storage device 612 and the memory 610 each include a tangible, non-transitory computer-readable storage medium. The data storage device 612 and the memory 610 may each include a high-speed random access memory, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid-state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices (such as internal hard disks and removable disks), magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices (such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), digital versatile disk read-only memory (DVD-ROM) disks) or other non-volatile solid-state storage devices.

[0056] Input / output devices 608 may include peripheral devices such as printers, scanners, display screens, etc. For example, input / output devices 608 may include: a display device such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor for displaying information to a user; a keyboard; and a pointing device such as a mouse or trackball through which a user can provide input to computer 602.

[0057] The image acquisition device 614 can be connected to the computer 602 to input image data (e.g., medical images) to the computer 602. It is possible to implement the image acquisition device 614 and the computer 602 as one device. It is also possible that the image acquisition device 614 and the computer 602 communicate wirelessly over a network. In a possible embodiment, the computer 602 can be remotely located relative to the image acquisition device 614.

[0058] Any or all of the systems and devices discussed herein may be implemented using one or more computers, such as computer 602 .

[0059] Those skilled in the art will recognize that the actual computer or computer system implementation may have other structures and may also include other components, and Figure 6 is a high-level representation of some components of such a computer for illustrative purposes.

[0060] The foregoing specific embodiments will be understood to be illustrative and exemplary in every respect, rather than restrictive, and the scope of the invention disclosed herein will not be determined by the specific embodiments, but by the claims interpreted according to the full scope permitted by the patent law. It will be understood that the embodiments shown and described herein are merely illustrative of the principles of the invention, and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the invention. Various other feature combinations may be implemented by those skilled in the art without departing from the scope and spirit of the invention.

Claims

1. A method for determining a 3D centerline of a blood vessel in one or more two-dimensional (2D) medical images, comprising: determining a current state observation of the artificial agent based on one or more image view sets, each image view set comprising one or more 2D medical images of a blood vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images; computing, using a machine learning model trained using deep reinforcement learning, policy values ​​for a plurality of actions for moving the artificial agent in three dimensions 3D based on the current state observation; moving the artificial agent according to a specific action among the plurality of actions based on the policy value; repeating the determining, calculating, and moving steps for a plurality of iterations to move the artificial agent along a 3D path between the starting position and the target position; as well as A 3D centerline of the blood vessel is determined as the 3D path.

2. The method according to claim 1, further comprising: A 2D centerline of the blood vessel is determined based on the 3D centerline of the blood vessel.

3. The method of claim 1 , wherein repeating the determining, calculating, and moving steps for a plurality of iterations to move the artificial agent along the 3D path between the starting position and the target position comprises: The determining, calculating and moving steps are repeated until the current position of the artificial agent is the target position or a maximum number of iterations have been performed.

4. The method of claim 1, wherein the plurality of actions comprises discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upward direction, and a downward direction. The method of claim 1 , wherein the plurality of actions comprises continuous movement in any direction.

6. The method of claim 1, wherein at least one of the one or more image view sets further comprises at least one of a vascularity image, an intensity image, one or more orientation images, a branch overlap score, and a keypoint detection. 7 . The method of claim 1 , wherein the one or more image view sets are a single image view set that includes an arrival time of a contrast agent and a blood flow velocity or a contrast agent propagation velocity.

8. The method of claim 1, wherein computing policy values ​​for a plurality of actions for moving the artificial agent in three dimensions (3D) based on the current state observation using a machine learning model trained using deep reinforcement learning comprises: The policy values ​​of the plurality of actions are calculated based on the current state observation using a trained deep neural network (DNN) trained using deep reinforcement learning.

9. The method according to claim 1, further comprising: detecting a proximal endpoint and a plurality of corresponding distal endpoints in a vessel tree; as well as Determining the current state observation, the calculating, the moving, the repeating, and determining the 3D centerline of the blood vessel are performed for each proximal end point-distal end point pair using the proximal end point as the starting position and the corresponding distal end point as the target position to determine the 3D centerline of the blood vessel tree.

10. The method of claim 1, wherein the one or more 2D medical images of the blood vessels are channels of angiographic x-ray images.

11. An apparatus for determining a 3D centerline of a blood vessel in one or more two-dimensional (2D) medical images, comprising: means for determining a current state observation of the artificial agent based on one or more image view sets, each image view set comprising one or more 2D medical images of a vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images; means for computing, using a machine learning model trained using deep reinforcement learning, policy values ​​for a plurality of actions for moving the artificial agent in three dimensions 3D based on the current state observation; means for moving the human agent according to a particular action among the plurality of actions based on the policy value; means for repeating said determining, said calculating and said moving steps for a plurality of iterations to move said artificial agent along a 3D path between said starting position and said target position; as well as A means for determining a 3D centerline of the blood vessel as the 3D path.

12. The apparatus according to claim 11, further comprising: Means for determining a 2D centerline of the blood vessel based on the 3D centerline of the blood vessel.

13. The device according to claim 11, wherein: The means for repeating the determining, calculating and moving steps for a plurality of iterations to move the artificial agent along the 3D path between the starting position and the target position comprises: Means for repeating the determining, calculating and moving steps until the current position of the artificial agent is the target position or a maximum number of iterations have been performed.

14. The apparatus of claim 11, wherein at least one of the one or more image view sets further comprises at least one of a vascularity image, an intensity image, one or more orientation images, a branch overlap score, and a keypoint detection.

15. A non-transitory computer readable medium storing computer program instructions which, when executed by a processor, cause the processor to perform operations comprising: determining a current state observation of the artificial agent based on one or more image view sets, each image view set comprising one or more two-dimensional 2D medical images of a blood vessel, a current position of the artificial agent in the one or more 2D medical images, and a start position and a target position in the one or more 2D medical images; computing, using a machine learning model trained using deep reinforcement learning, policy values ​​for a plurality of actions for moving the artificial agent in three dimensions 3D based on the current state observation; moving the artificial agent according to a specific action among the plurality of actions based on the policy value; repeating the determining, calculating, and moving steps for a plurality of iterations to move the artificial agent along a 3D path between the starting position and the target position; as well as A 3D centerline of the blood vessel is determined as the 3D path.

16. The non-transitory computer-readable medium of claim 15, wherein the plurality of actions include discrete movements in a forward direction, a backward direction, a left direction, a right direction, an upward direction, and a downward direction.

17. The non-transitory computer readable medium of claim 15, wherein the plurality of actions comprises continuous movement in any direction.

18. The non-transitory computer readable medium of claim 15, wherein the one or more image view sets are a single image view set that includes an arrival time of a contrast agent and a blood flow velocity or a contrast agent propagation velocity.

19. The non-transitory computer readable medium of claim 15, the operations further comprising: detecting a proximal endpoint and a plurality of corresponding distal endpoints in a vessel tree; as well as Determining the current state observation, the calculating, the moving, the repeating, and determining the 3D centerline of the blood vessel are performed for each proximal end point-distal end point pair using the proximal end point as the starting position and the corresponding distal end point as the target position to determine the 3D centerline of the blood vessel tree.

20. The non-transitory computer readable medium of claim 15, wherein the one or more 2D medical images of the blood vessels are channels of an angiographic x-ray image.