A system and method for angiography with reduced contrast agent and / or X-ray exposure using machine learning.

Abstract

Methods, systems, and computer readable media for reduced contrast and / or x-ray angiography using machine learning (e.g., deep learning) are provided. Briefly, the techniques described herein can use neural networks trained to preserve / maintain angiographic image quality while reducing the amount of potentially harmful chemical contrast and / or x-ray radiation during angiography. As a result, angiographic anatomy can be extracted from images using deep learning neural networks in reduced contrast and / or x-ray angiography. Reduction of chemical contrast and x-ray dose can be achieved based on operations performed before, during, and / or after angiography imaging.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority to U.S. Provisional Application No. 63 / 324,867, filed on March 29, 2022, the entire contents of which are incorporated herein by reference.

[0002] This disclosure generally relates to angiography, and more specifically, to systems and methods for performing angiography with reduced contrast agent and / or X - rays using machine learning.

Background Art

[0003] The heart pumps blood to the body through a series of expulsions of arteries. The blood reaches the veins through capillaries and returns to the heart. The presence and movement of blood within blood vessels (generally branched tubular structures) can be dynamically imaged by a technique called angiography.

[0004] In fluoroscopic angiography, a chemical contrast agent is injected as a bolus into the vascular system (e.g., the bloodstream), and a series of X - ray images are acquired. The chemical contrast agent can contain one or more of certain chemical substances in liquid form. The chemical contrast agent is denser than blood or tissue, and thus, the chemical contrast agent attenuates the passage of X - rays more than blood or tissue. The denser the chemical contrast agent, the sharper the contrast of the image imparted to the blood vessels during angiography by fluoroscopic angiography. Particularly high - density chemical contrast agents contain iodine as ions, and this type of contrast agent is called an "iodinated contrast agent". An example of an iodinated contrast agent is iohexol. Other chemical contrast agents do not contain iodine and are not iodinated.

[0005] Chemical contrast agents pass through the vascular system, blocking the passage of X-rays at a predetermined frame rate and forming a mold of the vascular structure containing the contrast agent. As a result, the spatiotemporal X-ray attenuation pattern creates a sequence of X-ray images obtained fluoroscopically on an X-ray sensor. This sequence is called angiography and is, for example, a series of images tracking the passage of a bolus of contrast agent. Angiographic images are typically two-dimensional in space and one-dimensional in time.

[0006] The injection of chemical contrast agents carries toxic side effects on internal organs such as the kidneys. However, reducing the dose of contrast agent to mitigate these toxic side effects can produce unsatisfactory images with a poor signal-to-noise ratio, resulting in incomplete angiography where vascular anatomical structures are not adequately visualized. Furthermore, reducing the amount of contrast agent used may require advancing the injection catheter further into the ductus arteriosus to concentrate the injected contrast agent within the anatomically relevant area. The need to advance the injection catheter further increases the risk of complications caused by the catheter damaging smaller vessels distal to the vascular tree. [Overview of the project]

[0007] Embodiments of the present invention are methods, systems, and computer-readable media for angiography with reduced contrast agents and / or X-rays using machine learning (e.g., deep learning). In short, the techniques described herein can use machine learning to maintain angiographic image quality while reducing the amount of potentially harmful chemical contrast agents and / or X-ray radiation during angiography. As a result, angiographic anatomy can be extracted from images in angiography with reduced contrast agents and / or X-rays. The reduction of chemical contrast agents and X-rays can be achieved based on operations performed before, during, and / or after angiography imaging.

[0008] Various mechanisms are provided for training machine learning models (e.g., deep learning neural networks) to generate higher-quality angiographic images with reduced chemical contrast agents and / or X-ray radiation. These mechanisms may include collecting training data acquired with reduced chemical contrast agents and / or X-ray radiation ("reduced training data") mixed with segmented standard full-volume data ("full-volume segmented data"). That is, a training dataset can be generated using full-volume segmented data acquired with full (standard) amounts of chemical contrast agents and X-ray radiation as known outputs, and unsegmented reduced training data, and a neural network can be trained on the full-volume segmented data. Training datasets may be generated using full (standard) and physically reduced amounts in animal (e.g., non-human) angiography, full (standard) and physically reduced amounts in realistic prosthetic organ models, and / or amounts where reduction has been computer-simulated in full human angiography data. Thus, in the neural network training process, multiple angiographic images acquired at full (standard) levels over time intervals may be used to segment the vessels, and the neural network (e.g., a convolutional network) may obtain standard-level segmentation results using reduced levels of image data. Therefore, the neural network can be trained on training data consisting of high-quality ground truth data of angiographic structures and actual or simulated degraded angiographic images acquired with reduced chemical contrast agents and / or X-ray radiation.

[0009] In the post-training deployment mode of the neural network, the neural network can acquire angiographic data obtained with reduced chemical contrast agents and / or X-ray radiation and generate image segmentation based on training for full-volume segmentation. For example, low-quality angiographic image data from physically reduced chemical contrast agents and / or X-ray radiation may be fed to a deep neural network, and the vascular structure may be estimated according to training on the low-quality angiographic images against higher-quality ground truth training data. In this way, the deep learning neural network can generate vascular segmentation of images obtained with physically reduced chemical contrast agents and / or X-ray radiation.

[0010] Neural networks can be applied to spans or sequences of angiographic images to identify vascular structures without increasing the amount of contrast agent and / or X-rays used in angiography. For example, a deep learning neural network can estimate vascular structures within an angiographic target image using one or more temporally adjacent and consecutive angiographic images. In this way, the neural network can be tuned to detect the spatiotemporal characteristics of blood vessels, including those containing contrast agent.

[0011] Alternative neural network configurations may be designed to optimize offline and / or real-time computations. For example, a neural network may be modified for real-time use, such that multiple angiographic images for data input include multiple temporally preceding angiographic images to generate segmentation of the most recent angiographic image.

[0012] A method according to a first aspect of the present invention comprises providing an angiographic target image obtained using a first amount of chemical contrast agent and / or X-ray radiation as input to a machine learning model via a processor, the machine learning model being trained with (a) a second angiographic image obtained using a second amount of chemical contrast agent and / or X-ray radiation as input, and (b) a third angiographic image obtained using a third amount of chemical contrast agent and / or X-ray radiation as a known output, the third amount being greater than the first and second amounts. The method further comprises obtaining an output from the machine learning model via a processor, the output being an angiographic image which is a processed version of the angiographic target image. Processing a first angiographic image obtained using a first amount of chemical contrast agent and / or X-ray radiation with a machine learning model trained using a second angiographic image obtained using a second amount of chemical contrast agent and / or X-ray radiation as training data, and a third angiographic image obtained using a third amount of chemical contrast agent and / or X-ray radiation (where the third amount is greater than the first amount) as known output, makes it possible to obtain diagnostically useful angiographic images from angiographic images obtained with reduced chemical contrast agent and / or X-ray radiation, thereby improving the safety of angiography.

[0013] In one embodiment, the target image is one of several angiographic images in a first subsequence of an angiographic image obtained using a first amount of chemical contrast agent and / or X-ray radiation, and provided to a machine learning model as input; the second angiographic image is one of several angiographic images in a second subsequence of an angiographic image obtained using a second amount of chemical contrast agent and / or X-ray radiation, and used to train the machine learning model; and by providing a subsequence of an angiographic image containing the target image obtained using a first amount of chemical contrast agent and / or X-ray radiation to a machine learning model trained using a subsequence of an angiographic image obtained using a second amount of chemical contrast agent and / or X-ray radiation, this embodiment can improve the accuracy of the output using spatiotemporal information from other images in the subsequence of the image containing the target image.

[0014] In one embodiment, a first subsequence of an angiographic image is provided to the machine learning model as a single vector. By providing the first subsequence of an angiographic image to the machine learning model as a single vector, this embodiment can reduce processing time.

[0015] In one embodiment, the first and second subsequences of the angiographic image each consist of an odd number of angiographic images. Using an odd number of angiographic images in each subsequence allows the target image to be acquired from the center of the first subsequence of the angiographic image, providing spatiotemporal information from before and after the target image, and potentially improving the accuracy of the output.

[0016] In one embodiment, the first subsequence of angiographic images consists of the same number of angiographic images as the second subsequence of angiographic images. By ensuring that the input in the deployment mode matches the input in the training mode, this embodiment can reduce the time required to train the machine learning model.

[0017] In one embodiment, the first and second subsequences each consist of five angiographic images, which is advantageous because the target image is located in the center of the subsequence and spatiotemporal information before and after the target image is available. In this embodiment, it has been found that diagnostically useful output can be provided with less processing than with subsequences of more than five images.

[0018] In one embodiment, a first subsequence of an angiographic image is one of a plurality of input subsequences, each of which is extracted from a different portion of an angiographic image sequence obtained using a first amount of chemical contrast agent and / or X-ray radiation. The plurality of input subsequences are provided as input to a machine learning model, and the machine learning model, via at least one processor, produces an output containing an angiographic image which is a processed version of each target image in each of the plurality of input subsequences. In this embodiment, it is possible to process multiple target images in an image sequence using a machine learning model, which can be diagnostically advantageous.

[0019] In one embodiment, a second subsequence of an angiographic image is one of several subsequences used to train a machine learning model, each of which is extracted from a different portion of an angiographic image sequence obtained using a second amount of chemical contrast agent and / or X-ray radiation. This embodiment allows the machine learning model to be trained to process multiple target images within the image sequence, which may be diagnostically advantageous.

[0020] In one embodiment, each angiographic image in a first subsequence of angiographic images is temporally continuous with at least one other angiographic image in the first subsequence, and each angiographic image in a second subsequence of angiographic images is temporally continuous with at least one other angiographic image in the second subsequence. Using temporally continuous images can improve the accuracy of the output by providing spatiotemporal information from the images to the target image.

[0021] In one embodiment, the first and second amounts of chemical contrast agent and / or X-ray radiation are the same. In this embodiment, the accuracy of the output can be improved.

[0022] In one embodiment, the first and second amounts are less than the amount required to obtain a diagnostically useful image from the X-ray imaging device, and the third amount is sufficient to obtain a diagnostically useful image from the X-ray imaging device. This embodiment may improve the safety of angiography by enabling the acquisition of angiographic images with reduced amounts of chemical contrast agent and / or X-ray radiation.

[0023] In one embodiment, the angiographic image of the second subsequence is an angiographic image of a non-human vascular structure. In this embodiment, the training cost of the machine learning model can be reduced.

[0024] In one embodiment, the angiographic image of the second subsequence is an angiographic image of an artificial blood vessel structure. This embodiment can reduce the training cost of the machine learning model.

[0025] In one embodiment, the second amount of chemical contrast agent and / or X-ray radiation is greater than the first amount of chemical contrast agent and / or X-ray radiation, and the angiographic image in the second subsequence is modified to simulate that it was obtained using a smaller amount of chemical contrast agent and / or X-ray radiation than the second amount. In this embodiment, the quality of the training data can be better controlled, and the training cost of the machine learning model can be reduced.

[0026] In one embodiment, the second amount of the contrast agent and / or the X-ray radiation is more than the first amount of the contrast agent and / or the X-ray radiation, and the second angiographic image is modified to simulate being obtained using an amount less than the second amount of the contrast agent and / or the X-ray radiation. In this embodiment, the quality of the training data can be better controlled, and the learning cost of the machine learning model can be reduced.

[0027] In one embodiment, the second angiographic image is modified by adding randomly generated noise to at least some of the pixels of the second angiographic image. In this embodiment, the quality of the training data can be better controlled, and the learning cost of the machine learning model can be reduced.

[0028] In one embodiment, the randomly generated noise is added only to the pixels determined to correspond to the blood vessels in the second angiographic image. In this embodiment, it helps to reduce the learning cost of the machine learning model and avoid registration errors caused by the movement of organs in the angiographic field before and after the administration of the contrast agent.

[0029] In one embodiment, the third angiographic image is a segmented angiographic image, and the output obtained from the machine learning model is an angiographic image that is a segmented version of the first angiographic image. In this embodiment, there is no need to perform a separate manual, semi-automatic, or automatic segmentation process, and the processing time can be shortened.

[0030] In one embodiment, the processed version of the first angiographic image can be displayed on a display. According to this embodiment, medical experts can view and analyze the output and provide medical opinions such as diagnosis.

[0031] According to another aspect, there are provided systems and computer program products that utilize substantially the same technology as the technology described above.

[0032] Other purposes and advantages of these technologies will become apparent from the specification and drawings. [Brief explanation of the drawing]

[0033] [Figure 1A] This is a side view showing an example of a rotational X-ray system that may be used in conjunction with embodiments of the present disclosure to acquire angiographic data. [Figure 1B] This is a partial schematic diagram showing an example of a rotational X-ray system that may be used in conjunction with embodiments of the present disclosure to acquire angiographic data. [Figure 2] This is a schematic diagram of a computer system or information processing device that may be used in conjunction with embodiments of the present disclosure. [Figure 3A] A flowchart illustrating a method for training a neural network based on the respective training data, according to an exemplary embodiment, is shown. [Figure 3B] A flowchart illustrating a method for training a neural network based on the respective training data, according to an exemplary embodiment, is shown. [Figure 3C] A flowchart illustrating a method for training a neural network based on the respective training data, according to an exemplary embodiment, is shown. [Figure 4] An exemplary embodiment shows a system configured to generate images of simulated vascular structures obtained with reduced chemical contrast agents and / or X-ray radiation. [Figure 5] An exemplary method for segmenting vascular objects within a single image from angiographic images obtained with reduced chemical contrast agents and / or X-ray radiation is shown according to an exemplary embodiment. [Figure 6] This example demonstrates how machine learning can be used to generate higher-quality angiographic images from angiographic target images located at various positions within a subsequence of angiographic images. [Modes for carrying out the invention]

[0034] Fluoroscopy with contrast agents is a commonly used medical imaging technique. In short, a physician accesses a blood vessel (typically the femoral artery) percutaneously, guides a catheter to the root artery of the organ in question, and injects a contrast agent in time with fluoroscopic imaging of an X-ray sequence.

[0035] Fluoroscopy angiography can be used to diagnose blockages in the coronary arteries of the heart and brain. If arterial blockage is diagnosed through angiography, treatment options may become available. For example, stents can be placed in calcified plaques in the coronary arteries of the heart, or thrombolytic agents can be directly applied to blood clots in the cerebral arteries.

[0036] Angiography may include foreground objects such as blood vessels. The sharper, more detailed, and clearer the foreground objects are relative to the background in an image, the higher the signal-to-noise ratio of the image, and thus the greater its diagnostic value. However, in some cases, even when chemical contrast agents are injected to improve the clarity of blood vessels within the anatomically visualized region in fluoroscopic angiography, the vascular structure in the angiographic image may not be sufficiently clear.

[0037] In standard practice, the image quality of angiography can be improved by increasing the dose of chemical contrast agent injected and / or increasing the dose of fluoroscopic X-ray radiation. However, the use of chemical contrast agents and X-ray radiation in angiographic imaging carries risks. For example, the procedure of placing a catheter in a blood vessel and moving it to the target organ to inject the chemical contrast agent carries risks. Furthermore, chemical contrast agents and X-ray radiation may be harmful to human and animal subjects (e.g., causing toxic side effects).

[0038] In general, chemical contrast agents can exhibit some toxicity when injected into the vascular circulatory system. Once injected, chemical contrast agents are excreted through biochemical and physiological processes by excretory organs such as the kidneys and liver. Depending on the dose, chemical contrast agents can be toxic to these excretory organs.

[0039] Chemical contrast agents can also place a significant mass load on the vascular system, stressing vascular structures such as the heart, and potentially inducing or worsening heart failure in patients with reduced cardiac pumping function. Furthermore, contrast agents can also burden the excretory organs.

[0040] Furthermore, some individuals develop an immune response to specific molecular structures contained in various chemical contrast agents, particularly iodine-based contrast agents. Histiocyte-mediated immune responses to injected contrast agents can become severe immediately and, if not recognized and treated promptly, can be fatal. For these reasons alone, increasing the dosage of chemical contrast agents to obtain higher-quality angiography also increases the risk of harm to the patient.

[0041] Furthermore, X-ray doses can be harmful to irradiated tissues, particularly those sensitive to radiation, such as the thyroid gland and reproductive organs. Depending on the radiation dose, it may induce chronic inflammation or damage the biomolecules that make up the tissue, potentially leading to a range of consequences from relatively mild skin inflammation along the X-ray path to cancer formation in the irradiated tissue. In particular, high-dose X-rays can directly damage biomolecules in tissues, including deoxyribonucleic acid (DNA) in cell nuclear organelles, potentially causing organ malformations and increasing the risk of tumor formation.

[0042] Therefore, techniques for reducing the amount of chemical contrast agent and / or X-ray radiation administered during angiography are described herein. In particular, angiographic image quality is maintained by clearly and accurately displaying the imaged vascular system in the foreground, even when both chemical contrast agent and X-ray radiation are reduced. This can be achieved using machine learning (e.g., deep learning) techniques, as will be described in more detail below.

[0043] Figures 1A, 1B, and 2 illustrate exemplary systems or apparatus that may be employed to carry out embodiments of the present invention. It should be understood that these systems and apparatus are merely illustrative examples of typical systems and apparatus, and that other hardware and software configurations may also be used with embodiments of the present invention. Therefore, it should be recognized that embodiments are not intended to be limited to the specific systems and apparatus illustrated herein, and that other suitable systems and apparatus can be employed without departing from the spirit and scope of the subject matter provided herein.

[0044] First, Figures 1A and 1B illustrate a rotating X-ray system 28 that may be used to obtain angiography in fluoroscopic angiography. When acquiring angiography, a chemical contrast agent is injected into the patient, who is positioned between the X-ray source and the detector, and the X-ray projection is captured by the X-ray detector as a two-dimensional image (i.e., an angiographic image or image frame). A sequence of such image frames constitutes an angiographic examination. In one embodiment, the sequence of angiographic image frames can be acquired at a rate faster than the patient's heart rate. For example, the patient's heart rate can be measured (e.g., using an electrocardiogram device), and the sequence of angiographic image frames can be acquired at a rate faster than the measured heart rate.

[0045] As shown in Figure 1A, an example of an angiography imaging system is shown in the form of a rotary X-ray system 28 including a base having a C-arm 30 that carries an X-ray source assembly 32 at one end and an X-ray detector array assembly 34 at the other end. The base allows the X-ray source assembly 32 and the X-ray detector array assembly 34 to be oriented to various positions and angles around a patient placed on a table 36, while providing the physician with access to the patient. The base includes a pedestal 38, which has a horizontal leg 40 extending below the table 36 and a vertical leg 42 extending upward from the end of the horizontal leg 40 spaced apart from the table 36. A support arm 44 is rotatably fixed to the upper end of the vertical leg 42 so as to rotate around a horizontal pivot axis 46.

[0046] The horizontal pivot axis 46 coincides with the centerline of the table 36, and the support arm 44 extends radially outward from the horizontal pivot axis 46, supporting the C-arm drive assembly 47 at its outer end. The C-arm 30 is slidably fixed to the C-arm drive assembly 47 and is connected to a drive motor (not shown) that slides the C-arm 30 so as to rotate around the C-axis 48 as indicated by arrow 50. The horizontal pivot axis 46 and the C-axis 48 intersect each other at the system isocenter 56 located above the table 36 and are perpendicular to each other.

[0047] The X-ray source assembly 32 is mounted on one end of the C-arm 30, and the X-ray detector array assembly 34 is mounted on the other end. The X-ray source assembly 32 emits an X-ray beam, which is directed towards the X-ray detector array assembly 34. Both assemblies 32 and 34 extend radially inward toward the horizontal pivot axis 46 so that the central beam passes through the system isocenter 56. Thus, the central beam can be rotated around the system isocenter about either the horizontal pivot axis 46 or the C-axis 48, or both, while acquiring X-ray attenuation data from a subject placed on the table 36.

[0048] The X-ray source assembly 32 includes an X-ray source that emits a beam of X-rays when energized. The central beam passes through the system isocenter 56 and strikes a two-dimensional flat-panel digital detector 58 housed in the X-ray detector array assembly 34. The two-dimensional flat-panel digital detector 58 may be, for example, a two-dimensional array of 2048 × 2048 detector elements. Each element generates an electrical signal representing the intensity of the X-rays it strikes, and therefore the attenuation of the X-rays as they pass through the patient. During scanning, the X-ray source assembly 32 and the detector array assembly 34 are rotated around the system isocenter 56 to acquire X-ray attenuation projection data from various angles. In some embodiments, the detector array is capable of acquiring 50 projections or image frames per second, which is a limiting factor determining the number of image frames that can be acquired for a given scanning path and speed.

[0049] Referring to Figure 1B, the rotation of assemblies 32, 34 and the operation of the X-ray source are managed by a control mechanism 60 of the X-ray system. The control mechanism 60 includes an X-ray controller 62 that provides power and timing signals to the X-ray source assembly 32. A data acquisition system (DAS) 64 within the control mechanism 60 samples data from the detector element and passes the data to an image reconstructor (or module) 65. The image reconstructor 65 receives the digitized X-ray data from the DAS 64 and performs high-speed image reconstruction according to the method of this disclosure. The reconstructed image is used as input to a computer 66, which stores the image in a mass storage device 69 or processes the image further. The image reconstructor 65 may be a standalone computer or may be integrated with the computer 66.

[0050] The control mechanism 60 also includes a base motor controller 67 and a C-axis motor controller 68. In response to operation commands from the computer 66, the motor controllers 67, 68 supply power to motors in the X-ray system that cause rotation around the horizontal pivot axis 46 and the C-axis 48, respectively. The computer 66 also receives commands and scanning parameters from the operator via a console 70 having a keyboard and other manually operable control devices. The associated display 72 allows the operator to view reconstructed image frames and other data from the computer 66. Commands supplied by the operator are used by the computer 66 under the direction of a stored program to provide control signals and information to the DAS 64, the X-ray controller 62, and the motor controllers 67, 68. Furthermore, the computer 66 operates a table motor controller 74 that controls the motorized table 36 to position the patient relative to the system isocenter 56.

[0051] Referring next to Figure 2, a block diagram is shown of a computer system or information processing device 80 (e.g., the image reconstructor 65 and / or computer 66 in Figure 1B) that can be incorporated into an angiography imaging system, such as the rotating X-ray system 28 in Figures 1A and 1B, and used as a standalone device for reducing contrast agents and / or X-rays in angiography by deep learning according to embodiments of the present invention. The information processing device 80 may be local or remote to the rotating X-ray system 28. In one example, the functions performed by the information processing device 80 may be offered as a Software-as-a-Service (SaaS) option. SaaS refers to a software application that is stored on one or more remote servers (e.g., in the cloud) and provides one or more services (e.g., angiography image processing) to remote users. Angiography images can be obtained directly from an angiography imaging system such as the rotating X-ray system 28 in Figures 1A and 1B, or from other sources such as physical storage media configured to store data.

[0052] In some embodiments, the computer system 80 includes a monitor or display 82, a computer system 84 (including a processor 86, a bus subsystem 88, a memory subsystem 90, and a disk subsystem 92), a user output device 94, a user input device 96, and a communication interface 98. The monitor 82 may include hardware and / or software elements configured to produce a visual representation or display of information. Some examples of the monitor 82 may include well-known display devices such as television monitors, cathode ray tube (CRT), liquid crystal displays (LCD), and light-emitting diode (LED) displays. In some embodiments, the monitor 82 may provide an input interface, such as incorporating touchscreen technology.

[0053] The computer system 84 may include one or more well-known computer components such as a central processing unit (CPU), memory or storage device, graphics processing unit (GPU), communication system, and interface card. As shown in Figure 2, the computer system 84 may include one or more processors 86 that communicate with numerous peripheral devices via a bus subsystem 88. The processors 86 may include commercially available central processing units, etc. The bus subsystem 88 may include a mechanism for allowing the various components and subsystems of the computer system 84 to communicate with each other as intended. Although the bus subsystem 88 is schematically shown as a single bus, multiple bus subsystems may be utilized in alternative embodiments of the bus subsystem. Peripheral devices that communicate with the processors 86 may include a memory subsystem 90, a disk subsystem 92, a user output device 94, a user input device 96, and a communication interface 98, etc.

[0054] The processor 86 may be implemented using one or more analog and / or digital electrical or electronic components, and may include a microprocessor, microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), programmable logic, and / or other analog and / or digital circuit elements configured to perform the various functions described herein, such as by executing instructions stored in the memory subsystem 90 and / or disk subsystem 92 or another computer program product.

[0055] The memory subsystem 90 and the disk subsystem 92 are examples of physical storage media configured to store data such as instructions that can be executed by one or more processors 86 to perform the operations described herein. The memory subsystem 90 may include a number of memories or memory devices, including random access memory (RAM) that volatilely stores program code, instructions, and data during program execution, and read-only memory (ROM) that stores fixed program code, instructions, and data. The disk subsystem 92 may include a number of file storage systems that provide persistent (non-volatile) storage for programs and data. Other types of physical storage media include floppy disks, removable hard disks, compact disk read-only memory (CD-ROM), digital video discs (DVDs), optical storage media such as barcodes, semiconductor memory such as flash memory, read-only memory (ROM), battery-backed volatile memory, and networked storage devices. The memory subsystem 90 and the disk subsystem 92 may be configured to store programming and data configurations that provide the functionality or features of the technology described herein. Software code modules and / or processor instructions that implement or provide functionality when executed by processor 86 may be stored in memory subsystem 90 and disk subsystem 92. Memory subsystem 90 may be a non-temporary computer-readable storage medium.

[0056] The user input device 96 may include hardware and / or software elements configured to receive input from the user for processing by components of the computer system 80. The user input device may include all possible types of devices and mechanisms for inputting information into the computer system 84. These may include keyboards, keypads, touchscreens, touch interfaces integrated into displays, audio input devices such as microphones and voice recognition systems, and / or other types of input devices. In various embodiments, the user input device 96 may include computer mice, trackballs, trackpads, joysticks, wireless remotes, drawing tablets, voice command systems, eye-tracking systems, and the like. In some embodiments, the user input device 96 is configured to allow the user to select or interact with objects, icons, text, etc., which may be displayed on the monitor 82 via commands, actions, or gestures, such as clicking a button.

[0057] The user output device 94 may include hardware and / or software elements configured to output information to the user from the components of the computer system 80. The user output device may include all possible types of devices and mechanisms for outputting information from the computer system 84. These may include displays (e.g., monitors 82), printers, touch or force feedback devices, audio output devices, and the like.

[0058] The communication interface 98 may include hardware and / or software elements configured to provide one-way or two-way communication with other devices. For example, the communication interface 98 may provide an interface between the computer system 84 and other communication networks and devices, such as via an internet connection.

[0059] The techniques described herein may enable a reduction in the amount of angiography required to obtain diagnostically useful angiographic images (e.g., the dose of chemical contrast agent and / or X-ray radiation). In particular, the amount of contrast agent and / or X-rays used in angiography may be reduced compared to the amount of contrast agent and / or X-rays used in standard / conventional angiography that would be required to obtain diagnostically useful angiographic images without the techniques described herein (e.g., without machine learning models such as the deep learning neural network described herein). For example, a “diagnostically useful” or “high-quality” angiographic image is one that provides data of sufficient quality to provide meaningful clinical information and / or enable treatment decisions. For example, a diagnostically useful angiographic image may have sufficient clarity to allow a medical professional to visually identify and segment the blood vessels in the image. For example, if chest pain is caused by insufficient blood flow in the coronary arteries supplying the myocardium, a diagnostically useful angiographic image of the coronary arteries would accurately show the segment in the coronary arteries of the heart that has stenosis impairing circulation to the myocardium.

[0060] The X-ray dose required to produce diagnostically useful images in angiography also varies depending on the patient's / subject's physical characteristics and the nature of the angiographic procedure. Methods for calculating X-ray doses are well known in the art. Typically, the total / standard dose of X-ray irradiation for interventional cardiac procedures in a fluoroscopy system is in the range of 8–10 millisieverts of radiation. A sievert corresponds to 1 joule of energy per kilogram of mass. Because X-rays are ionizing, it is always desirable to minimize X-ray exposure to the patient (and medical staff involved in the angiographic procedure) while providing useful visualization of the target tissue.

[0061] The main type of contrast agent used in angiography is the family of iodine contrast agents, which are ionic or preferably nonionic iodine contrast agents. Such agents are well known in the art and include: iohexol (Omnipaque®, GE Healthcare), iopromide (Ultravist®, Bayer Healthcare), iodixanol (Visipaque®, GE Healthcare), ioxagrate (Hexabrix®, Mallinckrodt Imaging), iotalaminic acid (Cyst-Conrey II®, Mallinckrodt Imaging), and iopamidol (Osbrand). E (trademark), Bracco Imaging). See also Lusic and Granstaff, "X-ray computed tomography contrast agents," Chemical Review Vol. 13, pp. 1641-1666 (2013). Other agents include gadolinium-based agents. See Ose et al., "Gadolinium as an alternative to iodide contrast medium for X-ray angiography in patients with severe allergies," Circulation Journal, Vol. 69, pp. 507-509 (2005).

[0062] The total (standard) amount of contrast agent varies depending on the properties of the drug, the patient's / subject's physical characteristics, and the nature of the angiography method. However, generally, the total (standard) amount of contrast agent needs to improve the visualization of the target tissue by increasing the absolute CT (computerized tomography) attenuation difference between the target tissue and the surrounding tissues and fluids. In fluoroscopic angiography, injected chemical contrast agents typically increase the CT attenuation of blood vessels to 2 to 10 times the baseline level without chemical contrast agents. In fluoroscopic angiography, the injection catheter is guided close to the target organ, resulting in a higher local volume but a lower systemic volume. In CT with intravenous injection of contrast agents, the contrast agent is diluted throughout the vascular system. Therefore, the local concentration of the contrast agent in some vessels may be lower. This increase in CT attenuation due to intravenous injection of chemical contrast agents is typically 1.2 to 4 times the baseline CT attenuation without chemical contrast agents. In CT angiography, it has been shown that a CT attenuation value of at least 1.2 times from baseline, and in fluoroscopic angiography, a CT attenuation value of at least 2 times from baseline, yields "diagnostically useful" or "high-quality" angiographic images. The imaging medium should contain a high molar percentage of X-ray attenuating atoms per drug (molecule, polymer, or particle) to reduce the amount and concentration required for imaging. The tissue retention time of the contrast agent should also be long enough to complete the CT scan and schedule instrument time in the diagnostic setting (e.g., 2-4 hours). Furthermore, the contrast agent should preferably satisfy the following criteria: (a) localize or target the tissue of interest and have a favorable biodistribution and pharmacokinetic profile; (b) dissolve easily or form a stable suspension under low-viscosity aqueous physiological conditions (appropriate pH and osmotic pressure); (c) be non-toxic; and (d) be eliminated from the body in a reasonably short time, usually within a few hours (less than 24 hours).

[0063] In some embodiments, the amount of iodine contrast agent used to obtain diagnostically useful angiographic images can be reduced by approximately 25% from the total amount of contrast agent typically injected (standard amount). Further reductions can be achieved by combining the techniques described herein with the spatiotemporal reconstruction techniques described in U.S. Patent Application No. 16 / 784,073, filed February 6, 2020. This document is incorporated herein by reference in its entirety. The spatiotemporal reconstruction of the image may be an input to a machine learning model, or the output of a machine learning model may be processed using spatiotemporal reconstruction.

[0064] To improve the clarity and sharpness of vascular systems imaged with reduced chemical contrast agents and X-ray irradiation, a deep learning neural network (e.g., having an input layer, an output layer, and three or more layers between the input and output layers) is provided, which has properties that facilitate the ability to detect vascular systems with reduced chemical contrast agents and / or X-rays. While the description herein focuses on deep learning neural networks, it should be understood that these techniques may be used with any suitable machine learning model, and deep learning neural networks are merely examples. The machine learning model may be implemented by any suitable machine learning technique (e.g., mathematical / statistical, classifier, feedforward, recurrent, convolution, or other neural networks). For example, a neural network having an input layer, one or more hidden layers (e.g., including any hidden layers), and an output layer may be used. Each layer may contain one or more nodes or neurons, and neurons in the input layer may receive input (e.g., image data or image feature vectors) and associate them with weight values. For example, each node in the input layer may receive, or encode, the relative brightness of one pixel in an angiographic image as input, and the relative brightness may be a floating-point number between 0 and 1. Neurons in the hidden and output layers are connected to one or more neurons in the preceding layer, and the preceding layer receives the output of the connected neurons as input. Each connection is associated with a weight value, and each neuron generates an output based on the combination of weights of the input to that neuron. The output of a neuron may further be based on a bias value of a particular type of neural network (e.g., a recurrent type of neural network). The weight (and bias) values ​​can be adjusted based on various training techniques.For example, machine learning of a neural network may be performed using a training set of contrast-enhanced and / or X-ray-reduced angiographic images as input, and the corresponding full set of high-resolution angiographic images as known outputs, and the neural network attempts to generate known outputs by adjusting weight (and bias) values ​​(e.g., using training techniques such as backpropagation) using errors from the outputs generated by the neural network (e.g., the difference between the generated output and the known output). In an exemplary embodiment, the known output at each node may be a pixel value representing brightness or intensity (e.g., a floating-point number between 0 and 1), or the probability (p-value) that a pixel is a blood vessel (e.g., a floating-point number between 0 and 1). A neural network trained using the latter technique (i.e., segmented angiographic images where the known outputs have p-values ​​between 0 and 1 for each pixel) can be advantageously used to generate segmented angiographic images from contrast-enhanced and / or X-ray-reduced angiographic images.

[0065] More specifically, during the training phase, a data adjustment system may be provided for training the deep learning neural network to perform well in settings of reduced chemical contrast agents and / or X-ray radiation. Thus, the training phase may enable the deep learning neural network to output high-quality angiographic images based on angiographic images obtained with reduced chemical contrast agents and / or X-ray radiation during the deployment phase.

[0066] In one embodiment, a deep learning network data training system is provided that facilitates the ability of a convolutional spatiotemporal network to detect pixels corresponding to blood vessels even with reduced chemical contrast agents and / or fluoroscopic X-ray radiation. The deep learning neural network can be trained using angiography training data. The angiography training data may include (1) a first set of angiography images obtained with conventional / standard amounts of chemical contrast agents and X-ray radiation, and (2) a second set of comparative angiography images with reduced chemical contrast agents and / or X-ray radiation. In one embodiment, the comparative angiography images with reduced chemical contrast agents and / or X-ray radiation can be used as input to the training set, and the angiography images obtained with conventional / standard amounts of chemical contrast agents and X-ray radiation can be used as known outputs. In one embodiment, feature vectors may be extracted from the images and used as input along with the corresponding known outputs of the training set. The feature vectors may include any appropriate features (e.g., pixel intensity).

[0067] In one embodiment, the comparative angiographic images as a second set may be identical or nearly identical to the corresponding images in the first set, except for differences in the amount of contrast agent and / or X-rays used in the angiography. For example, the same object may be captured in both angiographic images and may be substantially the same size, position, and orientation in both images. The angiographic images in the first set and the corresponding images in the second set may be similar enough to provide useful training data for a deep learning neural network, for example, this data could be used to train a deep learning neural network to generate diagnostically useful angiographic images based on diagnostically unhelpful angiographic images obtained from angiography with reduced contrast agent and / or X-rays.

[0068] Training data can be acquired based on any suitable angiography training images. For example, training data may be acquired from (1) a laboratory environment using animals undergoing approved angiography, (2) a laboratory environment using a physical model with fluids mechanically injected into a synthetic organ being angiographically imaged, and / or (3) full-quality human clinical angiography data computer-modified to simulate reduced chemical contrast agents and / or X-rays. The data training system can train a deep learning neural network using these training options individually or in any suitable combination.

[0069] Figure 3A shows a flowchart of an exemplary method 100 for training a deep learning neural network based on vascular structures such as non-human vascular structures (e.g., training data option (1) training data obtained in a laboratory environment using animals undergoing approved angiography). In this embodiment, in step 102, a first set of angiographic images of one or more non-human vascular structures is subjected to standard / conventional amounts of chemical contrast agent (e.g., for coronary arteries, about 10 ml of 300 mg of iodine / ml) and X-ray radiation (e.g., for angiography, about 400 dGy × cm²). 2 In step 104, a second set of angiographic images of one or more non-human vascular structures is obtained with reduced chemical contrast agent (e.g., chemical contrast agent at 1 / 4 the standard dose of iodine contrast agent) and X-ray radiation (e.g., X-ray radiation at 1 / 2 the standard dose). Then in step 106, a deep learning neural network is trained on the first and second sets.

[0070] In one embodiment, angiography may be obtained using experimental animals with conventional amounts of chemical contrast agent and X-ray radiation to establish an "optimal standard" reference set of angiographic images. Subsequently, angiographic examination results can be obtained with reduced chemical contrast agent and / or X-ray doses without moving the position of the animals or the base of the fluoroscopy imaging device.

[0071] Angiographic images obtained based on conventional contrast agents and / or X-ray doses may be manually employed by human technicians to segment vessels. These segmentations can serve as a guide for segmenting vessels that are difficult to see in stress angiography, which is similar or identical in all respects except that obtained with reduced chemical contrast agents and X-rays. As used herein, the term “segmentation” may mean manually, semi-automatically, or fully automatically identifying and representing (e.g., displaying) vascular elements in an angiographic image. “Segmentation” of vessels may mean processing the image so that vascular structures are represented as distinct from noise or other structures in the imaging field. For example, vascular structures in a “segmented” angiographic image may be represented as white objects against a black background. Angiographic images subject to segmentation may be referred to herein as “segmented angiographic images.” In some embodiments, segmented angiographic images obtained with conventional chemical contrast agent and X-ray radiation doses can be used as training data (e.g., known output) for angiographic images obtained with reduced contrast agents and / or X-rays.

[0072] Figure 3B shows a flowchart of an exemplary method 200 for training a deep learning neural network based on artificial blood vessel structures (e.g., training option (2) training data obtained in a laboratory environment using a physical model with fluid mechanically injected into an angiographically imaged artificial organ). In this example, in step 202, a first set of angiographic images of one or more artificial blood vessel structures is obtained with standard / conventional amounts of chemical contrast agent and X-ray radiation. In step 204, a second set of angiographic images of one or more artificial blood vessel structures is obtained with reduced amounts of chemical contrast agent and X-ray radiation. Then, in step 206, the deep learning neural network is trained on the first and second sets.

[0073] Artificial blood vessel structures may be artificial solid blood vessel organ phantoms or organoids that include a mechanical fluid pump to simulate pulsating arterial flow. The fluid can be delivered into the artificial blood vessel structure in a network of hollow tubular structures that share the size, shape, and branching pattern of blood vessels. One manufacturer of artificial blood vessels is Heartroid®, produced by JMC Co., Ltd. (Yokohama, Japan). Artificial blood vessels are suitable for producing systematic, standard, and reduced-contrast images with chemical contrast agents and X-ray radiation.

[0074] As a specific example, an organoid is placed in an angiography system, a fluid containing a chemical contrast agent is injected, and fluoroscopic X-ray images are acquired to generate a series of angiographic images of the chemical contrast agent flowing through the organoid's vascular channels. Next, by employing a reduced interaction range between the chemical contrast agent and X-ray radiation, a vast number of angiographic images can be obtained by varying the amount of chemical contrast agent and X-rays without changing the position of the organoid or the imaging base. Angiographic images obtained with large amounts of contrast agent and / or X-rays can serve as training data for angiographic images obtained with small amounts of contrast agent and / or X-rays. Therefore, a neural network can be given data with reduced chemical contrast agent and X-ray radiation and trained using vascular data obtained from angiography with large amounts of contrast agent and / or X-rays.

[0075] Images obtained with large amounts of contrast agent and / or X-rays can be enhanced by artificial editing or mathematical processing. For example, the techniques discussed in AF Frangi, WJ Niessen, KL Vincken, and MA Viergever, “Multiscale vessel enhancement filtering,” Lecture Notes in Computer Science (including sub-series Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 1496, pp. 130-137, 1998, are used. This document is incorporated herein by reference in its entirety.

[0076] Figure 3C shows a flowchart of an exemplary method 300 for training a deep learning network based on human vascular structures (e.g., training option (3) training data obtained from full-quality human clinical angiography data computer-modified to simulate reduced chemical contrast and / or X-rays). In this example, in step 302, a first set of angiography images of one or more human vascular structures is obtained with standard amounts of chemical contrast and X-ray radiation. In step 304, a second set of angiography images of one or more human vascular structures is obtained by simulating with reduced chemical contrast and / or X-ray radiation. Then, in step 306, a deep learning neural network is trained on the first and second sets.

[0077] In one embodiment, anonymized human angiography data can be obtained with standard amounts of chemical contrast agent and X-rays currently used to generate high-quality angiographic images. Vascular structures are extracted from these images by artificial editing or mathematical vascular structure filters. The angiographic images can then be computer-modified to simulate with reduced chemical contrast agent and reduced X-rays, based on the calculation method described, for example, M. Elhamiasl and J. Nuyts, "Low-dose X-ray CT simulation from an available higher-dose scan." Phys Med Biol, vol. 65, no. 13, p. 135010, 2020 07 08. This literature is incorporated herein by reference in its entirety.

[0078] Simulations of reduced X-ray radiation can be generated by adding a mixture of Poisson and Gaussian noise to data collected by an X-ray detector. The greater the added noise, the lower the simulated X-ray dose. To simulate reduced chemical contrast agents, angiographic images are characterized by a histogram of pixel values. When angiographic images are treated as normalized, areas with greater X-ray attenuation due to the imaged tissue are represented as brighter. In this case, when a bolus of contrast agent passes through the imaged vascular system, there are more pixels with bright values ​​due to the contrast agent in the imaged vessels, and the pixel histogram shifts to the right. Therefore, reduced chemical contrast agents can be simulated by adding Poisson and Gaussian noise to the pixels that have shifted to the right on the pixel histogram due to the passage of the contrast agent bolus. For example, a pre-contrast image of the angiographic field can be obtained using the total dose of X-ray radiation without the use of a chemical contrast agent (e.g., before the chemical contrast agent is administered), and a post-contrast image of the same angiographic field can be obtained using the total dose of both the chemical contrast agent and X-ray radiation (e.g., after the chemical contrast agent is administered). For corresponding pixels in the two images, the difference in brightness or intensity between two corresponding pixels in the image may be determined (e.g., by subtracting the values ​​of each pixel). The brightness difference between two corresponding pixels may be stored in a first matrix (e.g., as floating-point numbers between 0 and 1), and a second matrix of random numbers (e.g., floating-point numbers between 0 and 1 randomly generated using a Poisson or Gaussian distribution) may be generated to a size corresponding to the first matrix. For each non-zero cell in the first matrix, the random number of the corresponding cell in the second matrix may be subtracted from the non-zero cell. This result is stored in a third matrix, and by adding this matrix to the pixel values ​​of the pre-contrast image, an angiographic image with simulated reduction of contrast agent and / or X-rays can be obtained.

[0079] To avoid registration errors due to the movement of blood vessels in the angiographic field before and after administration of a chemical contrast agent, pixels representing blood vessels may be identified, and a large amount of random noise may be added to these pixels to simulate the reduced chemical contrast agent. In particular, the noise may have a negative mean to simulate the signal reduction due to the reduction of the contrast agent and a high standard deviation to simulate the signal noise reduction. Using DICOM pixel values ​​from 0 to 32,000 as an example (instead of floating-point numbers between 0 and 1), suppose the average pixel value of the image before the chemical contrast agent arrives is a first number (e.g., 50 ± 10). When the chemical contrast agent arrives, some pixels become brighter, representing blood vessels containing the chemical contrast agent. Thus, the average pixel value increases overall (e.g., to 60 ± 15). In such a case, pixel values ​​greater than the pre-contrast mean plus two standard deviations (e.g., 50 + 20) can be assumed to be blood vessel pixels (i.e., pixels representing blood vessels). Therefore, to simulate reduced chemical contrast agents, more random noise may be added, particularly to vascular pixels.

[0080] Any appropriate data preparation operations may be applied to computer-transform the simulated contrast agent and / or X-ray reduction. For example, one or more data augmentation operations may be performed to reduce overfitting in a deep learning neural network. Examples of such data augmentation operations include any translation and rotation of the training data. For example, the transformation of the simulated contrast agent and / or X-ray reduction may be incorporated into the data augmentation step.

[0081] Images with simulated contrast agent and / or X-ray reduction may preferably have a worse signal-to-noise ratio than unmodified images and can be provided to a neural network. The neural network can then use vascular structures obtained from images acquired with standard amounts of chemical contrast agent and X-ray as reference vascular data (i.e., known outputs) to be trained to be detected and generated by the neural network. Thus, data with simulated contrast agent and / or X-ray reduction can be used as input to train a deep neural network to generate angiographic images comparable to those obtained from full amounts of chemical contrast agent and X-ray. In one embodiment, the reference vascular data may include segmented angiographic images obtained with full amounts of chemical contrast agent and X-ray radiation for training a deep neural network to generate high-quality segmented angiographic images from corresponding non-segmented angiographic images obtained or simulated obtained with reduced amounts of chemical contrast agent and / or X-ray radiation.

[0082] Figure 4 shows an exemplary system 400 configured to generate images of simulated vascular structures with reduced chemical contrast and X-ray radiation, according to an exemplary embodiment. The exemplary system 400 includes a computer 402, a display 404 connected to the computer, a pointing device 406 such as a mouse or trackpad connected to the computer, and a keyboard 408 connected to the computer. The computer 402 may include a processor that communicates with memory and / or other non-transient data storage devices containing instructions for segmenting angiographic images, which are executable by the processor. The computer 402 may also store instructions for operating a neural network in training mode and / or deployment mode, which are executable by the processor. The computer 402 may also have a communication interface configured to send and receive angiographic data over a communication network such as a local area network or a wide area network. The system 400 may be utilized for multi-frame deep learning. In one embodiment, human coronary angiography is obtained. While this example employs angiography of the human heart, these techniques may be applied to angiography of other organs or animals. In the example in Figure 4, the neural network may be running in training mode.

[0083] Angiographic images stored in computer 402 can be displayed on the display 404 of computer system 400. An exemplary coronary angiographic image 410 is displayed on the display 404. Computer system 400 can obtain the image 410 by any conventional method. For example, the image 410 may be obtained directly from an angiography imaging system, such as the system shown in Figures 1A and 1B, via a wired, wireless, or communication network. In another embodiment, computer system 400 can obtain the image 410 from a remote source via a local area network, wide area network, or other type of communication network. In yet another embodiment, computer system 400 may upload images from a portable data storage device such as a USB thumb drive or DVD. In one embodiment, a human analyst examines the exemplary image 410 and interacts with the system using a graphical user interface device such as a mouse 406 to select (i.e., paint) pixels representing blood vessels in the segmented coronary angiographic image 412. In one embodiment, segmentation may be performed solely by a human analyst by painting on the angiographic image. In other examples, a mathematical algorithm or a deep learning segmentation algorithm may perform initial segmentation inferences. In yet another embodiment, a mathematical algorithm or a deep learning segmentation algorithm may perform autopainting.

[0084] Figure 4 further shows a simulated reduced coronary angiography image 414, which simulates a reduced X-ray dose compared to the angiography image 410. As described above, the angiography image 410 can be used to generate the simulated reduced coronary angiography image 414 using any appropriate technique for simulating a reduced X-ray dose. For example, a computer system 400 may be used to generate the simulated reduced coronary angiography image. Once generated, the simulated reduced coronary angiography image 414 can be used to train a deep learning neural network by referencing the segmentation of the same angiography image obtained with the full X-ray dose in a saved segmented coronary angiography image 416 (which may be the same image as segmented image 412).

[0085] In Figure 4, the angiographic image 410 and the segmented image 412 are displayed side by side, but in other embodiments, painting may be performed on the same image on the display 404. Therefore, painting may be superimposed on the angiographic image 410 to generate the segmented image 412.

[0086] In training mode, the neural network can acquire multiple temporally consecutive and adjacent images (e.g., five images) obtained in standard quantities, and multiple corresponding images obtained (or simulated obtained) in reduced quantities, as training data. Images can be selected from a DICOM (Digital Imaging and Communications in Medicine) file containing, for example, approximately 80 images in total. Each image is 512 x 512 pixels in size, and each pixel can be represented as an integer (e.g., approximately 0 to approximately 16,000) in the DICOM format.

[0087] In one embodiment, at least one of a plurality of temporally consecutive images (e.g., five images) obtained in standard quantities may be used as a known output of the neural network, and at least one of a plurality of corresponding images obtained (or simulated to be obtained) in reduced quantities of contrast agent and / or X-rays may be used to encode the input layer of the neural network. The neural network can form connections between individual neurons based on the training data, and each connection has one or more associated weights (e.g., floating-point numbers) to generate a known output from the input. In one embodiment, if the known output consists of a segmented angiographic image, the neural network may be trained to assign each pixel in the output image a floating-point number between 0 and 1 representing the probability that the pixel is part of a blood vessel. In another embodiment, the neural network may assign each pixel a plurality of floating-point numbers between 0 and 1 representing the probability that the pixel is part of a particular feature (e.g., a blood vessel, a catheter, and / or a bifurcation). For example, three floating-point numbers between 0 and 1 may be assigned, one floating-point number representing the probability that the pixel is part of a blood vessel, another floating-point number representing the probability that the pixel is part of a catheter, and another floating-point number representing the probability that the pixel is part of a branch point (for example, where one blood vessel branches into two or more blood vessels).

[0088] A checkpoint file that stores weights for a neural network can be used in unfolding / predictive mode. For example, training with images obtained with standard contrast agent and / or X-ray angiography may result in a first checkpoint file with one set of weights, while training with images obtained (or simulated obtained) with reduced contrast agent and / or X-ray angiography may result in a second checkpoint file with a different set of weights. The first checkpoint file can be applied to the input of one or more images obtained with standard contrast agent and / or X-ray angiography to obtain one or more segmented images, and the second checkpoint file can be applied to the input of one or more images obtained (or simulated obtained) with reduced contrast agent and / or X-ray angiography to obtain one or more segmented images. The second checkpoint file can be applied in a clinical setting where reduced contrast agent and / or X-ray angiography is administered to avoid giving patients the side effects of angiography with large amounts of contrast agent and / or X-rays while maintaining angiographic image quality.

[0089] Figure 5 shows an exemplary method 500 for segmenting vascular objects within a single image from angiographic images acquired with reduced amounts of chemical contrast agent or X-rays. In this example, a convolutional network extracts information from multiple angiographic images to perform segmentation. In the example in Figure 5, the neural network may be operating in unfolded mode.

[0090] Analyzing an entire sequence of angiographic images (e.g., performing calculations) can exceed the practical limits of a computer's memory and computing speed. Therefore, as illustrated in Figure 5, the techniques described herein enable the generation of high-quality images based on subsequences of images, within the constraints of that computer's memory and computing speed. As used herein, the term “sequence” may refer to an angiographic image, e.g., an entire set of images acquired fluoroscopically over the movement of a bolus of injected contrast agent. The term “subsequence” may refer to a secondary set of images provided to a deep learning neural network system for estimating vascular structure. The angiographic images within a “subsequence” may preferably be temporally continuous or continuous but separated by intervening images.

[0091] Figure 5 shows an example of segmenting a single angiographic image from a subsequence of noisy angiographic images acquired with reduced chemical contrast agents or reduced X-ray radiation doses. In this example, the angiographic image to be segmented is drawn from a subsequence of five temporally adjacent angiographic images 502(a)–(e) drawn from angiographic images with physically reduced chemical contrast agents and X-rays. In this example, the middle (third) image 502(c) in the subsequence of five angiographic images is the target image (image of interest). The target image may be the image from which a neural network generates a high-quality version.

[0092] The subsequences of the five angiographic images 502(a) to (e) may be combined and fed to the convolutional neural network 504 as a single input. For example, if each image consists of a 512 × 512 vector of pixel values, the five angiographic images may be input to or encoded into the neural network as a 5 × 512 × 512 vector of pixel values, and the neural network may be trained to generate only one high-quality and / or segmented angiographic image corresponding to one of the five angiographic images (e.g., the central image 502(c)).

[0093] The convolutional neural network 504 may be pre-trained with images that simulate noised contrast agent and / or X-ray reduction, rather than with vascular segmentation extracted from a standard amount of noise-free images (for example, as described above in relation to Figure 4). For example, a subsequence of low-resolution images containing a target image may be obtained to train the neural network to generate only one high-resolution and / or segmented angiographic image from five temporally consecutive low-resolution images. The target image is preferably positioned in the center of the subsequence such that there is one or more images before the target image and one or more images after the target image. In this way, the neural network can be trained to use spatiotemporal information when providing the output. In the subsequence, the number of images before the target image may differ from the number of images after the target image. However, in a preferred embodiment, the target image is positioned in the center of the subsequence such that there are an equal number of images on both sides of the target image. For example, in the case of five temporally consecutive images, the target image is preferably positioned between two temporally consecutive images on either side of the target image. In some embodiments, the neural network may be trained using multiple subsequences. For example, if there are 10 images available from the first angiography and 12 from the second angiography, the training set can be assembled with up to 14 unique subsequences, each consisting of 5 temporally consecutive images. That is, 6 unique subsequences of 5 temporally consecutive images can be assembled from the first angiography (because images 3 through 8 are each located in the center of 5 temporally consecutive images), and 8 subsequences of 5 temporally consecutive images can be assembled from the second angiography (because images 3 through 10 are each located in the center of 5 temporally consecutive images). The first, second, second-to-last, and last images in each series are not targets in this example because they are not located in the center of 5 temporally consecutive images.Using a subsequence of five temporally consecutive images has been shown to be advantageous in that it can generate high-quality and / or segmented images with less processing and relatively fewer images at both ends of the sequence that are not the target themselves. However, current techniques can also be adapted to use subsequences containing fewer than five temporally consecutive images or more than five temporally consecutive images. In general, increasing the number of images in a subsequence tends to increase the signal-to-noise ratio of the images generated by the neural network. At the same time, increasing the number of images in a subsequence increases the number of images that are missing at both ends.

[0094] While it is advantageous to use a temporally consecutive odd number of images as a subsequence for training a neural network to generate high-quality and / or segmented images corresponding to a target image in the center of a subsequence (for example, because they are based on equal amounts of spatiotemporal information before and after the target image), this technique can be adapted to generate high-quality and / or segmented images corresponding to a target image that is not in the center of a subsequence (for example, the first, second, fourth, or fifth image in a subsequence of five images).

[0095] Furthermore, this technology can be adapted to use subsequences consisting of an even number of images that are consecutive in time. The advantage of using subsequences consisting of an even number of images that are consecutive in time is that it provides greater flexibility regarding the position of the target image and the size of the subsequence (for example, it is possible to target the first or last image in an image sequence by using only a subsequence consisting of two images that are consecutive in time). It should also be understood that this technology can be adapted to use a combination of odd and even subsequences to generate high-quality and / or segmented images corresponding to corresponding targets located anywhere in the sequence. In another embodiment, multiple neural networks can be deployed, each neural network being trained on a different number of input images, from one to as many as the processing resources allow. By using multiple neural networks in this way, high-quality and / or segmented images can be generated from all images of angiography.

[0096] In one embodiment, the images in the training set may be subjected to data augmentation steps, such as translation and / or rotation, before being input into the neural network for training and good generalization. It should be understood that simulations with reduced chemical contrast agents and / or X-ray radiation may be incorporated as part of the data augmentation steps.

[0097] Because the trained convolutional neural network 504 operates on a subsequence of five noisy angiographic images 502(a)-(e), it can estimate a single segmented image 506 with a larger signal-to-noise ratio than if the trained convolutional neural network were operating on only a single noisy angiographic image. The segmented image 506 is derived from the five noisy angiographic images 502(a)-(e). 02 This can represent the deep learning neural network estimation of the vascular structure in the central (third) image in the subsequences (a) to (e).

[0098] More specifically, the neural network 504 loads floating-point numbers from a checkpoint file generated during training mode, and based on these floating-point numbers, it generates five noisy angiographic images 5 02 A segmented image 506 can be output from (a) to (e). The neural network may output one or more of three images: one image based on the probability that a pixel is part of a blood vessel, another image based on the probability that a pixel is part of a catheter, and another image based on the probability that a pixel is part of a bifurcation. These three images can be stacked and superimposed to generate a segmented image 506 that shows blood vessels, catheters, and / or bifurcations in high quality, for example, as if the target image were obtained from angiography with standard contrast and / or X-rays.

[0099] For example, the convolutional neural network 504 used for angiography segmentation is based on the "U-net architecture," which is described, for example, in O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015, N. Navab, J. Hornegger, WM Wells, and AF Frangi, Eds. Cham: Springer International Publishing, 2015, pp. 234-241, and this document is incorporated herein by reference in its entirety.

[0100] In a specific example, the source code for a deep learning neural network may be written in the Python language and generated as a U-net architecture in the PyTorch machine learning software library (see https: / / pytorch.org; the entire website is incorporated herein by reference). The U-net architecture may be three-dimensional in the sense that each image has two spatial dimensions, and the vascular system in an image may be estimated from both the image and temporally adjacent angiographic images. In one specific example, the PyTorch library for neural network machine learning may include a Python base class called nn.Module, as found at https: / / pytorch.org / docs / stable / generated / torch.nn.Module.html; the entire website is incorporated herein by reference. All neural network modules consist of the subclass nn.Module. A Python class called UNet may inherit from nn.Module. The Python class UNet may contain a convolutional neural network structure with spatiotemporal properties. It should be understood that the techniques described herein can be implemented using any appropriate machine learning mechanism.

[0101] It should be understood that any suitable programming language, library, toolset, and / or other mechanism may be used to deploy deep learning neural networks according to the embodiments provided herein. For example, the U-net structure may be extended to a complete three-dimensional structure that simultaneously estimates vascular structures in multiple temporally adjacent angiographic images.

[0102] The convolutional neural network 504 may have an encoder-decoder structure. The convolutional neural network 504 may also include jump connections between layers of the same size encoder and decoder. These jump connections may enable the output of segmented images 506 with a granularity similar to that of angiography subsequence inputs (e.g., subsequences of five angiography images 502(a)-(e)). The loss function used to train this architecture may be a linear combination of a classification loss function, cross-entropy, and dice loss function for sharp boundary detection.

[0103] To prevent the movement of blood vessels between angiography image frames from interfering with the angiography data, the structure of the convolutional neural network 504 can have a high spatiotemporal convolution density. That is, organs with large movements, such as the heart, can be imaged at 15 Hz and used to determine the vascular system in which the neighborhoods of five images are located in the central image. Organs with small movements, such as the brain, can be imaged at, for example, 6 Hz and used to determine the vascular system in which the neighborhoods of five images are located in the central image. In this way, the convolutional neural network 504 can explain both small movements (such as the movement of the brain, which is confined within the rigid container of the skull and whose movement is restricted) and large movements (such as the movement of the heart, a muscular organ that is constantly beating to pump blood into the arterial system).

[0104] In contrast to the deployment mode illustrated in Figure 5, the roles of the data source (input) and product (output) may be changed in the deep learning network training mode. For example, image 506 may represent a ground truth representation of vascular structure, such as that obtained from angiography with full-volume chemical contrast and X-ray radiation, and one or more images in subsequences 502(a) to (e) may be obtained from empirically reduced contrast agent and / or X-ray examinations from animal or physical organoid model angiography, or from reduced images simulated by a computer drawn from the full-volume ground truth image 506. In training mode, training may be performed based on both one or more images in subsequences 502(a) to (e) and the segmented image 506.

[0105] Figure 5 illustrates subsequences 502(a)-(e) as having five angiographic images, but please understand that the number five is merely illustrative. Depending on the situation, fewer or more images may be used for the same angiography. For example, more than five angiographic images may be used to estimate the segmentation of intermediate images in an angiographic sequence containing dozens of individual images. This can increase the signal-to-noise ratio in the angiographic images of interest compared to using only five angiographic images (at the cost of increased computer resources), thereby further reducing the amount of chemical contrast agent and / or X-ray radiation.

[0106] In this configuration, the vascular segmentation of angiographic images can be estimated based on fewer surrounding images as you move towards the beginning or end of the sequence. For example, the sixth image may be segmented based on the central position of five images (e.g., 4th, 5th, 6th, 7th, and 8th), while the fifth image, although it cannot be the center of the subsequence of five images, may be treated as being the center of the subsequence of three images (e.g., 4th, 5th, and 6th).

[0107] In addition, the angiographic image of interest (e.g., the target image) may be located at a different point in the subsequence (e.g., near the beginning or end of the subsequence) rather than in the middle of it. For example, during angiography catheter positioning, an angiographer may choose to perform intermittent real-time angiography injections while acquiring fluoroscopic images. The angiographic image to be viewed cannot be the central image in a real-time angiographic situation because future angiographic images have not yet been acquired. Instead, a deep learning network can perform signal-to-noise enhancement based on the target image and several images that precede it in time.

[0108] Figure 6 illustrates the subsetting of a sequence of angiographic images into multiple overlapping subsequences, each of which estimates vascular structures for a single angiographic image. More specifically, Figure 6 shows a first exemplary method for estimating a high-resolution image when the target image is in the center of the subsequence, and a second exemplary method for estimating a high-resolution image when the target image is not in the center of the subsequence. The first method is applicable to offline fluoroscopic angiography, and the second method may be applicable to real-time fluoroscopic angiography.

[0109] The first method 600 is shown in the upper panel of Figure 6. As shown, a sequence 602 of angiographic images with a total length n is generated. Within the sequence 602 of angiographic images, there is a subsequence 604 of five angiographic images, with the central image being the focus image for segmentation by the deep learning network. The neural network can generate a segmented image 606 corresponding to the central (third) angiographic image of the subsequence. This process can be repeated image by image for the sequence 602 of angiographic images until all images except the first two and the last two are focus images for deep learning segmentation. This can generate a sequence 608 of segmented images.

[0110] The second method 700 is illustrated in the lower panel of Figure 6. Here, the angiographic image of interest may be the first (most recent) angiographic image 704(a) of the subsequence of angiographic images 704(a)-(c). The neural network applies deep learning calculations to improve the signal-to-noise ratio of the target image 704(a) and generate a high-quality segmented version 706(a) of the target image, thereby maintaining image quality even when the dose of chemical contrast agent or X-ray dose is reduced. For example, the neural network may use the target image and one or more subsequent temporally consecutive images as input to the deep learning network calculations.

[0111] The present invention may include methods, systems, apparatus, and / or computer program products at any level of technical detail that can be integrated. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0112] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each arithmetic / processing unit, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include conductive transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each arithmetic / processing unit may receive computer-readable program instructions from the network and transfer them for storage in a computer-readable storage medium within each arithmetic / processing unit.

[0113] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0114] These computer-readable program instructions may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to manufacture a machine. This creates a means for instructions executed via the processor of a computer or other programmable data processing device to perform functions / operations specified in the blocks of a flowchart and / or block diagram. These computer-readable program instructions may be stored in a non-temporary computer-readable storage medium that can instruct a computer, a programmable data processing device, and / or other device to function in a particular manner. Thus, the computer-readable storage medium on which the instructions are stored consists of a product containing instructions that perform the modes of functions / operations specified in the blocks of a flowchart and / or block diagram.

[0115] Computer-readable program instructions may be loaded into a computer, other programmable data processing device, or other device, and a series of operations may be executed on the computer, other programmable device, or other device to generate a computer implementation process. This ensures that the instructions executed on the computer, other programmable device, or other device perform the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0116] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which may contain one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in a block may occur in a different order than shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, and blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in a block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs a specified function or operation, or a special-purpose combination of hardware and computer instructions.

[0117] The above description is intended to instruct those skilled in the art on how to carry out the subject matter of this application and is not intended to detail all obvious modifications and variations that would be apparent to those skilled in the art by reading the description. However, all such obvious modifications and variations are intended to be included within the scope of the invention as defined by the claims. The claims are intended to include components and processes in any order effective in satisfying the intended purpose therein, unless the context specifically indicates otherwise.

Claims

1. A method for providing a first sequence of angiographic images of a subject obtained fluoroscopically at a rate faster than the subject's heart rate using a first amount of chemical contrast agent and / or X-ray radiation, as input to a machine learning model via a processor, The machine learning model is trained using (a) a second sequence of angiographic images obtained or simulated to have been obtained fluoroscopically at a rate faster than the heart rate using a second amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as training input, and (b) a third sequence of angiographic images obtained fluoroscopically at a rate faster than the heart rate using a third amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as a known output, wherein the third amount is greater than the first and second amounts, the first sequence of angiographic images and the third sequence of angiographic images are obtained using a non-zero amount of chemical contrast agent, and the second sequence of angiographic images is obtained or simulated to have been obtained using a non-zero amount of chemical contrast agent. The process involves obtaining an output from the machine learning model, via the processor, which includes a segmented angiographic image, which is an image of interest from the first sequence of angiographic images. A method wherein the first sequence of angiographic images is provided to the machine learning model as a single vector.

2. The method according to claim 1, wherein the first and second sequences of angiographic images each consist of an odd number of angiographic images.

3. The method according to claim 2, wherein the image of interest is located in the center of the first sequence of angiographic images.

4. The method according to claim 1, wherein the first sequence of angiographic images consists of the same number of angiographic images as the second sequence of angiographic images.

5. The method according to claim 1, wherein the first and second sequences each consist of five angiographic images.

6. The first sequence of angiographic images is one of a plurality of input subsequences, each of which is extracted from a different portion of a larger sequence of angiographic images obtained using the first amount of chemical contrast agent and / or X-ray radiation. The method according to claim 1, further comprising providing the plurality of input subsequences as input to the machine learning model via at least one of the processors, and obtaining an output from the machine learning model via at least one of the processors, which includes a plurality of segmented angiographic images, the plurality of angiographic images of interest in the plurality of input subsequences.

7. The method according to claim 1, wherein the second sequence of angiographic images is one of a plurality of training subsequences used to train the machine learning model, each of the plurality of training subsequences being extracted from a different portion of a larger number of training sequences of angiographic images obtained using the second amount of chemical contrast agent and / or X-ray radiation.

8. The method according to claim 1, wherein each of the angiographic images in the first sequence of angiographic images is temporally continuous with at least one other angiographic image in the first sequence, and each of the angiographic images in the second sequence of angiographic images is temporally continuous with at least one other angiographic image in the second sequence.

9. The method according to claim 1, wherein the first and second amounts are the same.

10. The method according to claim 1, wherein the first and second amounts are less than the amount necessary to obtain a diagnostically useful image from the X-ray fluoroscopy imaging device, and the third amount is sufficient to obtain a diagnostically useful image from the X-ray fluoroscopy imaging device.

11. The method according to claim 1, wherein the angiographic image of the second sequence is an angiographic image of a non-human vascular structure.

12. The method according to claim 1, wherein the angiographic image of the second sequence is an angiographic image of an artificial blood vessel structure.

13. The method according to claim 1, wherein the second amount of chemical contrast agent and / or X-ray radiation is greater than the first amount of chemical contrast agent and / or X-ray radiation, and the angiographic image in the second sequence is a modified angiographic image that simulates being obtained using a smaller amount of chemical contrast agent and / or X-ray radiation than the second amount.

14. The method according to claim 1, wherein the second amount of chemical contrast agent and / or X-ray radiation is greater than the first amount of chemical contrast agent and / or X-ray radiation, and the second sequence of angiographic images is modified to simulate that obtained using a smaller amount than the second amount of chemical contrast agent and / or X-ray radiation.

15. The method according to claim 14, wherein the second sequence of angiographic images is modified by adding randomly generated noise to at least some of the pixels of the second sequence of angiographic images.

16. The method according to claim 15, wherein the randomly generated noise is added only to pixels that are determined to correspond to blood vessels in the second sequence of the angiographic image.

17. The method according to claim 1, wherein the angiographic images of the third sequence of angiographic images are segmented angiographic images, and the output obtained from the machine learning model is an angiographic image which is a segmented version of the image of interest.

18. The method according to claim 1, further comprising displaying the segmented image, which is the image of interest, on a display via the processor.

19. It is a system, One or more memory devices, It includes at least one processor connected to one or more of the memory devices, and at least one of the processors is A first sequence of angiographic images of a subject obtained fluoroscopically at a rate faster than the subject's heart rate using a first amount of chemical contrast agent and / or X-ray radiation is provided as input to a machine learning model, the machine learning model being trained with (a) a second sequence of angiographic images obtained or simulated to have been obtained fluoroscopically at a rate faster than the subject's heart rate using a second amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as training input, and (b) a third sequence of angiographic images obtained fluoroscopically at a rate faster than the subject's heart rate using a third amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as a known output, wherein the third amount is greater than the first amount, the first sequence of angiographic images and the third sequence of angiographic images are obtained using a non-zero amount of the chemical contrast agent, and the second sequence of angiographic images is obtained or simulated to have been obtained using a non-zero amount of the chemical contrast agent. From the aforementioned machine learning model, an output is obtained which includes segmented angiographic images, which are images of interest in the first sequence of angiographic images. The first sequence of angiographic images is provided to the machine learning model as a single vector in the system.

20. The system according to claim 19, wherein the image of interest is located in the center of the first sequence of angiographic images.

21. The system according to claim 19, wherein the first sequence of angiographic images is one of a plurality of input subsequences of angiographic images provided to the machine learning model, each of the plurality of input subsequences being extracted from different portions of a larger sequence of angiographic images obtained using a first amount of chemical contrast agent and / or X-ray radiation, and at least one processor is configured to provide the plurality of input subsequences as input to the machine learning model and to obtain from the machine learning model an output comprising a plurality of segmented angiographic images which are images of interest in the plurality of angiographic images in the plurality of input subsequences.

22. The system according to claim 19, wherein the first sequence of angiographic images consists of the same number of angiographic images as the second sequence of angiographic images.

23. The system according to claim 19, wherein the third amount of chemical contrast agent and / or X-ray radiation is greater than the second amount of chemical contrast agent and / or X-ray radiation.

24. The system according to claim 19, wherein the second amount of chemical contrast agent and / or X-ray radiation is greater than the first amount of chemical contrast agent and / or X-ray radiation, and the second sequence of angiographic images is modified to simulate that it was obtained using a smaller amount than the second amount of chemical contrast agent and / or X-ray radiation.

25. The system according to claim 19, wherein the first and second amounts are less than the amount necessary to obtain a diagnostically useful image from the X-ray imaging device, and the third amount is sufficient to obtain a diagnostically useful image from the X-ray imaging device.

26. The system according to claim 19, wherein the angiographic image of the third sequence of angiographic images is a segmented angiographic image.

27. The system according to claim 19, further comprising a display device, wherein at least one of the processors is configured to display a segmented image, which is the image of interest, on the display device.

28. A computer program product comprising one or more non-temporary computer-readable storage media in which instructions are stored, wherein the instructions are instruction execution by at least one processor, and by the instructions, at least one of the processors A first sequence of angiographic images of a subject obtained fluoroscopically at a rate faster than the subject's heart rate using a first amount of chemical contrast agent and / or X-ray radiation is provided as input to a machine learning model, the machine learning model being trained with (a) a second sequence of angiographic images obtained or simulated to have been obtained fluoroscopically at a rate faster than the subject's heart rate using a second amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as training input, and (b) a third sequence of angiographic images obtained fluoroscopically at a rate faster than the subject's heart rate using a third amount of chemical contrast agent and / or X-ray radiation, provided to the machine learning model as a known output, wherein the third amount is greater than the first amount, the first sequence of angiographic images and the third sequence of angiographic images are obtained using a non-zero amount of the chemical contrast agent, and the second sequence of angiographic images is obtained or simulated to have been obtained using a non-zero amount of the chemical contrast agent. From the aforementioned machine learning model, an output is obtained which includes segmented angiographic images, which are images of interest in the first sequence of angiographic images. The first sequence of angiographic images is provided to the machine learning model as a single vector in a computer program product.

29. The computer program product according to claim 28, wherein the image of interest is located in the center of the first sequence of angiographic images.

30. The computer program product according to claim 28, wherein the first sequence of angiographic images is one of a plurality of input subsequences of angiographic images provided to the machine learning model, each of the plurality of input subsequences being extracted from different parts of a larger sequence of angiographic images obtained using a first amount of chemical contrast agent and / or X-ray radiation, the instruction is executable by at least one of the processors, the instruction causes at least one of the processors to provide the plurality of input subsequences as input to the machine learning model, and the machine learning model produces an output comprising a plurality of segmented angiographic images, which are images of interest in the plurality of angiographic images in the plurality of input subsequences.

31. The computer program product according to claim 28, wherein the first sequence of angiographic images consists of the same number of angiographic images as the second sequence of angiographic images.

32. The computer program product according to claim 28, wherein the third amount of the chemical contrast agent and / or X-ray radiation is greater than the second amount of the chemical contrast agent and / or X-ray radiation.

33. The computer program product according to claim 28, wherein the second amount of chemical contrast agent and / or X-ray radiation is greater than the first amount of chemical contrast agent and / or X-ray radiation, and the second sequence of angiographic images is modified to simulate that it was obtained using a smaller amount of chemical contrast agent and / or X-ray radiation than the second amount.

34. The computer program product according to claim 28, wherein the first and second amounts are less than the amount necessary to obtain a diagnostically useful image from the X-ray imaging device, and the third amount is sufficient to obtain a diagnostically useful image from the X-ray imaging device.

35. The computer program product according to claim 28, wherein the angiographic images of the third sequence of angiographic images are segmented angiographic images, and the output obtained from the machine learning model is an angiographic image which is a segmented version of the image of interest.

36. The computer program product according to claim 28, wherein the instruction is executable by at least one of the processors, and by the instruction, at least one of the processors displays a segmented image, which is the image of interest, on a display device.

37. The method according to claim 1, wherein the plurality of angiographic images of the second sequence of angiographic images are obtained fluoroscopically.

38. The method according to claim 1, wherein the plurality of angiographic images in the second sequence of angiographic images are based on one or more images obtained fluoroscopically.

39. The system according to claim 19, wherein the plurality of angiographic images of the second sequence of angiographic images are obtained fluoroscopically.

40. The system according to claim 19, wherein the plurality of angiographic images in the second sequence of angiographic images are based on one or more images obtained fluoroscopically.

41. The system according to claim 19, wherein each of the angiographic images in the first sequence of angiographic images is temporally continuous with at least one other angiographic image in the first sequence, and each of the angiographic images in the second sequence of angiographic images is temporally continuous with at least one other angiographic image in the second sequence.

42. The computer program product according to claim 28, wherein the plurality of angiographic images of the second sequence of angiographic images are obtained fluoroscopically.

43. The computer program product according to claim 28, wherein the plurality of angiographic images in the second sequence of angiographic images are based on one or more images obtained fluoroscopically.

44. The computer program product according to claim 28, wherein each of the angiographic images in the first sequence of angiographic images is temporally continuous with at least one other angiographic image in the first sequence, and each of the angiographic images in the second sequence of angiographic images is temporally continuous with at least one other angiographic image in the second sequence.

45. The method according to claim 1, wherein the second amount is at least 25% of the third amount.

46. The method according to claim 1, wherein the first amount is at least 75% of the second amount.

Images