Program, information processing device, and model generation method

Abstract

To provide a program or the like that can generate a vascular image that reflects the dynamics of blood vessels from input pulsation information. [Solution] A program that acquires multiple X-ray fluoroscopic images of a patient's blood vessels and the patient's pulsation information, and causes a computer to execute a process in which it inputs the acquired X-ray fluoroscopic images and the pulsation information into a trained model that generates a video of blood vessels synchronized with the pulsation information when the X-ray fluoroscopic images and the pulsation information are input, and generates a video of blood vessels synchronized with the pulsation information.

Description

Technical Field

[0001] The present invention relates to a program, an information processing apparatus, and a model generation method. Background Art

[0002] In medical procedures using catheters, it is critical to insert the catheter into the target position safely and accurately. For this reason, a common technique is to perform angiography to check the shape, course, presence or absence of stenosis, and other conditions of blood vessels. Such angiographic images are referenced during the procedure.

[0003] Contrast agents have been used to visualize blood vessel images, but especially for hypertensive patients, contrast agents carry risks of causing transient blood pressure elevation and contrast-induced nephropathy, so their use tends to be kept to the minimum necessary. In addition, even when a contrast agent is used, the time during which the blood vessel image can be confirmed is short, and conventionally, blood vessels have been grasped using still images captured at the time of contrast agent injection. However, since blood vessels move minutely along with the heartbeat, if a mismatch occurs between the heartbeat-synchronized vascular motion and the distal end position of the catheter, friction or pressure due to contact with the vascular wall may occur, which may amplify risks such as damaging the vascular intima.

[0004] For example, Patent Document 1 discloses that an image representing a specific site is created by converting an X-ray fluoroscopic image of a region including the specific site of a subject using a trained model. However, during a procedure, it is difficult to accurately capture minute vascular motions, particularly dynamics such as arterial dilation and contraction accompanying the heartbeat, in real time, and there has been a demand for an assistance technique capable of grasping vascular dynamics in real time. Prior Art Documents Patent Documents

[0005] Patent Document 1 International Publication No. 2019 / 138438 Summary of Invention [Problems that the invention aims to solve]

[0006] The present invention has been made in view of the above circumstances, and aims to provide a program, etc., capable of generating a video of a vascular image corresponding to a patient's pulsation information. [Means for solving the problem]

[0007] The present invention, which solves the above problems, has the following configuration. [1] Multiple fluoroscopic images of the patient's blood vessels and the patient's pulsation information are obtained. When X-ray fluoroscopy images and pulse information are input, a trained model that generates a video of vascular images synchronized with the input pulse information is used to generate a video of vascular images synchronized with the acquired pulse information. A program that instructs a computer to perform a process. [2] The plurality of X-ray fluoroscopic images are a set of frame images of an X-ray fluoroscopic video, as described in [1]. [3] The program described in [1] or [2], wherein the plurality of X-ray fluoroscopic images are taken using a contrast agent. [4] A program described in any of [1] to [3] that displays positional information of a catheter inserted into a blood vessel in the generated video of the blood vessel. [5] The program further outputs an alarm if the catheter displayed in the output video of the vascular image deviates from a preset range. A program described in [4] that causes a computer to perform a process.

[0008] [6] An acquisition unit that acquires X-ray fluoroscopic images of the patient's blood vessels and the patient's pulsation information, Based on the acquired X-ray fluoroscopy images and pulse information, an output unit generates and outputs a video of the vascular image synchronized with the acquired pulse information. An information processing device having, The information processing device generates a video of a vascular image synchronized with the input pulsation information, based on the input X-ray fluoroscopy image and pulsation information, which is generated by a computer.

[0009] [7] Training data is obtained including multiple fluoroscopic images of the patient's blood vessels, the patient's pulsation information, and fluoroscopic videos corresponding to the pulsation information. Based on the aforementioned training data, when the X-ray fluoroscopic image and the corresponding pulsation information are input, an X-ray fluoroscopic video of the vascular image synchronized with the pulsation information is generated. A model generation method in which a computer performs the processing. [Effects of the Invention]

[0010] According to the present invention, it is possible to generate a video of the vascular image corresponding to the patient's pulsation information using that pulsation information. This allows the surgeon to constantly monitor the dynamics of specific vascular sites while suppressing the amount of contrast agent used during the procedure, thereby contributing to improved safety and efficiency of the procedure. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing the configuration of an information processing device related to one embodiment of the present invention. [Figure 2] This is a block diagram showing the configuration of an information processing device according to one embodiment of the present invention. [Figure 3] Figure 3(A) is a schematic diagram showing the configuration of a trained model M according to one embodiment of the present invention, and Figure 3(B) is a schematic diagram showing an example of images (22a, 22b) included in an X-ray fluoroscopic image(s). [Figure 4] This is a flowchart showing a training method for generating a trained model M according to one embodiment of the present invention. [Figure 5] This is a flowchart showing a training method for generating a trained model C according to one embodiment of the present invention. [Figure 6] This is a flowchart showing a video generation method according to one embodiment of the present invention. [Figure 7] This is a flowchart illustrating a catheter superimposition and deviation determination method according to an embodiment of the present invention. [Modes for Carrying Out the Invention]

[0012] In one embodiment, the present invention provides a trained model that generates a moving image of a vascular image reflecting vascular dynamics synchronized with pulsation information, based on information related to pulsation acquired from a patient (hereinafter referred to as "pulsation information"). The trained model used in the present embodiment is pre-trained using patient pulsation information and a plurality of X-ray fluoroscopic images, so as to generate a time-series vascular image reflecting the motion associated with vascular pulsation, that is, a moving image showing vascular dynamics, when pulsation information is input. Furthermore, in one embodiment, it is also possible to display the position information of the catheter present in the blood vessel in a superimposed manner on the generated moving image.

[0013] As used herein, "pulsation information" refers to information acquired from a patient during a medical procedure that reflects the rhythm or cycle of the patient's own heartbeat or pulse. Examples include, but are not limited to, electrocardiogram data, blood pressure waveform data, pulse oximetry (SpO2) pulse wave data, or information such as heart rate and heartbeat cycle extracted based on these, and other types of data can also be used as long as the data allows estimation of the vascular dynamics of the patient during the procedure.

[0014] In the present invention, "catheter" means an elongated tubular medical device to be inserted into a blood vessel. The catheter is a general term that includes "catheter tubes", which are flexible hollow tubular instruments, and "guide wires", which are flexible linear members used to insert and guide instruments to a target site in catheter treatment or endovascular surgery.

[0015] "Catheter tubes" are medical devices used for inserting therapeutic devices therethrough, injecting and discharging contrast media, drug solutions and the like, or acquiring biological information. Specific examples include, but are not limited to, angiographic catheters, balloon catheters, stent delivery catheters, ablation catheters, debulking catheters (including rotablaters, Diamondback devices and the like), aspiration catheters (including thrombus aspiration catheters), drainage catheters, endovascular treatment catheters (including those for coronary arteries, cerebral blood vessels, peripheral blood vessels and the like), intravascular ultrasound (IVUS) catheters, microcatheters, guiding catheters, sheath introducers, central venous catheters (CVC), pulmonary artery catheters (Swan-Ganz catheters and the like), feeding catheters, and endoscopic catheters. "Guide wires" are usually thin linear bodies made of metal (stainless steel, superelastic alloy, etc.) or synthetic resin, and are inserted in advance into blood vessels or the like to guide catheter tubes and other therapeutic devices to a target site. Specific examples include, but are not limited to, guide wires, steering wires, delivery wires and the like.

[0016] Hereinafter, a program, an information processing apparatus, and a model generation method according to the present invention will be described with reference to the drawings. It should be noted that the present invention is not limited to the following embodiments, and can be appropriately modified within a range that does not impair the effects of the present invention. [Configuration of Information Processing System] Figure 1 is a configuration diagram showing an embodiment of an information processing system according to the present invention. The information processing system of the present invention is mainly composed of the following constituent elements.

[0017] In one embodiment, in the model generation step described later, the information processing system 1 includes an information processing apparatus 100, a pulsation information acquisition apparatus 2, and an angiography apparatus 3, and may further include a display apparatus 20. In one embodiment, during the video generation stage using the trained model M described later, the information processing system includes an information processing device 100, a pulse information acquisition device 2, an angiography device 3 (hereinafter referred to as the imaging device 3), and a display device 20. Furthermore, a catheter information acquisition device 4 and an alarm device 21 may be included as needed. Each device shown in Figure 1 transmits and receives information such as data and control signals. Wired connections (e.g., USB cable, BNC cable, HDMI® cable) or wireless connections (e.g., Wi-Fi®, Bluetooth®, internet connection) can be used for connection (hereinafter referred to as wired / wireless connection).

[0018] The information processing device 100 is a device capable of processing various types of information and transmitting and receiving information. In the following description, a personal computer will be used as the information processing device 100, but it may also be a server computer connected by wired or wireless connection. The information processing device 100 incorporates a trained model M and acquires X-ray fluoroscopic images taken by the imaging device 3, or X-ray fluoroscopic images taken by the imaging device 3 and stored in the database, as well as pulsation information output from the pulsation information acquisition device 2. Using the trained model M, it generates a video of the vascular image synchronized with the pulsation information and outputs it to the display device 20. In one embodiment, the information processing device 100 may further have a function to acquire positional information of an intravascular catheter from a catheter information acquisition device 4 or the like, estimate that positional information using a trained model C, and superimpose it on the generated video of the vascular image. In one embodiment, the information processing device 100 may be configured to output an alarm when the intravascular catheter deviates from a preset range.

[0019] In this embodiment, the generation of the trained model M may be performed in advance before the procedure, or it may be performed in real time using data acquired during the procedure. Furthermore, the generation of vascular videos using the trained model M is performed during the patient's procedure.

[0020] The pulse information acquisition device 2 is a device that can acquire the patient's pulse information in real time during treatment. The pulse information acquisition device 2 includes, for example, a biomedical information measurement device such as an electrocardiograph, pulse wave meter, and blood pressure monitor.

[0021] The imaging device 3 is a device for capturing images of a patient's blood vessels to acquire pulsation information. The imaging device 3 is equipped with an X-ray source (X-ray tube) and an X-ray detector (FPD, etc.), and captures an X-ray fluoroscopic image or X-ray fluoroscopic video by detecting X-rays irradiated from the X-ray source with the X-ray detector. In this embodiment, multiple X-ray fluoroscopic images (hereinafter referred to as X-ray fluoroscopic image(s)) are used, but this may also be an X-ray fluoroscopic video composed of multiple frame images. The captured X-ray fluoroscopic image(s) may depict not only the blood vessels but also catheters inserted into the blood vessels (especially radiopaque markers). In the following, X-ray fluoroscopic images (groups) are described as representative examples, but the same applies to X-ray fluoroscopic videos, so the explanation will be omitted.

[0022] The catheter information acquisition device 4 is an optional device for acquiring information about the position and placement status of a catheter inserted into the patient's blood vessels. In one embodiment, the catheter information acquisition device 4 is a device that, like the imaging device 3, uses X-ray fluoroscopy to image a catheter inserted into a blood vessel. Note that if the catheter itself uses radiopaque markers or materials, imaging may be possible without the use of a contrast agent. In one embodiment, the catheter information acquisition device 4 may be a device that acquires positional information of a catheter inserted into a blood vessel using an electromagnetic field (electromagnetic navigation). In yet another embodiment, it may be a device that acquires information from an ultrasound probe, a catheter with a sensor, or the like.

[0023] The display device 20 is a device for visualizing the processing results of the information processing device 100. In this embodiment, it functions as an information display means that displays a video of the generated blood vessel image based on the output from the information processing device 100. In one embodiment, if the catheter position is superimposed on the video, the catheter position information is displayed simultaneously overlaid on the vascular image. The display device 20 can also display catheter position information, alarm status, input / output data, etc., as needed. Examples of display devices 20 include stationary display devices (such as LCD monitors) and portable display devices (such as tablet terminals), but are not limited to these.

[0024] The alarm device 21 is an optional device that notifies the operator of a warning when the information processing device 100 detects an abnormality. In this embodiment, the information processing device 100 notifies the operator of any abnormal catheter position (such as movement outside a predetermined range) in response to the output from the information processing device 100. The notification method can be any of the following, or a combination thereof: light, sound, vibration, warning display, etc. Specific examples include, but are not limited to, warning lights, vibration warning devices, alarms, and warning displays on a screen.

[0025] The information processing system of the present invention may have an input device (not shown) as needed. The input device may be, for example, a keyboard, mouse, trackball, touchpad, touch panel, microphone, gesture input device, etc., and will receive various instructions and operations (for example, instructions to start an examination, setting parameters, switching display content, setting a region of interest (ROI), etc.) from the operator (user such as a doctor or medical technologist).

[0026] [Configuration of the information processing device] Figure 2 is a block diagram showing an example configuration of the information processing device 100. The information processing device 100 is a computer and includes a control unit 10, a main memory unit 11, an auxiliary memory unit 12, and an input / output interface 13. Each unit is connected in a manner that allows communication via a data cable 15, such as a bus cable.

[0027] The control unit 10 is configured using one or more arithmetic processing units such as CPUs, GPUs, GPGPUs, and TPUs. The control unit 10 controls the entire device, interprets and executes programs, and outputs instructions to external devices. For example, the control unit 10 reads and executes computer programs P stored in the auxiliary storage unit 12, thereby performing various information processing and control processing.

[0028] The main memory unit 11 is a volatile memory such as SRAM, DRAM, or flash memory, and temporarily stores some or all of the data and computer programs necessary for the control unit 10 to perform calculations.

[0029] The auxiliary storage unit 12 is a non-volatile storage device such as an HDD (hard disk drive) or an SSD (solid-state drive). The auxiliary storage unit 12 may also be configured as an external storage device connected to the information processing device 100 by wired or wireless connection. The auxiliary storage unit 12 stores the computer program P, trained model M, trained model C, and other data necessary for the control unit 10 to perform processing. The trained model M may be incorporated as a program module into the AI ​​software. The auxiliary storage unit 12 may also store setting data indicating the normal range of the catheter (for example, vascular region data and thresholds including safety margins) used for determining catheter deviation. The auxiliary storage unit 12 has a database 14, which may store data necessary for generating X-ray fluoroscopic images (groups) and videos of the patient's own blood vessels, as well as data related to catheter position information. This data is transmitted to the information processing device 100 according to instructions from the control unit 10. Database 14 may contain a large amount of training data. Database 14 may be stored on an external server connected via wired or wireless connection, or on a removable non-volatile storage medium.

[0030] The input / output interface 13 is an interface for connecting to external devices such as the pulse information acquisition device 2, imaging device 3, catheter information acquisition device 4, alarm device 21, and display device 20, and transmits and receives information and control signals with these external devices. In one embodiment, the input / output interface 13 also functions as an interface for transmitting and receiving data with devices connected to an information communication network. For example, the input / output interface 13 may connect to an external server or database to transmit and receive information.

[0031] Computer program P describes the procedures for performing a series of processes according to this embodiment. For example, it describes the procedure for performing inference processing based on X-ray fluoroscopic images (group), and implements processing procedures such as acquiring necessary data, preprocessing, generating a vascular image video using a trained model, estimating and calculating catheter position information, superimposing (rendering) the estimated catheter position onto the vascular image video, determining deviation by comparing the catheter position with a preset range, outputting an alarm to the alarm device 21 in case of deviation, and outputting the final video (including superimposed display) to the display device 20. The trained model M (video generation model) and the trained model C (catheter position estimation model) are machine learning models trained using predetermined training data, and include trained parameters and network structures, and are used for generating vascular image videos (trained model M) and estimating catheter positions (trained model C). Image processing algorithms may also be used to estimate catheter positions in the generated videos, and these may be stored as part of the computer program P or separately in the auxiliary storage unit 12. In one embodiment, the trained model M and trained model C may be integrated into a computer program that describes a series of processing steps, as modules of a program that implements an inference algorithm.

[0032] The trained model M, trained model C, computer program P, data necessary for video generation, and setting data for catheter deviation detection may be stored on an external server connected via wired / wireless connection, or on a removable storage medium such as an optical disc or flash memory (hereinafter sometimes referred to as "external storage medium"). When the external storage medium is connected to the information processing device 100, the control unit 10 detects it and transfers the program, etc., to the main memory unit 11 to make it available for use by the information processing device 100.

[0033] [Pre-trained model M (video generation model)] A pre-trained model M suitable for this embodiment will be described. Figure 3 is a schematic diagram showing the configuration of a pre-trained model M according to one embodiment of the present invention. In one embodiment, the trained model M is a machine learning model that takes X-ray fluoroscopic images (group) 22 (a to n indicate multiple images) of the patient's own blood vessels and the patient's own pulse information as input and generates a video 23 of the blood vessels synchronized with the input pulse information (hereinafter referred to as the generated video). In one embodiment, it is also possible to implement additional functions such as superimposing catheter position information and catheter position deviation warnings based on the generated video 23.

[0034] In one embodiment, the X-ray fluoroscopic images (group) 22 input to the trained model M may be an X-ray fluoroscopic video consisting of a series of consecutive frames. An X-ray fluoroscopic video is a preferred input format because it captures the displacement of blood vessels over time. The input X-ray fluoroscopic images (group) 22 may be X-ray fluoroscopic images of the patient acquired by the imaging device 3 during the procedure, or they may be X-ray fluoroscopic images of the patient acquired by the imaging device 3 before the procedure and stored in the database 14. X-ray fluoroscopic images (group) 22 taken without contrast agent often have unclear vascular images. Therefore, X-ray fluoroscopic images (group) taken with contrast agent may be stored in the database 14 of the auxiliary storage unit 12, and retrieved from the database 14 as needed and input into the trained model M. Furthermore, the X-ray fluoroscopic images (group) 22 used as input may be those taken using a contrast agent. Images taken under contrast agent use tend to depict blood vessels more clearly, so by inputting these, the shape and movement of blood vessels can be grasped more clearly in the generated video.

[0035] Pulsation information is information about the patient's heartbeat. The trained model M is input with time-series data of pulsation information acquired from the pulsation information acquisition device 2. The input pulsation information should preferably use features that are highly related to the movement of blood vessels. For example, indicators that characterize the pulsation, blood pressure, and pulse wave amplitude information may be input as features. When using features as input data, it is desirable to train the model using those features as training data.

[0036] Examples of indicators that characterize a heartbeat include electrocardiogram waveforms and pulse waves. Examples of electrocardiogram waveforms include the RR interval, heart rate, and RR interval variability. Pulse wave parameters include, for example, peak interval, pulse rate, variation in peak interval, number of peaks within a given time, and changes in peak value. The above indicators, based on electrocardiogram waveforms and pulse waves, may be either one or both. This information is useful for understanding the trend of the pulsation cycle and serves as a criterion for synchronizing the vascular images in each input fluoroscopic image. Furthermore, since all of this information can be calculated with simple signal processing, it can be obtained relatively easily even during surgery when computational resources are limited, making it a suitable feature for real-time processing.

[0037] Examples of blood pressure and pulse wave amplitude information include systolic blood pressure, diastolic blood pressure, mean arterial pressure, and pulse pressure (the difference between systolic and diastolic blood pressure) measured from the patient using an arterial line during treatment. Pulse wave amplitude can also be used as a feature. These features contain information about pulse strength and vascular elasticity, making them extremely useful when analyzing correlations with vascular images. Either blood pressure or pulse wave amplitude information may be used, or both may be used. Furthermore, the indicators characterizing the pulse and the blood pressure and pulse wave amplitude information may be used individually or in combination. Extraction of indicators characterizing the pulse and information on blood pressure and pulse wave amplitude from the pulse information is achieved, for example, by a program module that performs filtering, peak detection, and numerical calculation processing.

[0038] In one embodiment, the generated video 23 produced by the trained model M is a video that predicts the positional changes of blood vessels that displace according to the pulse cycle of the input pulse information. Blood vessels in the body displace slightly with the pulse, and the generated video reproduces the dynamics of the blood vessels synchronized with the input pulse information in a time series. Therefore, by generating such a video, it becomes possible to constantly predict and visualize the minute movements of blood vessels synchronized with the pulse based on pulse information, without using contrast agents. This generated video may be combined (superimposed) with catheter position information that has been separately estimated or acquired in subsequent processing. In addition, if the catheter displayed in the generated video 23 deviates from a preset range (for example, contact with the blood vessel wall), an alarm may be output. Note that the number of images in the input X-ray fluoroscopy images (group) 22 and the number of frames in the output generated video 23 may or may not match. The trained model M described later can predict the dynamics of the vascular image, so it is possible to display a vascular image corresponding to pulsation even if the number of input X-ray fluoroscopy images (group) is small.

[0039] In one embodiment, the trained model M can be implemented as a model that reconstructs fluoroscopic images of moving blood vessels in a time series based on pulsation information. The generated video 23 is produced by deforming a reference frame (for example, the first frame or a frame corresponding to a specific cardiac phase) selected from the input fluoroscopic images (group) 22 in a time series based on pulsation information. In one embodiment, the trained model M may generate images of blood vessels using an image generation model such as a GAN, and a video may be created using multiple output images as frames. In one embodiment, the trained video generation model M may be a motion prediction network (hereinafter referred to as the first trained model M1) that predicts motion information (e.g., displacement vector field, deformation field) within an image from input data (X-ray fluoroscopy images (group) and pulsation information) at each time step based on deep learning. If necessary, it may also include a warping module that deforms (warps) a reference frame in time series based on this predicted motion information. In the following, a configuration comprising the first trained model M1 and a warping module will be described as the trained model M.

[0040] The first trained model M1 predicts the movement information of blood vessels corresponding to each time step of the pulsation information from the input X-ray fluoroscopy images (group) 22 and pulsation information (predicted movement information). In one embodiment, the first trained model M1 comprises an image encoder (e.g., CNN) that extracts spatial features from X-ray fluoroscopy images (group) 22, a time-series encoder (e.g., RNN, LSTM, Transformer) that extracts temporal features from pulsation information, and a decoder (e.g., a CNN with a U-Net structure) that integrates them to output motion information for each time step. Furthermore, the first pre-trained model M1 may be an optical flow estimation network (e.g., FlowNet, RAFT, etc.) that estimates correspondences and motion between images.

[0041] The warping module is a program that rearranges the pixels of a reference frame to the target position at each time step according to predicted motion information, and performs interpolation (e.g., bilinear interpolation) as needed. This generates a deformed vascular image frame corresponding to each time step. By continuously executing this process along the time series of the pulsation information, a video of the vascular image that reproduces how the blood vessels move in sync with the input pulsation information is output. Examples of warping modules include a bilinear interpolation module, a nearest neighbor interpolation module, and a bicubic interpolation module, depending on the interpolation method used.

[0042] In one embodiment, the information processing device 100 takes the patient's pulse information acquired in real time during the procedure and the X-ray fluoroscopy images (group) 22 acquired from the database 14 or the X-ray fluoroscopy images (group) 22 acquired in real time as input, and uses a trained model M to sequentially generate video frames 23a to n of vascular images synchronized with the input pulse information. In one embodiment, the generated video may be segmented to show vascular images or regions other than vascular images. In the catheter position deviation warning, deviation detection can be performed using the segmented region as a set region.

[0043] [Trained Model C (Catheter Position Estimation Model)] In one embodiment, a pre-trained model (hereinafter referred to as "pre-trained model C") can be used separately to estimate the positional information of a catheter inserted into a blood vessel, in order to superimpose the catheter position and, if necessary, to provide a deviation alarm. In one embodiment, the trained model C is a machine learning model that takes various sensor information and image data related to catheter position estimation as input and outputs information indicating the position and shape of the catheter within a blood vessel (hereinafter referred to as catheter position information) in a time series. This model C is used independently of or in conjunction with the trained model M that generates vascular image videos.

[0044] The input data for the trained model C will be one or more of the following information sources capable of capturing the position and shape of the catheter. The input data format will be selected depending on the information source and model architecture used (e.g., image sequences, time-series sensor data). Examples of image sequences include X-ray fluoroscopic images (group) and ultrasound images (group). Preprocessing such as noise reduction and contrast adjustment may be applied as needed. The X-ray fluoroscopic images (or group of images) may be the X-ray fluoroscopic images themselves used to generate vascular video, or X-ray images with enhanced catheter visibility. Ultrasound images such as intravascular ultrasound (IVUS) or echocardiography, which show the positioning of the catheter and its relationship with surrounding tissues, may also be used. The time-series sensor data may be time-series data indicating the three-dimensional position and shape of the catheter, obtained from electromagnetic sensors, fiber optic sensors (FBGs), etc., that are built into or attached to the catheter. Preprocessing such as calibration and noise filtering may be applied. These input pieces of information are fed into the trained model C, either individually or in combination. When using multiple information sources, processing may be performed to synchronize their time and ensure the consistency of their coordinate systems.

[0045] The catheter position information output by the trained model C is provided in a format usable for subsequent superimposition and deviation detection processes. Examples include a time-series list of 3D or 2D coordinate points representing the catheter's centerline, a segmentation mask indicating the region where the catheter is located, and the coordinates of key points such as the catheter tip or specific markers. In one embodiment, catheter position information is temporally synchronized with each frame of the vascular image video generated by the trained model M, and is represented in the same coordinate system as the vascular image video, or in a coordinate system that can be easily converted, for superimposed display purposes.

[0046] In one embodiment, the trained model C is implemented as a deep learning-based model. The specific architecture is selected depending on the type of input data. For image input, examples include segmentation networks based on convolutional neural networks (CNNs) (e.g., U-Net, DeepLab, etc.) to extract the catheter region from X-ray or ultrasound images, and object detection networks that directly detect catheter tips, markers, etc. (e.g., YOLO, SSD, R-CNN systems, etc.). When using video (frame sequence) as input, an architecture combining a CNN and a time-series model (e.g., LSTM) is given as an example. When sensor data is used as input, models suitable for time-series data processing, such as recurrent neural networks (RNNs), LSTMs (Long Short-Term Memory), GRUs (Gated Recurrent Units), or Transformers, are exemplified for processing time-series sensor data and estimating the continuous position and shape of the catheter. In one embodiment, a model may be used to integrate multiple different types of inputs (e.g., images and sensor data), consisting of an encoder corresponding to each input and a decoder or fusion network that fuses their features to produce a final output. The architecture of these pre-trained models C may differ from the first pre-trained model M1 (motion prediction network) that predicts the movement of vascular images, depending on its purpose and primary input data, or it may share some network structures (e.g., image feature extraction unit).

[0047] In inference processing, the information processing device 100 estimates catheter position information using a separately trained or prepared trained model C (or rule-based algorithm), and uses the results in combination with a vascular image video (generated video) generated by the trained model M for the above-mentioned superimposed display and deviation detection / alarm processing.

[0048] [How to generate a pre-trained model M (video generation model)] As an example of how to generate a pre-trained model, we will explain the use of supervised learning. However, other learning methods and combinations thereof, such as unsupervised learning, semi-supervised learning, reinforcement learning, transfer learning, and ensemble learning, can also be used depending on the purpose and available data.

[0049] The training data includes multiple X-ray fluoroscopic images (groups) of blood vessels, pulsation information, and videos of blood vessels synchronized with the pulsation information (hereinafter referred to as "synchronized videos"), and the trained model M is generated using this training data. Because the relationship between pulsation information and vascular dynamics differs from person to person, the X-ray fluoroscopic images (groups), pulsation information, and synchronized videos used as training data will be limited to data from the patient themselves.

[0050] X-ray fluoroscopic images (groups) are preferable because they allow for clearer visualization of blood vessels, and therefore, images taken with contrast agents of a portion of the patient's own blood vessels (for example, a specific area or specific blood vessel) are preferable. The X-ray fluoroscopic images (or group of images) are X-ray fluoroscopic images taken at regular or arbitrary intervals, but they may also be a group of frames in a video. In one embodiment, the multiple X-ray fluoroscopic images may be an X-ray fluoroscopic video of any length.

[0051] Images can be used as training data either as is or after preprocessing. Preprocessing images can improve the accuracy of blood vessel recognition by the learned model. Examples of preprocessing include image normalization, noise reduction, contrast enhancement, smoothing, filtering, edge detection, and region of interest (ROI) setting. One type of preprocessing may be applied, or multiple types may be combined (hereinafter referred to collectively as preprocessing). These processes are carried out by the program according to instructions from the control unit 10. In one embodiment, an ROI may be set on the vascular image. By setting an ROI on the vascular image, features of the vascular image can be efficiently extracted during training.

[0052] Pulse information refers to information about the pulse obtained from the patient mentioned above, and may be raw data, or it may be characterized by indicators that characterize the pulse, blood pressure, and pulse wave amplitude information. The pulse information may be subjected to the filtering process described above, and indicators that characterize the pulse, as well as blood pressure and pulse wave amplitude information, may be extracted and used.

[0053] The synchronized video is ground truth data, created by synchronizing X-ray fluoroscopic images (groups) and pulsation information acquired simultaneously from the patient. Specifically, the time axes of both are synchronized based on a network time synchronization method or time information such as timestamps included in DICOM, and each X-ray fluoroscopic image is associated with the pulsation information at each time step. Preferably, the pulsation information to be synchronized is the input pulsation information, and if the above-mentioned features are used, it is preferable that the synchronized video is also synchronized with the above-mentioned features. Furthermore, time information may be added to the input X-ray fluoroscopy images (or groups of images) and pulsation information. In addition, the training data prepared in this manner may be subjected to quality evaluation by experts, or additional information required for a specific model configuration (e.g., vascular region masks) may be added. In such cases, manual work, rule-based programs, or other pre-trained models may be used as appropriate.

[0054] In one embodiment, the first trained model M1 (motion prediction network), which is the main learning target constituting the trained model M, is generated by performing the following learning. Furthermore, a learning model is referred to as a "trained model" when training is complete, and as a "learning model" before training or during the training process. The first trained model M1 (motion prediction network), which is not yet trained, receives as input: X-ray fluoroscopic images (or a group of images) of the patient themselves (which may have been pre-processed) and corresponding pulsation information (raw data or featured time-series data) obtained from the training dataset. Then, based on the input X-ray fluoroscopic images (group) and pulsation information, the first learning model M1 is trained to output predicted blood vessel movement information (e.g., displacement vector field or deformation field) corresponding to each time step of the pulsation information. This predicted movement information is learned with the aim that the video generated after the warping process described later will reproduce the dynamics of the blood vessels in the synchronized video, which is the ground truth data.

[0055] During training, the warping module applies the predicted motion information output by the first training model M1 to the corresponding reference frame in the training dataset. This generates a predicted synchronized video. Next, the error between the generated predicted synchronized video and the correct labeled synchronized video in the training data is evaluated using an appropriate loss function (e.g., pixel-level similarity between corresponding video frames, structural similarity, etc.), and learning proceeds by optimizing the internal parameters of the first learning model M1 to minimize the error. For this optimization, for example, backpropagation and related optimization algorithms (e.g., Adam, SGD) are used. This learning is performed in a substantially end-to-end manner, as the parameters of the motion prediction network are updated through warping based on the error of the final video output. This learning process optimizes the first trained model M1 (motion prediction network) to accurately predict motion information showing the dynamics of blood vessels synchronized with the pulse, based on the input X-ray fluoroscopy images (group) and patient-specific pulse information. This results in a trained model M (movement generation model) consisting of the optimized first trained model M1 and a warping module. By using the obtained trained model M, it becomes possible to predict and visualize subtle positional changes and movements of blood vessels based on pulsation information, and generate videos.

[0056] [Method for generating a pre-trained model C (catheter position estimation model)] As an example of how to generate a pre-trained model C, we will explain the case using supervised learning, but other learning methods can also be used.

[0057] The training data uses a training dataset that includes pairs of input data necessary for catheter position estimation (e.g., X-ray fluoroscopy images showing the catheter, ultrasound images, time-series data from catheter-mounted sensors, etc.; hereinafter referred to as catheter input data) and corresponding ground truth catheter position and shape information (e.g., annotated coordinate point clouds or segmentation masks, etc.; hereinafter referred to as ground truth catheter labels). Training is performed using this training dataset to generate a trained model C. Note that the position of contrast markers may also be used as a feature in the catheter input data. Training data is constructed with consideration for the versatility of the model and its suitability to specific situations, and may include procedural data from specific patients, data from multiple patients, or experimental data using physical phantoms or data generated from computer simulations.

[0058] If the catheter input data consists of fluoroscopic or ultrasound images, preprocessing (e.g., noise reduction, contrast adjustment, region of interest extraction, image normalization, data augmentation, etc.) is performed as needed before use for training. This preprocessing is done to facilitate the extraction of catheter features from the images by the learning model C during the learning process. If the catheter input data is sensor data (e.g., position coordinates of an electromagnetic sensor, shape data of an FBG sensor), preprocessing such as sensor-specific calibration correction, noise filtering (e.g., Kalman filter), coordinate system unification, and time-axis interpolation / resampling may be performed. These preprocessing steps are selected and applied to improve estimation accuracy.

[0059] The correct catheter label is the true catheter position and shape information that serves as the learning target, and it is preferable that the correct catheter label is temporally synchronized with the corresponding catheter input data.

[0060] The learning model C receives pre-processed catheter input data and is trained to output predicted catheter position and shape information (e.g., predicted coordinate point cloud, predicted segmentation mask, etc.) at each time step (or each input data point) based on that data.

[0061] During training, catheter input data is first acquired from the training dataset in units such as minibatches, and then input into the training model C (initial state or during training) to obtain predicted catheter position and shape information. Next, the error between this prediction result and the corresponding ground truth catheter label included in the training data is evaluated using an appropriate loss function for the task. Then, based on the calculated error, the intrinsic parameters of the learning model C are updated using backpropagation and optimization algorithms. This entire process is repeated multiple times across the entire training dataset, and the model continues to learn until its performance reaches the target level.

[0062] The information processing device 100 may have pre-trained models M, including a first pre-trained model M1 optimized for generating vascular image videos, and a pre-trained model C for catheter position estimation, prepared internally (for example, in the auxiliary storage unit 12) in an executable format after undergoing the learning process described above. Alternatively, the first pre-trained model M1, which has been trained in advance by transfer learning or the like, may be optimized (fine-tuned) using the patient's own data. Furthermore, during actual use, the information processing device 100 may generate a pulse-synchronized vascular image video using a trained model M in response to newly input patient X-ray fluoroscopic images and pulse information. In one embodiment, the computer program P performs, in parallel with or following the video generation, the estimation of the catheter position using a trained model C, the overlaying of the estimation result onto the video, and the detection and alarm processing for deviations from a preset range.

[0063] In one embodiment, the following are examples of deviation detection and alarm processing. In one embodiment, based on the catheter's position information, an alarm is output if the position of the catheter tip (or contrast marker) moves outside a predetermined position range. Deviation detection and alarm processing may be implemented in a computer program P using a rule-based algorithm. In one embodiment, deviation detection is performed by setting a position range and determining the position.

[0064] In one embodiment, the position range may be set using the predicted motion information (or warped blood vessel, hereinafter the same) output by the trained model M (or the first trained model M1). Since the predicted motion information is information that predicts the dynamics of the blood vessel (e.g., displacement vector field, deformation field), it is set as the range in which the catheter can move. Then, deviation detection is performed based on the predicted motion information and the catheter position information output by the trained model C. For example, based on the positional relationship between the two, it is determined whether the catheter tip (or contrast marker) position is within the blood vessel range based on the predicted motion information, and if it is determined that the catheter tip position is outside the range, an alarm is issued. Furthermore, a safe range for catheter movement may be set, for example, based on the intravascular space. This range may be set manually by a physician or based on past case data.

[0065] [Process for generating the pre-trained model M (video generation model)] The process of generating the trained model M is performed by the control unit of the learning device according to a computer program stored in a memory device such as an auxiliary memory unit. The learning process may be performed on the information processing device 100 or on a separate learning computer (learning device). An example of performing the learning process on the information processing device 100 is described below.

[0066] Figure 4 is a flowchart showing an example of the process for generating a trained model M (first trained model M1 + warping module). The following describes an example in which the control unit 10 of the information processing device 100 performs this learning process. First, the control unit 10 obtains the training data necessary for learning from the database 14 (step S1). Training data may be obtained from an external database via the input / output interface 13, or from a database 14 stored in the auxiliary storage unit 12.

[0067] Next, the control unit 10 performs the above preprocessing on the acquired training data as needed (step S2). Furthermore, the control unit 10 selects a reference frame from the input X-ray fluoroscopy images (group).

[0068] Next, the control unit 10 performs a learning process to optimize the parameters of the first trained model M1 using the training data (step S3). In this step, the following processes will be executed in order. (1) The control unit 10 inputs the X-ray fluoroscopic images and pulse information of the training data into the first trained model M1 and obtains "predicted motion information". (2) The control unit 10 inputs this "predicted motion information" and the "reference frame" selected in step S2 to the warping module and generates a "predicted synchronized video". (3) The control unit 10 compares the generated "predicted synchronized video" with the correct data "synchronized video" included in the training data, and calculates the error between the two using a loss function (e.g., pixel-level error between frames, structural similarity error, etc.). (4) The control unit 10 updates (optimizes) the parameters of the first learning model M1 based on the calculated error, for example, using backpropagation and an optimization algorithm (e.g., Adam, SGD). In this process, the error is backpropagated through the warping module.

[0069] The control unit 10 determines whether or not the learning termination conditions have been met (step S4). Termination conditions may include reaching a predetermined number of epochs (the number of times the entire training dataset has been trained), the loss function becoming sufficiently small (or converging), or the model's performance on separately prepared validation data reaching a target value (or no further improvement being observed). If it is determined that the termination condition has not been met (step S4: NO), the control unit 10 returns to the process of step S1, executes each step using, for example, the next batch of data, and continues learning using the model with updated parameters. If the control unit 10 determines that the termination condition has been met (step S4: YES), it terminates the series of learning processes.

[0070] The above process yields a trained model M consisting of an optimized first trained model M1 (motion prediction network) and a warping module (end). This trained model M is trained to take X-ray fluoroscopic images and pulsation information as input and output a video of vascular images synchronized with the pulsation. In one embodiment, this trained model M is used to generate a basic vascular image video that serves as the basis for realizing functions such as superimposing catheter position and deviation warnings. The trained model M can be further improved and optimized in performance by performing additional training and fine-tuning using new data after deployment, as needed.

[0071] [Generation process of trained model C (catheter position estimation model)] The process for generating the trained model C will be explained using an example where the training process is performed in the information processing device 100, similar to the process for generating the trained model M.

[0072] Figure 5 is a flowchart showing an example of the process for generating the trained model C (a model for catheter position estimation). Note that the explanation of the process for generating the trained model M is omitted here. The following describes an example in which the control unit 10 of the information processing device 100 performs this learning process. First, the control unit 10 obtains the training data necessary for training the trained model C from the database 14 (or an external database, external storage medium, etc.) (step S11).

[0073] Next, the control unit 10 performs preprocessing on the acquired training data catheter input data as necessary (step S12). Note that if you use a pre-processed dataset as training data, this step may be omitted.

[0074] Next, the control unit 10 performs a learning process to optimize the parameters of the learning model C during the learning process using the pre-processed training data (step 13). In this step, the following processes will be executed in order. (1) The control unit 10 inputs the catheter input data of the training data into the learning model C and obtains predicted catheter position and shape information (hereinafter referred to as the prediction result). (2) The control unit 10 compares this prediction result with the corresponding ground truth catheter label included in the training data and calculates the error between the two using a loss function appropriate for the task (e.g., MSE for position coordinates, Dice loss for segmentation, etc.). (3) Based on the calculated error, the control unit 10 updates (optimizes) the parameters of the trained model C using backpropagation and an optimization algorithm (e.g., Adam, SGD, etc.).

[0075] The control unit 10 determines whether or not the learning termination conditions have been met (step S14). The termination conditions are the same as above, so the explanation will be omitted. If it is determined that the termination condition has not been met (step S14: NO), the control unit 10 executes each step using the next training data and continues learning using the model with updated parameters. If the control unit 10 determines that the termination condition has been met (step S14: YES), it terminates the series of learning processes.

[0076] The above process yields an optimized, trained model C (end). This trained model C is trained to estimate the position and shape of the catheter with high accuracy based on catheter input data, and is stored in an executable format in the auxiliary storage unit 12, etc. The stored trained model C is called by the computer program P during inference processing and used to provide catheter position information necessary for superimposing the catheter position and deviation detection / alarm processing.

[0077] [Inference Processing (Video Generation Processing)] Figure 6 is a flowchart showing an example of a processing procedure that generates a pulse-synchronized vascular image video using a trained model M (video generation model), and further performs superimposed display of catheter information and alarm output. In one embodiment, the input data includes a group of X-ray fluoroscopic images of the patient's blood vessels and time-series data of the patient's pulse acquired during the procedure. The following processes are executed by the control unit 10 of the information processing device 100 shown in Figures 1 and 2, according to the computer program P stored in the auxiliary storage unit 12. The computer program incorporates the trained model M (including the optimized first trained model M1 and a warping module) obtained through the previous learning process.

[0078] The control unit 10 acquires X-ray fluoroscopic images (groups) including the vascular image of the target patient from, for example, the imaging device 3 or the database 14 (step S21). The X-ray fluoroscopic images (group) may be acquired in real time, for example, during vascular treatment or diagnostic procedures, or they may be pre-recorded data. Simultaneously, the control unit 10 acquires the patient's pulsation information corresponding to the X-ray fluoroscopic images in real time, for example, from the pulsation information acquisition device 2.

[0079] The control unit 10 performs preprocessing on the acquired X-ray fluoroscopy image and pulsation information as needed (step S22). Image preprocessing of X-ray fluoroscopic images includes, for example, noise reduction, normalization, and contrast enhancement. This preprocessing uses the same methods as those used during model training, or methods that have been adapted for inference. Furthermore, the control unit 10 selects a reference frame from the input X-ray fluoroscopy images (group) to be used as the basis for warping (for example, the first frame of the input sequence or a frame corresponding to a specific cardiac phase).

[0080] The control unit 10 inputs the pre-processed X-ray fluoroscopic images (group) and pulsation information to the first trained model M1 (motion prediction network) (step S23). The first pre-trained model M1 estimates and outputs "predicted motion information" (e.g., displacement vector field or deformation field) that represents the movement of blood vessels corresponding to each time step of the input pulsation information, based on the input data.

[0081] The control unit 10 inputs the predicted motion information output from the first trained model M1 in step S23 and the "reference frame" selected in step S22 to the warping module (step S24). The warping module generates a deformed vascular image frame corresponding to each time step by deforming a reference frame chronologically according to predicted motion information. This generates (reconstructs) a video of the vascular image synchronized with the input pulsation information.

[0082] The control unit 10 sends the final video frame generated in step S24 to an output destination such as the display device 20 for display (step S25).

[0083] The control unit 10 determines whether or not to continue the video generation process (step S26). For example, if real-time input data is supplied and continuous monitoring and display are required (step S26: YES), the control unit 10 returns to step S21 and performs a series of processes using the newly acquired data (repeat steps S21 to S25). On the other hand, if processing for a specific dataset is completed, if there is a user instruction, or if a series of procedures (techniques) is finished, and it is determined that continuation is unnecessary (step S26: NO), the control unit 10 terminates the video generation process (end).

[0084] Through the processing described above, the dynamics of the blood vessels, synchronized with the pulsation, are estimated by the trained model M based on the input X-ray fluoroscopy images and pulsation information, and visualized as a video.

[0085] [Overlay display of catheters and warning display] In one embodiment, the positional information of an intravascular catheter (hereinafter referred to as "catheter") can also be superimposed and displayed on the generated video of the vascular image. When superimposing the catheter and displaying a warning, the following processing is performed immediately following the video generation process. Specifically, the catheter position information obtained in the following steps is drawn onto the video generation process described above. The control unit 10 may temporarily save the generated video, perform the catheter position estimation process described below (steps S31 to S33), and then superimpose the catheter onto the generated video (step S34), or it may perform the video generation process and the catheter position estimation process in parallel and then superimpose the catheter onto the video. Figure 7 is a flowchart showing a catheter superposition method according to one embodiment. The following processes are executed by the control unit 10 of the information processing device 100 shown in Figures 1 and 2, according to the computer program P stored in the auxiliary storage unit 12. The computer program incorporates the trained model C obtained through the previous learning process, logic for performing catheter position estimation and superimposed drawing, and logic for performing deviation detection, alarm output, etc., as needed.

[0086] The control unit 10 acquires information regarding the position and shape of the catheter from the catheter information acquisition device 4 (e.g., another X-ray device, ultrasound device, electromagnetic sensor system, etc.) (step S31). This data is acquired in real time, for example, during vascular treatment or diagnostic procedures.

[0087] The control unit 10 performs preprocessing on the acquired information regarding the position and shape of the catheter, if necessary (step S32). Preprocessing includes the above-mentioned preprocessing steps such as filtering and feature extraction. If catheter-related information is in the form of image data, similar image preprocessing may be applied. Preprocessing of sensor data, etc., includes the above-mentioned preprocessing such as calibration and noise reduction. This preprocessing uses the same methods as those used during model training, or methods that have been adapted for inference.

[0088] Based on the catheter-related information preprocessed in step S32, the control unit 10 uses a trained model C included in the computer program P to calculate and estimate the position and shape information (e.g., coordinate point cloud) of the catheter corresponding to the current time step (step S33). The control unit 10 inputs the catheter input data (e.g., X-ray images, sensor data, etc.) acquired in step S31 and preprocessed in step S32 into the trained model C. The trained model C calculates and outputs predicted catheter position and shape information (e.g., estimated centerline coordinates, catheter area mask, etc.) corresponding to the current time step based on the input data. The control unit 10 acquires the predicted catheter position and shape information output from this model C and temporarily stores or directly uses it for the subsequent steps S34 (overlay drawing process) and S35 (deviation determination process).

[0089] The control unit 10 superimposes the catheter position and shape information for the corresponding time obtained in step S33 onto the vascular image frame generated by the trained model M using the rendering unit included in the computer program P (step S34). The control unit 10 sends the final video frame in which the catheter position is superimposed in step S34 to an output destination such as a display device 20 for display.

[0090] The control unit 10 determines whether the catheter position (predicted catheter position) obtained in step S33 deviates from a preset normal range (e.g., inside the estimated vascular region based on predicted vascular movement information, or within a safety margin therefor) (step S35). If it is determined that a deviation has occurred (step S35: YES), the control unit 10 issues an alarm output instruction to the alarm device 21 via the input / output I / F 13. If it is determined that there is no deviation (step S35: NO), no alarm processing is performed, and the process proceeds to, for example, the determination of whether to continue the video generation process (step S26 in Figure 6, etc.), or the control unit 10 returns to step S31 and repeats the series of processes using the newly acquired data.

[0091] In one embodiment, when an alarm is issued in step S35, this fact may also be displayed on the display device 20 (e.g., flashing warning color, message display). This allows users such as doctors and technicians to visually confirm the dynamic relationship between the pulsation of the blood vessels and the position of the catheter (when superimposed) as a video.

[0092] If an alarm output instruction is received, the catheter superimposition drawing process (and video generation process) may be terminated. Also, if instructed by the operator, or if it is determined that continuation is unnecessary, such as when a series of procedures (procedures) is completed, the control unit 10 terminates the video generation process (end).

[0093] In one embodiment, catheter position information is superimposed in real time along with the dynamics of the blood vessel, and an alarm is output if the position deviates from a preset safety range. This makes it possible, for example, to observe the dynamics of blood vessels and the positional relationship of the catheter in detail without using contrast agents, to improve the safety of the procedure while reducing the risk of vascular perforation, and to obtain information that can lead to quantitative evaluation of hemodynamics and support for diagnosis. [Explanation of Symbols]

[0094] 1. Information Processing System 2. Pulse information acquisition device 3. Angiography equipment (imaging device) 4. Catheter information acquisition device 10 Control Unit 11 Main memory 12 Auxiliary storage 13 Input / Output Interfaces 14 Databases 15 Data Cables 20 Display device 21 Alarm device 22 X-ray fluoroscopic images (group) 22a~n X-ray fluoroscopic images 23 Generated Videos 23a~n Video frames 100 Information Processing Devices 120 Blood vessels

Claims

1. Multiple X-ray fluoroscopic images of the patient's blood vessels and the patient's pulsation information are acquired. When X-ray fluoroscopic images and pulse information are input, a trained model generates a video of vascular images synchronized with the input pulse information. A program that instructs a computer to perform a process.

2. The program according to claim 1, wherein the plurality of X-ray fluoroscopic images are a group of frame images of an X-ray fluoroscopic video.

3. The program according to claim 1 or 2, wherein the plurality of X-ray fluoroscopic images are taken using a contrast agent.

4. The program according to claim 1, wherein the generated video of the blood vessel image displays positional information of a catheter inserted into the blood vessel.

5. The program outputs an alarm if the catheter displayed in the generated video of the vascular image deviates from a preset range. The program according to claim 4, which causes a computer to perform a process.

6. An acquisition unit that acquires X-ray fluoroscopic images of the patient's blood vessels and the patient's pulsation information, Based on the acquired X-ray fluoroscopy images and pulsation information, an output unit generates and outputs a video of the vascular image synchronized with the acquired pulsation information. An information processing device having, The aforementioned information processing device generates a video of a vascular image synchronized with the input pulsation information, based on the input X-ray fluoroscopy image and pulsation information, which is generated by a computer.

7. Training data is acquired that includes multiple X-ray fluoroscopic images of the patient's blood vessels, the patient's pulsation information, and X-ray fluoroscopic videos corresponding to the pulsation information. Based on the aforementioned training data, when the X-ray fluoroscopic image and the corresponding pulsation information are input, an X-ray fluoroscopic video of the vascular image synchronized with the pulsation information is generated. A model generation method in which a computer performs the processing.

Images