Information processing method, program, and information processing device

Abstract

Provided are an information processing method and the like that appropriately associate an image captured by an image acquisition catheter with an image captured by angiography even for a bent blood vessel.　The information processing method causes a computer to execute a process for: acquiring cross-sectional image information related to a plurality of cross-sectional images (71) of a blood vessel; acquiring, in perspective images (61), sensor position information related to the positions of a sensor used for capturing the respective cross-sectional images (71); acquiring designation of the visual field direction; and outputting either the shape of the blood vessel seen through in the acquired visual field direction or the angle relating to the visual field direction on the basis of the cross-sectional image information and the sensor position information.

Description

Information Processing Method, Program, and Information Processing Apparatus

[0001] The present invention relates to an information processing method, a program, and an information processing apparatus.

[0002] A program for superimposing a three-dimensional image of blood vessels on an image taken by angiography has been proposed (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2022-55170

[0004] The three-dimensional image of blood vessels used in Patent Document 1 is created on the premise that the catheter for image acquisition is in a straight state. Therefore, when used for blood vessels that bend with a relatively small radius of curvature, such as coronary arteries, the range in which it can be appropriately superimposed on the image taken by angiography is short.

[0005] On one aspect, an object is to provide an information processing method or the like that can appropriately associate an image taken by an image acquisition catheter with an image taken by angiography even for a bent blood vessel.

[0006] The present invention is an information processing method in which a computer executes a process of: (1) acquiring tomographic image information regarding a plurality of tomographic images of blood vessels; acquiring sensor position information regarding the position of a sensor used for taking each of the tomographic images in a fluoroscopic image; acquiring a designation of a viewing direction; and outputting the shape of the blood vessel viewed from the acquired viewing direction or an angle regarding the viewing direction based on the tomographic image information and the sensor position information.

[0007] Here, in an embodiment of the present invention, (2) in the information processing method described in (1) above, it is preferable that the designation of the viewing direction is acquired based on an input from a user.

[0008] (3) In the information processing method described in (1) above, it is preferable that the designation of the viewing direction is automatically acquired.

[0009] (4) In the information processing method according to any one of (1) to (3) above, it is preferable that the plurality of tomographic images are taken by three-dimensional intravascular scanning.

[0010] (5) The information processing method described in any one of (1) to (4) above preferably involves repeatedly calculating the inclination of the three-dimensional scanning axis between the two transverse images with respect to the fluoroscopic image, based on the distance between the two transverse images in the intravascular three-dimensional scanning and the distance the sensor has moved between the acquisition of the two transverse images in the fluoroscopic image, and then calculating the curvature of the blood vessel in which the intravascular three-dimensional scanning was performed based on the series of inclinations calculated.

[0011] (6) The information processing method described in (5) above preferably includes a three-dimensional model of the blood vessel created based on the plurality of transverse layer images, and the shape of the blood vessel is the shape of a deformed three-dimensional model obtained by deforming the three-dimensional model based on the calculated bending state.

[0012] (7) The information processing method described in (6) above preferably obtains the position of the shooting center, which is the center of the shooting space in which the perspective image can be taken, and outputs the shape of the deformed three-dimensional model rotated around the position of the shooting center, or the rotation angle of the deformed three-dimensional model.

[0013] (8) The information processing method described in (6) or (7) above preferably involves determining the position of the deformed three-dimensional model in the organ model by pattern matching the deformed three-dimensional model with blood vessels included in the organ model relating to the shape of the organ and the blood vessels running around the organ, obtaining the position of the organ center which is the center of the organ model, and outputting the shape of the deformed three-dimensional model rotated around the organ center, or the rotation angle of the deformed three-dimensional model.

[0014] (9) Preferably, the program causes a computer to perform the following processes: acquire transverse image information relating to multiple transverse images of a blood vessel; acquire sensor position information relating to the position of the sensor used to capture each of the transverse images in the fluoroscopic image; acquire a specification of the field of view direction; and based on the transverse image information and the sensor position information, output the shape of the blood vessel viewed fluoroscopy from the acquired field of view direction, or the angle relating to the field of view direction.

[0015] (10) Preferably, an information processing device having a control unit, wherein the control unit acquires transverse image information relating to a plurality of transverse images of a blood vessel, acquires sensor position information relating to the position of each sensor used to capture the transverse image in the fluoroscopic image, acquires a specification of the field of view direction, and outputs the shape of the blood vessel viewed fluoroscopy from the acquired field of view direction, or an angle relating to the field of view direction, based on the transverse image information and the sensor position information.

[0016] In one respect, this provides an information processing method that can appropriately match images taken with an imaging catheter with images taken by angiography, even for curved blood vessels.

[0017] This is an explanatory diagram illustrating the configuration of the information processing system. This is an explanatory diagram illustrating the outline of the procedure for creating estimated contrast-enhanced images. This is an explanatory diagram illustrating intravascular three-dimensional scanning. This is an explanatory diagram illustrating the procedure for calculating the scanning position of the transverse layer image from the angiographic image. This is an explanatory diagram illustrating the scanning position of the transverse layer image in the angiographic image. This is an explanatory diagram illustrating the method for calculating the inclination of the image acquisition catheter relative to the angiographic image. This is a flowchart illustrating the processing flow of the program. This is an example screen. This is a flowchart illustrating the processing flow of the program in Embodiment 2. This is a flowchart illustrating the processing flow of the program in Embodiment 3. This is an example of an organ model and a deformed three-dimensional model matched to the organ model. This is an explanatory diagram illustrating the configuration of the information processing system in Embodiment 4. This is a schematic diagram illustrating the three-dimensional relationship between blood vessels and angiographic images in Embodiment 5. This is an explanatory diagram illustrating the positive and negative signs of the blood vessel angle φ. This is a flowchart illustrating the processing flow of the subroutine for calculating the blood vessel angle at each shooting position in Embodiment 5. This is an explanatory diagram illustrating the method for calculating the reference coordinate S0 in Embodiment 6. This is an explanatory diagram illustrating the relationship between the shooting direction of the angiography device and the blood vessel being observed. This is an explanatory diagram illustrating the reference line in Embodiment 7. This flowchart explains the processing flow of the subroutine for calculating the angle of blood vessels at each imaging position in Embodiment 7. This flowchart explains the processing flow of the subroutine for blood vessel detection. This flowchart explains the processing flow of the subroutine for positive / negative determination.

[0018] [Embodiment 1] Figure 1 is an explanatory diagram illustrating the configuration of the information processing system 10. In this embodiment, the information processing system 10 used to support endovascular treatments such as PCI (Percutaneous Coronary Intervention) will be described as an example. In the following description, tasks performed by paramedical staff such as nurses or clinical engineers based on the instructions of a physician may also be described as tasks performed by a physician. Unless otherwise specified, "blood vessels" refers to coronary arteries.

[0019] The information processing system 10 includes a catheter system 30, an angiography device 41, and an information processing device 20. The catheter system 30 includes a catheter control device 31, an image acquisition catheter 32, and an MDU (Motor Driving Unit) 33. The image acquisition catheter 32 is connected to the catheter control device 31 via the MDU 33.

[0020] The image acquisition catheter 32 is a so-called three-dimensional scanning catheter that acquires a transverse image 71 (see Figure 2) while moving the scanning plane axially by radial scanning. Details of the configuration of the image acquisition catheter 32 will be described later. In the following description, the side of the image acquisition catheter 32 furthest from the MDU 33 will be referred to as the tip side. The catheter control device 31 outputs catheter images such as a transverse image 71 of the blood vessel and a three-dimensional model 75 (see Figure 2).

[0021] The angiography apparatus 41 comprises a C-arm 421 and an X-ray tube 422 and a flat-panel detector 423 fixed to both ends thereof. X-rays emitted from the X-ray tube 422 pass through the patient's body and are detected by the flat-panel detector 423. Various image processing is performed in real time on the detected X-rays inside the angiography apparatus 41, and real-time images, which are fluoroscopic images, are output sequentially. In principle, real-time images are taken continuously during endovascular treatment.

[0022] A physician or a technologist acting under a physician's instructions may change the orientation of the C-arm 421 as needed. In this embodiment, the C-arm 421 is rotatable around two axes: an axis passing through the center point of the arc constituting the C-arm 421 and perpendicular to the arc, and an axis passing through the center point of the arc and bisecting the arc. Therefore, the angiography apparatus 41 can image the imaging space 44, which is a roughly spherical space sandwiched between the X-ray tube 422 and the flat-panel detector 423, from any orientation. In endovascular treatment of heart disease, it is customary to align the center of the imaging space 44 with the center of the patient's heart. In the following description, the center of the imaging space 44 may be referred to as the imaging center.

[0023] The physician administers a contrast agent into the patient's blood vessels from an angiography agent delivery device (not shown in the diagram) as needed. As the contrast agent flows into the blood vessels within the field of view of the angiography device 41, a real-time image clearly depicting the course of the blood vessels is captured. The real-time image during contrast administration is referred to as the angiography image 61 (see Figure 2). The angiography image 61 is an example of a fluoroscopic image. As mentioned above, when performing endovascular treatment for heart disease, by aligning the imaging center with the patient's heart, it is possible to prevent the heart from moving out of the angiography image 61 even if the orientation of the C-arm 421 is changed.

[0024] The information processing device 20 comprises a control unit 21, a main memory 22, an auxiliary memory 23, a communication unit 24, a display unit 25, an input unit 26, and a bus. The control unit 21 is an arithmetic control device that executes the program of this embodiment. One or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), or multi-core CPUs are used in the control unit 21. The control unit 21 is connected to each hardware component of the information processing device 20 via the bus.

[0025] The main memory 22 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory. The main memory 22 temporarily stores information necessary during processing performed by the control unit 21 and the program currently being executed by the control unit 21.

[0026] The auxiliary storage device 23 is a storage device such as SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 23 stores the program to be executed by the control unit 21, and various data necessary for the execution of the program. The communication unit 24 is an interface for communication between the information processing device 20 and the network.

[0027] The display unit 25 is, for example, a liquid crystal display device or an organic EL (Electro-Luminescence) display device. The input unit 26 is, for example, an input device such as a keyboard, mouse, trackball, or microphone. The display unit 25 and the input unit 26 may be stacked together to form a touch panel.

[0028] The display unit 25 may be a connection interface that connects the information processing device 20 to an external display device. A communication unit 24 that connects data to an external display device via a network may realize the functions of the display unit 25 and the input unit 26.

[0029] The information processing device 20 in this embodiment is an information device such as a general-purpose personal computer, tablet, smartphone, or server computer. The information processing device 20 may also be a mainframe computer, a virtual machine operating on a mainframe computer, a cloud computing system, a quantum computer, or multiple personal computers performing distributed processing. The information processing device 20 may be configured integrally with, for example, a catheter control device 31, an angiography device 41, or a hospital information system (HIS) not shown in the figures.

[0030] In the following explanation, we will mainly use the case where the control unit 21 performs software-based processing as an example. The processes described using flowcharts may each be implemented by dedicated hardware.

[0031] Figure 2 is an explanatory diagram illustrating the procedure for creating an estimated projection-enhanced image. The physician inserts a guiding catheter 39 (see Figure 3) up to just before the treatment site. The guiding catheter 39 is a relatively thick tube and serves as a passage for the physician to safely insert the image acquisition catheter 32 and various treatment catheters, etc., up to the vicinity of the treatment site.

[0032] The physician inserts the image acquisition catheter 32 via the guiding catheter 39 to the vicinity of the treatment site. The configuration of the image acquisition catheter 32 will be described later. The physician starts intravascular three-dimensional scanning. The angiography device 41 continuously acquires multiple angiographic images 61. The catheter system 30 continuously acquires multiple transverse images 71. The control unit 21 acquires the angiographic images 61 and the transverse images 71, respectively.

[0033] The timing of the acquisition of the angiography image 61 and the transverse image 71 are not synchronized. The frame rates of the two are also different. However, the internal clock of the angiography device 41 and the internal clock of the catheter system 30 are synchronized, and it is possible to know the chronological order of acquisition based on the timestamps recorded in the angiography image 61 and the transverse image 71. Furthermore, by pre-setting the delay caused by the transmission and reception of image signals from the angiography device 41 on a frame-by-frame basis in the catheter system 30, it is also possible to know the chronological order of acquisition of the angiography image 61 and the transverse image 71.

[0034] After a single intravascular three-dimensional scan is completed, the angiography device 41 creates a three-dimensional model 75 based on a series of transverse images 71. The creation of a three-dimensional model 75 based on a series of transverse images 71 taken using an image acquisition catheter 32 is a conventional procedure, so a detailed explanation will be omitted. The control unit 21 acquires the three-dimensional model 75 from the angiography device 41. The control unit 21 may also create the three-dimensional model 75 based on the transverse images 71.

[0035] Furthermore, it is desirable that the three-dimensional model 75 be created using, for example, the portion corresponding to the lumen region. This is because it makes it easier to establish a correspondence with the shape of the blood vessel depicted in the angiographic image 61.

[0036] It should be noted that conventional three-dimensional models 75 are based on the assumption that the catheter 32 used for image acquisition during intravascular three-dimensional scanning is straight. Therefore, the central axis 751 of the three-dimensional model 75 is straight. The central axis 751 is an example of a three-dimensional scanning axis.

[0037] Generally, the orientation of the C-arm 421 is not changed during intravascular three-dimensional scanning. Therefore, in multiple angiographic images 61 taken during intravascular three-dimensional scanning, the same blood vessel is captured in substantially the same configuration.

[0038] The sensor 322 (see Figure 3) contains a material with lower X-ray transmittance than the contrast agent, such as metal or ceramics. Therefore, each angiography image 61 displays a sensor image 63 in which the sensor 322 used to capture the transverse image 71 is depicted. The control unit 21 detects the sensor image 63 from each angiography image 61. The sensor image 63 is an example of sensor position information relating to the position of the sensor 322 used to capture each transverse image 71.

[0039] Specifically, the control unit 21 detects the sensor image 63, for example, by pattern matching. The control unit 21 may also detect the sensor image 63 based on the difference between two consecutive angiography images 61.

[0040] The control unit 21 determines the location in the angiography image 61 where each transverse image 71 was taken, based on a comparison of the timestamps of the transverse images 71 and the timestamps of the angiography images 61. Specifically, the control unit 21 obtains the timestamps of, for example, two angiography images 61. The control unit 21 obtains the timestamp of the transverse images 71 taken between the acquisition times of the two angiography images 61.

[0041] The control unit 21 can calculate the location in the angiography image 61 where the transverse layer image 71, which was taken between the acquisition times of the two angiography images 61, was taken by interpolating the positions of the sensor images 63 depicted in each of the two angiography images 61. By repeating the above process, the control unit 21 can establish a correspondence between the angiography image 61, the transverse layer image 71, and the three-dimensional model 75.

[0042] More precisely, even when the imaging timing of the angiography image 61 is synchronized with the electrocardiogram, there will still be a deviation in the arrangement and shape of the blood vessels imaged in the plurality of angiography images 61 due to the influence of pulsation, respiration, etc. Therefore, it is desirable to determine at which position of each angiography image 61 each tomographic image 71 was taken, taking into account the influence of deformation and movement of the blood vessels in each angiography image 61.

[0043] For example, the control unit 21 may detect feature points such as blood vessel bifurcations from each angiography image 61, and determine at which position of each angiography image 61 each tomographic image 71 was taken based on the detected feature points. The control unit 21 may also determine at which position in each angiography image 61 each tomographic image 71 was taken using any other arbitrary method.

[0044] Note that after the control unit 21 associates at which position in the angiography image 61 each tomographic image 71 was taken based on the time stamp, the control unit 21 may correct the associated result based on the positions of feature points such as the positions of blood vessel bifurcations depicted in the angiography image 61 and the tomographic image 71 respectively. As described above, an information processing method that can perform more accurate association can be realized.

[0045] The control unit 21 calculates the inclination of the central axis 751 of the three-dimensional model 75 with respect to the angiography image 61 based on the association between the angiography image 61, the tomographic image 71, and the three-dimensional model 75. The method for calculating the inclination will be described later. The control unit 21 generates a deformed three-dimensional model 76 by bending the three-dimensional model 75.

[0046] The user, for example, operates the input unit 26 to change the orientation of the deformed three-dimensional model 76 or the orientation of the projection plane 45 on the screen. It is desirable that the control unit 21 restricts the range within which the projection plane 45 can be changed so that the perpendicular line passing through the central part of the projection plane 45 passes through the imaging center when the intravascular three-dimensional scan is performed. By providing such a restriction, an information processing system 10 that simulates imaging using the blood vessel imaging device 41 can be provided.

[0047] The control unit 21 superimposes on the projection plane 45 an estimated contrast image created by vertically projecting the deformed three-dimensional model 76 onto the projection plane 45, and displays it on the display unit 25. Adjusting the orientation of the three-dimensional object on the screen and projecting the three-dimensional object onto a plane have been conventionally performed using CAD (Computer Aided Design) software, CG (Computer Graphics) creation software, etc., and thus detailed description thereof is omitted.

[0048] Specifically, the user adjusts the orientations of the deformed three-dimensional model 76 and the projection plane 45 so that, for example, in the estimated contrast image superimposed on the projection plane 45, the stenosis part of the blood vessel looks thickest, that is, the major axis of the stenosis part is parallel to the projection plane 45. The control unit 21 displays the deformed three-dimensional model 76 rotated based on the operation by the user on the display unit 25. The control unit 21 may display the rotation angle of the deformed three-dimensional model 76 on the display unit 25.

[0049] The user may also adjust the orientations of the deformed three-dimensional model 76 and the projection plane 45 so that, on the projection plane 45, the stenosis part of the blood vessel looks thinnest, that is, the minor axis of the stenosis part is parallel to the projection plane 45. In addition, while viewing the estimated contrast image, the user may adjust the orientations of the deformed three-dimensional model 76 and the projection plane 45 so that an angiogram 61 useful for endovascular treatment can be taken.

[0050] The control unit 21 may calculate the relationship between the deformed three-dimensional model 76 and the projection plane 45 when, for example, the stenosis part of the blood vessel looks thickest or thinnest, and create an estimated contrast image.

[0051] As described above, the positional relationship between the deformed three-dimensional model 76 and the projection plane 45 when a desired estimated contrast image is taken is determined. The control unit 21 calculates the arrangement of the projection plane 45 when the deformed three-dimensional model 76 is arranged at the angle at the time of performing the three-dimensional intravascular scan. Specifically, the control unit 21 performs a coordinate transformation to return the deformed three-dimensional model 76 to the angle at the time of imaging by rotating both around the imaging center while fixing the positional relationship between the deformed three-dimensional model 76 and the projection plane 45. The perpendicular line to the calculated projection plane 45 corresponds to the viewing direction of the blood vessel imaging device 41.

[0052] As described later, the control unit 21 outputs information regarding the field of view direction of the angiography device 41. The user operates the angiography device 41 based on the outputted information to acquire an actual angiographic image 61. The control unit 21 may also transmit information regarding the field of view direction to the angiography device 41 based on the user's instructions. The user observes the actual state of the blood vessels depicted in the acquired angiographic image 61.

[0053] According to this embodiment, the number of imaging sessions required to obtain the desired angiographic image can be reduced. Therefore, it is possible to provide an information processing method that reduces the radiation exposure of patients and medical personnel. By reducing the number of imaging sessions, the amount of contrast agent administered to the patient can also be reduced. Therefore, it is possible to provide an information processing method that reduces the burden on the patient's body.

[0054] Figure 3 is an explanatory diagram illustrating intravascular three-dimensional scanning. First, the structure of the image acquisition catheter 32 will be described. The upper part of Figure 3 shows the tip of the image acquisition catheter 32 protruding from the tip of the guiding catheter 39. The image acquisition catheter 32 is either an OCT (Optical Coherence Tomography) catheter or an IVUS (Intravascular Ultrasound) catheter. The image acquisition catheter 32 may also be a dual-sensor type catheter that has the functions of both an OCT catheter and an IVUS catheter.

[0055] The image acquisition catheter 32 comprises a sensor 322, a shaft 323, and a sheath 324. The sheath 324 is a resin tube with its tip bent in a roughly crank shape. A guide wire insertion hole (not shown) is provided on the tip side of the bend.

[0056] A shaft 323 is inserted inside the sheath 324. A sensor 322 is fixed to the tip of the shaft 323. If the image acquisition catheter 32 is an OCT catheter, the sensor 322, which is an OCT sensor, is a ball lens that emits light from a light source and receives reflected light. If the image acquisition catheter 32 is an IVUS catheter, the sensor 322, which is an IVUS sensor, is an ultrasonic sensor that transmits and receives ultrasonic waves.

[0057] The explanation continues using Figures 1 and 3. The MDU 33 moves forward and backward inside the sheath 324 while rotating the sensor 322 and shaft 323. Radial scanning is performed as the sensor 322 rotates while transmitting and receiving light or ultrasound. The catheter control device 31 generates one image for each rotation of the sensor 322. The generated image is a transverse layer image 71 centered on the sensor 322 and approximately perpendicular to the sheath 324. The acquisition of the transverse layer image 71 refers to the series of processes from the transmission and reception of data by the sensor 322 to the generation of a single transverse layer image 71.

[0058] By performing a pullback operation that rotates the sensor 322 while pulling it toward the MDU 33 at a constant speed, the catheter control device 31 achieves three-dimensional scanning by continuously capturing multiple transverse layer images 71 that are approximately perpendicular to the sheath 324. In three-dimensional scanning, the scan line moves in a spiral shape. However, as shown in the lower part of Figure 3, the catheter control device 31 creates one transverse layer image 71 for each rotation of the sensor 322. The interval ΔL between adjacent transverse layer images 71 is constant, which is the distance the sensor 322 is pulled back during one rotation of the shaft 323.

[0059] The catheter control device 31 can also automatically measure parameters such as the inner diameter of the blood vessel or the diameter of the EEM (External Elastic Membrane). The catheter control device 31 can also generate transverse and longitudinal images with various regions extracted and marked, such as plaque areas, calcified areas, or stent placement areas.

[0060] Since the automatic measurement and automatic extraction of various regions by the catheter control device 31 have been used conventionally, a detailed explanation will be omitted.

[0061] The functions and configuration of the catheter control device 31 are the same as those of conventional imaging diagnostic devices for OCT or IVUS, so a detailed explanation will be omitted. Note that the control unit 21 may also implement the functions of the catheter control device 31.

[0062] Figure 4 is an explanatory diagram illustrating the procedure for calculating the scanning position of the transverse layer image 71 from the angiography image 61. The control unit 21 extracts sensor images 63 from each of the multiple angiography images 61 taken during the intravascular three-dimensional scanning.

[0063] As mentioned above, since the intravascular three-dimensional scanning is performed with the C-arm 421 stationary, the shape of the blood vessels depicted in each angiographic image 61 is identical. Therefore, the control unit 21 can integrate information regarding the positions of multiple sensor images 63 onto a single angiographic image 61.

[0064] The control unit 21 interpolates the position of the sensor 322 based on the timestamp of the angiography image 61 and the timestamp of the transverse image 71, thereby calculating which position on the angiography image 61 corresponds to the position of the sensor 322 when each transverse image 71 was taken.

[0065] Figure 5 is an explanatory diagram illustrating the scanning position of the transverse image 71 in the angiographic image 61. In Figure 5, the position of the sensor 322 that captured the transverse image 71 during pullback is indicated by the symbol P. The distance ΔP between adjacent symbols P is not constant.

[0066] Figure 6 is an explanatory diagram illustrating the method for calculating the inclination of the image acquisition catheter 32 with respect to the angiographic image 61. Figure 6 shows the blood vessel 69 in the portion sandwiched between two transverse layer images 71 and the angiographic image 61. Because the frame rate of the intravascular three-dimensional scanning is sufficiently high, the blood vessel 69 is of a length that can be approximated by a straight line.

[0067] The angiographic image 61 is a plane parallel to the flat-panel detector 423 at the time of acquisition. The angle between the blood vessel 69 and the angiographic image 61 is denoted as φ. The ends of the blood vessel 69 are indicated by the symbols P and A. The blood vessel 69 is projected onto the angiographic image 61 by X-rays emitted perpendicular to the angiographic image 61. The ends of the projected angiographic image 61 are indicated by the symbols P and B.

[0068] As explained using Figure 3, the length between PA is ΔL. The distance ΔP between PB is given by equation (1): ΔP = ΔL cosφ ‥‥‥ (1)

[0069] From equation (1), even if ΔL is constant, ΔP changes depending on the angle φ between the blood vessel and the angiography image 61. Therefore, the angle φ between the blood vessel and the angiography image 61 can be calculated using the interval ΔP between the sensor images 63 as explained using Figure 4. By sequentially calculating and integrating the angles φ in each adjacent angiography image 61, the control unit 21 can calculate the curvature of the blood vessel.

[0070] It should be noted that whether φ is a positive or negative value cannot be determined solely by the formula. However, the control unit 21 can determine whether φ is a positive or negative value based on anatomical information about the blood vessel on which the intravascular ultrasound examination was performed.

[0071] Figure 7 is a flowchart illustrating the program's processing flow. The control unit 21 acquires angiography images 61 from the angiography device 41 (step S501). The control unit 21 acquires information regarding catheter images from the catheter system 30 (step S502). The information regarding catheter images is an example of transverse image information, including multiple transverse images 71 and a three-dimensional model 75 generated from the transverse image information.

[0072] The control unit 21 may acquire the angiographic image 61 and the transverse layer image 71 in real time and in parallel. Even when acquiring them in parallel, the catheter control device 31 generates a three-dimensional model 75 after the series of intravascular three-dimensional scans are completed and transmits it to the information processing device 20.

[0073] The control unit 21 detects the sensor image 63 from each angiography image 61 (step S503). The control unit 21 calculates which position on the angiography image 61 corresponds to the sensor 322 at the time each transverse image 71 was taken (step S504). The control unit 21 calculates the angle of the blood vessel at the position where each transverse image 71 was taken, as explained using Figure 6 (step S505).

[0074] The control unit 21 deforms the three-dimensional model 75 based on the angle calculated in step S505 to create a deformed three-dimensional model 76 (step S506). The control unit 21 obtains the angle of the projection plane 45 relative to the deformed three-dimensional model 76 (step S507). The control unit 21 can simulate the operation of the angiography apparatus 41 by rotating the projection plane 45 with respect to the center of the imaging space 44, as explained using Figure 1.

[0075] The control unit 21 generates a projection image, i.e., an estimated contrast-enhanced image, by projecting the deformed three-dimensional model 76 onto the projection surface 45 in a field of view direction perpendicular to the projection surface 45 (step S508). Through the above process, the control unit 21 obtains the field of view direction on which the deformed three-dimensional model 76 is projected in step S507, and generates an estimated contrast-enhanced image based on the field of view direction in step S508.

[0076] The projection diagram generated in step S508 corresponds to the result of simulating the angiographic image 61 taken by the angiography device 41. The control unit 21 displays the generated projection diagram on the display unit 25 (step S509).

[0077] The control unit 21 determines whether or not to terminate the process (step S510). For example, if it receives a termination instruction from the user, the control unit 21 determines to terminate the process. If it determines not to terminate the process (NO in step S510), the control unit 21 returns to step S507. If it determines to terminate the process (YES in step S510), the control unit 21 terminates the process.

[0078] Figure 8 shows an example screen. The control unit 21 is in the initial state of the screen that the control unit 21 displays on the display unit 25 in step S509 of the program described using Figure 7. The deformed three-dimensional model 76 and the angiography image 61 are displayed in perspective. The positional relationship between the two is the same as that during intravascular three-dimensional scanning. In the state shown in Figure 8, the projection plane 45 overlaps with the angiography image 61.

[0079] An angle indicator 66 is displayed at the bottom of the screen. The upper indicator shows the lateral direction, that is, the angle of the patient's body from left to right. The lower indicator shows the head-foot direction, that is, the angle of the patient's body from head to toe or from toe to head. Specifically, the rightmost point of the lateral indicator represents RAO (Right Anterior Oblique) 90 degrees, indicating the angle of fluoroscopy from the patient's right side to their left side.

[0080] Similarly, the left end of the lateral indicator represents LAO (Left Anterior Oblique) 90 degrees, indicating the angle of fluoroscopy from the patient's left hand side to their right hand side. The right end of the cephalofoot indicator represents CRA (Cranial) 90 degrees, indicating the angle of fluoroscopy from the patient's head side to their feet side. The left end of the cephalofoot indicator represents CAU (Caudal) 90 degrees, indicating the angle of fluoroscopy from the patient's feet side to their head side.

[0081] The imaging angle of the angiography device 41 is indicated by sliders that move left and right on each indicator. The control unit 21 may also display the lateral and cephalofoot angles numerically near the sliders, corresponding to the slider positions. The lateral and cephalofoot angles are examples of information regarding the field of view of the angiography device 41.

[0082] The imaging angle of the angiography device 41 is parallel to the perpendicular line of the projection plane 45. The control unit 21 maintains the relative positional relationship between the projection plane 45 and the deformed three-dimensional model 76, and returns the deformed three-dimensional model 76 to the angle it was at when the intravascular three-dimensional scanning was performed. The control unit 21 calculates the angle of the perpendicular line of the projection plane 45 and displays it on the angle indicator 66.

[0083] Three setting buttons 67 and one reset button 68 are located in the lower left corner of the screen. Each of the setting buttons 67 is set to a frequently used shooting angle for acquiring angiographic images 61. When the user selects a setting button 67, the control unit 21 changes the orientation of the deformed three-dimensional model 76 and the projection plane 45, and the slider of the angle indicator 66 to the set shooting angle. When the user selects a reset button 68, the control unit 21 returns the screen shown in Figure 8 to its initial state, that is, the state showing the shooting angle when intravascular three-dimensional scanning was performed.

[0084] When the user moves the deformable three-dimensional model 76 or the projection surface 45, the control unit 21 erases the angiography image 61 and displays the deformable three-dimensional model 76 and the projection surface 45, which simulates the angiography image 61 onto which the deformable three-dimensional model 76 is projected, on the screen.

[0085] In Figure 8, the deformed three-dimensional model 76 shows the surface shape of the external elastic membrane of a blood vessel. The control unit 21 can simulate angiography image 61 by extracting the portion corresponding to the lumen region of the blood vessel from the same deformed three-dimensional model 76 and projecting it onto the projection plane 45. The control unit 21 may also display the deformed three-dimensional model 76 showing the surface shape of the lumen region of the blood vessel on the screen shown in Figure 8.

[0086] When the user moves the deformable three-dimensional model 76 or projection plane 45 on the screen, the control unit 21 calculates the corresponding angle and displays a slider on the angle indicator 66 at the corresponding position. The user may also move the slider on the angle indicator 66 left or right. When the control unit 21 receives the slider operation, it changes the orientation of the deformable three-dimensional model 76 or projection plane 45 according to the slider's position. In other words, the control unit 21 accepts input from the user regarding the field of view.

[0087] The control unit 21 may change the relationship between the deformed three-dimensional model 76 and the projection plane 45 according to a predetermined rule, such as fixing one of the shooting angles in the head-to-foot direction and the shooting angle in the lateral direction, and repeatedly changing the other in increments of 5 degrees, and continuously display it on the screen shown in Figure 8.

[0088] In this case, the relationship between the deformed three-dimensional model 76 and the projection plane 45, that is, the field of view direction when projecting the deformed three-dimensional model 76, may be stored in the auxiliary storage device 23, for example, in the form of a table. The control unit 21 uses the field of view directions sequentially acquired from the auxiliary storage device 23 in step S507 of the program described using Figure 7. Through the above process, the control unit 21 can automatically acquire the field of view direction specification from the auxiliary storage device 23.

[0089] It is desirable that the user can set and change the change range and change increments as appropriate. The control unit 21 may use a program (not shown) to automatically generate a table that covers the change range set by the user and record it in the auxiliary storage device 23.

[0090] For example, the user can visually monitor the screen and stop the display at the appropriate location. The user can determine the appropriate shooting angle with minimal effort. The control unit 21 may, for example, extract the position where the width of the constricted area is maximum or minimum as instructed by the user and display it on the display unit 25.

[0091] According to this embodiment, it is possible to provide an information processing method that can appropriately associate images taken by the catheter system 30 with images taken by the angiography device 41, even for curved blood vessels.

[0092] According to this embodiment, a deformed three-dimensional model 76 that reproduces the actual shape of a blood vessel can be created based on multiple transverse layer images 71 captured by intravascular three-dimensional scanning. By displaying an image of the deformed three-dimensional model 76 projected onto a plane, the user can appropriately determine the scanning angle of the angiography device 41.

[0093] According to this embodiment, the number of imaging sessions required to obtain the desired angiographic image can be reduced. Therefore, it is possible to provide an information processing method that reduces the radiation exposure of patients and medical personnel. By reducing the number of imaging sessions, the amount of contrast agent administered to the patient can also be reduced. Therefore, it is possible to provide an information processing method that reduces the burden on the patient's body.

[0094] [Embodiment 2] This embodiment relates to an information processing method that allows a user to fine-tune the shape of a deformable three-dimensional model 76. Parts common to Embodiment 1 will not be described.

[0095] Figure 9 is a flowchart illustrating the processing flow of the program in Embodiment 2. Steps S501 to S506 are the same as the flowchart of Embodiment 1 explained using Figure 1, so their explanation is omitted.

[0096] The control unit 21 displays the screen described using Figure 8 on the display unit 25 (step S521). In step S521, the screen is in its initial state, with the projection plane 45 and the angiography image 61 superimposed on each other. The positional relationship between the deformed three-dimensional model 76 and the angiography image 61 is the same as that during intravascular three-dimensional scanning.

[0097] The control unit 21 determines whether or not it has received a deformation instruction for the deformable three-dimensional model 76 (step S522). The user gives a deformation instruction, for example, by operating a right-click menu. Specifically, the user gives a deformation instruction when they determine that there is a difference between the shape of the automatically generated deformable three-dimensional model 76 and the actual shape of the blood vessel.

[0098] If the control unit determines that a modification instruction has been received (YES in step S522), it accepts a deformation instruction from the user (step S523). Since deformation instructions for models representing three-dimensional shapes have been conventionally performed using CAD software and CG creation software, a detailed explanation will be omitted.

[0099] The control unit 21 modifies the deformed three-dimensional model 76 based on the deformation instruction (step S524). The control unit 21 returns to step S521.

[0100] If it is determined that no deformation instruction has been received (NO in step S522), the control unit 21 determines the angles of the deformed three-dimensional model 76 and the projection plane 45 (step S507). The subsequent processing is the same as the processing flow of Embodiment 1 described using Figure 7, so the explanation is omitted.

[0101] [Embodiment 3] This embodiment relates to an information processing method for displaying a deformed three-dimensional model 76 along with an organ model 79 showing the shape of organs and blood vessels running around the organs. Parts common to Embodiment 1 will not be described.

[0102] Figure 10 is a flowchart illustrating the processing flow of the program in Embodiment 3. Steps S501 to S506 are the same as the flowchart of Embodiment 1 explained using Figure 1, so their explanation is omitted.

[0103] The control unit 21 determines the position of the deformed three-dimensional model 76 on the organ model 79 by pattern matching the deformed three-dimensional model 76 with the blood vessels included in the organ model 79 (see Figure 11) (step S541). The control unit 21 may also determine the position of the deformed three-dimensional model 76 on the organ model 79 based on instructions from the user.

[0104] The organ model 79 is created, for example, by preoperative CT (Computed Tomography) and shows the vascular structure of the patient themselves. If individual differences in vascular structure can be ignored, or if an organ model 79 of the patient themselves has not been created, the organ model 79 may be a model that shows the general shape of the organ.

[0105] The control unit 21 determines the display angle of the organ model 79 (step S542). For example, the control unit 21 determines the display angle of the organ model 79 so that the deformed three-dimensional model 76 placed on the organ model 79 is positioned in the center of the front view. The control unit 21 may also determine the display angle of the organ model 79 based on instructions from the user.

[0106] When the control unit 21 receives instructions from the user, it acquires the position of the organ center, which is the center of the organ model 79. Based on the user's operation, the control unit 21 rotates the organ model 79 around the organ center. As a result, the user can get the feeling of finding the appropriate display angle while rotating the entire organ model 79.

[0107] The control unit 21 determines the projection angle (step S543). Specifically, based on user instructions, the control unit 21 determines the position and angle of the projection plane 45 to be placed near the organ model 79.

[0108] The control unit 21 generates a projection image, i.e., an estimated contrast-enhanced image, by projecting the deformed three-dimensional model 76 onto the projection plane 45 (step S544). The control unit 21 displays the generated estimated contrast-enhanced image and organ model 79 on the display unit 25 (step S545).

[0109] The control unit 21 determines whether or not to terminate the process (step S546). For example, if it receives a termination instruction from the user, the control unit 21 determines to terminate the process. If it determines not to terminate the process (NO in step S546), the control unit 21 returns to step S542. If it determines to terminate the process (YES in step S546), the control unit 21 terminates the process.

[0110] Furthermore, in step S542, it is desirable for the control unit 21 to restrict the range of change in the position and angle of the projection plane 45 so that the center of the projection plane 45 passes through the center of the organ. By adding such constraints, the projection diagram generated in step S544 becomes a diagram projecting a deformed three-dimensional model 76 rotated around the center of the organ.

[0111] Figure 11 shows an example of an organ model 79 and a deformed three-dimensional model 76 matched to the organ model 79. The organ model 79 is a three-dimensional model of the heart, with three coronary arteries running across its surface. The deformed three-dimensional model 76 is matched to a portion of the right coronary artery.

[0112] According to this embodiment, by matching and displaying the organ model 79 and the deformed three-dimensional model 76, even paramedical staff or visitors can quickly understand which part of the organ is being treated with endovascular therapy.

[0113] For example, even in patients whose blood vessel structure differs from that of the average person due to past cardiac surgery or congenital heart disease, doctors and other medical professionals can quickly determine the necessary procedures and provide appropriate treatment by matching and displaying an organ model 79 generated from the patient's own data with a deformed three-dimensional model 76.

[0114] [Embodiment 4] This embodiment relates to a configuration in which an information processing system 10 is realized by operating a general-purpose computer 90 and a program 97 in combination. The parts that are common with Embodiment 1 will not be described.

[0115] Figure 12 is an explanatory diagram illustrating the configuration of the information processing system 10 of Embodiment 4. The information processing system 10 includes a catheter system 30, an angiography device 41, and a computer 90. The computer 90 includes the aforementioned control unit 21, main memory 22, auxiliary memory 23, communication unit 24, display unit 25, input unit 26, and bus, as well as a reading unit 29.

[0116] The program 97 is recorded on a portable recording medium 96. The control unit 21 reads the program 97 via the reading unit 29 and saves it to the auxiliary storage device 23. The control unit 21 may also read the program 97 stored in a semiconductor memory 98, such as a flash memory, installed in the computer 90. Furthermore, the control unit 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown) and save it to the auxiliary storage device 23.

[0117] Program 97 is installed as a control program for the computer 90, loaded into the main memory 22, and executed. Thus, the information processing system 10 described in Embodiment 1 is realized. Program 97 in this embodiment is an example of a program product.

[0118] Program 97 may be provided on a recording medium or distributed from an external computer. The computer program can be deployed to run on a single computer or at a single site, or distributed across multiple sites and interconnected by a communication network.

[0119] [Embodiment 5] This embodiment relates to a method for determining whether the angle φ described using Figure 6 and equation (1) is a positive or negative value. The parts that are common with Embodiment 1 will not be explained.

[0120] Figure 13 is a schematic diagram illustrating the three-dimensional relationship between the blood vessels 69 and the angiographic image 61 in Embodiment 5. In this embodiment, the heart is approximated and treated as a spherical simplified heart model 78. As mentioned above, three coronary arteries run along the surface of the heart. These coronary arteries are also approximated and treated as being located on the surface of the simplified heart model 78.

[0121] In intravascular three-dimensional scanning, an image acquisition catheter 32 is often inserted into one of these three coronary arteries to perform the three-dimensional scan. The path of the image acquisition catheter 32 is shown by a thick line on the surface of the simplified cardiac model 78, and the positions where the three-dimensional scan was performed are indicated by the labels Q1 to Q4. As explained using Figure 3, Q1 to Q4 are arranged at equal intervals on the surface of the simplified cardiac model 78.

[0122] The angiographic image 61 is an image obtained by projecting the simplified cardiac model 78 and the image acquisition catheter 32 perpendicularly to the angiographic image 61. Q1 to Q4 are projected onto P1 to P4, respectively. In Figure 13, four P and four Q are schematically shown, but the following explanation will describe the case where there are N P and N Q.

[0123] For illustrative purposes, the X and Y axes are defined on the angiographic image 61, and the Z axis is defined perpendicular to the angiographic image 61. As shown in Figure 13, XYZ are right-handed orthogonal coordinates with their origin on the angiographic image 61. The coordinates of the nth point P, Pn, are denoted as (PXn, PYn, 0). The coordinates of Qn corresponding to each Pn are denoted as (PXn, PYn, QZn), where n is an integer.

[0124] A reference coordinate S0 is defined on the angiographic image 61. The reference coordinate S0 is, for example, the centroid from P1 to PN, and is calculated by equation (2).

[0125]

[0126] Figure 14 is an explanatory diagram illustrating the positive and negative signs of the blood vessel angle φ. In Figure 14, a schematic diagram is used in which the plane perpendicular to the angiographic image 61, passing through P1 to P4, is unfolded on a plane. P1 to P4 are represented as a straight line.

[0127] Let's explain using the section from P1 to P2 as an example. For the sake of explanation, in Figure 13, move the simplified heart model 78 in the negative direction of the Z axis to place P1 and Q1 in the same position. P1 and Q1 correspond to the symbol P in Figure 6. P2 corresponds to the symbol B in Figure 6. Q2 corresponds to the symbol A in Figure 6.

[0128] As explained using Figure 6, the angle φ is calculated by equation (1) based on the interval ΔL at which the image acquisition catheter 32 takes transverse images 71 and ΔP, which is the distance between P1 and P2 on the angiographic image 61. However, equation (1) alone cannot determine whether φ is a positive or negative value.

[0129] In this embodiment, for each Pn from n=1 to n=(N-1), a group is used which is obtained by exhaustively combining the cases where φn is positive and the cases where it is negative. For example, in Figure 14, No. 1 shows the case where φ1, φ2, and φ3 are all positive. No. 2 shows the case where φ1 and φ2 are positive and φ3 is negative. No. 3 shows the case where φ1 and φ3 are positive and φ2 is negative.

[0130] If we ignore the case where φn is zero, the group of combinations of positive and negative φn has 2 to the power of (N-1) elements. If, among the N φn, there are A φn that are zero, then the group of combinations of positive and negative φn has 2 to the power of (N-A-1) elements.

[0131] In other words, the factorial of (N-1) is the maximum value of the elements in the group formed by the combination of positive and negative φn. As shown in Nos. 1 to 3, the spatial arrangement of Q1 to QN differs depending on the combination of positive and negative φn. Among these arrangements, the one closest to the surface of the spherical simplified heart model 78 is the result of correctly determining the positive and negative φn.

[0132] Based on the above principles, the outline of the process by which the control unit 21 determines the sign of φn will be explained. The control unit 21 generates all possible combinations of positive and negative signs for φn, as explained using Figure 14. For each combination, the control unit 21 calculates the variance of the distance between the reference coordinate S0 and each point from Q0 to QN. The smaller the variance, that is, the smaller the variation in the distance between the reference coordinate S0 and each point from Q0 to QN, the closer the reference coordinate S0 is to the center of the simplified heart model 78.

[0133] The control unit 21 determines the reference coordinate S by changing the value of the Z coordinate of the reference coordinate S0 within a predetermined range along a line passing through the reference coordinate S0 and perpendicular to the angiographic image 61. Specifically, the control unit 21 determines a series of reference coordinates S by changing only the value of the Z coordinate of the reference coordinate S0 in increments equivalent to, for example, five pixels of the angiographic image 61. The control unit 21 repeatedly performs the process of calculating the variance of the distance between the reference coordinate S and each point from Q0 to QN for each combination of positive and negative φn.

[0134] The control unit 21 extracts the combination of the Z coordinate value of the reference coordinate S and the sign of φn that minimizes the variance calculated by the above process. The extracted result is the result of correctly determining the sign of φn. As a result, the control unit 21 can automatically determine the sign of φn.

[0135] Figure 15 is a flowchart illustrating the processing flow of the subroutine for calculating the angle of blood vessels at each imaging position in Embodiment 5. The control unit 21 executes the flowchart in Figure 15 instead of step S505 of the program described using Figure 7. Note that in step S504 of the program described using Figure 7, the coordinates from P1 to PN have already been calculated.

[0136] The control unit 21 identifies the reference coordinate S0 based on equation (2) (step S551). The control unit 21 creates a group that covers all combinations of positive and negative values ​​of φn (step S552). As described above, the created group contains a maximum of (N-1) factorial elements.

[0137] The control unit 21 calculates the coordinates of each point from Q1 to QN for each element created in step S551 (step S553). Specifically, as shown in Figure 14, the control unit 21 calculates the coordinates of each point in order from Q1. The control unit 21 temporarily records the combination of positive and negative φn and the coordinate of Qn in the auxiliary storage device 23.

[0138] The control unit 21 selects one element, that is, a pair of positive and negative φn combinations, from the group created in step S552 (step S554). The control unit 21 obtains the coordinates of Qn calculated in step S553 corresponding to the selected element.

[0139] The control unit 21 calculates the distance between the reference coordinate S0 and each Qn. The control unit 21 calculates the variance of the calculated distances (step S555). Since the method for calculating the distance between two points with known coordinates, and the method for calculating the variance of multiple distances are known, a detailed explanation is omitted. The control unit 21 temporarily records the combination of positive and negative φn, the coordinates of Qn, and the variance calculated in step S555 in the auxiliary storage device 23 (step S556).

[0140] The control unit 21 determines whether it has finished processing all elements included in the group created in step S552 (step S557). If it determines that it has not finished (NO in step S557), the control unit 21 returns to step S554. If it determines that it has finished (YES in step S557), the control unit 21 determines whether it has finished processing the reference coordinates S within a predetermined range (step S558).

[0141] If it is determined that the process is not complete (NO in step S558), the control unit 21 moves the reference coordinate S in the positive direction of the Z axis (step S559). Specifically, the control unit 21 changes only the value of the Z coordinate of the reference coordinate S. The control unit 21 returns to step S554.

[0142] If it is determined that the process has finished (YES in step S558), the control unit 21 determines the combination of positive and negative values ​​of φn that minimizes the variance recorded in step S556 of the series of loops (step S560). The control unit 21 then terminates the process.

[0143] According to this embodiment, an information processing device 20 can be provided that can automatically determine whether the angle φ, as explained using Figure 6 and equation (1), is a positive or negative value. Therefore, an information processing device 20 can be provided that can quickly associate images taken by an image acquisition catheter with images taken by angiography.

[0144] The size of the heart is generally said to be about the size of a fist. Even when the heart is enlarged or atrophied due to disease, its size is still within the range of about half to three fists.

[0145] If the range in which the coordinates of Qn calculated in step S553 are distributed is extremely large or extremely small compared to the dimensions of the heart, the control unit 21 may delete the corresponding positive and negative combinations of φn from the group created in step S552. The control unit 21 may also accept input of information regarding a threshold for deleting positive and negative combinations of n. By removing combinations that are not actually possible in the early stages of processing, an information processing device 20 with a low information processing load can be provided.

[0146] In step S560, the control unit 21 may extract a predetermined number of positive and negative combinations of φn, starting with the smallest variance, and then select a combination in which the arrangement of Qn is close to the actual shape of a blood vessel. For example, the control unit 21 may select a combination in which the shape formed by connecting Qn with straight lines is small in difference from the shape of a blood vessel included in the organ model 79 described using Figure 11.

[0147] [Embodiment 6] This embodiment relates to an information processing system 10 that performs the identification of reference coordinates in step S551 of the program described with reference to Figure 15 using an angiography image 61 taken with an angiography device 41. The parts common to Embodiment 5 will not be described.

[0148] Figure 16 is an explanatory diagram illustrating the method for calculating the reference coordinate S0 in Embodiment 6. Figure 16 shows a so-called negative display angiography image 61 in which areas with high X-ray absorption are displayed in black. The areas without hatching at the edges of the angiography image 61 are regions where no image is depicted.

[0149] Figure 16A shows the pixel values ​​on the angiographic image 61 divided into three stages. The vertical hatching represents areas where the pixel value is between 0 and 50, i.e., areas close to black. The areas with vertical hatching depict blood vessels through which contrast agent is flowing.

[0150] Horizontal hatching represents areas where the pixel value is between 151 and 255, i.e., areas that are close to white. The areas with horizontal hatching are image noise. Downward-sloping hatching represents areas where the pixel value is between 51 and 150, i.e., gray areas. The areas with downward-sloping hatching depict biological tissue such as the heart. The control unit 21 acquires the angiography image 61 shown in Figure 16A from the angiography device 41.

[0151] Figure 16B is a diagram in which the region with a pixel value between 51 and 150 is left, and the rest of the region is made white. The control unit 21 determines whether the pixel value of each pixel constituting the angiography image 61 is within a predetermined range, and changes the pixel value of any pixels outside the range to 255, i.e., white.

[0152] Note that the range of pixel values ​​used for determination is an example. The control unit 21 may, for example, display a luminance histogram on the display unit 25 showing the number of pixels corresponding to each pixel value from 0 to 255, and accept the user's specification of a threshold value.

[0153] Figure 16C is a diagram in which only the largest region of a continuous biological tissue area is retained, and the other regions are shown in white. The control unit 21 extracts the portion composed of a continuous sequence of pixels other than white, and then extracts the portion with the maximum number of constituent pixels. The control unit 21 calculates the centroid of the extracted pixels. The calculated centroid is the reference coordinate S0 in this embodiment.

[0154] According to this embodiment, even when the length of the blood vessel 69 captured by the image acquisition catheter 32 is short, an information processing system 10 can be provided that can stably and reproducibly determine the position of the reference coordinate S0.

[0155] [Modification] The control unit 21 may obtain the coordinates of the foot of the perpendicular line drawn from the center of gravity of the organ model 79 to the angiography image 61 from the angiography device 41, along with the angiography image 61. The control unit 21 can use the obtained coordinates of the foot of the perpendicular line as the reference coordinate S0.

[0156] [Embodiment 7] This embodiment relates to an information processing system 10 that determines the coronary artery into which the image acquisition catheter 32 is inserted based on the imaging angle of the angiography device 41, and determines whether φn is positive or negative. The parts that are common with Embodiment 1 will not be explained.

[0157] Figure 17 is an explanatory diagram illustrating the relationship between the imaging direction of the angiography device 41 and the blood vessel 69 being observed. As explained using Figure 8, the imaging angle of the angiography device 41 is expressed in two directions: lateral and cephalopedus. Figure 17 shows the general relationship between the imaging angle of the angiography device 41 and the coronary artery into which the image acquisition catheter 32 is inserted.

[0158] In Figure 17, the horizontal direction indicates the lateral angle. "RAO90°" means the angle of fluoroscopy from the patient's right hand side to the left hand side, and "LAO90°" means the angle of fluoroscopy from the patient's left hand side to the right hand side. In Figure 17, the vertical direction indicates the head direction. "CRA90°" means the angle of fluoroscopy from the patient's head side to the foot side, and "CAU90°" means the angle of fluoroscopy from the patient's foot side to the head side.

[0159] If the coronary artery (CAU) is in the range of 90 to 15 degrees, the imaging catheter 32 is often inserted into the LCX (Left Circumflex Artery). If the CAU is in the range of 15 degrees and the CRA is in the range of 90 degrees, and the LAO is in the range of 15 to 90 degrees, the imaging catheter 32 is often inserted into the RCA (Right Coronary Artery). If the CAU is in the range of 15 degrees and the CRA is in the range of 90 degrees, and the RAO is in the range of 90 degrees and the LAR is in the range of 90 degrees, the imaging catheter 32 is often inserted into the LAD (Left Anterior Descending Artery).

[0160] Based on the above, the control unit 21 can determine which blood vessel 69 the angiography device 41 is inserted into, based on the imaging angle acquired from the angiography device 41. However, since there are individual differences in the course of the coronary arteries, even if the imaging angle is classified as "LCX" in Figure 17, for example, the image acquisition catheter 32 may be inserted into the RCA.

[0161] Therefore, it is desirable that the control unit 21 can accept correction instructions from the user after displaying the coronary arteries determined based on Figure 17 on the display unit 25. The control unit 21 may also accept the user's specification of coronary arteries without performing the determination process based on Figure 17.

[0162] Figure 18 is an explanatory diagram illustrating the reference line SL in Embodiment 7. In this embodiment as well, the heart is approximated and treated as a spherical simplified heart model 78. The three coronary arteries mentioned above all follow the surface of the simplified heart model 78. The blood vessel 69 into which the image acquisition catheter 32 is inserted is shown by a thick line.

[0163] As shown in Figure 18, we will explain using the example where the blood vessel 69 into which the image acquisition catheter 32 is inserted is located on the other side of the heart, i.e., on the side of the X-ray tube 422, as seen from the angiographic image 61. In Figure 18, the sensor 322 takes a transverse image 71 while moving from Q1 to Q2.

[0164] The control unit 21 extracts from the simplified cardiac model 78 the position of the blood vessel 69 into which the image acquisition catheter 32 is inserted that is furthest from the angiographic image 61, i.e., the position closest to the flat-panel detector 423. In the following description, the extracted position will be referred to as the change point QC. Between Q1 and QC, the angle φ between the blood vessel 69 and the angiographic image 61 is positive, and after QC, the angle φ between the blood vessel 69 and the angiographic image 61 is negative.

[0165] In the following explanation, the foot of the perpendicular line drawn from the change point QC to the angiographic image 61 will be referred to as the change point foot PC. The control unit 21 determines the reference line SL connecting the reference coordinate S0 and the change point foot PC. The side of the reference line SL closer to P1 will be referred to as the near side, and the side closer to P5 will be referred to as the far side. Between P1 and PC, the angle φ between the blood vessel 69 and the angiographic image 61 is positive, and beyond PC, the angle φ between the blood vessel 69 and the angiographic image 61 is negative.

[0166] Figure 19 is a flowchart illustrating the processing flow of the subroutine for calculating the angle of blood vessels at each imaging position in Embodiment 7. The control unit 21 executes the flowchart in Figure 15 instead of step S505 of the program described using Figure 7. Note that in step S504 of the program described using Figure 7, the coordinates from P1 to PN have already been calculated.

[0167] The control unit 21 obtains the imaging angle of the angiography image 61 from the angiography device 41 (step S611). The control unit 21 activates a subroutine for vessel determination (step S612). The subroutine for vessel determination determines the vessel into which the image acquisition catheter 32 is inserted based on the imaging angle of the angiography image 61. The processing flow of the subroutine for vessel determination will be described later.

[0168] The control unit 21 determines the positional relationship between the heart and the blood vessels in the simplified heart model 78 based on the imaging angle of the angiography image 61 and the blood vessels determined in step S612 (step S613). The positional relationship is, for example, whether more than half of the blood vessels 69 are on the X-ray tube 422 side or the flat-panel detector 423 side of the heart.

[0169] The control unit 21 identifies the reference coordinate S0 (step S614). The method for identifying the reference coordinate S0 is the same as the method described in Embodiment 5 or Embodiment 6. The control unit 21 activates a positive / negative determination subroutine (step S615). The positive / negative determination subroutine determines whether the angle φ of the blood vessel 69 is positive or negative. The processing flow of the positive / negative determination subroutine will be described later. The control unit 21 terminates processing.

[0170] Figure 20 is a flowchart illustrating the processing flow of the blood vessel identification subroutine. The blood vessel identification subroutine determines the blood vessel 69 into which the image acquisition catheter 32 is inserted, based on the acquisition angle of the angiography image 61.

[0171] The control unit 21 determines whether the angle in the head-foot direction of the angiographic image 61 acquired in step S611 of the program described using Figure 19 is between 15 degrees and 90 degrees Celsius (step S631). If it is determined that the CAU is between 15 degrees and 90 degrees Celsius (YES in step S631), the control unit 21 determines that the blood vessel 69 into which the image acquisition catheter 32 is inserted is the left circumflex artery (LCX) (step S632).

[0172] If it is determined that the CAU is not between 15 degrees and 90 degrees (NO in step S631), the control unit 21 determines whether the lateral angle of the angiography image 61 acquired in step S611 is between 15 degrees and 90 degrees (LAO) (step S633).

[0173] If the LAO is determined to be between 15 and 90 degrees (YES in step S633), the control unit 21 determines that the blood vessel 69 into which the image acquisition catheter 32 is inserted is the right coronary artery (RCA) (step S634). If the LAO is not between 15 and 90 degrees (NO in step S633), the control unit 21 determines that the blood vessel 69 into which the image acquisition catheter 32 is inserted is the left anterior descending artery (LAD) (step S635).

[0174] After the completion of step S632, step S634, or step S635, the control unit 21 terminates the process.

[0175] Figure 21 is a flowchart illustrating the processing flow of the positive / negative determination subroutine. Based on the determination result of step S613 of the program described using Figure 19, the control unit 21 determines whether the blood vessel 69 into which the image acquisition catheter 32 is inserted is on the X-ray tube 422 side of the heart (step S651).

[0176] If it is determined that the X-ray tube 422 is the target (YES in step S651), the control unit 21 acquires the position of the sensor 322 on the angiographic image 61 (step S652). The control unit 21 acquires the coordinates one by one from P1 in the loop starting from step S652 and then executes the subsequent processing.

[0177] The control unit 21 determines whether the acquired coordinates are on the near side of the reference line SL, i.e., on the P1 side (step S653). If it is determined to be on the near side (YES in step S653), the control unit 21 determines that the angle φ at P1 is positive (step S654). If it is determined not to be on the near side (NO in step S653), the control unit 21 determines that the angle φ at P1 is negative (step S655).

[0178] The control unit 21 determines whether or not processing of all sensor positions, i.e., point P, has been completed (step S656). If it determines that processing has not been completed (NO in step S656), the control unit 21 returns to step S652. If it determines that processing has been completed (YES in step S656), the control unit 21 terminates the processing.

[0179] If it is determined that the X-ray tube 422 is not the correct location (NO in step S651), the control unit 21 obtains the position of the sensor 322 on the angiography image 61 (step S662). The control unit 21 then obtains the coordinates of each sensor one by one, starting from P1, in the loop that begins in step S662, and executes the subsequent processing.

[0180] The control unit 21 determines whether the acquired coordinates are on the near side of the reference line SL, i.e., on the P1 side (step S663). If it is determined to be on the near side (YES in step S663), the control unit 21 determines that φ at P1 is negative (step S664). If it is determined not to be on the near side (NO in step S663), the control unit 21 determines that φ at P1 is positive (step S665).

[0181] The control unit 21 determines whether or not processing of all sensor positions, i.e., point P, has been completed (step S666). If it determines that processing has not been completed (NO in step S666), the control unit 21 returns to step S662. If it determines that processing has been completed (YES in step S666), the control unit 21 terminates the processing.

[0182] According to this embodiment, by using a simplified heart model 78, it is possible to provide an information processing device 20 that determines the sign of φ so as not to deviate significantly from the actual shape of the heart and coronary arteries.

[0183] The technical features (constituent elements) described in each embodiment are combinable with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0184] The independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. Furthermore, while the claims use a multi-claim format in which claims refer to two or more other claims (multi-claim format), this is not the only option. A multi-claim format in which at least one multi-claim is referenced (multi-multi-claim format) is also acceptable.

[0185] 10 Information Processing System 20 Information Processing Device 21 Control Unit 22 Main Memory Unit 23 Auxiliary Memory Unit 24 Communication Unit 25 Display Unit 26 Input Unit 29 Reading Unit 30 Catheter System 31 Catheter Control Device 32 Image Acquisition Catheter 322 Sensor 323 Shaft 324 Sheath 33 MDU 39 Guiding Catheter 41 Angiography Device 421 C-Arm 422 X-ray Tube 423 Planar Detector 44 Imaging Space 45 Projection Plane 61 Angiographic Image (Fluoroscopy Image) 63 Sensor Image 66 Angle Indicator 67 Setting Button 68 Reset Button 69 Blood Vessel 71 Transverse Layer Image 75 Three-Dimensional Model 751 Central Axis 76 Deformed Three-Dimensional Model 78 Simplified Heart Model 79 Organ Model 90 Computer 96 Portable Recording Medium 97 Program 98 Semiconductor Memory

Claims

1. An information processing method in which a computer performs the following steps: acquires transverse image information relating to multiple transverse images of a blood vessel; acquires sensor position information relating to the position of the sensor used to capture each of the transverse images in the fluoroscopic image; acquires a specified field of view direction; and outputs the shape of the blood vessel viewed fluoroscopy from the acquired field of view direction, or an angle relating to the field of view direction, based on the transverse image information and the sensor position information.

2. The information processing method according to claim 1, wherein the specification of the field of view direction is obtained based on input from the user.

3. The information processing method according to claim 1, wherein the specification of the field of view direction is automatically acquired.

4. The information processing method according to claim 1, wherein the plurality of transverse images are acquired by intravascular three-dimensional scanning.

5. An information processing method according to any one of claims 1 to 4, which involves repeatedly calculating the inclination of the three-dimensional scanning axis between the two transverse images with respect to the fluoroscopic image, based on the distance between the two transverse images in the intravascular three-dimensional scanning and the distance the sensor moved between the acquisition of the two transverse images in the fluoroscopic image, and calculating the curvature of the blood vessel in which the intravascular three-dimensional scanning was performed based on the series of inclinations calculated.

6. The information processing method according to claim 5, wherein the transverse layer image information includes a three-dimensional model of the blood vessel created based on the plurality of transverse layer images, and the shape of the blood vessel is the shape of a deformed three-dimensional model obtained by deforming the three-dimensional model based on the calculated bending state.

7. The information processing method according to claim 6, which obtains the position of the shooting center, which is the center of the shooting space in which the perspective image can be taken, and outputs the shape of the deformed three-dimensional model rotated around the position of the shooting center, or the rotation angle of the deformed three-dimensional model.

8. The information processing method according to claim 6, which determines the position of the deformed three-dimensional model in the organ model by pattern matching the deformed three-dimensional model with blood vessels included in an organ model relating to the shape of the organ and the blood vessels running around the organ, obtains the position of the organ center which is the center of the organ model, and outputs the shape of the deformed three-dimensional model rotated around the organ center, or the rotation angle of the deformed three-dimensional model.

9. A program that causes a computer to perform the following processes: acquire transverse image information relating to multiple transverse images of a blood vessel; acquire sensor position information relating to the position of the sensor used to capture each of the transverse images in the fluoroscopic image; acquire a specified field of view direction; and based on the transverse image information and the sensor position information, output the shape of the blood vessel viewed fluoroscopy from the acquired field of view direction, or the angle relating to the field of view direction.

10. An information processing device having a control unit, wherein the control unit acquires transverse image information relating to a plurality of transverse images of a blood vessel, acquires sensor position information relating to the position of each sensor used to capture the transverse image in the fluoroscopic image, acquires a specification of the field of view direction, and outputs the shape of the blood vessel viewed fluoroscopy from the acquired field of view direction, or an angle relating to the field of view direction, based on the transverse image information and the sensor position information.

Images