Information processing method, program, and information processing device

Abstract

The present invention provides an information processing method and the like that can associate an area in a contrast-enhanced image and a transverse tomographic image in which an image of the area is captured.　In the information processing method, a computer executes processes for: acquiring a plurality of transverse tomographic images that have been generated by three-dimensional scanning using a catheter (32) for image acquisition and an angiographic image; acquiring the position of a first landmark that is located outside a three-dimensional scanning range and the position of a second landmark located within the three-dimensional scanning range; determining a route of the catheter for acquiring the image; calculating the frame number corresponding to a first tomographic image depicting the first landmark; determining the frame number corresponding to a second tomographic image depicting the second landmark; and specifying the positions of the respective transverse tomographic images by dividing the route by the number of frames from the first tomographic image to the second tomographic image.

Description

Information Processing Method, Program, and Information Processing Apparatus

[0001] The present invention relates to an information processing method, a program, and an information processing apparatus.

[0002] Programs and the like for associating a collateral in a tomographic image taken from inside a blood vessel using an image acquisition catheter with a collateral in a contrast image taken from outside the body have been proposed (Patent Document 1).

[0003] International Publication No. 2023 / 100838

[0004] According to Patent Document 1, it is possible to identify a tomographic image in which the same collateral as the collateral detected in the contrast image is photographed.

[0005] However, lesions such as thrombus or calcification can occur even in places where the collateral does not branch. In Patent Document 1, for places where the collateral does not branch, the place in the contrast image and the tomographic image in which the place is photographed cannot be associated.

[0006] In one aspect, an object is to provide an information processing method or the like capable of associating a place in a contrast image with a tomographic image in which the place is photographed.

[0007] The present invention is an information processing method in which a computer performs the following steps: (1) acquires a plurality of transverse layer images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially, and an angiographic image in which the image acquisition catheter is depicted; acquires the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image; determines the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image; calculates which frame of the plurality of transverse layer images corresponds to the first tomographic image in which the first landmark is depicted, assuming that three-dimensional scanning was performed up to the position of the first landmark; determines which frame of the plurality of transverse layer images corresponds to the second tomographic image in which the second landmark is depicted; and identifies the position of each transverse layer image by dividing the path into the number of frames from the first tomographic image to the second tomographic image.

[0008] Herein, in embodiments of the present invention, (2) the information processing method of (1) above is further preferably such that the first landmark is a tip marker provided on the image acquisition catheter.

[0009] (3) In the information processing method of (1) or (2) above, it is preferable that the second landmark is the tip of the guiding catheter through which the image acquisition catheter is inserted.

[0010] (4) The information processing method described in any one of (1) to (3) above preferably further displays the angiographic image and accepts a specification regarding the position of the first landmark and the position of the second landmark in the displayed angiographic image.

[0011] (5) The information processing method described in any one of (1) to (4) above further preferably generates a path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image, superimposes the generated path onto the angiographic image, and accepts instructions to modify the path.

[0012] (6) The information processing method described in any one of (1) to (5) above is further preferably carried out by dividing the path equally by the number of frames from the first tomographic image to the second tomographic image.

[0013] (7) The information processing method described in any one of (1) to (6) above is further preferably an OCT (Optical Coherence Tomography) catheter for image acquisition.

[0014] (8) The information processing method described in any one of (1) to (6) above is further preferably an IVUS (Intravascular Ultrasound) catheter for image acquisition.

[0015] (9) The information processing method described in any one of (1) to (6) above is further preferably a dual-sensor type in which the image acquisition catheter has both an OCT sensor and an IVUS sensor.

[0016] (10) Preferably, the program causes a computer to perform the following processes: (10) Acquire multiple transverse images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane in the axial direction, and an angiographic image in which the image acquisition catheter is depicted; acquire the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image; determine the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image; calculate which frame of the multiple transverse images corresponds to the first tomographic image in which the first landmark is depicted, assuming that three-dimensional scanning was performed up to the position of the first landmark; determine which frame of the multiple transverse images corresponds to the second tomographic image in which the second landmark is depicted; and divide the path by the number of frames from the first tomographic image to the second tomographic image to determine the position of each transverse image.

[0017] (11) Preferably, an information processing device having a control unit, wherein the control unit acquires a plurality of transverse layer images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane in the axial direction, and an angiographic image in which the image acquisition catheter is depicted, acquires the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image, determines the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image, calculates which frame of the plurality of transverse layer images corresponds to the first tomographic image in which the first landmark is depicted assuming that three-dimensional scanning was performed up to the position of the first landmark, determines which frame of the plurality of transverse layer images corresponds to the second tomographic image in which the second landmark is depicted, and identifies the position of each transverse layer image by dividing the path by the number of frames from the first tomographic image to the second tomographic image.

[0018] In one respect, this provides an information processing method that can associate a location in a contrast-enhanced image with a transverse layer image in which that location is captured.

[0019] This is an explanatory diagram illustrating the overview of the information processing system. This is an explanatory diagram illustrating the relationship between the angiographic image and each part of the image acquisition catheter. This is an explanatory diagram illustrating the relationship between the angiographic image and the transverse image taken by the image acquisition catheter. This is an example screen illustrating the operation procedure. This is an example screen illustrating the operation procedure. This is an example screen illustrating the operation procedure. This is an example screen illustrating the operation procedure. This is an explanatory diagram illustrating the procedure for estimating the position of the transverse image. This is an explanatory diagram illustrating the GC judgment model. This is a flowchart illustrating the processing flow of the program. This is a flowchart illustrating the processing flow of the subroutine for calculating the corresponding position. This is an example screen. This is an example screen. This is an explanatory diagram illustrating a schematic diagram regarding the positions of the first landmark marker, the second landmark marker, and the side branch marker. This is an explanatory diagram illustrating the correspondence between the angiographic image and the longitudinal image. This is a flowchart illustrating the processing flow of the subroutine for correspondence. This is an explanatory diagram illustrating the procedure for correspondence between the angiographic image of Embodiment 3 and the transverse image taken by the image acquisition catheter. This is an explanatory diagram illustrating the overview of the information processing system of Embodiment 4.

[0020] [Embodiment 1] Figure 1 is an explanatory diagram illustrating the overview of the information processing system 10. In this embodiment, the information processing system 10 used to support endovascular treatments such as PCI (Percutaneous Coronary Intervention) will be described as an example. In the following description, tasks performed by paramedical staff such as nurses or clinical engineers based on the instructions of a physician may also be described as tasks performed by a physician.

[0021] The information processing system 10 includes a catheter system 30, an angiography device 41, and an information processing device 20. The catheter system 30 includes a catheter control device 31, an image acquisition catheter 32, and an MDU (Motor Driving Unit) 33. The image acquisition catheter 32 is connected to the catheter control device 31 via the MDU 33.

[0022] Details of the configuration of the image acquisition catheter 32 will be described later. In the following description, the side of the image acquisition catheter 32 furthest from the MDU 33 will be referred to as the tip side. Catheter images such as transverse images 71 (see Figure 3) and longitudinal images 72 (see Figure 12) of the blood vessel are output from the catheter control device 31.

[0023] The angiography apparatus 41 comprises a C-arm 421 and an X-ray tube 422 and a flat-panel detector 423 fixed to both ends thereof. X-rays emitted from the X-ray tube 422 pass through the patient's body and are detected by the flat-panel detector 423. Various image processing is performed in real time on the detected X-rays inside the angiography apparatus 41, and real-time images are output sequentially. In principle, real-time image acquisition is performed continuously during endovascular treatment.

[0024] The physician administers a contrast agent into the patient's blood vessels from an angiography device (not shown in the diagram) as needed. As the contrast agent flows into the blood vessels within the field of view of the angiography device 41, a real-time image clearly depicting the course of the blood vessels is captured. The real-time image during contrast administration is referred to as the angiography image 61 (see Figure 2).

[0025] The information processing device 20 comprises a control unit 21, a main memory 22, an auxiliary memory 23, a communication unit 24, a display unit 25, an input unit 26, and a bus. The control unit 21 is an arithmetic control device that executes the program of this embodiment. One or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), or multi-core CPUs are used in the control unit 21. The control unit 21 is connected to each hardware component of the information processing device 20 via the bus.

[0026] The main memory 22 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory. The main memory 22 temporarily stores information necessary during processing performed by the control unit 21 and the program currently being executed by the control unit 21.

[0027] The auxiliary storage device 23 is a storage device such as SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 23 stores the GC (Guiding Catheter) determination model 55, the program to be executed by the control unit 21, and various data necessary for the execution of the program. The GC determination model 55 may be stored in an external mass storage device connected to the information processing device 20. The communication unit 24 is an interface for communication between the information processing device 20 and the network.

[0028] The display unit 25 is, for example, a liquid crystal display device or an organic EL (Electro-Luminescence) display device. The input unit 26 is, for example, an input device such as a keyboard, mouse, trackball, or microphone. The display unit 25 and the input unit 26 may be stacked together to form a touch panel.

[0029] The display unit 25 may be a connection interface that connects the information processing device 20 to an external display device. A communication unit 24 that connects data to an external display device via a network may realize the functions of the display unit 25 and the input unit 26.

[0030] The information processing device 20 in this embodiment is an information device such as a general-purpose personal computer, tablet, smartphone, or server computer. The information processing device 20 may also be a mainframe computer, a virtual machine operating on a mainframe computer, a cloud computing system, a quantum computer, or multiple personal computers performing distributed processing. The information processing device 20 may be configured integrally with, for example, a catheter control device 31, an angiography device 41, or a hospital information system (HIS) not shown in the figures.

[0031] In the following explanation, we will mainly use the case where the control unit 21 performs software-based processing as an example. The processes and models described using flowcharts may each be implemented by dedicated hardware.

[0032] Figure 2 is an explanatory diagram illustrating the relationship between the angiographic image 61 and the various parts of the image acquisition catheter 32. The physician inserts the guiding catheter 39 up to just before the treatment site. The guiding catheter 39 is a relatively thick tube and serves as a passage for the physician to safely insert the image acquisition catheter 32 and various treatment catheters, etc., up to the vicinity of the treatment site.

[0033] The left side of Figure 2 shows the tip of the image acquisition catheter 32 protruding from the tip of the guiding catheter 39. The image acquisition catheter 32 is either an OCT catheter or an IVUS catheter. The image acquisition catheter 32 may also be a dual-sensor type catheter that has the functions of both an OCT catheter and an IVUS catheter.

[0034] The image acquisition catheter 32 comprises a tip marker 321, a sensor 322, a shaft 323, and a sheath 324. The sheath 324 is a resin tube with its tip bent in a roughly crank shape. A guide wire insertion hole (not shown) is provided on the tip side of the bend.

[0035] A shaft 323 is inserted inside the sheath 324. A sensor 322 is fixed to the tip of the shaft 323. If the image acquisition catheter 32 is an OCT catheter, the sensor 322, which is an OCT sensor, is a ball lens that emits light from a light source and receives reflected light. If the image acquisition catheter 32 is an IVUS catheter, the sensor 322, which is an IVUS sensor, is an ultrasonic sensor that transmits and receives ultrasonic waves.

[0036] The tip marker 321 is fixed to the tip of the sheath 324. The tip marker 321 is made of a material with low X-ray transmittance, such as metal.

[0037] The explanation continues using Figures 1 and 2. The MDU 33 rotates the sensor 322 and shaft 323 inside the sheath 324. Radial scanning is performed as the sensor 322 rotates while transmitting and receiving light or ultrasound. The catheter control device 31 generates one image for each rotation of the sensor 322. The generated image is a transverse layer image 71 (see Figure 3) centered on the sensor 322 and approximately perpendicular to the sheath 324. In the following explanation, the series of processes from the transmission and reception of data by the sensor 322 to the generation of one transverse layer image 71 may be referred to as the acquisition of the transverse layer image 71.

[0038] The MDU 33 can also be moved forward and backward while rotating the sensor 322 and shaft 323 inside the sheath 324. By performing a pullback operation in which the sensor 322 is pulled toward the MDU 33 at a constant speed while being rotated, the catheter control device 31 realizes three-dimensional scanning in which multiple transverse layer images 71, which are substantially perpendicular to the sheath 324, are continuously captured at predetermined intervals.

[0039] In other words, the image acquisition catheter 32 is a so-called three-dimensional scanning catheter that acquires transverse images 71 while moving the scanning plane axially by radial scanning. In the following explanation, one transverse image 71 may be referred to as one frame. A series of transverse images 71 taken during the pullback operation are numbered sequentially from the first transverse image 71 as frame 1, frame 2, frame 3, and so on. The axial three-dimensional scanning range is the range from the first transverse image 71 taken in one pullback operation to the last transverse image 71.

[0040] The catheter control device 31 can also generate longitudinal images 72 (see Figure 12) based on multiple transverse images 71 acquired by three-dimensional scanning. The catheter control device 31 can also automatically measure parameters such as the inner diameter of the blood vessel or the diameter of the EEM (External Elastic Membrane). The catheter control device 31 can also generate transverse images 71 and longitudinal images 72 in which various regions such as plaque areas, calcified areas, or stent placement areas have been extracted and marked.

[0041] Since the automatic measurement by the catheter control device 31 and the automatic extraction of various regions have been conventionally used, detailed explanations thereof are omitted.

[0042] Since the functions and configuration of the catheter control device 31 are the same as those of the conventionally used OCT or IVUS image diagnostic devices, detailed explanations thereof are omitted. Note that the control unit 21 may realize the functions of the catheter control device 31.

[0043] An angiogram image 61 is shown on the right side of FIG. 2. In FIG. 2 and subsequent figures, regions with lower X-ray transmittance are indicated by finer hatching. Regions without hatching are regions with higher X-ray transmittance. In the regions indicated by the finer hatching, for example, the tip marker 321, the region where the blood vessel wall is calcified, and stents, etc. are depicted.

[0044] From the angiogram image 61, the positions of the first landmark 621 corresponding to the tip marker 321 and the second landmark 622 corresponding to the tip of the guiding catheter 39 are acquired. Thereafter, a path line 633 corresponding to the path through which the image acquisition catheter 32 passes between the first landmark 621 and the second landmark 622 is acquired. The method for acquiring the first landmark 621, the second landmark 622, and the path line 633 will be described later.

[0045] FIG. 3 is an explanatory diagram for explaining the relationship between the angiogram image 61 and the cross-sectional images taken by the image acquisition catheter 32. By one pull-back operation, a plurality of cross-sectional images 71 substantially perpendicular to the sheath 324 are taken as shown on the left side of FIG. 3. The interval ΔL between adjacent cross-sectional images 71 is the distance by which the sensor 322 is pulled back while the shaft 323 makes one rotation.

[0046] When viewed on the angiogram image 61, each cross-sectional image 71 is a plane perpendicular to the path line 633. By determining which cross-sectional image 71 each position on the path line 633 corresponds to, it is possible to associate the location in the angiogram image 61 with the cross-sectional image 71 in which that location was captured. That is, the control unit 21 can specify the position of each cross-sectional image 71 in the angiogram image 61. The following describes the specific procedure.

[0047] FIGS. 4 to 7 are examples of screens for explaining the operation procedure. After the user finishes capturing the cross-sectional image 71 by the pullback operation, the user operates the input unit 26 to display the screen shown in FIG. 4 on the display unit 25. The control unit 21 may detect the end of the pullback operation and automatically display the screen shown in FIG. 4 on the display unit 25.

[0048] In the screen shown in FIG. 4, a blood vessel image column 66 is arranged on the left side and a manual column 69 is arranged on the right side. A horizontal line and a first slider 671 are arranged below the blood vessel image column 66.

[0049] In the manual column 69, the operation procedure is displayed by an illustration and a simple explanatory text. In the lower center of the manual column 69, "page number / total number of pages" such as "1 / 4" is displayed. That is, FIG. 4 is a screen for accepting the operation of the first stage out of a total of four stages of work. A next button 682 is displayed on the lower right side of the manual column 69.

[0050] In the blood vessel image column 66, a blood vessel image captured in real time by the blood vessel imaging device 41 during or before and after the pullback operation is displayed. Here, the blood vessel image includes the angiogram image 61 captured when the contrast agent is flowing through the blood vessel and the non-angiogram image of the blood vessel captured when the contrast agent is not flowing through the blood vessel.

[0051] When the user moves the first slider 671 left and right, the blood vessel images displayed on the screen change in the order in which they were captured. The first stage is a stage for accepting the selection of the blood vessel images to be used in subsequent processing.

[0052] The user operates the first slider 671 to select an angiographic image 61 in which the contrast agent is flowing through the blood vessels at an appropriate concentration and the lesion is clearly depicted. The selection of the angiographic image 61 is confirmed when the user selects the next button 682. In subsequent processing, the angiographic image 61 selected by the user is used.

[0053] Let's move to Figure 5 and continue the explanation. As indicated by "2 / 4" in the lower center of the manual section 69, Figure 5 is the screen for accepting the second of four steps of the operation. A back button 683 is displayed on the lower left side of the manual section 69. If the back button 683 is selected, the control unit 21 returns to the previous step, that is, the screen explained using Figure 4.

[0054] The vascular image section 66 displays the angiographic image 61 selected in the first stage. The user operates the cursor 619 to specify the position of the second landmark 622 corresponding to the tip of the guiding catheter 39. Specifically, the user places the cursor 619 at the desired position and performs an operation such as clicking. After that, the user selects the next button 682. The control unit 21 accepts the selection of the next button 682, and the position of the second landmark 622 is confirmed.

[0055] When the control unit 21 receives an operation such as a click, it is desirable to display a circular second landmark marker 632 (see Figure 6) centered on the position of the second landmark 622. The user can confirm whether or not the second landmark 622 has been designated to the desired position by visually observing the second landmark marker 632.

[0056] In Figure 5, an enlarged area 615 is displayed around the cursor 619, which magnifies a portion of the angiographic image 61 by approximately 1.5 to 2 times in both the vertical and horizontal directions. By using the enlarged area 615, the user can precisely specify the position of the second landmark 622.

[0057] The control unit 21 may use a known algorithm, such as a Voronoi diagram, to extract the location with the lowest X-ray transmittance near the user-specified location and use it as the location for the second landmark 622.

[0058] Although not shown in the diagram, in the third of the four steps, the user operates the cursor 619 to specify the position of the first landmark 621 corresponding to the tip marker 321. As a result, the control unit 21 completes acquiring the positions of the first landmark 621 and the second landmark 622. Note that the order of the second and third steps may be reversed.

[0059] Let's move to Figure 6 and continue the explanation. As indicated by "4 / 4" in the lower center of the manual section 69, Figure 6 is the screen for accepting the fourth and final stage of the four-stage process. Note that a back button 683 is displayed on the lower left of the manual section 69, and a confirm button 684 is displayed on the lower right.

[0060] The control unit 21 automatically generates the path of the image acquisition catheter 32 between the first landmark 621 and the second landmark 622 acquired in the second and third stages. The control unit 21 displays the automatically generated path line 633 in the vascular image field 66.

[0061] The control unit 21 generates a path line 633 that represents the shortest time to connect two points, for example, from the second landmark 622 to the first landmark 621, when the contrast-enhanced image is considered as a velocity field corresponding to the contrast intensity. The control unit 21 may also automatically generate the path line 633 based on any other algorithm.

[0062] The user visually inspects the path line 633 to determine whether the path for the image acquisition catheter 32 has been generated appropriately. If it is determined that it is not appropriate, the user modifies the path line 633 by adding points that the path line 633 passes through, as described in manual section 69. Figure 7 shows an example of the modified path line 633. Since the method of modifying the curve by adding points has been used conventionally, a detailed explanation is omitted.

[0063] Subsequently, the user selects the confirmation button 684. The control unit 21 accepts the selection of the confirmation button 684, and the route line 633 is confirmed. The control unit 21 then performs subsequent processing based on the confirmed first landmark 621, second landmark 622, and route line 633.

[0064] Figure 8 is an explanatory diagram illustrating the procedure for estimating the position of the transverse image 71. For the sake of simplicity, the start and end points of each range along the longitudinal direction of the image acquisition catheter 32 are set midway between the two transverse images 71.

[0065] The upper part of Figure 8 shows the image acquisition catheter 32 in the pullback operation start position, i.e., when the sensor 322 is at its furthest tip. The image acquisition catheter 32 captures transverse layer images 71 within the scanning range. As explained using Figure 3, the interval between transverse layer images 71 is ΔL, and the total number of transverse layer images 71 actually captured N1 is calculated by equation (1). N1 = L1 / ΔL ... (1) L1 is the length of the scanning range.

[0066] Beyond the pullback start position, the transverse layer image 71 is not captured. In the following explanation, the area between the pullback start position and the tip marker 321 is referred to as the non-scanning range. Assuming that transverse layer images 71 can be captured at intervals of ΔL even in the non-scanning range, the number N2 of transverse layer images 71 virtually captured in the non-scanning range is calculated by equation (2). N2 = L2 / ΔL ... (2) L2 is the length of the non-scanning range.

[0067] Here, the length L2 of the non-scanning range is determined by the specifications of the image acquisition catheter 32. Therefore, if the effects of individual differences are to be considered, the user may measure and input the length of the non-scanning range. As mentioned above, when the first transverse image 71 taken during the pullback operation is counted as the first frame, the virtual transverse images 71 within the non-scanning range are counted sequentially from the proximal end of the non-scanning range as frame 0, frame -1, frame -2, and so on. The transverse image 71 at the very tip of the non-scanning range corresponds to the (N2-1)th frame.

[0068] In the following explanation, the first tomographic image will refer to the hypothetical transverse image 71 at the very tip of the non-scanning range, that is, the image in which the tip marker 321 should be depicted if a transverse image 71 were actually obtainable. Similarly, the second tomographic image will refer to the transverse image 71 at the very tip of the GC range, that is, the image in which the tip of the guiding catheter 39 is depicted. The first tomographic image is the transverse image 71 corresponding to the position of the first landmark 621 in the transverse image 71, and the second tomographic image is the transverse image 71 corresponding to the position of the second landmark 622 in the transverse image 71. The number of transverse images 71 from the first to the second tomographic image will be referred to as the number of frames from the first to the second tomographic image.

[0069] In this embodiment, the tip of the guiding catheter 39 is located within the operating range. That is, near the end of the pullback operation, the transverse image 71 is captured from inside the guiding catheter 39. Since the guiding catheter 39 does not easily transmit light or ultrasound, the transverse image 71 captured from inside the guiding catheter 39 does not have an image quality of a usable level. Therefore, it is possible to determine whether the transverse image 71 was captured from inside the guiding catheter 39 or from outside the guiding catheter 39 based on the transverse image 71 itself. A specific example of the determination method will be described later.

[0070] If there are N3 transverse images 71 taken from inside the guiding catheter 39, the length L3 of the portion of the scanning range that is inside the guiding catheter 39 is calculated by equation (3): L3 = N3 × ΔL (3)

[0071] The imaging range from the pullback start position to the tip of the guiding catheter 39 is the range from the start of the pullback operation to the tip of the guiding catheter 39. The length L4 of the imaging range and the number N4 of transverse images 71 within the imaging range are calculated using equations (4) and (5). L4 = L1 - L3 ... (4) N4 = N1 - N3 ... (5)

[0072] A schematic representation of a linearly extended path line 633 is shown at the bottom of Figure 8. The length of the path line 633 in the angiographic image 61, i.e., the length between the first landmark 621 and the second landmark 622, is denoted as L5. L5 corresponds to the combined range of the non-scanning area L2 and the imaging area L4.

[0073] Therefore, the positions obtained by dividing the path line 633 into (N2:N4) on the angiographic image 61 correspond to the boundary LB between the scanned range and the non-scanned range. As explained using Figure 8, the boundary LB corresponds to the start position of the pullback operation of the sensor 322, that is, the position when the sensor 322 is at its furthest tip. Therefore, for any angiographic image 61 selected by the user, the control unit 21 can estimate the start position of the pullback operation of the sensor 322 on the path line 633. The positions obtained by equally dividing the distance from the second landmark 622 to the boundary LB into N4 parts along the path line 633 correspond to the center position of each transverse layer image 71. The normal of each transverse layer image 71 is parallel to the path line 633. Thus, the position of the transverse layer image 71 in the angiographic image 61 is estimated.

[0074] This division is equivalent to dividing the entire path line 633 into (N2 + N4) equal parts, that is, dividing the path line 633 equally by the number of frames from the first tomographic image corresponding to the tip marker 321 to the second tomographic image corresponding to the tip of the guiding catheter 39. However, in practice, computational costs can be saved by omitting the division for the portion corresponding to the unscanned area where the transverse layer image 71 has not been acquired.

[0075] Figure 9 is an explanatory diagram illustrating the GC determination model 55. The GC determination model 55 is a model that receives a transverse image 71 and outputs whether the transverse image 71 was taken from inside the guiding catheter 39 or from outside the guiding catheter 39.

[0076] The GC judgment model 55 is a program that determines, for example, whether the inner surface of the guiding catheter 39 is depicted in the transverse layer image 71, using methods such as pattern recognition. The GC judgment model 55 may also be a learning model generated by supervised learning using training data recorded by associating the transverse layer image 71 with a flag indicating whether it was taken from inside or outside the guiding catheter 39.

[0077] The GC determination model 55 may output whether the transverse image 71 was taken from inside the guiding catheter 39 or from outside the guiding catheter 39. For example, as illustrated in Figure 9, if the probability of the image being taken from outside the guiding catheter 39 is greater than the probability of it being taken from inside the guiding catheter 39, the GC determination model 55 may be configured to output the conclusion that the image was taken from outside the guiding catheter 39.

[0078] Figure 10 is a flowchart illustrating the program's processing flow. The program shown in Figure 10 is activated after the pullback operation is completed, either based on user instructions or automatically. The control unit 21 acquires the catheter image taken in a single pullback from the catheter control device 31 (step S501). As mentioned above, the catheter image includes a transverse image 71 and a longitudinal image 72. The catheter image may also include a three-dimensional model constructed based on multiple transverse images 71.

[0079] The control unit 21 acquires vascular images captured in real time from the angiography device 41 during and immediately before and after the pullback operation (step S502). The control unit 21 displays the screen described using Figure 4 on the display unit 25 (step S503). The control unit 21 accepts the selection of angiography image 61 via the operation of the first slider 671 and the next button 682 (step S504).

[0080] The control unit 21 displays the screen described using Figure 5 on the display unit 25 and accepts the selection of the first landmark 621 and the second landmark 622 (step S505). As described using Figure 6, the control unit 21 generates and displays a route line 633 connecting the first landmark 621 and the second landmark 622 (step S506).

[0081] As explained using Figure 7, the control unit 21 receives a modification instruction from the user regarding the route line 633 (step S507). Upon receiving the modification instruction, the control unit 21 generates the modified route line 633. The control unit 21 then starts a subroutine for calculating corresponding positions (step S508). The subroutine for calculating corresponding positions calculates which point on the first landmark marker 631 corresponds to the center of each cross-sectional layer image 71. The processing flow of the subroutine for calculating corresponding positions will be described later.

[0082] The control unit 21 displays the relationship between the angiography image 61 and the transverse and longitudinal images 72 based on the calculation results of the corresponding position calculation subroutine (step S509). A specific example of the display screen will be described later.

[0083] Figure 11 is a flowchart illustrating the processing flow of the subroutine for calculating corresponding positions. The subroutine for calculating corresponding positions calculates which point on the first landmark marker 631 corresponds to the center of each cross-sectional image 71.

[0084] The control unit 21 extracts the last transverse image 71 taken from among the transverse images 71 taken by the pullback operation (step S521). The control unit 21 inputs the transverse image 71 to the GC determination model 55 and obtains the output of the GC determination model 55. Based on the obtained output, the control unit 21 determines whether or not the transverse image 71 being processed was taken from inside the guiding catheter 39 (step S522).

[0085] If it is determined that the image was taken from inside the guiding catheter 39 (YES in step S522), the control unit 21 extracts the previous image, i.e., the transverse layer image 71 from the tip side (step S523). The control unit 21 returns to step S522.

[0086] If it is determined that the image was taken from outside the guiding catheter 39 (NO in step S522), the control unit 21 calculates N4, which is the number of transverse images 71 taken within the imaging range (step S524). Specifically, the control unit 21 calculates N4 by subtracting the number of transverse images 71 taken inside the guiding catheter 39, N3, from N1, which is the number of transverse images 71 taken in one pullback operation, based on equation (5).

[0087] The control unit 21 calculates the number N2 of transverse layer images 71 virtually captured in the non-scanning range based on equation (2) (step S525). The control unit 21 calculates the length of the path line 633 (step S526). Since the method for calculating the length of a curve drawn on the image is well known, a detailed explanation is omitted.

[0088] The control unit 21 calculates the position of the boundary LB that divides the length of the path line 633 calculated in step S526 into N4:N2 (step S527). N4 is on the side of the second landmark 622, and L2 is on the side of the first landmark 621. Since the method for calculating the position of dividing the length of the curve in a predetermined ratio is well known, a detailed explanation is omitted.

[0089] The control unit 21 associates the center of each transverse layer image 71 within the imaging range with the corresponding position on the path line 633 (step S528). Specifically, the control unit 21 calculates the positions that evenly divide the area from the second landmark 622 to the boundary LB into N4 parts along the path line 633. The control unit 21 sequentially associates and records the calculated positions with the transverse layer images 71. After that, the control unit 21 terminates the process.

[0090] Figure 12 shows an example screen. The angiography image 61 selected by the user in step S504 of the program described using Figure 10 is displayed from the center to the upper left of the screen. Various indicators superimposed on the angiography image 61 will be described later. However, the side branch marker 635, shown by the dashed line, will not be described in this embodiment.

[0091] The OCT tomography image 711 is displayed in the upper right of the screen. The IVUS tomography image 712 is displayed below the OCT tomography image 711. The OCT tomography image 711 and the IVUS tomography image 712 are examples of transverse images 71. If the image acquisition catheter 32 is not a catheter for a dual sensor system, either the OCT tomography image 711 or the IVUS tomography image 712, or any one of the transverse images 71, will be displayed on the screen.

[0092] The control unit 21 may also superimpose indicators showing the shape of, for example, the vascular lumen, EEM, plaque area, or calcified area onto the OCT tomographic image 711 and the IVUS tomographic image 712.

[0093] A longitudinal section image 72 is displayed at the bottom of the screen. The left side is the tip of the image acquisition catheter 32. The hatched area is the area where a certain amount of plaque is present or greater. Two plaque areas are depicted in the longitudinal section image 72. A thick horizontal line, the first area marker 641, is displayed above the plaque area on the tip side. The first area marker 641 indicates the extent of the calcified area.

[0094] Based on the corresponding position calculated in step S528 as described using Figure 11, the control unit 21 displays a first region marker 641 indicating the location of the transverse image 71 including the calcified region on the outside of the blood vessel through which the image acquisition catheter 32 has passed. The user can understand the status of the calcified region based on the first region marker 641 in the longitudinal image 72 and the first region marker 641 in the angiography image 61.

[0095] In the longitudinal tomographic image 72, a second slider 672 with a circle above the vertical line is displayed between two plaque regions. When the user operates the input unit 26 to move the second slider 672 left or right, the control unit 21 displays a transverse image marker 636 at the corresponding location on the angiographic image 61. The control unit 21 displays the transverse image 71 corresponding to the position of the second slider 672 on the right side of the angiographic image 61.

[0096] By sliding the second slider 672, the user can confirm which position on the longitudinal tomographic image 72 corresponds to which position on the angiographic image 61, and can also refer to the transverse tomographic image 71 at that position.

[0097] In the longitudinal section image 72, distal markers 638 and proximal markers 639 are shown outside the two plaque regions. The distal marker 638 has the letter "D" indicating distal at the lower right end of a vertical bar. The proximal marker 639 has the letter "P" indicating proximal at the lower left end of a vertical bar.

[0098] When the user operates the input unit 26 to move the tip marker 638 on the longitudinal tomographic image 72 left or right, the control unit 21 displays a roughly U-shaped tip marker 638 at the corresponding position in the angiographic image 61. Similarly, when the user operates the input unit 26 to move the proximal marker 639 on the longitudinal tomographic image 72 left or right, the control unit 21 displays the proximal marker 639 at the corresponding position in the angiographic image 61.

[0099] The control unit 21 displays the distance between the tip marker 638 and the proximal marker 639 at the bottom of the longitudinal tomographic image 72. The control unit 21 may also display values ​​such as the longitudinal and transverse inner diameter of the blood vessel at the position corresponding to the tip marker 638 on the lower left side of the tip marker 638. Similarly, the control unit 21 may also display values ​​such as the longitudinal and transverse inner diameter of the blood vessel at the position corresponding to the proximal marker 639 on the lower right side of the proximal marker 639.

[0100] The control unit 21 may accept user input on the angiographic image 61, such as the operation of the transverse tomographic marker 636, the proximal marker 638, and the proximal marker 639. If so, the control unit 21 displays the second slider 672, the proximal marker 638, and the proximal marker 639 at the corresponding positions on the longitudinal tomographic image 72.

[0101] For example, before implanting a stent, the user uses a screen like the one illustrated in Figure 12 to examine the condition of the blood vessel and determine the specifications and implantation location of the stent to be implanted.

[0102] Figure 13 is an example screen. Figure 13 shows an example screen displaying data after a pullback operation has been performed following stent placement. Explanations of aspects common to Figure 12 are omitted.

[0103] In the longitudinal section image 72, the longitudinal section of the stent is depicted as a dashed line between the leading edge marker 638 and the proximal end marker 639. Second region markers 642 are displayed at three locations inside the stent. The second region markers 642 are markers displayed at locations where, for example, stent expansion failure or stent malposition occurs. The second region markers 642 may also be markers automatically generated by the control unit 21 based on predetermined criteria.

[0104] The control unit 21 displays a second region marker 642 at the corresponding location in the angiographic image 61. It is desirable that the control unit 21 use a different color for the second region marker 642 in the longitudinal tomographic image 72 and the angiographic image 61 than for other indicators, such as the first region marker 641. This allows the user to easily determine the correspondence between indicators based on color.

[0105] According to this embodiment, it is possible to provide an information processing method that can associate a location in a contrast-enhanced image with a transverse image in which that location is captured. Users can easily understand the correspondence between the catheter image, which includes the transverse image 71 and the longitudinal image 72, and the angiography image 61, thereby enabling them to smoothly perform treatment procedures such as PCI.

[0106] [Embodiment 2] This embodiment relates to an information processing method for determining the spacing between transverse images 71 in an angiographic image 61 based on the position where the side branches of a blood vessel diverge. Parts common to Embodiment 1 will not be explained.

[0107] The control unit 21 uses any method, such as the method disclosed in Patent Document 1, to detect the locations where side branches diverge from the blood vessel through which the pathway line 633 has passed, for both the catheter image and the angiography image 61. The side branch markers 635 shown by dashed lines in the angiography image 61 in Figures 12 and 13 are examples of markers that indicate the locations where side branches have been detected in the angiography image 61. The control unit 21 may also accept user specifications for the locations where side branches diverge.

[0108] The control unit 21 may, for example, display a side branch marker 635 on the angiographic image 61 indicating the location where the side branch diverges, as shown in the screen examples in Figures 12 and 13. Similarly, the control unit 21 may display a marker on the longitudinal tomographic image 72 indicating the location where the side branch diverges.

[0109] Figure 14 is an explanatory diagram illustrating a schematic diagram of the positions of the first landmark marker 631, the second landmark marker 632, and the lateral branch marker 635.

[0110] Figure 14A shows the image acquisition catheter 32 in the pullback operation start position, i.e., when the sensor 322 is at its furthest tip. The positions of the side branches detected from the catheter image are indicated by arrows extending upward from the image acquisition catheter 32. Side branching is detected at two locations within the imaging range.

[0111] This state is schematically represented by a single horizontal line and a total of four vertical lines extending from positions corresponding to the branching of both ends and the side branches. The vertical lines at both ends correspond to the tip marker 321 and the tip of the guiding catheter 39. The two vertical lines in the center correspond to the positions where the side branches diverge. In the following explanation, the schematic diagram in Figure 14A may be referred to as a schematic diagram based on a catheter image.

[0112] Figure 14B schematically shows a linearly extended pathway line 633. The locations where the collateral branches detected from the angiographic image 61 branch off are indicated by arrows extending upward from the vessel through which the pathway line 633 passes. Collateral branching is detected at three locations within the range of the pathway line 633.

[0113] This state is schematically represented by a single horizontal line and a total of five vertical lines extending from the ends and the positions corresponding to the branching. The vertical lines at the ends correspond to the first landmark 621 and the second landmark 622. The three vertical lines in the center correspond to the positions where the side branches diverge. In the following explanation, the schematic diagram in Figure 14B may be referred to as a schematic diagram based on the angiographic image 61.

[0114] Figure 15 is an explanatory diagram illustrating the correspondence between angiographic images 61 and longitudinal tomographic images 72. Figure 15A is a diagram in which a schematic diagram based on catheter images and a schematic diagram based on angiographic images 61 are placed side by side. In the schematic diagram based on catheter images, the unit of length is the number of frames in the catheter images, while in the schematic diagram based on angiographic images 61, the unit of length is the number of pixels in the angiographic images 61. Thus, in Figure 15A, lengths based on different modalities are being compared, resulting in a difference between the two lengths.

[0115] In the schematic diagram based on the catheter image, the right end corresponds to the tip marker 321, and the left end corresponds to the tip of the guiding catheter 39. The positions where the side branches diverge are indicated by the labels B1 and B2, respectively, starting from the tip marker 321.

[0116] Let's take the example of a case where one transverse image 71 is taken between the tip marker 321 and B1, two transverse images 71 are taken between B1 and B2, and three transverse images 71 are taken between B2 and the tip of the guiding catheter 39. In the catheter image, the spacing of the transverse images 71 is the same in all parts.

[0117] The right end of the schematic diagram based on the angiographic image 61 corresponds to the first landmark 621, and the left end corresponds to the second landmark 622. The locations where the collateral branches diverge are indicated by the symbols C1, C2, and C3, respectively, starting from the first landmark 621.

[0118] Figure 15B shows a schematic diagram based on a catheter image and a schematic diagram based on angiography image 61 with their lengths aligned. The control unit 21 selects two straight lines such that the sum of the length of the straight line connecting B1 to any one of C1 to C3 and the length of the straight line connecting B2 to any one of C1 to C3 is minimized.

[0119] Figure 15C shows the result of selecting a straight line. As indicated by the dashed line, the straight line connecting B1 and C2, and the straight line connecting B2 and C3 have been selected. This means that B1 and C2, and B2 and C3, are associated with the same branch.

[0120] The control unit 21 determines the positions of the cross-sectional images 71 such that, within the path line 633, one R-size cross-sectional image 71 is placed at equal intervals in the portion between the first landmark 621 and C2, two R-size cross-sectional images 71 are placed in the portion between C2 and C3, and three R-size cross-sectional images 71 are placed in the portion between C3 and the second landmark 622.

[0121] Based on the above, the spacing between transverse images 71 in the angiographic image 61 can be determined based on the location where the side branches of the blood vessels diverge. For example, the larger the angle between the course of the blood vessel and the flat-panel detector 423, the shorter the length of the blood vessel in the angiographic image 61 will appear than its actual length. In such locations, the spacing between the transverse images 71 is calculated to be narrower, thereby improving the accuracy of the correspondence between the angiographic image 61 and the catheter image.

[0122] Figure 16 is a flowchart illustrating the processing flow of the mapping subroutine. The mapping subroutine is executed in place of step S528 of the mapping position calculation subroutine described using Figure 11. The mapping subroutine calculates and maps the intervals of the transverse layer images 71 in the path line 633 based on the locations where the side branches diverge, as described using Figure 15.

[0123] The control unit 21 determines the location where the side branch diverges in the catheter image (step S541). The control unit 21 also determines the location where the side branch diverges in the angiography image 61 (step S542). As described above, the method for determining the location where the side branch diverges in the catheter image and the angiography image 61 is well known, so a detailed explanation will be omitted.

[0124] As explained using Figure 15, the control unit 21 determines the combination of the branching position in the catheter image and the branching position in the angiography image 61 (step S543). The control unit 21 then performs subsequent processing for each range delimited by the combined branching.

[0125] The control unit 21 obtains the number of transverse images 71 included in one range based on the catheter image (step S544). The control unit 21 calculates the length of the pathway lines 633 included in one range based on the angiography image 61 (step S545).

[0126] The control unit 21 associates the center of each cross-sectional image 71 within the processing range with its corresponding position on the path line 633 (step S546). Specifically, the control unit 21 calculates the positions where the cross-sectional images 71 are arranged at equal intervals along the path line 633 within the processing range. The control unit 21 sequentially associates and records the calculated positions with the cross-sectional images 71.

[0127] The control unit 21 determines whether or not the processing for the entire range has been completed (step S547). If it determines that it has not been completed (NO in step S547), the control unit 21 returns to step S544. If it determines that it has been completed (YES in step S547), the control unit 21 terminates the processing.

[0128] According to this embodiment, the spacing between transverse images 71 in the angiographic image 61 can be determined based on the position where the side branches of the blood vessels diverge. Even when the blood vessels are significantly tortuous in three dimensions, this provides an information processing method that can accurately correlate catheter images and angiographic images 61.

[0129] [Embodiment 3] This embodiment relates to an information processing method that uses an approximation formula to associate the distance along the path line 633 in the angiography image 61 with the distance in the catheter image. The parts that are common with Embodiment 2 will not be explained.

[0130] Figure 17 is an explanatory diagram illustrating the procedure for correlating angiographic images 61 in Embodiment 3 with transverse images 71 taken with the image acquisition catheter 32. The explanation will be given using Figure 17A. The horizontal axis shows the distance from the second landmark 622 along the path line 633. The vertical axis shows the distance from the tip of the guiding catheter 39 along the image acquisition catheter 32.

[0131] The black diamond indicates the tip position of the guiding catheter 39, i.e., the position of the second landmark 622. The white diamond indicates the position of the tip marker 321, i.e., the position of the first landmark 621. In subsequent processing, the black diamond will be used as the origin. The straight line connecting the black diamond and the white diamond is shown as a dashed line.

[0132] Figure 17 illustrates an example where the blood vessel branches at five points in the angiographic image 61 and at three points in the catheter image. The black and white circles together show all possible combinations of branching in the angiographic image 61 and branching in the catheter image.

[0133] However, the possibility of a branch on the angiographic image 61 being associated with a branch on the catheter image in a combination far removed from the dashed line, as shown by the symbol PA, is extremely low. This is because if such a combination were to be associated, appropriate associations would not be possible for other branches.

[0134] In Figure 17, combinations where the possibility of correspondence is considered are shown with black circles, and combinations where the possibility of correspondence is ignored are shown with white circles. The lengths in the angiographic images shown on the horizontal axis and the lengths in the catheter images shown on the vertical axis are both normalized to a total length of 1.

[0135] In the catheter image, the branch closest to the tip of the guiding catheter 39 may correspond to two of the branches detected in the angiography image 61. Similarly, in the catheter image, the central branch may correspond to three of the branches detected in the angiography image 61.

[0136] The control unit 21 calculates an approximate formula for each branch in the catheter image that corresponds to one of the branches detected in the angiography image 61, using a known approximation method such as the least squares method. The control unit 21 calculates the cost of the calculated approximation formula based on equation (6).

[0137] C is the cost. R is the number of branches in the catheter image. ri is the residual. p is the number of branches that are not associated with the branches detected in the catheter image. w is a value indicating the weighting of p.

[0138] The weighting parameter w is explained below. w is between 0 and 1. When it is important that the branch detected in the catheter image corresponds to the branch detected in the angiography image 61, a value close to 1 is set for w. When it is important that the position of the branch detected in the catheter image is close to the position of the branch detected in the angiography image 61, a small value such as 0.01 is set for w.

[0139] The value of w is determined, for example, by a preliminary study using the catheter system 30 and angiography device 41 that will actually be used. The higher the accuracy of detecting side branches in the catheter image and angiography image 61, the smaller the appropriate value of w will be.

[0140] Figures 17B and 17C show an example of a combination for calculating the cost. For each of the three branches in the catheter image, one of the branches in the angiography image 61 is indicated by a black circle. The control unit 21 linearly approximates the coordinates indicated by the black circles with a linear function y = ax + b.

[0141] The control unit 21 calculates the residual ri, which is the distance from the approximation formula, for each black circle. The control unit 21 calculates the cost C based on equation (6). Note that R is 3 and p is 0. The control unit 21 extracts the combination with the smallest cost C.

[0142] The approximation formula calculated for the extracted combinations correlates the distance from the second landmark 622 along the path line 633 with the distance from the tip of the guiding catheter 39 along the image acquisition catheter 32. Specifically, in the approximation formula y = ax + b, x is the distance from the second landmark 622 along the path line 633, and y is the distance from the tip of the guiding catheter 39 along the image acquisition catheter 32.

[0143] In Figure 17, an example of approximation using a linear function is shown, but the approximation formula may also be a quadratic function, cubic function, or the like.

[0144] [Modification 1] When a material with low X-ray transmittance is used for the sensor 322, or when a material with low X-ray transmittance is placed near the sensor 322, the sensor 322 is depicted on the angiographic image 61. This modification relates to the information processing system 10 when the sensor 322 is depicted on the angiographic image 61. The parts common to Embodiment 3 will not be explained.

[0145] In the first step of the operation procedure described using Figures 4 to 7, after receiving the selection of angiography image 61, the control unit 21 determines whether the sensor 322 is located near the pullback start position in the selected angiography image 61.

[0146] Specifically, the control unit 21 determines that the sensor 322 is located near the start position of the pullback operation when the time when the selected angiography image 61 was taken and the time when the pullback operation started are approximately the same. The control unit 21 may also determine that the sensor 322 is located near the start position of the pullback operation when the difference between the selected angiography image 61 and the angiography image 61 taken immediately before the time when the pullback operation started is small.

[0147] If the control unit determines that the sensor 322 is located near the pullback start position in the selected angiographic image 61, in the third step of the operation procedure, the control unit 21 displays a message in the manual section 69, for example, "Click the sensor." The control unit 21 accepts the position of the landmark corresponding to the sensor 322 instead of the first landmark 621 corresponding to the tip marker 321. In the following description, this landmark will be referred to as the third landmark. The third landmark corresponds to the furthest tip of the imaging range L4.

[0148] The control unit 21 may use a known algorithm, such as a Voronoi diagram, to extract the location with the lowest X-ray transmittance near the user-specified location and use it as the location of the third landmark. The control unit 21 may also precisely determine the location of the boundary LB, as explained using Figure 8, on the angiographic image 61 by pattern matching between the X-ray image near the user-specified location and the shape of the sensor 322 stored for each model of the image acquisition catheter 32.

[0149] The control unit 21 determines a path line 633 between the second landmark 622 received in the second stage and the third landmark received in the third stage. The length L5 of the path line 633 in the angiography image 61 corresponds to the imaging range L4 of the image acquisition catheter 32.

[0150] According to this modified example, the range on the angiographic image 61 corresponding to the imaging range L4 coincides with the path line 633, so the distance along the path line 633 can be accurately correlated with the distance in the catheter image.

[0151] If the control unit 21 determines that the sensor 322 is depicted at the pullback start position on the angiographic image 61 selected in the first stage, it may accept user instructions regarding whether to specify the first landmark 621 or the third landmark in the third stage. For example, the control unit 21 displays a button in the manual field 69 of the third stage to accept the user's selection of whether to specify the first landmark 621 or the third landmark. The user can then visually inspect the angiographic image 61 and appropriately decide whether to specify the first landmark 621 or the third landmark.

[0152] The control unit 21 does not determine whether the sensor 322 is located near the pullback start position in the angiography image 61 selected in the first stage, and may accept user instructions in the third stage regarding whether to specify the first landmark 621 or the third landmark. In the first stage, the user selects the angiography image 61 in which the sensor 322 is located near the pullback start position, and in the third stage, selects the third landmark. Because the processing steps by the control unit 21 are few, an information processing system 10 that can be easily operated by an experienced user can be provided.

[0153] [Modification 2] In Embodiment 2, as in Modification 1, the control unit 21 may accept the designation of a third landmark instead of the first landmark 621. Since no calculations regarding the non-scanning range L2 are required, the processing speed is fast, and an information processing system 10 that accurately determines the transverse layer image 71 in the angiography image 61 can be provided.

[0154] [Embodiment 4] This embodiment relates to a configuration in which an information processing system 10 is realized by operating a general-purpose computer 90 and a program 97 in combination. The parts that are common with Embodiment 1 will not be described.

[0155] Figure 18 is an explanatory diagram illustrating the outline of the information processing system 10 of Embodiment 4. The computer 90 includes the control unit 21, main memory 22, auxiliary memory 23, communication unit 24, display unit 25, input unit 26, and bus mentioned above, as well as a read unit 29.

[0156] The program 97 is recorded on a portable recording medium 96. The control unit 21 reads the program 97 via the reading unit 29 and saves it to the auxiliary storage device 23. The control unit 21 may also read the program 97 stored in a semiconductor memory 98, such as a flash memory, installed in the computer 90. Furthermore, the control unit 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown) and save it to the auxiliary storage device 23.

[0157] Program 97 is installed as a control program for the computer 90, loaded into the main memory 22, and executed. Thus, the information processing system 10 described in Embodiment 1 is realized. Program 97 in this embodiment is an example of a program product.

[0158] Program 97 may be provided on a recording medium or distributed from an external computer. The computer program can be deployed to run on a single computer or at a single site, or distributed across multiple sites and interconnected by a communication network.

[0159] The technical features (constituent elements) described in each embodiment are combinable with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0160] The independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. Furthermore, while the claims use a multi-claim format in which claims refer to two or more other claims (multi-claim format), this is not the only option. A multi-claim format in which at least one multi-claim is referenced (multi-multi-claim format) is also acceptable.

[0161] 10 Information Processing System 20 Information Processing Device 21 Control Unit 22 Main Memory Unit 23 Auxiliary Memory Unit 24 Communication Unit 25 Display Unit 26 Input Unit 29 Reading Unit 30 Catheter System 31 Catheter Control Device 32 Image Acquisition Catheter 321 Tip Marker 322 Sensor 323 Shaft 324 Sheath 33 MDU 39 Guiding Catheter 41 Angiography Device 421 C-Arm 422 X-ray Tube 423 Planar Detector 55 GC Judgment Model 61 Angiographic Image 615 Magnified Area 619 Cursor 621 First Landmark 622 Second Landmark 631 First Landmark Marker 632 Second Landmark Marker 633 Path Line 635 Side Branch Marker 636 Transverse Layer Image Marker 638 Tip Marker 639 Proximal marker 641 First region marker 642 Second region marker 66 Vascular image section 671 First slider 672 Second slider 682 Next button 683 Back button 684 Confirm button 69 Manual section 71 Transverse image 711 OCT tomography 712 IVUS tomography 72 Longitudinal image 90 Computer 96 Portable recording medium 97 Program 98 Semiconductor memory

Claims

1. An information processing method in which a computer performs the following steps:

1. Acquire multiple transverse images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially, and an angiographic image in which the image acquisition catheter is depicted; acquire the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image; determine the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image; calculate which frame of the multiple transverse images corresponds to the first tomographic image in which the first landmark is depicted, assuming that three-dimensional scanning was performed up to the position of the first landmark; determine which frame of the multiple transverse images corresponds to the second tomographic image in which the second landmark is depicted; and identify the position of each transverse image by dividing the path by the number of frames from the first tomographic image to the second tomographic image.

2. The information processing method according to claim 1, wherein the first landmark is a tip marker provided on the image acquisition catheter.

3. The information processing method according to claim 1, wherein the second landmark is the tip of the guiding catheter through which the image acquisition catheter is inserted.

4. The information processing method according to claim 1, which displays the angiographic image and receives a specification regarding the position of the first landmark and the position of the second landmark in the displayed angiographic image.

5. The information processing method according to claim 1, comprising generating a path for the image acquisition catheter from the first landmark to the second landmark in the angiographic image, superimposing the generated path onto the angiographic image, and receiving instructions to modify the path.

6. The information processing method according to claim 1, wherein the division of the path is carried out by dividing the path equally by the number of frames from the first tomographic image to the second tomographic image.

7. The information processing method according to any one of claims 1 to 6, wherein the image acquisition catheter is an OCT (Optical Coherence Tomography) catheter.

8. The information processing method according to any one of claims 1 to 6, wherein the image acquisition catheter is an IVUS (Intravascular Ultrasound) catheter.

9. The information processing method according to any one of claims 1 to 6, wherein the image acquisition catheter is a dual-sensor type having both an OCT sensor and an IVUS sensor.

10. A program that causes a computer to perform the following processes: acquire multiple transverse images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane axially, and an angiographic image in which the image acquisition catheter is depicted; acquire the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image; determine the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image; calculate which frame of the multiple transverse images corresponds to the first tomographic image in which the first landmark is depicted, assuming that three-dimensional scanning was performed up to the position of the first landmark; determine which frame of the multiple transverse images corresponds to the second tomographic image in which the second landmark is depicted; and divide the path by the number of frames from the first tomographic image to the second tomographic image to identify the position of each transverse image.

11. An information processing device having a control unit, wherein the control unit acquires a plurality of transverse layer images generated by three-dimensional scanning using an image acquisition catheter that acquires images while moving the scanning plane in the axial direction, and an angiographic image in which the image acquisition catheter is depicted, the position of a first landmark located outside the three-dimensional scanning range of the image acquisition catheter and the position of a second landmark located within the three-dimensional scanning range of the image acquisition catheter in the angiographic image, respectively, the path of the image acquisition catheter from the first landmark to the second landmark in the angiographic image, the calculation of which frame of the plurality of transverse layer images corresponds to the first tomographic image in which the first landmark is depicted assuming that three-dimensional scanning was performed up to the position of the first landmark, the determination of which frame of the plurality of transverse layer images corresponds to the second tomographic image in which the second landmark is depicted, and the information processing device that identifies the position of each transverse layer image by dividing the path by the number of frames from the first tomographic image to the second tomographic image.

Images