Program, image processing method and image processing device

Abstract

Provided are a program and the like that make it possible to provide easily interpreted images. According to the present invention, a computer, following a program, acquires a signal data set for first inspection waves that are radiated in a direction that is inclined at a first angle relative to the long axis direction of an imaging core of a catheter. Then, on the basis of the acquired signal data set, the computer constructs an image that observes a direction that is inclined at a second angle relative to the long axis direction of the imaging core, the second angle being different from the first angle.

Description

[Technical Field]

[0001] The present invention relates to a program, an image processing method, and an image processing device. [Background technology]

[0002] Intravascular treatments, such as percutaneous coronary intervention (PCI), are being performed as minimally invasive treatments for angina pectoris, myocardial infarction, and the like. PCI is a treatment method in which a catheter is inserted into a blood vessel through the wrist, elbow, or groin, and a balloon or stent is used to dilate a stenotic portion of a coronary artery. The catheter is equipped with, for example, a sensor for intravascular ultrasound (IVUS) examination, allowing the surgeon to confirm the stenotic portion within the blood vessel using IVUS images taken from inside the blood vessel using the IVUS sensor. Some catheters are also equipped with a sensor for optical coherence tomography (OCT) using near-infrared light. Patent Document 1 discloses a catheter equipped with an IVUS sensor and an OCT sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-95624 Summary of the Invention [Problem to be solved by the invention]

[0004] In the catheter disclosed in Patent Document 1, the IVUS sensor and the OCT sensor are provided at different positions in the longitudinal direction of the catheter (the direction in which the blood vessel runs). Therefore, there is a discrepancy between the observation position by the IVUS sensor and the observation position by the OCT sensor at the same imaging timing. This makes interpretation of the IVUS and OCT images cumbersome, and there is a risk of reducing the accuracy of procedures performed while checking the IVUS and OCT images.

[0005] In addition, a contrast marker made of a radiopaque material is attached to the catheter, and the position of the catheter is confirmed using the contrast marker that appears in the X-ray image, while the stenosis site is confirmed using the IVUS image or OCT image. Even in this configuration, the IVUS sensor or OCT sensor and the contrast marker are provided at different positions along the longitudinal axis of the catheter, resulting in a misalignment between the position of the contrast marker in the X-ray image taken at the same time and the observation position of the IVUS sensor or OCT sensor. Therefore, even in this configuration, interpretation of the IVUS image or OCT image becomes complicated.

[0006] In one aspect, an object is to provide a program or the like that can provide images that are easy to interpret based on images taken using a catheter. [Means for solving the problem]

[0007] A program according to one aspect causes a computer to acquire a signal data set from a first inspection wave irradiated in a direction tilted at a first angle with respect to the longitudinal axis direction of an imaging core used in a catheter, and to construct an image based on the acquired signal data set, the image being observed in a direction tilted at a second angle different from the first angle with respect to the longitudinal axis direction. [Effects of the Invention]

[0008] In one aspect, it is possible to provide an image that is easy for the surgeon to interpret based on an image taken using a catheter. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic apparatus. [Figure 2] FIG. 1 is an explanatory diagram illustrating an overview of a catheter for diagnostic imaging. [Figure 3] FIG. 2 is an explanatory diagram showing a cross section of a blood vessel through which a sensor portion is inserted. [Figure 4A] FIG. 2 is an explanatory diagram illustrating a tomographic image. [Figure 4B] FIG. 2 is an explanatory diagram illustrating a tomographic image. [Figure 5] FIG. 1 is a block diagram illustrating an example of the configuration of an image processing device. [Figure 6A] FIG. 10 is an explanatory diagram illustrating the deviation of the observation positions of the IVUS sensor and the OCT sensor. [Figure 6B] FIG. 10 is an explanatory diagram illustrating the deviation of the observation positions of the IVUS sensor and the OCT sensor. [Figure 7A] 10A and 10B are explanatory diagrams illustrating a process for correcting a deviation of an observation position. [Figure 7B] 10A and 10B are explanatory diagrams illustrating a process for correcting a deviation of an observation position. [Figure 8] 10 is a flowchart illustrating an example of a correction processing procedure for a tomographic image. [Figure 9] 10A and 10B are explanatory diagrams of a method for calculating the amount of deviation of the observation position according to the distance from the rotation center of the sensor unit. [Figure 10] 10A and 10B are explanatory diagrams showing examples of calculation results of the number of correction frames according to each distance. [Figure 11] FIG. 10 is a block diagram showing an example of the configuration of an image processing device according to a second embodiment. [Figure 12] 10 is a flowchart showing an example of a correction processing procedure for a tomographic image according to the second embodiment. [Figure 13] FIG. 11 is a block diagram showing an example of the configuration of an image processing apparatus and a server according to a third embodiment. [Figure 14] FIG. 2 is a schematic diagram showing an example of the configuration of a catheter DB. [Figure 15] 11 is a flowchart showing an example of a correction processing procedure for a tomographic image according to the third embodiment. [Figure 16] FIG. 1 is a schematic diagram illustrating an example of the configuration of a learning model. [Figure 17] 10 is a flowchart illustrating an example of a learning model generation processing procedure. [Figure 18] 13 is a flowchart showing an example of a correction processing procedure for a tomographic image according to the fourth embodiment. [Figure 19] 10 is an explanatory diagram illustrating the deviation of the observation position of the IVUS sensor from the marker. FIG. [Figure 20] 10A and 10B are explanatory diagrams illustrating a process for correcting a deviation of an observation position. [Figure 21] 10A and 10B are explanatory diagrams illustrating a process for correcting a deviation of an observation position. [Figure 22] 13 is a flowchart showing an example of a correction processing procedure for a tomographic image according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The program, image processing method, and image processing device of the present disclosure will be described in detail below with reference to the drawings illustrating embodiments thereof. In the following embodiments, cardiac catheterization, which is an intravascular treatment, will be described as an example. However, the hollow organs that are the subject of catheterization treatment are not limited to blood vessels, and may be other hollow organs such as the bile duct, pancreatic duct, bronchi, and intestines.

[0011] (Embodiment 1) FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic device 100. In this embodiment, an imaging diagnostic device using a dual-type catheter equipped with the functions of both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) will be described. The dual-type catheter is provided with a mode for acquiring ultrasound tomographic images using only IVUS, a mode for acquiring optical coherence tomographic images using only OCT, and a mode for acquiring tomographic images using both IVUS and OCT, and these modes can be switched for use. Hereinafter, ultrasound tomographic images and optical coherence tomographic images will be referred to as IVUS images and OCT images, respectively, as appropriate. IVUS images and OCT images will also be collectively referred to as tomographic images.

[0012] The diagnostic imaging apparatus 100 of this embodiment includes an intravascular inspection apparatus 101, an angiography apparatus 102, an image processing device 3, a display device 4, and an input device 5. The intravascular inspection apparatus 101 includes a diagnostic imaging catheter 1 and an MDU (Motor Drive Unit) 2. The diagnostic imaging catheter 1 is connected to the image processing device 3 via the MDU 2. The display device 4 and the input device 5 are connected to the image processing device 3. The display device 4 is, for example, a liquid crystal display or an organic EL display, and the input device 5 is, for example, a keyboard, a mouse, a trackball, or a microphone. The display device 4 and the input device 5 may be stacked together to form a touch panel. Alternatively, the input device 5 and the image processing device 3 may be integrated. Furthermore, the input device 5 may be a sensor that accepts gesture input, gaze input, or the like.

[0013] The angiography device 102 is connected to the image processing device 3. The angiography device 102 is an angiography device that uses X-rays to capture blood vessels from outside the patient's body while injecting a contrast agent into the patient's blood vessels, thereby obtaining an angio image, which is a fluoroscopic image of the blood vessels. The angiography device 102 is equipped with an X-ray source and an X-ray sensor, and captures an X-ray fluoroscopic image (X-ray image) of the patient by the X-ray sensor receiving X-rays emitted from the X-ray source. Note that the diagnostic imaging catheter 1 is provided with a marker made of a radiopaque material that does not transmit X-rays, and the position of the diagnostic imaging catheter 1 (marker) is visualized in the angio image. The angiography device 102 outputs the captured angio image to the image processing device 3, which displays it on the display device 4 via the image processing device 3. Note that the display device 4 displays the angio image and a tomographic image captured using the diagnostic imaging catheter 1.

[0014] FIG. 2 is an explanatory diagram outlining the diagnostic imaging catheter 1. The upper dashed-dotted area in FIG. 2 is an enlarged version of the lower dashed-dotted area. The diagnostic imaging catheter 1 includes a probe 11 and a connector 15 disposed at the end of the probe 11. The probe 11 is connected to the MDU 2 via the connector 15. In the following description, the side of the diagnostic imaging catheter 1 farther from the connector 15 will be referred to as the distal end, and the connector 15 side will be referred to as the proximal end. The probe 11 includes a catheter sheath 11a, the distal end of which is provided with a guidewire insertion portion 14 through which a guidewire can be inserted. The guidewire insertion portion 14 forms a guidewire lumen and is used to receive a guidewire previously inserted into a blood vessel and guide the probe 11 to the affected area via the guidewire. The catheter sheath 11a forms a continuous tubular portion extending from the connection portion with the guidewire insertion portion 14 to the connection portion with the connector 15. A shaft 13 is inserted inside the catheter sheath 11a, and a sensor unit 12 is connected to the tip side of the shaft 13.

[0015] The sensor unit 12 has a housing 12d, the distal end of which is hemispherically shaped to reduce friction and snagging with the inner surface of the catheter sheath 11a. The housing 12d contains an ultrasound transmitting / receiving unit 12a (hereinafter referred to as the IVUS sensor 12a) that transmits ultrasound into a blood vessel and receives reflected waves from the blood vessel, and an optical transmitting / receiving unit 12b (hereinafter referred to as the OCT sensor 12b) that transmits near-infrared light into the blood vessel and receives reflected light from the blood vessel. In the example shown in FIG. 2, the IVUS sensor 12a is provided at the distal end of the probe 11, and the OCT sensor 12b is provided at the proximal end. The sensors are disposed on the central axis of the shaft 13 (on the two-dot chain line in FIG. 2) and are spaced a distance x apart along the axial direction (longitudinal direction of the shaft 13). In the diagnostic imaging catheter 1, the IVUS sensor 12a and the OCT sensor 12b are attached such that the direction of transmission and reception of ultrasound or near-infrared light is approximately 90 degrees relative to the axial direction of the shaft 13 (the radial direction of the shaft 13). It is desirable that the IVUS sensor 12a and the OCT sensor 12b are mounted slightly offset from the radial direction so as not to receive waves or light reflected from the inner surface of the catheter sheath 11a. In this embodiment, as shown by the arrow in Fig. 2, for example, the IVUS sensor 12a is mounted so that the direction of ultrasound irradiation is inclined toward the proximal end with respect to the radial direction, and the OCT sensor 12b is mounted so that the direction of near-infrared light irradiation is inclined toward the distal end with respect to the radial direction.

[0016] An electric signal cable (not shown) connected to the IVUS sensor 12a and an optical fiber cable (not shown) connected to the OCT sensor 12b are inserted into the shaft 13. The probe 11 is inserted into a blood vessel from the distal end side. The sensor unit 12 and the shaft 13 are movable forward and backward within the catheter sheath 11a and can also rotate circumferentially. The sensor unit 12 and the shaft 13 rotate around the central axis of the shaft 13 as the axis of rotation. The diagnostic imaging device 100 uses an imaging core formed by the sensor unit 12 and the shaft 13 to measure the state inside the blood vessel using ultrasound tomographic images (IVUS images) taken from inside the blood vessel or optical coherence tomographic images (OCT images) taken from inside the blood vessel.

[0017] The MDU 2 is a drive unit to which the probe 11 (diagnostic imaging catheter 1) is detachably attached via a connector unit 15, and controls the operation of the diagnostic imaging catheter 1 inserted into a blood vessel by driving a built-in motor in response to operation by a medical professional. For example, the MDU 2 performs a pull-back operation to rotate the sensor unit 12 and shaft 13 inserted into the probe 11 in a circumferential direction while pulling them toward the MDU 2 at a constant speed. The sensor unit 12 rotates while moving from the distal end to the proximal end due to the pull-back operation, and thereby scans the inside of the blood vessel continuously at predetermined time intervals, thereby continuously capturing multiple transverse cross-sectional images approximately perpendicular to the probe 11 at predetermined intervals. The MDU 2 outputs reflected wave data of ultrasound received by the IVUS sensor 12a and reflected light data received by the OCT sensor 12b to the image processing device 3.

[0018] The image processing device 3 acquires reflected wave data of ultrasound received by the IVUS sensor 12a and reflected light data received by the OCT sensor 12b via the MDU 2. The image processing device 3 generates ultrasound line data, which is a signal data set, from the reflected wave data of ultrasound, and constructs an ultrasound tomographic image (IVUS image) of a transverse layer of the blood vessel based on the generated ultrasound line data. The image processing device 3 also generates optical line data, which is a signal data set, from the reflected light data, and constructs an optical tomographic image (OCT image) of a transverse layer of the blood vessel based on the generated optical line data. Here, the signal data sets acquired by the IVUS sensor 12a and the OCT sensor 12b and the tomographic image constructed from the signal data sets will be described. FIG. 3 is an explanatory diagram showing a cross section of a blood vessel through which the sensor unit 12 is inserted, and FIGS. 4A and 4B are explanatory diagrams illustrating tomographic images.

[0019] First, referring to FIG. 3 , the operation of the IVUS sensor 12a and the OCT sensor 12b within a blood vessel and the signal data sets (ultrasound line data and optical line data) acquired by the IVUS sensor 12a and the OCT sensor 12b will be described. When tomographic image capture begins with the sensor unit 12 and the shaft 13 inserted within the blood vessel, the sensor unit 12 rotates in the direction indicated by the arrow, with the central axis of the shaft 13 serving as the center of rotation. At this time, the IVUS sensor 12a transmits and receives ultrasound waves at each rotation angle. Lines 1, 2, ... 512 indicate the transmission and reception directions of ultrasound waves at each rotation angle. In this embodiment, the IVUS sensor 12a intermittently transmits and receives ultrasound waves 512 times while rotating 360 degrees (one rotation) within the blood vessel. The IVUS sensor 12a acquires one line of data (signal data) in the transmission and reception direction by transmitting and receiving ultrasound waves once, and therefore, 512 ultrasound line data extending radially from the center of rotation can be obtained during one rotation. The 512 ultrasound line data are dense near the center of rotation, but become sparser as they move away from the center of rotation. Therefore, the image processing device 3 generates pixels in the empty spaces of each line by well-known interpolation processing, thereby constructing a two-dimensional ultrasound tomographic image (IVUS image) as shown in Figure 4A.

[0020] Similarly, the OCT sensor 12b also transmits and receives near-infrared light (measurement light) at each rotation angle. Since the OCT sensor 12b also transmits and receives measurement light 512 times while rotating 360 degrees inside the blood vessel, 512 optical line data extending radially from the center of rotation can be obtained during one rotation. For the optical line data, the image processing device 3 generates pixels in the empty spaces of each line by well-known interpolation processing, thereby constructing a two-dimensional optical coherence tomographic image (OCT image) similar to the IVUS image shown in FIG. 4A.

[0021] A two-dimensional tomographic image constructed from 512 pieces of line data in this manner is referred to as one frame of an IVUS image or an OCT image. Since the sensor unit 12 scans while moving inside a blood vessel, one frame of an IVUS image or an OCT image is acquired at each position of one rotation within the range of movement. That is, one frame of an IVUS image or an OCT image is acquired at each position from the distal end to the proximal end of the probe 11 within the range of movement, so that multiple frames of IVUS images or OCT images are acquired within the range of movement, as shown in FIG. 4B . In this embodiment, the IVUS sensor 12a and the OCT sensor 12b are configured to acquire 512 pieces of line data, but the number of line data acquired by the IVUS sensor 12a and the OCT sensor 12b is not limited to 512.

[0022] The diagnostic imaging catheter 1 has markers that are opaque to X-rays in order to confirm the positional relationship between an IVUS image obtained by the IVUS sensor 12a or an OCT image obtained by the OCT sensor 12b, and an angio image obtained by the angiography device 102. In the example shown in FIG. 2, a marker 14a is provided at the tip of the catheter sheath 11a, for example, at the guidewire insertion portion 14, and a marker 12c is provided on the shaft 13 side of the sensor unit 12. When the diagnostic imaging catheter 1 configured in this manner is photographed with X-rays, an angio image is obtained in which the markers 14a and 12c are visualized. The positions at which the markers 14a and 12c are provided are merely examples; the marker 12c may be provided on the shaft 13 instead of the sensor unit 12, and the marker 14a may be provided at a location other than the tip of the catheter sheath 11a.

[0023] 5 is a block diagram showing an example of the configuration of the image processing device 3. The image processing device 3 is a computer, and includes a control unit 31, a main memory unit 32, an input / output I / F 33, an auxiliary memory unit 34, and a reading unit 35. The control unit 31 is configured using one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), a tensor processing unit (TPU), etc. The control unit 31 is connected to each hardware unit constituting the image processing device 3 via a bus.

[0024] The main memory unit 32 is a temporary storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and temporarily stores data required for the control unit 31 to execute arithmetic processing.

[0025] The input / output I / F 33 is an interface to which the intravascular inspection device 101, the angiography device 102, the display device 4, and the input device 5 are connected. The control unit 31 acquires reflected wave data of ultrasound and reflected light data of measurement light from the intravascular inspection device 101 and acquires an angio image from the angiography device 102 via the input / output I / F 33. The control unit 31 generates ultrasound line data from the reflected wave data acquired from the intravascular inspection device 101 and further constructs an IVUS image. The control unit 31 also generates light line data from the reflected light data acquired from the intravascular inspection device 101 and further constructs an OCT image. The control unit 31 also displays a medical image on the display device 4 by outputting a medical image signal of the IVUS image, OCT image, or angio image to the display device 4 via the input / output I / F 33. The control unit 31 also receives information input to the input device 5 via the input / output I / F 33.

[0026] The auxiliary storage unit 34 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The auxiliary storage unit 34 stores the computer program P executed by the control unit 31 and various data required for the processing of the control unit 31. The auxiliary storage unit 34 may be an external storage device connected to the image processing device 3. The computer program P may be written to the auxiliary storage unit 34 during the manufacturing stage of the image processing device 3, or may be distributed by a remote server device and acquired by the image processing device 3 via communication and stored in the auxiliary storage unit 34. The computer program P may be readably recorded on a recording medium 30 such as a magnetic disk, optical disk, or semiconductor memory, or may be read from the recording medium 30 by the reading unit 35 and stored in the auxiliary storage unit 34.

[0027] The image processing device 3 may be a multi-computer including multiple computers. The image processing device 3 may also be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the image processing device 3 is described as being a single computer. In this embodiment, an angiography device 102 that captures two-dimensional angioimages is connected to the image processing device 3, but the device is not limited to the angiography device 102 as long as it captures images of a patient's hollow organs and the diagnostic imaging catheter 1 from multiple directions outside the body.

[0028] In the image processing device 3 of this embodiment, the control unit 31 reads and executes a computer program P stored in the auxiliary storage unit 34 to generate ultrasound line data from the reflected wave data received by the IVUS sensor 12a and generate optical line data from the reflected light data received by the OCT sensor 12b. Furthermore, the control unit 31 performs processing to construct an IVUS image based on the ultrasound line data and processing to construct an OCT image based on the optical line data. Since the IVUS sensor 12a and the OCT sensor 12b have different observation positions at the same imaging timing, as described below, the control unit 31 performs processing to correct the difference in observation positions in the IVUS image and the OCT image when constructing the IVUS image and the OCT image. Therefore, the image processing device 3 of this embodiment can construct IVUS images and OCT images with the same observation positions, providing images that are easy to interpret.

[0029] 6A and 6B are explanatory diagrams illustrating the deviation of the observation positions of the IVUS sensor 12a and the OCT sensor 12b. Fig. 6A shows the sensor unit 12 as viewed from the radial direction of the shaft 13, and Fig. 6B shows the sensor unit 12 as viewed from the tip side of the probe 11. In Figs. 6A and 6B, the solid arrows indicate the direction of transmission and reception of ultrasound by the IVUS sensor 12a, and the dashed arrows indicate the direction of transmission and reception of measurement light by the OCT sensor 12b.

[0030] In the sensor unit 12 of this embodiment, the IVUS sensor 12a transmits and receives ultrasound in a direction inclined toward the base end with respect to the radial direction of the shaft 13. In the example shown in FIG. 6A, the direction of transmission and reception of ultrasound forms an angle α with respect to the direction toward the base end on the central axis of the shaft 13. The OCT sensor 12b transmits and receives measurement light in a direction inclined toward the tip end with respect to the radial direction of the shaft 13. The direction of transmission and reception of measurement light forms an angle β with respect to the direction toward the tip end on the central axis of the shaft 13. In the IVUS sensor 12a and the OCT sensor 12b, the ultrasound path and the measurement light path intersect as shown in FIG. 6A. The same object is observed at the intersection, but at other locations, there is a misalignment in the axial direction of the shaft 13 between the respective observation positions. The misalignment of the observation positions is smallest at the intersection of the ultrasound and measurement light and increases with increasing distance from the intersection. Furthermore, the order of observation differs because the OCT sensor 12b observes the area closer to the sensor unit 12 than the intersection position before the IVUS sensor 12a, and the IVUS sensor 12a observes the area farther from the intersection position before the OCT sensor 12b. Such a deviation in the observation position differs depending on the distance x between the arrangement positions of the sensors 12a and 12b, the mounting angles of the sensors 12a and 12b (angle α in the transmission and reception direction of ultrasound, angle β in the transmission and reception direction of measurement light), and the distance from the rotation center of the sensor unit 12 to the observation position. Because the distance x between the arrangement positions of the sensors 12a and 12b and the mounting angle include individual differences that occur during the manufacturing process of the diagnostic imaging catheter 1, the amount of deviation in the observation position at each distance from the rotation center of the sensor unit 12 to the observation position differs for each diagnostic imaging catheter 1.

[0031] 6B, the direction of transmission and reception of ultrasound by the IVUS sensor 12a and the direction of transmission and reception of measurement light by the OCT sensor 12b are also misaligned in the rotational direction (circumferential direction) of the sensor unit 12. In the example shown in FIG. 6B, the direction of transmission and reception of measurement light is misaligned by an angle φ, with the clockwise direction being the positive direction, based on the direction of transmission and reception of ultrasound. Therefore, the observation positions by the IVUS sensor 12a and the observation positions by the OCT sensor 12b are also misaligned in the circumferential direction, and the misalignment (angular difference) of the observation positions in the circumferential direction also differs depending on the mounting angles of the sensors 12a and 12b. Therefore, the amount of misalignment of the observation positions in the circumferential direction also differs for each diagnostic imaging catheter 1.

[0032] Therefore, when constructing IVUS images and OCT images from ultrasound line data and optical line data, the image processing device 3 of this embodiment constructs IVUS images and OCT images in which the deviation of the observation position by the sensors 12a and 12b as described above has been corrected. Figures 7A and 7B are explanatory diagrams illustrating the process of correcting the deviation of the observation position. Figure 7A is an explanatory diagram of the deviation of the observation position in the axial direction of the shaft 13, and Figure 7B is an explanatory diagram illustrating the process of correcting the OCT image so that the observation position in the OCT image coincides with the observation position in the IVUS image.

[0033] In FIG. 7A, 1, 2, and 3 indicate observation positions with distances r1, r2, and r3 from the rotation center of the sensor unit 12. FIG. 7A shows that the OCT sensor 12b observed observation position 1 in the n-1th frame, observation position 2 in the nth frame, and observation position 3 in the n+1th frame, while the IVUS sensor 12a observed observation positions 1 to 3 in the nth frame. In this situation, as shown in the top row of FIG. 7B, observation positions 1 to 3 are captured in the nth frame of the IVUS image, but observation positions 1 to 3 are not captured in the n-1th and n+1th frames. On the other hand, as shown in the second row from the top of FIG. 7B, the OCT image shows observation position 1 in the n-1th frame, observation position 2 in the nth frame, and observation position 3 in the n+1th frame. That is, in the example shown in FIG. 7A, the observation timing of the OCT sensor 12b for observation position 1 is one frame earlier than that of the IVUS sensor 12a, and the observation timing for observation position 3 is one frame later. Therefore, the image processing device 3 can correct the deviation of the observation positions by correcting the IVUS images and OCT images to a state in which each observation position 1 to 3 is observed at the same observation timing (a state in which the tomographic images are captured with the same frame number).

[0034] Note that the observation positions are shifted not only in the axial direction of the shaft 13 but also in the circumferential direction (direction of rotation) as shown in FIG. 6B . Therefore, as shown in the third row from the top of FIG. 7B , observation positions 1 to 3 are captured in the OCT image at positions shifted by an angle φ in the circumferential direction. Therefore, as shown in the bottom row of FIG. 7B , the image processing device 3 of this embodiment performs a correction process to eliminate the shift between the observation positions in the OCT image and the observation positions in the IVUS image by combining regions in multiple frames of OCT images to construct one frame of OCT image. In the following, a correction is performed on the OCT image to match the observation target in the OCT image with the observation target in the IVUS image; however, a similar correction may also be performed on the IVUS image to match the observation target in the IVUS image with the observation target in the OCT image.

[0035] 8 is a flowchart showing an example of a correction process procedure for a tomographic image. The control unit 31 of the image processing device 3 performs the following process in accordance with a computer program P stored in the auxiliary storage unit 34. Note that the distance x between the arrangement positions of the sensors 12a and 12b shown in FIG. 6A, the angle α (second angle) of the transmission and reception direction of the ultrasonic waves by the IVUS sensor 12a, the angle β (first angle) of the transmission and reception direction of the measurement light by the OCT sensor 12b, and the circumferential deviation amount (angle φ) in the transmission and reception direction of the ultrasonic waves and the measurement light shown in FIG. 6B are stored in the main storage unit 32 or the auxiliary storage unit 34. In the following, a correction process is performed to align the object of observation with an IVUS image generated from reflected wave data of the ultrasonic waves (second inspection wave) transmitted by the IVUS sensor 12a (second transmission and reception unit) for an OCT image generated from reflected light data of the measurement light (first inspection wave) transmitted by the OCT sensor 12b (first transmission and reception unit).

[0036] When intravascular imaging processing by the intravascular inspection device 101 is started, the control unit 31 (acquisition unit) of the image processing device 3 acquires reflected ultrasound wave data from the IVUS sensor 12a via the MDU 2 and generates ultrasound line data from the acquired reflected ultrasound wave data. The control unit 31 also acquires reflected light data from the OCT sensor 12b via the MDU 2 and generates optical line data from the acquired reflected light data (S11). The control unit 31 interpolates pixels of the ultrasound line data to construct a two-dimensional IVUS image (S12). Then, based on the optical line data, the control unit 31 constructs an OCT image (hereinafter referred to as an OCT corrected image) in which the IVUS image and the observation target are aligned. Specifically, the control unit 31 first calculates the frame pitch during imaging by the sensor unit 12 (S13). The frame pitch is the distance between each frame of the tomographic image (the distance between the observation positions of each frame in the axial direction of the shaft 13 (the irradiation interval of the measurement light)) and is calculated based on the movement speed and rotation speed of the sensor unit 12 during imaging. Specifically, it is calculated from the movement speed (e.g., movement distance per second (unit: mm)) and rotation speed (e.g., number of rotations per second (unit: times)) of the sensor unit 12 due to the pullback operation. The control unit 31 calculates the frame pitch from, for example, (movement speed / rotation speed).

[0037] Next, the control unit 31 calculates the amount of deviation of the observation position at each distance from the rotation center of the sensor unit 12 (S14). Here, the control unit 31 calculates the amount of deviation of the observation position in the axial direction of the shaft 13, i.e., the amount of deviation between the position where the ultrasonic wave is emitted by the IVUS sensor 12a and the position where the measurement light is emitted by the OCT sensor 12b. FIG. 9 is an explanatory diagram of a method for calculating the amount of deviation of the observation position according to the distance from the rotation center of the sensor unit 12. In FIG. 9, the amount of deviation of the observation position in an area where the distance from the rotation center of the sensor unit 12 is L is expressed as Δx (unit: mm). The amount of deviation Δx in the axial direction is expressed by the following equation 1. Note that x (unit: mm) is the amount of deviation in the axial direction of the arrangement positions of the sensors 12a and 12b. Also, Δx IVUS (Unit: mm) is expressed by the following formula 2, Δx OCT (Unit: mm) is expressed by the following Equation 3. Therefore, by substituting Equations 2 and 3 into Equation 1, Equation 4 is obtained, and the deviation amount Δx of the observation position in the area where the distance from the rotation center of the sensor unit 12 is L is calculated using Equation 4.

[0038] Δx=x-Δx IVUS -Δx OCT …(Formula 1) Δx IVUS =L×cotα…(Formula 2) Δx OCT =L×cotβ…(Formula 3) Δx=xL×(cotα+cotβ) …(Equation 4)

[0039] Next, the control unit 31 calculates the amount of correction (number of correction frames) in the axial direction at each distance from the center of rotation of the sensor unit 12 based on the amount of deviation of the observation position in the axial direction of the shaft 13 (S15). The number of correction frames is calculated using the following equation 5 based on the amount of deviation of the observation position in the axial direction calculated in step S14 and the frame pitch calculated in step S13.

[0040] Number of correction frames = Δx / frame pitch ... (Equation 5)

[0041] Instead of calculating the deviation Δx of the observation position and the number of correction frames at each distance from the center of rotation of the sensor unit 12, the control unit 31 may calculate the distance L from the center of rotation at each point where the deviation Δx of the observation position is a multiple of the frame pitch. For example, when the frame pitch is 0.1 mm, the control unit 31 calculates each distance L where the deviation Δx of the observation position is 0.1 mm, 0.2 mm, 0.3 mm, and so on. In this case, the number of correction frames corresponding to each calculated distance L can be specified as 1, 2, 3, and so on. FIG. 10 is an explanatory diagram showing an example of the calculation result of the number of correction frames corresponding to each distance L. The example shown in FIG. 10 shows the calculation result when the movement speed of the sensor unit 12 is 10 mm / s, the rotation speed is 6000 rpm (100 fps), the axial distance x between the sensors 12a and 12b is 0.80 mm, the angle α of the transmission and reception direction of the ultrasonic waves is 85°, and the angle β of the transmission and reception direction of the measurement light is 84°. The number of correction frames calculated in this way indicates the number of frames of the OCT image when the object of observation captured in one frame of IVUS image is captured by the OCT sensor 12b. By using such a number of correction frames, the deviation of the observation position in the axial direction can be corrected in frame pitch units.

[0042] Next, the control unit 31 calculates the amount of correction in the circumferential direction (the number of correction lines) based on the amount of deviation of the observation position in the rotational direction of the sensor unit 12 (S16). The number of correction lines is calculated based on the amount of deviation (angle φ) in the circumferential direction between the transmission and reception direction of ultrasound by the IVUS sensor 12a and the transmission and reception direction of measurement light by the OCT sensor 12b. Note that one frame of OCT image is composed of 512 line data aligned in the circumferential direction, so the control unit 31 can calculate the number of correction lines using the following equation (6). As a result, when the transmission and reception direction is deviated by the angle φ, the OCT image can be corrected forward by the number of correction lines to align the observation position of the OCT image with the observation target of the IVUS image, thereby performing a correction that eliminates the deviation of the observation position in the circumferential direction.

[0043] Number of correction lines = 512 × φ / 360° ... (Equation 6)

[0044] The control unit 31 (image construction unit) constructs an OCT-corrected image in which the observation target is aligned with the IVUS image based on the number of correction frames corresponding to each distance calculated in step S15 and the number of correction lines calculated in step S16 (S17). For example, the control unit 31 subtracts the number of correction lines calculated in step S16 from each line number of the optical line data. If the line number after subtraction is a negative number, 512 is added so that each line number after subtraction has a value between 1 and 512. This associates each line of the optical line data with each line of the ultrasound line data, and the deviation of the observation position in the circumferential direction based on the IVUS image is corrected according to the association. The control unit 31 then performs correction processing according to the number of correction frames calculated in step S15 using the optical line data for each line number after subtraction to construct an OCT-corrected image for each frame. The construction process of the OCT-corrected image for the nth frame will be described below.

[0045] In the case of the calculation results shown in FIG. 10, for example, for the region up to 0.78 mm from the center of rotation (the intermediate point between 0.52 mm and 1.04 mm), pixel values ​​are assigned based on the data values ​​corresponding to the region up to 0.78 mm from the center of rotation in the optical line data of the (n-7) frame. Furthermore, for the region from 0.78 to 1.30 mm from the center of rotation (the intermediate point between 1.04 mm and 1.56 mm), pixel values ​​are assigned based on the data values ​​corresponding to the region from 0.78 to 1.30 mm from the center of rotation in the optical line data of the (n-6) frame. By performing this process for each region according to the distance from the center of rotation, data for each line of the optical line data is associated with data for each line of the ultrasound line data. Based on the associated details, OCT images for frames corresponding to the amount of misalignment in each region are combined to construct one frame of an OCT-corrected image. In areas farther from the center of rotation than the position where the deviation of the observation position is 0, pixel values ​​are assigned based on the data values ​​of the optical line data in the (n+number of correction frames)th frame. For example, among areas farther from the center of rotation than the position where the deviation of the observation position is 0, for areas where the calculated number of correction frames is 1, pixel values ​​based on the data values ​​of the corresponding area in the optical line data in the (n+1)th frame are assigned. The OCT-corrected image constructed in this manner is a tomographic image in which the deviation of the observation position in the axial direction based on the IVUS image has been corrected. The control unit 31 performs correction processing in the rotational and axial directions of the sensor unit 12 through the above-mentioned processing, and also constructs a two-dimensional OCT image (OCT-corrected image) by interpolating pixels between each line of the optical line data through interpolation processing.

[0046] By performing the correction process as described above, the control unit 31 constructs multiple frames of OCT-corrected images from the optical line data and generates OCT-corrected images in which the observation position is aligned with the IVUS image. The IVUS image and OCT-corrected image constructed by the above process are displayed on, for example, the display device 4 and presented to the surgeon using the diagnostic imaging catheter 1.

[0047] 2 and 6A, in this embodiment, the sensors 12a and 12b are arranged so that the direction of transmission and reception of ultrasound by the IVUS sensor 12a intersects with the direction of transmission and reception of measurement light by the OCT sensor 12b, but this configuration is not limited to this. For example, even if the sensors 12a and 12b are arranged so that the direction of transmission and reception of ultrasound is inclined toward the distal end with respect to the radial direction of the shaft 13 and the direction of transmission and reception of measurement light is inclined toward the proximal end, the deviation of the observation positions in the IVUS image and the OCT image can be corrected by similar processing.

[0048] (Embodiment 2) This section describes an imaging diagnostic device in which information used to correct a tomographic image is measured during the manufacture of the imaging diagnostic catheter 1, and a code obtained by encoding each measured value is added to the imaging diagnostic catheter 1. The information used to correct a tomographic image includes the distance x between the sensors 12a and 12b in the axial direction of the shaft 13, the transmission and reception direction (angle α) of ultrasound by the IVUS sensor 12a, the transmission and reception direction (angle β) of measurement light by the OCT sensor 12b, and the amount of deviation (angle φ) between the transmission and reception directions of ultrasound and measurement light in the rotation direction of the sensor unit 12.

[0049] The diagnostic imaging device 100 of this embodiment can be realized by the same devices as those in the diagnostic imaging device 100 of embodiment 1, and therefore a description of the same configuration will be omitted. Note that in the diagnostic imaging device 100 of this embodiment, the configuration of the image processing device 3 is slightly different from that of embodiment 1, and therefore only the different points will be described.

[0050] 11 is a block diagram showing an example of the configuration of the image processing device 3 of embodiment 2. The image processing device 3 of this embodiment includes a code reader 36 in addition to the configuration of the image processing device 3 of embodiment 1 shown in FIG. 5. The code reader 36 is a device that reads one-dimensional codes such as barcodes and two-dimensional codes such as QR Code (registered trademark), decodes the read code to obtain code information, and sends the obtained code information to the control unit 31.

[0051] In the diagnostic imaging apparatus 100 of this embodiment, the information (x, α, β, φ) used in the correction process of the tomographic image as described above is measured during the manufacturing process of the diagnostic imaging catheter 1, and the measured values ​​and a serial number assigned to the diagnostic imaging catheter 1 are coded. In the example shown in FIG. 11 , a QR code C is attached to the connector portion 15. The code C may be printed directly on the connector portion 15, or a sticker with the code C printed thereon may be affixed to the connector portion 15. In the diagnostic imaging apparatus 100 configured as described above, for example, when an operator connects the diagnostic imaging catheter 1 to the MDU 2 via the connector portion 15, the operator causes the code reader 36 to read the code C, and the image processing device 3 acquires the information (x, α, β, φ) used in the correction process of the tomographic image.

[0052] 12 is a flowchart showing an example of a correction processing procedure for a tomographic image according to the second embodiment. The processing shown in FIG. 12 is the processing shown in FIG. 8 with step S21 added before step S11. Explanation of the same steps as in FIG. 8 will be omitted. In the diagnostic imaging device 100 of this embodiment, the control unit 31 of the image processing device 3 first reads code information from the code C using the code reader 36 (S21). The code information includes each of the numerical values ​​(x, α, β, φ) measured during the manufacturing process of the diagnostic imaging catheter 1 and the serial number of the diagnostic imaging catheter 1, and each of the read information is stored in the main memory unit 32.

[0053] Thereafter, the intravascular imaging process by the intravascular inspection device 101 starts, and the control unit 31 executes the processes from step S11 onwards. Note that in this embodiment, when calculating the frame pitch during imaging in step S13, when calculating the amount of deviation of the observation position in the axial direction of the shaft 13 and the number of corrected frames in steps S14 and S15, and further when calculating the number of corrected lines in the rotation direction of the sensor unit 12 in step S16, each piece of information read from the code C is used.

[0054] By the above-described processing, in this embodiment as well, the axial and rotational deviations of the sensor unit 12 are corrected at the observation positions of the IVUS sensor 12a and the OCT sensor 12b, and an OCT corrected image can be constructed in which the observation positions are aligned with the IVUS image.

[0055] This embodiment provides the same effects as those of the first embodiment. Furthermore, in this embodiment, the distance x between the sensors 12a and 12b, the direction of transmission and reception of ultrasound by the IVUS sensor 12a (angle α), the direction of transmission and reception of measurement light by the OCT sensor 12b (angle β), and the amount of deviation (angle φ) between the transmission and reception directions of ultrasound and measurement light in the rotational direction of the sensor unit 12 are measured during the manufacturing process of the diagnostic imaging catheter 1, coded, and attached to the diagnostic imaging catheter 1 (e.g., connector unit 15). Therefore, the image processing device 3 can obtain various pieces of information used in correction processing to be performed on the tomographic image by reading the code C attached to the diagnostic imaging catheter 1 with the code reader 36. Since this information includes installation errors (individual differences) that occur for each catheter 1 during the manufacturing process, attaching such a code C to the catheter 1 enables accurate correction processing. Note that the modifications described in the first embodiment can also be applied to this embodiment. For example, instead of constructing an OCT-corrected image in which the observation target is matched to the IVUS image from the optical line data, a process of constructing an IVUS-corrected image in which the observation target is matched to the OCT image from the ultrasound line data may be performed. Even with such a configuration, the same effects as those of this embodiment can be obtained.

[0056] (Embodiment 3) In the imaging diagnostic apparatus 100 of embodiment 2, an imaging diagnostic apparatus configured so that the image processing device 3 acquires information used in correcting a tomographic image from a server will be described. The imaging diagnostic apparatus 100 of this embodiment can be realized by the same devices as those in the imaging diagnostic apparatus 100 of embodiment 1, and therefore a description of the same configuration will be omitted. Note that the configuration of the image processing device 3 in the imaging diagnostic apparatus 100 of this embodiment is slightly different from that of embodiment 2. In this embodiment, information (x, α, β, φ) used in correcting a tomographic image is managed by a server connected to a network such as the Internet. Therefore, the image processing device 3 can execute the same correction processing as in embodiments 1 and 2 by acquiring information used in the correction processing of the tomographic image from the server via the network.

[0057] Fig. 13 is a block diagram showing an example of the configuration of the image processing device 3 and server 6 of embodiment 3. The image processing device 3 of this embodiment includes a communication unit 37 in addition to the configuration of the image processing device 3 of embodiment 2 shown in Fig. 11. The communication unit 37 is a communication module for connecting the image processing device 3 to the network N by wired communication or wireless communication, and transmits and receives information to and from other devices via the network N.

[0058] The server 6 is configured using, for example, a server computer or a personal computer. The server 6 may be configured to perform distributed processing by providing multiple servers 6, or may be implemented by multiple virtual machines within a single server, or may be implemented using a cloud server. The server 6 includes a control unit 61, a main memory unit 62, an input / output I / F 63, an auxiliary memory unit 64, a reading unit 65, and a communication unit 66. The control unit 61, the main memory unit 62, the input / output I / F 63, the auxiliary memory unit 64, the reading unit 65, and the communication unit 66 of the server 6 are similar in configuration to the control unit 31, the main memory unit 32, the input / output I / F 33, the auxiliary memory unit 34, the reading unit 35, and the communication unit 37 of the image processing device 3, and therefore, description thereof will be omitted. The auxiliary memory unit 64 of the server 6 stores a catheter DB 64a (described later) in addition to the computer program executed by the control unit 61. The catheter DB 64a may be stored in another storage device connected to the server 6 or in another storage device with which the server 6 can communicate.

[0059] 14 is a schematic diagram showing an example of the configuration of the catheter DB 64a. The catheter DB 64a stores each of the numerical values ​​(x, α, β, φ) measured during the manufacturing process of the diagnostic imaging catheter 1, in association with the serial number assigned to the diagnostic imaging catheter 1. Specifically, the following are registered: the axial distance x between the sensors 12a and 12b attached to the diagnostic imaging catheter 1, the angle α of the transmission and reception direction of ultrasound by the IVUS sensor 12a, the angle β of the transmission and reception direction of measurement light by the OCT sensor 12b, and the deviation (angle φ) of the transmission and reception direction of ultrasound and measurement light in the rotation direction of the sensor unit 12.

[0060] In the diagnostic imaging apparatus 100 configured as described above, information (x, α, β, φ) used in the correction process of tomographic images is measured during the manufacturing process of the diagnostic imaging catheter 1, and each measured value is registered in the server 6 together with a serial number assigned to the diagnostic imaging catheter 1. Meanwhile, the serial number assigned to the diagnostic imaging catheter 1 is coded, and a code C is added to the connector section 15 as in FIG. 11 . In the diagnostic imaging apparatus 100 of this embodiment as well, the operator causes the code reader 36 to read the code C when connecting the diagnostic imaging catheter 1 to the MDU 2 via the connector section 15. As a result, the image processing device 3 obtains the serial number of the diagnostic imaging catheter 1, and obtains the information (x, α, β, φ) used in the correction process of tomographic images from the server 6 based on the serial number.

[0061] FIG. 15 is a flowchart showing an example of a procedure for correcting a tomographic image according to the third embodiment. The process shown in FIG. 15 is the process shown in FIG. 12 with steps S31 to S34 added between steps S21 and S11. Explanation of the same steps as in FIG. 12 will be omitted. Note that in FIG. 15, the left side shows the process performed by the image processing device 3, and the right side shows the process performed by the server 6. In the diagnostic imaging device 100 of this embodiment, the control unit 31 of the image processing device 3 first reads code information from the code C using the code reader 36 (S21). The code information here is the serial number of the diagnostic imaging catheter 1, and the control unit 31 requests information (x, α, β, φ) to be used in the correction process to be performed on the tomographic image acquired using the diagnostic imaging catheter 1 based on the read serial number from the server 6 (S31).

[0062] When the control unit 61 of the server 6 acquires the serial number of the diagnostic imaging catheter 1 from the image processing device 3, it reads out each piece of information (x, α, β, φ) corresponding to the serial number from the catheter DB 64a (S32). Then, the control unit 61 transmits the read out information (x, α, β, φ) to the image processing device 3 via the network N (S33). The control unit 31 of the image processing device 3 stores each piece of information (x, α, β, φ) acquired from the server 6 in the main memory unit 32 (S34).

[0063] Thereafter, the intravascular inspection device 101 starts intravascular imaging processing, and the control unit 31 executes the processing from step S11 onwards. Note that in this embodiment, when calculating the frame pitch during imaging in step S13, when calculating the amount of deviation of the observation position in the axial direction of the shaft 13 and the number of corrected frames in steps S14 and S15, and when calculating the number of corrected lines in the rotational direction of the sensor unit 12 in step S16, each piece of information acquired from the server 6 is used. Through the above-described processing, also in this embodiment, deviations in the axial direction and rotational direction of the sensor unit 12 are corrected at the observation positions of the IVUS sensor 12a and the OCT sensor 12b, and a corrected OCT image can be constructed in which the observation positions are aligned with the IVUS image.

[0064] In this embodiment, the same effects as those of the above-described embodiments can be obtained. In this embodiment, each piece of information (x, α, β, φ) used in the correction process performed on the tomographic image is managed by the server 6 in association with the serial number of the diagnostic imaging catheter 1. Therefore, the image processing device 3 can acquire each piece of information used in the correction process to be performed on the tomographic image from the server 6. This makes it possible to perform the correction process with high accuracy in this embodiment as well, and to provide IVUS images and OCT images in which the observation target is matched. Note that the modified examples described in the above-described embodiments can also be applied to this embodiment as appropriate.

[0065] (Embodiment 4) An image diagnostic device 100 will be described, in which the image processing device 3 is configured to identify each piece of information used in correction processing to be performed on the IVUS image or the OCT image based on the IVUS image and the OCT image. The image diagnostic device 100 of this embodiment can be realized by devices similar to the devices in the image diagnostic device 100 of embodiment 1, so a description of similar configurations will be omitted. Note that in the image diagnostic device 100 of this embodiment, the image processing device 3 stores a learning model 34M in the auxiliary storage unit 34, in addition to the configuration of the image processing device 3 of embodiment 1 shown in FIG.

[0066] 16 is a schematic diagram showing an example of the configuration of the learning model 34M. The learning model 34M is a machine learning model that receives as input IVUS images and OCT images generated by the image processing device 3 based on ultrasound line data and optical line data acquired using the diagnostic imaging catheter 1, and outputs information used in correction processing to be performed on the IVUS images or OCT images. The information used in the correction processing includes the distance x between the sensors 12a and 12b in the axial direction of the shaft 13 of the diagnostic imaging catheter 1, the transmission and reception direction of ultrasound (angle α), the transmission and reception direction of measurement light (angle β), and the deviation amount (angle φ) between the transmission and reception directions of ultrasound and measurement light in the rotation direction of the sensor unit 12. The learning model 34M outputs the probability of a preset option for each piece of information (x, α, β, φ).

[0067] The learning model 34M is configured, for example, by a convolutional neural network (CNN), which is a neural network generated by deep learning. The learning model 34M may be configured using algorithms other than CNN, such as a recurrent neural network (RNN), a generative adversarial network (GAN), a decision tree, a random forest, or a support vector machine (SVM), or may be configured by combining multiple algorithms. The image processing device 3 generates the learning model 34M in advance by performing machine learning to learn predetermined training data. The image processing device 3 then inputs IVUS images and OCT images into the learning model 34M and acquires information to be used for correction processing to be performed on the IVUS images or OCT images.

[0068] The learning model 34M receives, for example, an IVUS image and an OCT image acquired by a single pullback operation using the MDU2. The learning model 34M may receive one frame of an IVUS image and an OCT image acquired at the same time, or may receive IVUS images and OCT images acquired in time series sequentially. The learning model 34M includes an input layer to which the IVUS image and the OCT image are input, an intermediate layer that extracts features from the input information, and an output layer that outputs each piece of information used in the correction process. One output layer is provided for each piece of information (x, α, β, φ).

[0069] The input layer has multiple input nodes, and each pixel of the IVUS image and the OCT image is input to each input node. The intermediate layer uses predetermined functions, thresholds, etc. to perform operations such as filtering and compression on the tomographic image input via the input layer to calculate output values, and outputs the calculated output values ​​to the output layer. When the intermediate layer has multiple layers, the nodes of each layer calculate output values ​​based on the input tomographic image using functions, thresholds, etc. between each layer, and sequentially input the calculated output values ​​to nodes in the subsequent layer. The intermediate layer sequentially inputs the output values ​​of the nodes of each layer to nodes in the subsequent layer, thereby providing the final calculated output value to the output layer.

[0070] The learning model 34M has output layers including an inter-sensor distance output layer, an ultrasonic wave angle output layer, a measurement light angle output layer, and a circumferential deviation output layer. The inter-sensor distance output layer outputs the distance x between sensors 12a and 12b, the ultrasonic wave angle output layer outputs the transmission and reception direction of ultrasonic waves (angle α), the measurement light angle output layer outputs the transmission and reception direction of measurement light (angle β), and the circumferential deviation output layer outputs the deviation amount (angle φ) between the transmission and reception directions of ultrasonic waves and measurement light in the rotational direction. The inter-sensor distance output layer has multiple output nodes each associated with a distance option (first distance, second distance, etc.), and each output node outputs the probability that the associated distance should be determined. The output value from each output node of the inter-sensor distance output layer is, for example, a value between 0 and 1, and the sum of the probabilities output from each output node is 1.0 (100%). Similarly, the ultrasound angle output layer, measurement light angle output layer, and circumferential displacement output layer each have multiple output nodes associated with angles (first angle, second angle, etc.) provided as options, and each output node outputs the probability that the associated angle should be determined. In each of the ultrasound angle output layer, measurement light angle output layer, and circumferential displacement output layer, the output value from each output node is, for example, a value between 0 and 1, and the sum of the probabilities output from each output node is 1.0 (100%). The angles set as options for the ultrasound angle output layer, measurement light angle output layer, and circumferential displacement output layer are each different. With the above-mentioned configuration, when an IVUS image and an OCT image are input, the learning model 34M outputs each piece of information (x, α, β, φ) of the diagnostic imaging catheter 1 that captured the IVUS image and the OCT image.

[0071] When the learning model 34M described above is used, the image processing device 3 determines the distance associated with the output node that outputs the maximum output value (discrimination probability) among the output values ​​from the inter-sensor distance output layer as the inter-sensor distance in the diagnostic imaging catheter 1. Furthermore, the image processing device 3 determines the angle associated with the output node that outputs the maximum output value among the output values ​​from the ultrasound angle output layer as the transmission / reception angle of the ultrasound by the IVUS sensor 12a. Furthermore, the image processing device 3 determines the angle associated with the output node that outputs the maximum output value among the output values ​​from the measurement light angle output layer as the transmission / reception angle of the measurement light by the OCT sensor 12b. Furthermore, the image processing device 3 determines the angle associated with the output node that outputs the maximum output value among the output values ​​from the circumferential deviation amount output layer as the circumferential deviation amount of the ultrasound and the measurement light. A selection layer that selects and outputs the option with the highest probability may be provided after each output layer. In this case, each output layer has one output node that outputs the option (distance or angle) with the highest discrimination probability.

[0072] The learning model 34M can be generated by preparing training data including IVUS images and OCT images acquired using the diagnostic imaging catheter 1 and each piece of information (x, α, β, φ) (correct labels) measured, for example, during the manufacturing process of the diagnostic imaging catheter 1, and using this training data to perform machine learning on an untrained learning model. When the IVUS images and OCT images included in the training data are input, the learning model 34M learns so that, in each output layer, the output value from the output node corresponding to the correct label (each piece of information) included in the training data approaches 1 and the output values ​​from the other output nodes approaches 0. Specifically, the learning model 34M performs calculations in the intermediate layer based on the input IVUS images and OCT images to calculate output values ​​from the output nodes of each output layer. The learning model 34M then compares the calculated output value of each output node with a value corresponding to the correct label (1 for the output node corresponding to the correct value and 0 for the other output nodes) and optimizes parameters used in the calculation process in the intermediate layer so that each output value approaches the value corresponding to the correct label. The parameters are, for example, weights (coupling coefficients) between neurons, etc. The method for optimizing the parameters is not particularly limited, but the steepest descent method, the error backpropagation method, etc. can be used.

[0073] The learning model 34M may be learned by another learning device. The learned learning model 34M generated by learning by the other learning device is downloaded from the learning device to the image processing device 3, for example, via a network or via the recording medium 30, and stored in the auxiliary storage unit 34.

[0074] The process of learning the training data and generating the learning model 34M will be described below. Fig. 17 is a flowchart showing an example of the process procedure for generating the learning model 34M. The following process is performed by the control unit 31 of the image processing device 3 in accordance with the computer program P stored in the auxiliary storage unit 34, but may also be performed by another learning device.

[0075] The control unit 31 of the image processing device 3 acquires training data to which correct information (x, α, β, φ) is assigned for the IVUS images and OCT images (S41). The IVUS images and OCT images for the training data may be IVUS images and OCT images taken during catheter treatment. The correct information may be information measured, for example, during the manufacturing process, on the diagnostic imaging catheter 1 from which the IVUS images and OCT images were taken. Note that the training data may be prepared in advance, with the IVUS images and OCT images for the training data and each piece of information on the diagnostic imaging catheter 1 being registered in a training data DB (not shown). In this case, the control unit 31 may acquire the training data from the training data DB.

[0076] The control unit 31 performs a learning process for the learning model 34M using the acquired training data (S42). Here, the control unit 31 inputs the IVUS image and OCT image included in the training data into the learning model 34M and acquires output values ​​for each piece of information: the sensor-to-sensor distance x, the ultrasound angle α, the measurement light angle β, and the circumferential deviation φ. The control unit 31 compares the output value for each piece of output information with a value (0 or 1) corresponding to each piece of correct information, and optimizes parameters used for calculation processing in the intermediate layer so that the two values ​​approximate each other. Specifically, the control unit 31 trains the learning model 34M so that, in each output layer, the output value from the output node corresponding to the correct value approaches 1 and the output values ​​from the other output nodes approach 0.

[0077] The control unit 31 determines whether or not there is unprocessed data (S43). For example, if training data has been registered in advance in a training data DB, the control unit 31 determines whether or not there is unprocessed training data among the training data stored in the training data DB. If it is determined that there is unprocessed data (S43: YES), the control unit 31 returns to the processing of step S41, and the learning processing performs the processing of steps S41 to S42 based on the unprocessed training data. If it is determined that there is no unprocessed data (S43: NO), the control unit 31 ends the series of processes.

[0078] By the above-described processing, a learning model 34M is obtained that has been trained to input IVUS images and OCT images and output each piece of information (x, α, β, φ) of the diagnostic imaging catheter 1 used to capture the IVUS images and OCT images. The learning model 34M can be further optimized by repeatedly performing the learning processing using the training data described above. Furthermore, a learning model 34M that has already been trained can be retrained by performing the above-described processing, and in this case, a learning model 34M with higher discrimination accuracy can be obtained.

[0079] In the diagnostic imaging device 100 of this embodiment, an IVUS image and an OCT image are acquired using the diagnostic imaging catheter 1, and then the learning model 34M is used to identify each piece of information (x, α, β, φ) in the diagnostic imaging catheter 1 from the acquired IVUS image and OCT image. Then, the image processing device 3 performs correction processing on the IVUS image or the OCT image using the identified pieces of information (x, α, β, φ), thereby obtaining an IVUS image and an OCT image in which the observation target is matched.

[0080] Fig. 18 is a flowchart showing an example of a correction process procedure for a tomographic image according to the fourth embodiment. The process shown in Fig. 18 is the process shown in Fig. 8 with steps S51 and S52 added between steps S12 and S13. Explanation of the same steps as in Fig. 8 will be omitted. In the image processing device 3 of this embodiment, the control unit 31 performs the processes of steps S11 and S12 shown in Fig. 8, and then interpolates pixels of the optical line data acquired in step S11 by interpolation processing to construct a two-dimensional OCT image (S51). Note that the IVUS image and OCT image here are images in which a shift occurs in the observation position.

[0081] Next, the control unit 31 inputs the IVUS image constructed in step S12 and the OCT image constructed in step S51 into the learning model 34M, and identifies each piece of information (x, α, β, φ) in the diagnostic imaging catheter 1 that captured the IVUS image and the OCT image based on the output values ​​from the learning model 34M (S52). For example, the control unit 31 identifies the output node that outputs the maximum output value among the output nodes of each output layer in the learning model 34M, and identifies the value (distance or angle) associated with the identified output node as each piece of information in the diagnostic imaging catheter 1. The pieces of information (x, α, β, φ) thus identified are stored in, for example, the main memory unit 32.

[0082] Thereafter, the control unit 31 executes the processes from step S13 onward. In this embodiment, the information identified using the learning model 34M is used when calculating the frame pitch during imaging in step S13, when calculating the amount of deviation of the observation position in the axial direction of the shaft 13 and the number of corrected frames in steps S14 and S15, and when calculating the number of corrected lines in the rotational direction of the sensor unit 12 in step S16. By the above-described processes, in this embodiment as well, deviation of the observation positions in the IVUS sensor 12a and the OCT sensor 12b is corrected, and an IVUS image and a corrected OCT image with the observation positions aligned can be provided.

[0083] This embodiment provides the same effects as the above-described embodiments. Furthermore, in this embodiment, the information (x, α, β, φ) used in the correction process to be performed on the IVUS image or OCT image is identified from the IVUS image and the OCT image using the learning model 34M. Therefore, by performing the correction process using such information, it becomes possible to construct an OCT-corrected image in which the observation position matches the IVUS image. The modified examples described in the above-described embodiments can also be applied to this embodiment, as appropriate.

[0084] In this embodiment, the image processing device 3 locally performs the process of identifying each piece of information (each piece of information used in the correction process) in the diagnostic imaging catheter 1 using the learning model 34M, but this configuration is not limited to this. For example, a server may be provided that performs the process of identifying each piece of information using the learning model 34M. In this case, the image processing device 3 may be configured to transmit the IVUS image and the OCT image to the server, and the server may acquire each piece of information identified from the IVUS image and the OCT image. Even with such a configuration, the same process as in this embodiment is possible, and the same effects can be obtained.

[0085] In the present embodiment, when an IVUS image and an OCT image are input, the learning model 34M may be configured to output the number of correction frames and the number of correction lines (correction amount) used in the correction process instead of the information (x, α, β, φ) used in the correction process. In this case, the image processing device 3 can generate a corrected IVUS-corrected image or an OCT-corrected image by using the learning model 34M to identify the correction amount to be used in the correction process and performing the correction process using the identified correction amount. Furthermore, when an IVUS image and an OCT image are input, the learning model 34M may be configured to output an IVUS-corrected image or an OCT-corrected image that has been subjected to the correction process. In this case, the image processing device 3 can acquire a corrected IVUS-corrected image or an OCT-corrected image by using the learning model 34M.

[0086] (Embodiment 5) In the imaging diagnostic apparatus 100 of the first to fourth embodiments described above, the image processing device 3 is configured to construct an OCT-corrected image in which the observation target is matched to the IVUS image. In this embodiment, an image processing device 3 is described that constructs an IVUS-corrected image or an OCT-corrected image in which the observation target is matched to the position of a marker 12c (contrast marker) visualized on an angio image. The imaging diagnostic apparatus 100 of this embodiment can be realized by the same devices as those in the imaging diagnostic apparatus 100 of the first embodiment, and therefore a description of the same components will be omitted. Below, an example will be described in which the imaging diagnostic catheter 1 includes both the IVUS sensor 12a and the OCT sensor 12b. However, the imaging diagnostic catheter 1 of this embodiment may also include only the IVUS sensor 12a or only the OCT sensor 12b. Below, a process for constructing an IVUS-corrected image in which the observation target is matched to the marker 12c on an angio image will be described. However, an OCT-corrected image in which the observation target is matched to the marker 12c on an angio image can also be constructed by a similar process.

[0087] 19 is an explanatory diagram illustrating the deviation of the observation position of the IVUS sensor 12a from the marker 12c. Fig. 19 shows the sensor unit 12 viewed from the radial direction of the shaft 13. When such an imaging diagnostic catheter 1 is imaged using an angiography device 102, an angioimage is obtained in which the marker 12c is visualized, as shown in the lower diagram of Fig. 19. In the sensor unit 12 of this embodiment, the IVUS sensor 12a transmits and receives ultrasound waves at an angle α relative to the direction toward the base end on the central axis of the shaft 13. Therefore, as shown in Fig. 19, the ultrasound path and the imaging range of the marker 12c intersect, and the observation position of the IVUS sensor 12a is deviated in the axial direction of the shaft 13 from the position of the marker 12c at locations other than the intersection. The deviation of the observation position of the IVUS sensor 12a relative to the position of the marker 12c varies depending on the distance y between the placement positions of the IVUS sensor 12a and the marker 12c, the mounting angle of the IVUS sensor 12a (angle α in the transmission and reception direction of ultrasound), and the distance L from the center of rotation of the sensor unit 12.

[0088] When constructing an IVUS image from ultrasound line data, the image processing device 3 of this embodiment constructs an IVUS-corrected image in which the deviation of the observation position of the IVUS sensor 12a from the position of the marker 12c as described above is corrected. Figures 20 and 21 are explanatory diagrams illustrating the process of correcting the deviation of the observation position. Figure 20 is an explanatory diagram of the deviation of the observation position of the IVUS sensor 12a in the axial direction of the shaft 13, and Figure 21 is an explanatory diagram of an IVUS-corrected image in which the observation position is aligned with the position of the marker 12c. 12c in Figure 20 indicates the position of the marker 12c at the timing of capturing the nth frame of the IVUS image, and 1 to 5 in Figure 20 indicate positions r1 to r5, which are distances from the center of rotation of the sensor unit 12, at the position (imaging range) of the marker 12c in the nth frame. Figure 20 shows that the IVUS sensor 12a observed observation position 5 in the n-2 frame, observation position 4 in the n-1 frame, observation position 3 in the n frame, observation position 2 in the n+1 frame, and observation position 1 in the n+2 frame.

[0089] In this situation, as shown in the upper part of Fig. 21, in the IVUS image, observation position 5 is photographed in the n-2th frame, observation position 4 is photographed in the n-1th frame, observation position 3 is photographed in the nth frame, observation position 2 is photographed in the n+1th frame, and observation position 1 is photographed in the n+2th frame. That is, in the example shown in Figs. 20 and 21, the observation timing of the IVUS sensor 12a for observation position 5 is two frames before the arrival timing of the marker 12c, and the observation timing of the IVUS sensor 12a for observation position 4 is one frame before the arrival timing of the marker 12c. Furthermore, the observation timing of the IVUS sensor 12a for observation position 2 is one frame after the arrival timing of the marker 12c, and the observation timing of the IVUS sensor 12a for observation position 1 is two frames after the arrival timing of the marker 12c. Therefore, as shown in the lower part of Figure 21, the image processing device 3 can generate an IVUS corrected image in which the observation position is aligned with the position of the marker 12c by synthesizing each region (region corresponding to the distance from the center of rotation) in multiple frames of IVUS images to construct one frame of IVUS image.

[0090] 22 is a flowchart showing an example of a correction process procedure for a tomographic image according to embodiment 5. It is assumed that the distance y between the IVUS sensor 12a and the marker 12c and the transmission / reception angle α of the ultrasonic waves by the IVUS sensor 12a shown in FIG.

[0091] When intravascular imaging processing by the intravascular inspection device 101 is started, the control unit 31 of the image processing device 3 acquires reflected ultrasonic wave data from the IVUS sensor 12a via the MDU 2 and generates ultrasonic line data from the acquired reflected ultrasonic wave data (S61). Next, the control unit 31 calculates the frame pitch during imaging by the sensor unit 12 (S62). The calculation of the frame pitch is the same as step S13 in FIG. 8. Next, the control unit 31 calculates the amount of deviation of the observation position of the IVUS sensor 12a from the position of the marker 12c at each distance from the center of rotation of the sensor unit 12 (S63). The amount of deviation Δy (unit: mm) of the observation position in the region where the distance from the center of rotation of the sensor unit 12 is L is calculated using the following equation 7. Note that y (unit: mm) is the axial distance between the IVUS sensor 12a and the marker 12c.

[0092] Δy=y-Δx IVUS =yL×cotα…(Equation 7)

[0093] Next, the control unit 31 calculates the amount of correction (number of correction frames) in the axial direction at each distance from the rotation center of the sensor unit 12 based on the amount of deviation of the observation position of the IVUS sensor 12a relative to the marker 12c (S64). The number of correction frames is calculated using the following equation 8 based on the amount of deviation of the observation position calculated in step S63 and the frame pitch calculated in step S62.

[0094] Number of correction frames = Δy / frame pitch ... (Equation 8)

[0095] In this embodiment, instead of calculating the deviation Δy at each distance from the center of rotation of the sensor unit 12, the control unit 31 may calculate the distance L from the center of rotation at each location where the deviation Δy is a multiple of the frame pitch. For example, if the frame pitch is 0.1 mm, the control unit 31 calculates each distance L where the deviation Δy of the observation position is 0.1 mm, 0.2 mm, 0.3 mm, etc. In this case, the number of correction frames corresponding to each calculated distance L can be specified as 1, 2, 3, etc.

[0096] The control unit 31 constructs an IVUS-corrected image in which the observation target is aligned with the position of the marker 12c based on the number of correction frames corresponding to each distance calculated in step S64 (S65). For example, the control unit 31 constructs one frame of an IVUS-corrected image by combining regions of multiple frames of IVUS images using ultrasound line data according to the number of correction frames corresponding to the distance from the center of rotation of the sensor unit 12. In the example shown in FIGS. 20 and 21, one frame of an IVUS-corrected image is constructed by assigning pixel values ​​based on data values ​​corresponding to each region of five ultrasound line data from the n-2th frame to the n+2th frame, in order of the region farthest from the center of rotation of the sensor unit 12. In addition to performing the correction process described above, the control unit 31 interpolates pixels between each line of the ultrasound line data to construct a two-dimensional IVUS image (IVUS-corrected image). The IVUS-corrected image constructed in this manner is an IVUS image in which the contrast range of the marker 12c is the imaging target, and therefore an IVUS-corrected image capturing the location of the marker 12c on the angio image can be provided. In this way, the photographing position of the IVUS corrected image and the position of the marker 12c on the angio image coincide with each other, so that an IVUS image that is easy to interpret can be provided, and it is expected that the accuracy of the procedure performed while checking the IVUS image will be improved. The IVUS corrected image constructed by the above-mentioned processing is displayed on, for example, the display device 4 and presented to the surgeon using the diagnostic imaging catheter 1.

[0097] This embodiment achieves the same effects as the above-described embodiments. Furthermore, this embodiment can provide an IVUS-corrected image in which the observation target is matched with the marker 12c on the angio image. Therefore, even when the diagnostic imaging device 100 is used in a mode in which only IVUS images are acquired, it is possible to present an IVUS image that is easy to interpret. Note that, in this embodiment, a similar process can also be used to construct an OCT-corrected image in which the observation target is matched with the marker 12c on the angio image. In this case, even when the diagnostic imaging device 100 is used in a mode in which only OCT images are acquired, it is possible to present an OCT image that is easy to interpret.

[0098] The configuration of this embodiment can also be applied to the diagnostic imaging apparatus 100 of embodiments 2 and 3, and similar effects can be obtained when applied to embodiments 2 and 3. When applied to embodiments 2 and 3, the information used in the correction process (distance y between the IVUS sensor 12a and the marker 12c, and transmission / reception angle α of the ultrasound by the IVUS sensor 12a) can be obtained by reading the code C attached to the diagnostic imaging catheter 1 or by obtaining it from the server 6. The modified examples described in each of the above-mentioned embodiments can also be applied to this embodiment as appropriate.

[0099] The image processing device 3 in the above-described first to fourth embodiments performs a correction process to match the observation position of the IVUS sensor 12a with the observation position of the OCT sensor 12b. Alternatively, as in the fifth embodiment, for example, a correction process to match the observation position with the position of the marker 12c (enhanced area) may be performed on each of the IVUS image and the OCT image, thereby matching the observation position of the IVUS sensor 12a with the observation position of the OCT sensor 12b. In this case, even if the IVUS image acquisition process and the OCT image acquisition process are performed separately, it is possible to match the observation target in the IVUS image with the observation target in the OCT image by performing a correction process to match the observation position with the position of the marker 12c on the obtained IVUS image and OCT image.

[0100] In each of the above-described embodiments, the IVUS sensor 12a that captures intravascular tomographic images using ultrasound and the OCT sensor 12b that captures intravascular tomographic images using near-infrared light are used, but the present invention is not limited to such a configuration. For example, instead of the IVUS sensor 12a or the OCT sensor 12b, various sensors that can observe the state of blood vessels may be used, such as a sensor that receives Raman scattered light from within the blood vessel to capture intravascular tomographic images, or a sensor that receives excitation light from within the blood vessel to capture intravascular tomographic images.

[0101] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0102] 1 Diagnostic imaging catheter 2 MDU 3. Image processing device 4 Display device 5 Input Devices 6 Server 31 Control Unit 36 Code reader 37 Communications Department 101 Intravascular inspection device 102 Angiography equipment 34M Learning Model 64a Catheter DB C Code P Computer Program

Claims

1. acquiring a signal data set by a first inspection wave irradiated in a direction inclined at a first angle with respect to a longitudinal axis direction of an imaging core used in the catheter; Based on the acquired signal data set, an image is constructed in which an observation target is a direction tilted at a second angle different from the first angle with respect to the long axis direction. Have the computer execute the process, the imaging core includes a first transceiver that irradiates the first inspection wave and receives a reflected wave of the first inspection wave, and a second transceiver that irradiates a second inspection wave in a direction inclined at the second angle with respect to the long axis direction and receives a reflected wave of the second inspection wave, The image to be constructed is constructed based on a signal data set generated by the first inspection wave, and is an image to be observed in a direction in which the second inspection wave is irradiated. program.

2. Based on a circumferential angular difference about the long axis direction between the irradiation direction of the first inspection wave by the first transmitting / receiving unit and the irradiation direction of the second inspection wave by the second transmitting / receiving unit, a correction is performed on the signal data set generated by the first inspection wave or the signal data set generated by the second inspection wave to eliminate the circumferential angular difference. The program according to claim 1 , which causes the computer to execute a process.

3. corresponding each signal data included in the signal data set generated by the first inspection wave to each signal data included in the signal data set generated by the second inspection wave based on the first angle, the second angle, and the distance in the long axis direction between the arrangement positions of the first transceiver unit and the second transceiver unit provided in the imaging core; Based on the associated contents, an image is constructed from the signal data set generated by the first inspection wave, the image being observed in the direction in which the second inspection wave is irradiated.

3. The program according to claim 1, which causes the computer to execute processing.

4. acquiring the first angle, the second angle, and the distance in the long axis direction at the arrangement positions of the first transceiver unit and the second transceiver unit of the imaging core, which are measured during the manufacture of the catheter; An image is constructed from a signal data set generated by the first inspection wave based on the acquired first angle, the second angle, and the distance in the long axis direction, the image being observed in a direction in which the second inspection wave is irradiated.

4. The program according to claim 1, which causes the computer to execute a process.

5. calculating a distance in the long axis direction between an irradiation position of the first inspection wave and an irradiation position of the second inspection wave at each distance from the imaging core to an observation object based on the first angle, the second angle, and a distance in the long axis direction between an arrangement position of the first transceiver unit and the second transceiver unit of the imaging core; calculating a correction amount for constructing an image, the observation target of which is the direction in which the second inspection wave is irradiated, from a signal data set generated by the first inspection wave based on the calculated distance to the irradiation position and the irradiation interval of the first inspection wave; Based on the calculated correction amount, an image is constructed from the signal data set generated by the first inspection wave, the image being observed in the direction in which the second inspection wave is irradiated.

5. The program according to claim 1, which causes the computer to execute a process.

6. inputting the acquired signal data set based on the first inspection wave and the signal data set based on the second inspection wave into a learning model that has been trained to output the first angle, the second angle, and the distance in the long axis direction at the arrangement positions of the first transceiver unit and the second transceiver unit of the imaging core when the signal data set based on the first inspection wave and the signal data set based on the second inspection wave are input, and identifying the first angle, the second angle, and the distance in the long axis direction; Based on the identified first angle, the second angle, and the distance in the long axis direction, an image is constructed from a signal data set generated by the first inspection wave, with the direction in which the second inspection wave is irradiated as an observation target.

6. The program according to claim 1, which causes the computer to execute a process.

7. A signal data set is obtained by irradiating a first test wave in a direction inclined at a first angle with respect to a longitudinal axis direction of an imaging core used in a catheter; Based on the acquired signal data set, an image is constructed in which an observation target is a direction tilted at a second angle different from the first angle with respect to the long axis direction. Have the computer execute the process, the imaging core includes a first transmitting / receiving unit that irradiates the first inspection wave and receives a reflected wave of the first inspection wave, and a contrast marker that is made of an X-ray opaque material; causing the computer to execute a process of associating each signal data included in the signal data set generated by the first inspection wave with an X-ray image of the contrast marker, based on the distance in the long-axis direction between the first transmitting / receiving unit provided in the imaging core and the arrangement position of the contrast marker and the first angle; The image to be constructed is an image constructed from the signal data set generated by the first inspection wave based on the associated content, and the image is an image in which the contrast range of the contrast marker in the X-ray image is the observation target. program.

8. acquiring a signal data set by a first inspection wave irradiated in a direction inclined at a first angle with respect to a longitudinal axis direction of an imaging core used in the catheter; Based on the acquired signal data set, an image is constructed in which an observation target is a direction tilted at a second angle different from the first angle with respect to the long axis direction. The computer executes the processing, the imaging core includes a first transceiver that irradiates the first inspection wave and receives a reflected wave of the first inspection wave, and a second transceiver that irradiates a second inspection wave in a direction inclined at the second angle with respect to the long axis direction and receives a reflected wave of the second inspection wave, The image to be constructed is constructed based on a signal data set generated by the first inspection wave, and is an image to be observed in a direction in which the second inspection wave is irradiated. Image processing methods.

9. an acquisition unit that acquires a signal data set generated by a first inspection wave irradiated in a direction inclined at a first angle with respect to a longitudinal axis direction of an imaging core used in the catheter; an image constructing unit that constructs an image of an observation target in a direction tilted at a second angle different from the first angle with respect to the long axis direction based on the acquired signal data set; Equipped with the imaging core includes a first transceiver that irradiates the first inspection wave and receives a reflected wave of the first inspection wave, and a second transceiver that irradiates a second inspection wave in a direction inclined at the second angle with respect to the long axis direction and receives a reflected wave of the second inspection wave, The image constructing unit constructs an image, the observation target of which is a direction in which the second inspection wave is irradiated, based on a signal data set generated by the first inspection wave. Image processing device.

Images