Image diagnostic device, catheter image display method, and computer program

Abstract

Provided are an image diagnostic device, a catheter image display method, and a computer program that are capable of appropriately presenting information regarding the inside and outside of a luminal organ in the diagnosis or treatment of the organ by the use of a medical catheter. This image diagnostic device comprises: a display unit; a connection unit that is connected to an image diagnostic catheter which is for a luminal organ and which comprises an ultrasonic transmission / reception unit at the tip-end side thereof; and a processing unit that, on the basis of a signal obtained from the image diagnostic catheter, generates a plurality of tomographic images at different depths in the radial direction of the luminal organ and causes the display unit to display the generated tomographic images at different depths.

Description

Image diagnostic device, catheter image display method, and computer program

[0001] The present invention relates to an image diagnostic apparatus, a catheter image display method, and a computer program for processing medical images.

[0002] Medical catheters are used for diagnosing or treating lesions in hollow organs such as blood vessels and vasculature. Diagnostic medical catheters are equipped with ultrasonic sensors or light-receiving sensors and moved into the organ, and images based on signals obtained from the sensors are used for diagnosis.

[0003] Diagnostic imaging of vascular organs, particularly blood vessels, is essential for safely and reliably performing procedures such as percutaneous coronary intervention (PCI). For this reason, intravascular imaging techniques such as intravascular ultrasound (IVUS) and optical coherence tomography (OCT) using medical catheters have become widespread, along with angiography, which uses contrast media to capture images from outside the body.

[0004] In recent years, the number of cases in which IVUS is used for not only arterial but also venous diseases has been increasing. Patent Document 1 discloses a technique for evaluating deep vein thrombosis by inserting an ultrasound imaging catheter into a vein and determining the acuteness of the thrombus.

[0005] Patent document 2 discloses a method for displaying an ultrasound image obtained by inserting an ultrasound imaging catheter into a vein, while identifying anatomical features of the venous blood vessel and displaying an indicator for identifying the orientation of the ultrasound image.

[0006] Patent Document 3 discloses that when displaying an ultrasound image obtained by inserting an ultrasound imaging catheter into a blood vessel, a user can select an image type depending on, for example, whether to observe the coronary artery structure or the peripheral vein structure, etc. In Patent Document 3, the system sets gain, contrast, etc. depending on the selected image type.

[0007] Patent No. 7181276 Special Publication No. 2022-509391 Publication Special Publication No. 2022-509395

[0008] When targeting veins, information is required about the size of the venous blood vessel diameter, the effect of pulsation, and other blood vessels outside the blood vessels (e.g., arteries), but it is not easy to present both information inside the blood vessels and information outside the blood vessels with equal detail in a single tomographic image.

[0009] An object of one aspect of the present disclosure is to provide an imaging diagnostic device, a catheter image display method, and a computer program that can appropriately present information about the inside and outside of an organ when diagnosing or treating a hollow organ using a medical catheter.

[0010] (1) An imaging diagnostic device according to one embodiment of the present disclosure includes a display unit, a connector connected to a diagnostic imaging catheter for a tubular organ having an ultrasound transmitter / receiver at its tip, and a processor that generates multiple tomographic images of the tubular organ at different radial depths based on signals obtained from the diagnostic imaging catheter and displays the generated tomographic images at different depths on the display unit.

[0011] (2) In the imaging diagnostic device of (1) above, the different depths may include a first depth suitable for observing the inside of the hollow organ and a second depth suitable for observing other organs located outside the hollow organ.

[0012] (3) In the image diagnostic device of (1) or (2) above, the processing unit may adjust at least one of the tomographic images at different depths in at least one of image settings of gain, contrast, STC (Sensitivity Time Control), and gamma correction according to the corresponding depth, and display the adjusted image on the display unit.

[0013] (4) In the imaging diagnostic device of any one of (1) to (3) above, the processing unit may generate a longitudinal cross-sectional image of the tubular organ in the axial direction based on a signal obtained from the imaging diagnostic catheter, and display the longitudinal cross-sectional image on the display unit together with the cross-sectional images at different depths.

[0014] (5) In the imaging diagnostic device according to any one of (1) to (4) above, the processing unit may accept settings of the different depths.

[0015] (6) In the imaging diagnostic device of (2) above, the processing unit may display a tomographic image of the second depth only when displaying a tomographic image of a position among different positions in the axial direction of the tubular organ where the other organ is determined to be present within a distance corresponding to the second depth from the tubular organ.

[0016] (7) In the imaging diagnostic device of (2) above, the processing unit may accept a selection operation to display / hide each of the tomographic image at the first depth and the tomographic image at the second depth, and change the display mode of the tomographic image being displayed on the display unit in accordance with the accepted operation.

[0017] (8) In the diagnostic imaging device of (6) or (7) above, the processing unit may change the display of tomographic images at different depths while the diagnostic imaging catheter is moving in the axial direction of the hollow organ.

[0018] (9) In one embodiment of the catheter image display method of the present disclosure, a computer generates multiple cross-sectional images of a tubular organ at different radial depths based on signals obtained from a diagnostic imaging catheter of the tubular organ that is equipped with an ultrasound transmitter / receiver unit at its tip, and outputs the generated cross-sectional images at different depths to a display device.

[0019] (10) A computer program according to one embodiment of the present disclosure causes a computer to perform a process of generating multiple cross-sectional images of a tubular organ at different radial depths based on signals obtained from a diagnostic imaging catheter for the tubular organ that is equipped with an ultrasound transmitter / receiver at its tip, and outputting the generated cross-sectional images at different depths to a display device.

[0020] According to the present disclosure, it is possible to appropriately present information about the inside and outside of a hollow organ when diagnosing or treating the organ using a medical catheter.

[0021] 1 is a schematic diagram of an image diagnostic apparatus. FIG. 1 is an explanatory diagram showing the operation of a catheter. FIG. 2 is a block diagram showing the configuration of an image processing apparatus. FIG. 3 is a schematic diagram of a segmentation model. FIG. 4 is a flowchart showing an example of an information processing procedure by an image processing apparatus. FIG. 5 is a flowchart showing an example of an information processing procedure by an image processing apparatus. FIG. 6 is a flowchart showing an example of an information processing procedure by an image processing apparatus. FIG. 7 is a diagram showing an example of a contrast adjustment process for a tomographic image. FIG. 8 is a diagram showing an example of a gamma adjustment process for a tomographic image. FIG. 9 is a diagram showing an example of a gain adjustment process for a tomographic image. FIG. 10 is a diagram showing an example of an STC adjustment process for a tomographic image. FIG. 11 is a diagram showing an example of a screen displayed on a display device. FIG. 12 is a flowchart showing an example of a setting screen. FIG. 13 is a flowchart showing an example of an information processing procedure by an image processing apparatus of a second embodiment. FIG. 14 is a flowchart showing an example of an information processing procedure by an image processing apparatus of the second embodiment. FIG. 15 is a diagram showing an example of a screen displayed on a display device in the second embodiment. FIG. 16 is a flowchart showing an example of an information processing procedure by an image processing apparatus of a third embodiment. FIG. 17 is a flowchart showing an example of an information processing procedure by an image processing apparatus of the third embodiment. FIG. 18 is a flowchart showing an example of an information processing procedure by an image processing apparatus of the third embodiment. FIG. 19 is a flowchart showing an example of an information processing procedure by an image processing apparatus of the third embodiment. FIG. 19 is a diagram showing an example of a screen displayed on a display device 4 in the third embodiment.

[0022] Embodiments of an imaging diagnostic apparatus, a catheter image display method, and a computer program according to the present disclosure will be described below with reference to the drawings. In the following embodiments, information processing will be described for blood vessels as an example of a hollow organ, but the hollow organ is not limited to blood vessels.

[0023] 1 is a schematic diagram of an imaging diagnostic apparatus 100. The imaging diagnostic apparatus 100 includes a catheter 1, an MDU (Motor Drive Unit) 2, an image processing device 3, a display device 4, and an input device 5.

[0024] The catheter 1 is a flexible medical tube known as an imaging catheter, through which a shaft with an imaging device 11 connected to the tip is inserted. The imaging device 11 and the shaft inside the catheter 1 are connected to the MDU 2 and the image processing device 3 via a connector 12 on the base end side.

[0025] The imaging device 11 (see FIG. 2) of the catheter 1 includes an ultrasound probe including an ultrasound transducer and an ultrasound sensor for IVUS.

[0026] A signal obtained by the imaging device 11 of the catheter 1 is output to the base end side of the catheter 1 via a signal line arranged inside the shaft. The image processing device 3 to which the catheter 1 is connected operates the imaging device 11 of the catheter 1, processes the signal obtained from the imaging device 11, and displays the image generated by the processing on the display device 4.

[0027] The MDU 2 is a driving unit attached to the base end of the catheter 1, and controls the operation of the catheter 1 by driving an internal motor in response to the operation of a doctor or an examination operator.

[0028] The image processing device 3 generates an image by converting the distribution of radially reflected waves from the inside of the tubular organ into brightness based on a signal obtained from the imaging device 11 of the catheter 1, and then generates a tomographic image by polar coordinate conversion of the generated 360-degree image (see FIG. 2 ). The image processing device 3 outputs the generated tomographic image and information obtained by processing the tomographic image to a built-in display unit 35 or an externally connected display device 4. The image processing device 3 is, for example, a medical device such as an intravascular imaging diagnostic device, an angiography device, an external monitor, or an electrocardiograph. The image processing device 3 may be a smartphone, a tablet terminal, a laptop personal computer (PC), or a desktop PC, and functions based on a software program as a medical device such as an intravascular imaging diagnostic device depending on its intended use. The configuration of the image processing device 3 will be described in detail below.

[0029] The display device 4 uses a liquid crystal display panel, an organic EL (Electro Luminescence) display panel, etc. The display device 4 displays the medical images generated by the image processing device 3 and information related to the medical images.

[0030] The input device 5 is an input interface that accepts operations for the image processing device 3. The input device 5 may be a keyboard, a mouse, or the like, or may be a touch panel, soft keys, hard keys, or the like built into the display device 4. The input device 5 may also accept operations based on voice input. In this case, the input device 5 uses a microphone and a voice recognition engine.

[0031] Fig. 2 is an explanatory diagram showing the operation of the catheter 1. In Fig. 2, the catheter 1 is inserted into a tubular blood vessel L by a doctor or an examination operator along a guide wire W that is inserted into a vein shown in Fig. 2. Driven by the MDU 2, the catheter 1 moves within the blood vessel L as indicated by the arrow in the figure, and the imaging device 11 scans the inside of the blood vessel in a spiral pattern.

[0032] In the diagnostic imaging apparatus 100 of this embodiment, the image processing device 3 acquires a signal for each scan output from the imaging device 11 of the catheter 1. Each scan involves emitting a detection wave from the imaging device 11 in the radial direction and detecting reflected light, resulting in spiral scanning. The detection wave is emitted with an intensity that reaches the outside of the blood vessel L. The processing unit 30 of the image processing device 3 logarithmically converts the waveform of the reflected wave and obtains a brightness distribution for each scan by converting the amplitude after logarithmic conversion into brightness values.

[0033] The image processing device 3 generates a tomographic image (cross-sectional image) (I1 in FIG. 2) by polar coordinate transformation (inverse transformation) of a rectangular image (I0 in FIG. 2) in which the brightness distribution for each scan is aligned in the radial direction for every 360 degrees and arranged in a rectangular shape. The tomographic image I1 is also called a frame image. The reference point (center) of the tomographic image I1 corresponds to the range of the catheter 1 (not imaged). The image processing device 3 may use specific hardware to perform the process of generating the tomographic image I1 from the signal obtained from the imaging device 11.

[0034] The image processing device 3 may further generate a longitudinal image (longitudinal cross-sectional image) I2 in which the pixel values ​​on a straight line (indicated by a thick arrow) at an arbitrary angle passing through the reference point of the tomographic image I1 are arranged along the length direction (long axis direction) of the blood vessel by the catheter 1.

[0035] The diagnostic imaging device 100 of the present disclosure is used by a physician to identify lesions and anatomical features inside and outside a vein, which is a hollow organ. To this end, an examination operator or a physician visually checks, in real time, a tomographic image I1 obtained by the imaging device 11 on the display device 4 while moving the catheter 1. Since it is difficult to change the display depth of a single tomographic image I1 in real time, the image processing device 3 displays tomographic images at different depths on the display device 4.

[0036] When a vein is the insertion target, it is important to observe both the vein itself (shown in FIG. 2 ) and the organs outside the vein, such as arteries. Therefore, the image processing device 3 further generates, from the generated tomographic image I1, a tomographic image I11 at a first display depth, which primarily observes the inside of the vein, and a tomographic image I12 at a second display depth, which primarily observes the outside of the vein. The image processing device 3 appropriately uses the obtained rectangular image I01, tomographic image I1, tomographic image I11, tomographic image I12, and long-axis image, performs image processing on each, and outputs images that make it easier to understand the anatomical features of the vein and the state of the lesion. The processing performed by the image processing device 3 is described in detail below.

[0037] 3 is a block diagram showing the configuration of the image processing device 3. The image processing device 3 is a computer, and includes a processing unit 30, a storage unit 31, and an input / output I / F 32.

[0038] The processing unit 30 includes one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), tensor processing units (TPUs), etc. The processing unit 30 has a built-in non-temporary storage medium such as a random access memory (RAM), and performs calculations based on a computer program P3 stored in the storage unit 31 while storing data generated during processing in the non-temporary storage medium.

[0039] The storage unit 31 is a non-volatile storage medium such as a hard disk or flash memory. The storage unit 31 stores the computer program P3 read by the processing unit 30, setting data, and the like. The setting data includes a first display depth and a second display depth. For example, if the first display depth is set to 10 mm for veins in the lower limbs, the second display depth may be set to 60 mm. The first display depth and the second display depth may be stored in combination with multiple versions and associated with data identifying the combination. A combination of a first display depth of 2 mm and a second display depth of 60 mm, or a combination of a first display depth of 5 mm and a second display depth of 60 mm may be possible. The setting data includes image settings for gain, contrast, STC (Sensitivity Time Control), and gamma correction for each of the first display depth and the second display depth. The storage unit 31 also stores a trained segmentation model 31M. The segmentation model 31M will be described later.

[0040] The computer program P3 and the segmentation model 31M may be copies of the computer program P9 and the segmentation model 91M stored in a non-temporary storage medium 9 outside the device, read out via the input / output I / F 32. The computer program P3 and the segmentation model 31M may be distributed by a remote server device, acquired by the image processing device 3 via a communication unit (not shown), and stored in the storage unit 31.

[0041] The input / output I / F 32 is an interface to which the catheter 1, the display device 4, and the input device 5 are connected. The processing unit 30 acquires signal data output from the imaging device 11 via the input / output I / F 32. The processing unit 30 outputs screen data of a screen including the generated tomographic images I1, I11, I12 and / or long-axis images to the display device 4 via the input / output I / F 32. The processing unit 30 accepts operation information input to the input device 5 via the input / output I / F 32.

[0042] 4 is a schematic diagram of the segmentation model 31M. The segmentation model 31M is a model trained to output an image showing the regions of one or more objects appearing in an input tomographic image I1 (or tomographic images I11 and I12). The segmentation model 31M is, for example, a model that performs semantic segmentation. The segmentation model 31M is designed to output an image in which each pixel in the input image is tagged with data indicating which object the pixel is within.

[0043] The segmentation model 31M uses, for example, a so-called U-net, in which a convolution layer, a pooling layer, an upsampling layer, and a softmax layer are symmetrically arranged, as shown in FIG. 4 . When a tomographic image I1 generated by a signal from the catheter 1 is input, the segmentation model 31M outputs a tag image IS1. The tag image IS1 is obtained by tagging the pixels at the positions of the blood vessel lumen area, the membrane area corresponding to the area between the blood vessel lumen boundary and the blood vessel boundary, including the tunica media, the area in which the guidewire W and its reflection are captured, and the area corresponding to the catheter 1, with different pixel values ​​(shown by different types of hatching and solid color in FIG. 4 ). The segmentation model 31M further identifies the area of ​​lipid plaque formed in the blood vessel. The segmentation model 31M identifies the area in which fibrous plaque or calcified plaque is captured.

[0044] As described above, the segmentation model 31M is exemplified by semantic segmentation and U-net, but it goes without saying that it is not limited to this. Alternatively, the segmentation model 31M may be a model that realizes individual recognition processing using instance segmentation or the like. The segmentation model 31M is not limited to being U-net-based, and may also be a model based on SegNet, R-CNN, or an integrated model with other edge extraction processing.

[0045] The processing unit 30 identifies the blood (lumen area), intima area, adventitia area, etc. of the blood vessels shown in the tomographic image I1 based on pixel values ​​in a tag image IS1 obtained by inputting the tomographic image I1 into the segmentation model 31M and their coordinates within the image. By identifying the blood vessel area, the processing unit 30 detects the lumen boundary and blood vessel boundary of the blood vessels shown in the tomographic image I1. Strictly speaking, the blood vessel boundary is the external elastic membrane (EEM) between the tunica media and adventitia of the blood vessel.

[0046] The processing unit 30 may identify the range of lipid plaque and each of fibrous plaque and calcified plaque based on pixel values ​​and coordinates within the tag image IS1 obtained by inputting the tomographic image I1 into the segmentation model 31M.

[0047] The processing unit 30 of the image processing device 3 identifies the lumen boundary of the vascular lumen area from each range identified for each of the tomographic images I1, I11, and I12, and calculates values ​​such as the maximum diameter, minimum diameter, and average inner diameter inside the lumen boundary. Furthermore, the processing unit 30 can calculate the ratio of the cross-sectional area of ​​calcified plaque, fibrous plaque, and lipid plaque identified for each of the IVUS tomographic images I1, I11, and I12 to the area inside the vascular boundary (hereinafter referred to as plaque burden). Specifically, the plaque burden is calculated for each of the tomographic images I1, I11, and I12 using the formula "1 - (lumen area / vascular boundary area)." The image processing device 3 of the present disclosure may output graphs of the distribution of the average lumen diameter and the distribution of plaque burden relative to the position in the longitudinal direction of the blood vessel.

[0048] The processing procedure by the image processing device 3 will be described with reference to a flowchart. Figures 5 and 6 are flowcharts showing an example of the information processing procedure by the image processing device 3. When an operation to start scanning is performed on the image diagnostic device 100 and a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing device 3 starts the following processing.

[0049] Each time the processing unit 30 acquires a predetermined amount (e.g., 360 degrees) of signal data from the imaging device 11 of the catheter 1 (step S101), it performs polar coordinate transformation (inverse transformation) on the rectangular image I0 arranged in a rectangle to generate a tomographic image I1 (step S102).

[0050] The processing unit 30 inputs the tomographic image I1 to the segmentation model 31M (step S103). The processing unit 30 identifies the recognition result of the area captured in the tomographic image I1 based on the tag image IS output from the segmentation model 31M (step S104). In step S104, the processing unit 30 identifies data indicating anatomical features such as the maximum value, minimum value, and mean lumen of the range inside the lumen boundary of the vein being observed. In step S104, the processing unit 30 may determine whether an artery is captured outside the vein.

[0051] The processing unit 30 stores the signal data acquired in step S101, the tomographic image I1, and the recognition results identified in step S104 in the memory unit 31 in association with data on the position on the long axis within the blood vessel (vein) (step S105).

[0052] The processing unit 30 generates a tomographic image I11 at a first display depth and a tomographic image I12 at a second display depth, which are different in the radial direction, from the tomographic image I1 generated in step S102 (step S106). Since the data on the first display depth and the second display depth are included in the setting data of the storage unit 31 as described above, the processing unit 30 reads these data and generates the tomographic image I11 and the tomographic image I12 from the tomographic image I1.

[0053] The processing unit 30 performs image processing on the generated tomographic image I11 at the first display depth based on the image settings corresponding to the first display depth (step S107). The image processing is at least one of gain adjustment, contrast adjustment, STC adjustment (i.e., brightness adjustment), and gamma correction.

[0054] The processing unit 30 performs image processing on the tomographic image I12 at the second display depth based on the image settings corresponding to the second display depth (step S108). The image processing in step S108 is similarly at least one of gain adjustment, contrast adjustment, STC adjustment (i.e., brightness adjustment), and gamma correction.

[0055] The processing unit 30 outputs the tomographic image I11 at the first display depth after performing the image processing of step S107 and the tomographic image I12 at the second display depth after performing the image processing of step S108 so that they can each be displayed in real time on the screen displayed on the display device 4 (step S109).

[0056] The processing unit 30 stores the post-image-processing tomographic image I11 at the first display depth and the post-image-processing tomographic image I12 at the second display depth in the storage unit 31 in association with data on the longitudinal axis of the blood vessel (vein) (step S110). Since the tomographic image I1 is already stored in the storage unit 31, the processing of step S110 is not essential.

[0057] The processing unit 30 outputs the text indicating the recognition result identified in step S104 to fit within the screen displayed on the display device 4 in step S109 (step S111).

[0058] The processing unit 30 generates a long-axis image I2, which is a longitudinal cross-sectional image at a predetermined angle, based on the tomographic image I1 stored in association with data on the long axis of the blood vessel through the scanning up to this point (step S112). In step S112, the processing unit 30 may generate a tomographic image I11 at the first display depth, a tomographic image I12 at the second display depth, or either one of the long-axis images. The processing unit 30 outputs the generated long-axis image I2 so that it can be displayed in real time on the screen of the display device 4 (step S113).

[0059] The processing unit 30 determines whether scanning by the imaging device 11 of the catheter 1 has been completed (step S114). If scanning is being performed automatically in step S114, the processing unit 30 determines whether scanning has been performed for a set length. If the processing unit 30 is being operated by an examination operator or a doctor in step S114, the processing unit 30 determines whether a stop button has been pressed.

[0060] If it is determined that the scanning is not complete (S114: NO), the processing unit 30 returns the process to step S101 and generates the next tomographic image I1. If it is determined that the scanning is complete (S114: YES), the processing unit 30 ends the process.

[0061] During or after the scan is completed, the tomographic image I1 is stored in association with the position on the long axis of the blood vessel in step S105. Therefore, the processing unit 30 may sequentially generate long axis images I2 based on these images and output them to the display device 4 for display.

[0062] The processing procedures shown in Figures 5 and 6 are merely examples, and the processing order may be partially interchanged as long as there are no discrepancies. In the processing procedures shown in Figures 5 and 6, the processing unit 30 generates a tomographic image I11 at the first display depth and a tomographic image I12 at the second display depth (S106). However, step S106 may be omitted. When display at the first display depth is selected, the tomographic image I1 may be expanded within the range of the first display depth, and image processing for the first display depth may be performed on the display device 4 before being displayed. When display at the second display depth is selected, the tomographic image I1 may be expanded within the range of the second display depth, and image processing for the second display depth may be performed on the display device 4 before being displayed. In other words, the processing unit 30 may perform image processing depending on the display depth at which the image is to be displayed on the display device 4.

[0063] FIG. 7 is a diagram showing an example of the contrast adjustment process for the tomographic images I1, I11, and I12. The contrast is adjusted by changing the dynamic range of the luminance distribution. FIG. 7 shows how the contrast increases as the dynamic range is reduced. When the dynamic range of the ultrasound intensity of the imaging device 11 is reduced, the gradation of soft tissue with weak reflection intensity becomes clearer, while tissue that is harder than a certain level appears at approximately the same high brightness. Therefore, the setting data is initially set to adjust the contrast more strongly for the first display depth, which targets lesions such as plaque within the blood vessel, and to adjust the contrast more weakly for the second display depth, which is intended to detect the presence or absence of other blood vessel walls outside the blood vessel.

[0064] FIG. 8 shows an example of gamma adjustment processing for tomographic images I1, I11, and I12. Gamma adjustment is a correction that nonlinearly converts brightness levels. By adjusting the gamma value, the difference in the high-brightness range is reduced and the difference in the low-brightness range is increased, or conversely, the difference in the low-brightness range is reduced and the difference in the high-brightness range is increased. In the gamma adjustment shown in FIG. 8, the original tomographic image on the left is converted to the corrected tomographic image on the right by reducing the difference in the low-brightness range and increasing the difference in the high-brightness range. In this case, pixel values ​​of medium brightness are darker, and even slight changes in the high-brightness range result in a significant change in brightness, resulting in a higher brightness. The setting data initially sets the gamma value for the first display depth to be increased as shown in FIG. 8, and the gamma value for the second display depth to be set to a value smaller than that for the second display depth.

[0065] 9 is a diagram showing an example of the processing contents of gain adjustment for tomographic images I1, I11, and I12. Gain adjustment involves amplifying the amplitude of the waveform of the reflected ultrasound waves, and adjusting the brightness and amplitude up or down. Adjusting to increase the gain increases the brightness value of the small amplitude portion of the reflected wave (low brightness) corresponding to soft tissue, making the entire image brighter. Adjusting to decrease the gain decreases the brightness value of the medium amplitude portion of the reflected wave (medium brightness), making the entire image darker. The setting data is initially set to decrease the gain for the first display depth and increase the gain for the second display depth.

[0066] FIG. 10 shows an example of the STC adjustment process for tomographic images I1, I11, and I12. In the STC adjustment, gain is adjusted to vary for each display depth. In FIG. 10, the display depth is indicated by an arrow, and the range from near the catheter 1 to the outside is indicated from the top to the bottom of the arrow. In the STC adjustment shown in FIG. 10, the gain is increased in the range from the middle to the outside compared to the area near the catheter. For example, for the first display depth, which is intended to observe the inside of a blood vessel, the membrane area is brightened. Initially, the setting data is set so that for the first display depth, the gain is increased in the middle to display the membrane area brightly, as shown in FIG. 10, and for the second display depth, which is intended to observe the outside of the blood vessel, the gain is increased from the middle to the outside to make it easier to visually determine whether other blood vessels (arteries) exist outside the blood vessel.

[0067] The memory unit 31 of the image processing device 3 stores the contents of the various image processing operations shown in Figures 7 to 10, distinguishing between image settings corresponding to a first display depth in which the inside of blood vessels is the main observation target, and image settings corresponding to a second display depth in which the outside of blood vessels is also the observation target.

[0068] Fig. 11 is a diagram showing an example of a screen 400 displayed on the display device 4. The screen 400 includes tomographic images I11 and I12 at different display depths. In the example of Fig. 11, the tomographic image I11 at the first display depth has a display depth of 10 mm, and the tomographic image I12 at the second display depth has a display depth of 60 mm.

[0069] In the example of the screen 400 in Fig. 11, a tomographic image I11 at a first display depth, which is primarily an observation target of the inside of a blood vessel, is image-processed to reduce brightness and adjust gain for clarity, whereas a tomographic image I12 at a second display depth, which is primarily an observation target of the outside of a blood vessel, is image-processed to increase brightness to emphasize brightness and clearly show whether or not other blood vessels are present on the outside.

[0070] The screen 400 further includes text 401 of data indicating anatomical features identified based on the recognition results for the tomographic image I1. In the example of Fig. 11, the text indicates the maximum and minimum values ​​of the lumen diameter and the maximum and minimum values ​​of the vascular boundary at the current position on the long axis while being displayed in real time.

[0071] The screen 400 further includes a long-axis image I2 at a default angle to the current position on the long axis. A cursor 402 indicating the current position on the long axis is superimposed on the long-axis image I2.

[0072] The screen 400 includes a setting button 403. When the setting button 403 is selected, the setting button 403 may be selectable at any time, or may be disabled so as not to be selectable while blood vessels are being scanned.

[0073] 11, when the outside of a blood vessel is also to be observed, a tomographic image I11 at a first display depth in which the inside of the blood vessel is the main observation target, and a tomographic image I12 at a second display depth in which the outside of the blood vessel is the main observation target are generated from a tomographic image I1, and both are displayed. Moreover, because appropriate image processing is performed on both the inside and outside of the blood vessel due to the difference in display depth, information about the inside and outside of the blood vessel can be displayed appropriately.

[0074] As described above, the first display depth and the second display depth are stored as setting data in the storage unit 31. The setting data can be changed by the examination operator or doctor before starting scanning or during operation. Fig. 12 is a flowchart showing an example of a processing procedure for accepting the setting of the display depth. When the setting button is selected by the input device 5, the processing unit 30 of the image processing device 3 executes the following processing.

[0075] The processing unit 30 displays a setting screen (step S301) and accepts input of setting values ​​for the first display depth and the second display depth included in the setting screen via the input device 5 (step S302). The processing unit 30 accepts input of image setting parameters for each of the first display depth and the second display depth to be set (step S303). In step S303, the processing unit 30 may accept input for each of gain, contrast, STC (brightness), and gamma correction via dials or slide bars provided on the input device 5.

[0076] The processing unit 30 stores the setting values ​​of the first and second display depths and the parameters related to image settings as setting data in the storage unit 31 (step S304), and ends the process.

[0077] Fig. 13 is a diagram showing an example of the setting screen 430. As shown in Fig. 13, the setting screen 430 includes a first area 431 including slide bars for adjusting the display depth setting value and slide bars for adjusting gain, contrast, STC, and gamma correction for each of the first and second display depths, and a second area 432. The STC adjustment is performed by accepting a mask value in a range corresponding to the catheter 1 and gain adjustment parameters (see Fig. 10) for each depth based on the mask value.

[0078] The setting screen 430 includes a setting button 433. When the setting button 433 is selected, the processing unit 30 stores the input setting values ​​and parameters.

[0079] By making the setting data including the first display depth and the second display depth adjustable, it becomes possible for the examination operator and the doctor to display the blood vessels (veins) to be observed at settings that are easy to view.

[0080] Second Embodiment In a second embodiment, the image processing device 3 does not constantly output the tomographic image I11 at the first display depth and the tomographic image I12 at the second display depth, but switches between the images as needed.

[0081] The configuration of the diagnostic imaging device 100 in the second embodiment is the same as that of the diagnostic imaging device 100 in the first embodiment, except for the processing procedures and display contents described below. Therefore, the same components of the diagnostic imaging device 100 in the second embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0082] 14 and 15 are flowcharts showing an example of an information processing procedure by the image processing device 3 of the second embodiment. The processing unit 30 of the image processing device 3 starts the following processing when an operation to start scanning of the image diagnostic apparatus 100 is performed by the input device 5 and a signal is output from the imaging device 11 of the catheter 1. Of the processing procedures shown in the flowcharts of Fig. 14 and Fig. 15, steps common to the processing procedures shown in the flowcharts of Fig. 5 and Fig. 6 of the first embodiment are assigned the same step numbers and detailed descriptions thereof will be omitted.

[0083] The processing unit 30 of the image processing device 3 stores the signal data, the tomographic image I1 generated from the signal data, and the recognition result based on the segmentation model 31M in the storage unit 31 (S105). Based on the stored data, the processing unit 30 determines whether or not another organ, specifically an artery, exists within a distance corresponding to the second display depth outside the organ into which the catheter 1 is inserted (step S121). In step S121, the processing unit 30 determines whether or not an artery exists by determining whether or not an area recognized as an artery is included in the tomographic image I1 generated from the reflection of ultrasound up to the second display depth.

[0084] If it is determined that another organ is present within the distance of the second display depth (S121: YES), the processing unit 30 generates a tomographic image I11 of a first display depth that differs in the radial direction and a tomographic image I12 of a second display depth from the tomographic image I1 (S106).

[0085] The processing unit 30 performs image processing on the generated tomographic image I11 at the first display depth based on the image settings corresponding to the first display depth (S107), and performs image processing on the tomographic image I12 at the second display depth based on the image settings corresponding to the second display depth (S108).

[0086] The processing unit 30 outputs the processed tomographic image I11 at the first display depth and the processed tomographic image I12 at the second display depth so that they can be displayed in real time on the screen of the display device 4 (S109). The processing unit 30 stores the processed tomographic image I11 at the first display depth and the processed tomographic image I12 at the second display depth (S110).

[0087] The processing unit 30 outputs text indicating the recognition result to the display device 4 (step S111). The processing unit 30 generates a long axis image I2 (step S112), outputs the generated long axis image I2 to the display device 4 (step S113), and proceeds to step S114.

[0088] If it is determined in step S121 that no other organs exist within the distance of the second display depth (S121: NO), the processing unit 30 generates a tomographic image I11 at the first display depth, which mainly focuses on the inside of a blood vessel, from the tomographic image I1 (step S122).The processing unit 30 then performs image processing on the generated tomographic image I11 at the first display depth based on the image settings corresponding to the first display depth (step S123).

[0089] The processing unit 30 outputs the tomographic image I11 at the first display depth after the image processing of step S123 so that it can be displayed in real time on the screen of the display device 4 (step S124). The processing unit 30 stores the tomographic image I11 at the first display depth after the image processing in the storage unit 31 in association with data on the position on the long axis of the blood vessel (step S125). Since the tomographic image I1 is already stored in the storage unit 31, the processing of step S125 is not essential.

[0090] The processing unit 30 proceeds to step S111. As a result, the tomographic image I11 at the first display depth and the tomographic image I12 at the second display depth are displayed together on the display device 4 only when another organ is captured within the range of the second display depth.

[0091] 16 and 17 are diagrams showing examples of a screen 400 displayed on the display device 4 in the second embodiment. The screen 400 in Fig. 16 and the screen 400 in Fig. 17 show examples that change in response to the position of the catheter 1 in the blood vessel into which the catheter 1 is inserted or the passage of time. Similar to the example of the screen 400 shown in Fig. 11 of the first embodiment, the screen 400 shown in Fig. 16 and 17 includes at least a tomographic image I11, text 401 of data indicating anatomical features, a long-axis image I2, and a cursor 402.

[0092] 16 shows an example of a display when it is determined that no other blood vessels (arteries) exist outside the blood vessel within a distance corresponding to the second display depth on the distal side of the blood vessel into which the catheter 1 has been inserted. In FIG. 16, since it is determined that no other blood vessels exist outside the blood vessel, the tomographic image I11 at the first display depth is displayed large and at a low brightness so that the contrast inside the blood vessel is clear. This makes it possible to observe the state of the blood vessel interior and blood cells.

[0093] Fig. 17 shows a display example when it is determined that another blood vessel (artery) is present. In Fig. 17, in addition to the tomographic image I11 at the first display depth shown in Fig. 16, a tomographic image I12 at the second display depth, which allows a wider range to be observed, is displayed in parallel. When the processing unit 30 determines that another blood vessel is present and generates the tomographic image I12, the processing unit 30 may display the tomographic image I12 with a sound effect to notify the presence of the other blood vessel by making the image appear next to the tomographic image I11. The examination operator or doctor can continue to observe the tomographic image I11 while grasping the arrangement of the blood vessels and recognizing that there is a high possibility that another blood vessel exists outside, without having to change the settings themselves.

[0094] Third Embodiment In a third embodiment, a selection operation switches between display and non-display for each of a tomographic image I11 at a first display depth and a tomographic image I12 at a second display depth.

[0095] The configuration of the diagnostic imaging device 100 in the third embodiment is the same as that of the diagnostic imaging device 100 in the first embodiment, except for the processing procedures and display contents described below. Therefore, the same components of the diagnostic imaging device 100 in the third embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0096] 18 to 20 are flowcharts showing an example of an information processing procedure by the image processing device 3 of the third embodiment. The processing unit 30 of the image processing device 3 starts the following processing when an operation to start scanning of the image diagnostic device 100 is performed by the input device 5 and a signal is output from the imaging device 11 of the catheter 1. Of the processing procedures shown in the flowcharts of FIGS. 18 to 20, steps that are common to the processing procedures shown in the flowcharts of FIGS. 5 and 6 of the first embodiment are assigned the same step numbers and detailed descriptions thereof will be omitted.

[0097] The processing unit 30 of the image processing device 3 stores the signal data, the tomographic image I1 generated from the signal data, and the recognition result based on the segmentation model 31M in the storage unit 31 (S105). Before generating tomographic images at different display depths, the processing unit 30 determines whether the mode is a first mode in which both the tomographic image I11 at the first display depth and the tomographic image I12 at the second display depth are displayed, a second mode in which only the tomographic image I11 at the first display depth is displayed, or a third mode in which only the tomographic image I12 at the second display depth is displayed (step S131). Initially, the first mode is stored in the storage unit 31 as setting data.

[0098] If it is determined to be in the first mode (S131: first mode), the processing unit 30 generates a tomographic image I11 with a first display depth and a tomographic image I12 with a second display depth that differ radially from the tomographic image I1 (S106).

[0099] The processing unit 30 performs image processing on the generated tomographic image I11 at the first display depth based on the image settings corresponding to the first display depth (S107), and performs image processing on the tomographic image I12 at the second display depth based on the image settings corresponding to the second display depth (S108).

[0100] The processing unit 30 outputs the processed tomographic image I11 at the first display depth and the processed tomographic image I12 at the second display depth so that they can be displayed in real time on the screen of the display device 4 (S109). The processing unit 30 stores the processed tomographic image I11 at the first display depth and the processed tomographic image I12 at the second display depth (S110).

[0101] The processing unit 30 outputs text indicating the recognition result to the display device 4 (step S111). The processing unit 30 generates a long-axis image I2 (S112) and outputs the generated long-axis image I2 to the display device 4 (S113).

[0102] The processing unit 30 determines whether a display / hide selection operation has been received while either the tomographic image I11 or I12 is being displayed on the display device 4 (step S132). The processing unit 30 may receive the display / hide selection operation using display / hide buttons corresponding to the first display depth and the second display depth, respectively, provided on the input device 5. The processing unit 30 may display a cursor on the screen displayed on the display device 4 and receive the display / hide selection from a menu that is displayed when the cursor is placed on either the tomographic image I11 or I12 with an operation equivalent to, for example, right-clicking a mouse.

[0103] If it is determined in step S132 that the display / non-display selection operation has not been received (S132: NO), the processing unit 30 proceeds to step S114.

[0104] If it is determined in step S132 that a display / hide selection operation has been received (S132: YES), the processing unit 30 determines one of the first mode, second mode, and third mode depending on the target selected for display (step S133). In step S133, when both the tomographic image I11 at the first display depth and the tomographic image I12 at the second display depth are initially displayed in the first mode, if a selection operation to hide either one of the images is performed, the processing unit 30 determines the second mode or the third mode depending on the target. In the second mode, in which only the tomographic image I11 at the first display depth is displayed, hiding the tomographic image I11 at the first display depth is disabled. Similarly, in the third mode, in which only the tomographic image I12 at the second display depth is displayed, hiding the tomographic image I12 at the second display depth is disabled. After the determination in step S133, the processing unit 30 proceeds to step S114.

[0105] If it is determined in step S131 that the mode is the second mode (S131: second mode), the processing unit 30 generates a tomographic image I11 at a first display depth, which mainly focuses on the inside of a blood vessel, from the tomographic image I1 (step S134).The processing unit 30 then performs image processing on the generated tomographic image I11 at the first display depth based on the image settings corresponding to the first display depth (step S135).

[0106] The processing unit 30 outputs the tomographic image I11 at the first display depth after the image processing of step S133 so that it can be displayed in real time on the screen displayed on the display device 4 (step S136). The processing unit 30 stores the tomographic image at the first display depth after the image processing in the memory unit 31 in association with data on the position on the long axis of the blood vessel (step S137). Since the tomographic image I1 is already stored in the memory unit 31, the processing of step S137 is not essential. The processing unit 30 proceeds to step S111.

[0107] If it is determined in step S131 that the mode is the third mode (S131: third mode), the processing unit 30 generates a tomographic image I12 at a second display depth, which mainly focuses on the outside of the blood vessel, from the tomographic image I1 (step S138).The processing unit 30 performs image processing on the generated tomographic image I12 at the second display depth based on the image settings corresponding to the second display depth (step S139).

[0108] The processing unit 30 outputs the tomographic image I12 at the second display depth after the image processing of step S133 so that it can be displayed in real time on the screen displayed on the display device 4 (step S140). The processing unit 30 stores the tomographic image I12 at the second display depth after the image processing in the memory unit 31 in association with the position data on the long axis of the blood vessel (step S141). Since the tomographic image I1 is already stored in the memory unit 31, the processing of step S141 is not essential. The processing unit 30 proceeds to step S111.

[0109] Fig. 21 is a diagram showing an example of a screen 400 displayed on the display device 4 in the third embodiment. Similar to the example of the screen 400 shown in Fig. 11 of the first embodiment, the screen 400 in Fig. 21 includes at least a tomographic image I11, text 401 of data indicating anatomical features, a long-axis image I2, and a cursor 402.

[0110] 21 includes a menu 405 that is displayed when a specific operation (right-clicking the mouse) is performed after hovering the cursor over the tomographic image I12, one of the tomographic images I11 and I12 at different display depths on the screen displayed on the display device 4. In the menu 405 on the screen 400 shown in Fig. 21, "Display" is selected, but when the examination operator or the doctor selects "Hide" using the input device 5, the screen 400 changes to a screen that displays only the tomographic image I11 at the first display depth, as shown in Fig. 16 of the second embodiment.

[0111] The selection of whether to display or not the tomographic image I11 at the first display depth and the tomographic image I12 at the second display depth may be performed by voice recognition using a voice input / output unit included in the input device 5. This allows the examination operator or doctor to switch the display content according to the position and observation content while operating the catheter 1 and visually checking the tomographic images I11 and I12 that have been subjected to appropriate image processing on the display device 4.

[0112] (Modification) In the first to third embodiments, a specific example of processing was described using an example screen 400 including cross-sectional images I11 and I12 and a long-axis image I2. The display of the cross-sectional images I11 and I12 and the long-axis image I2 may be combined with a three-dimensional image. FIG. 22 is a diagram showing an example of a screen 400 in a modification. In the modification, the screen 400 includes a three-dimensional image 406 generated based on the recognition result and a three-dimensional cursor 407 indicating a position on the long axis of the three-dimensional image 406. The three-dimensional image 406 such as the screen 400 in FIG. 22 may be output continuously, as shown in the first embodiment, or may be output together with the cross-sectional image I12 only when it is determined that another organ (artery) is present outside the blood vessel and within the second display depth, as shown in the second embodiment.

[0113] As described above, in cases where it is necessary to understand the structure of the target organ relative to surrounding organs, such as when observing veins, a tomographic image I11 at a first display depth and a tomographic image I12 at a second display depth may be generated and displayed as appropriate. As in the modified example, outputting a three-dimensional structure using a three-dimensional image allows for a more appropriate understanding of the target organ. Furthermore, by performing appropriate image processing on the inside and outside of the blood vessels due to the difference in display depth, information about the inside and outside of the blood vessels can be appropriately displayed.

[0114] The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims.

[0115] 100 Image diagnostic device 1 Catheter 3 Image processing device 30 Processing unit 31 Storage unit P3 Computer program 4 Display device (display unit)

Claims

1. Display unit and A connector that connects to a catheter for imaging diagnostics of tubular organs, which has an ultrasound transmitting and receiving unit at its tip, A processing unit generates multiple tomographic images of different depths in the radial direction of the tubular organ based on signals obtained from the aforementioned diagnostic imaging catheter, and displays the generated tomographic images of different depths on a display unit. A diagnostic imaging device equipped with the following features.

2. The different depths include a first depth suitable for observing the inside of the tubular organ and a second depth suitable for observing other organs located outside the tubular organ. The medical imaging apparatus according to claim 1.

3. The processing unit adjusts at least one of the tomographic images of different depths with respect to at least one of the image settings for gain, contrast, STC (Sensitivity Time Control), and gamma correction, according to the corresponding depth, and displays it on the display unit. The diagnostic imaging apparatus according to claim 1 or 2.

4. The aforementioned processing unit, Based on the signals obtained from the aforementioned diagnostic imaging catheter, a longitudinal tomographic image in the axial direction of the tubular organ is generated. The cross-sectional images of different depths are displayed on the display unit together with the aforementioned cross-sectional images of different depths. The diagnostic imaging apparatus according to claim 1 or 2.

5. The processing unit accepts the setting of different depths. The diagnostic imaging apparatus according to claim 1 or 2.

6. The aforementioned processing unit, The tomographic image of the second depth is displayed only when displaying a tomographic image of a position in the axial direction of the tubular organ where the other organ is determined to be located within a distance corresponding to the second depth from the tubular organ. The image diagnostic apparatus according to claim 2.

7. The aforementioned processing unit, The system accepts a selection operation to show / hide the first depth tomographic image and the second depth tomographic image, respectively. The display mode of the tomographic image currently displayed on the display unit is changed according to the received operation. The image diagnostic apparatus according to claim 2.

8. The aforementioned processing unit, During the axial movement of the imaging catheter through the tubular organ, the display of tomographic images at different depths is changed. The medical imaging apparatus according to claim 6 or 7.

9. Computers Based on signals obtained from a catheter for imaging diagnostics of tubular organs equipped with an ultrasonic transmitting and receiving unit at its tip, multiple tomographic images of different depths in the radial direction of the tubular organ are generated. The generated tomographic images at different depths are output to a display device. Catheter image display method.

10. On the computer, Based on signals obtained from a catheter for imaging diagnostics of tubular organs equipped with an ultrasonic transmitting and receiving unit at its tip, multiple tomographic images of different depths in the radial direction of the tubular organ are generated. The generated tomographic images at different depths are output to a display device. A computer program that executes a process.