COMPUTER PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS

Abstract

To provide a computer program, an information processing method, and an information processing device capable of presenting information on an anatomical feature point in a more comprehensible manner.SOLUTION: A computer program causes a computer for acquiring a plurality of cross-sectional images of a hollow organ to execute processing for; acquiring data divided into different ranges including the lumen and membrane of the hollow organ for each of the plurality of cross-sectional images; determining whether or not the cross-sectional image is a candidate for an image capturing a second tube branched from a first tube on the basis of an organ border corresponding to an outer contour of the membrane of the hollow organ for each of the plurality of cross-sectional images; executing detection processing including determination on presence or absence of the second tube, specification of a border of the first tube, and calculation of an angle of the second tube for the cross-sectional image determined to be the candidate; specifying the position of the branch part in a longitudinal direction of the hollow organ on the basis of a result of the detection processing; and displaying the cross-sectional images of the hollow organ, vertical cross-sectional images, or three-dimensional images, and data on the second tube.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing device relating to processing of medical images. [Background technology]

[0002] In medical examinations, images of the subject are either taken directly or obtained by imaging the results of measurements using electromagnetic waves, etc. In particular, in the examination of hollow organs, various techniques are used that use images obtained by moving an imaging element inside the organ.

[0003] Diagnostic imaging of vascular organs, especially blood vessels, is essential for safe and reliable treatment such as percutaneous coronary intervention (PCI). For this reason, in addition to angiography, which uses contrast media to take images from outside the body, intravascular imaging techniques such as IVUS (Intra Vascular Ultra Sound) and OCT (Optical Coherence Tomography) / OFDI (Optical Frequency Domain Imaging) using catheters are becoming widespread.

[0004] In the above-mentioned image diagnosis, it is not easy to obtain accurate diagnostic information from the captured medical images. In order to assist in the interpretation of medical images, various techniques have been proposed that use image analysis or machine learning to correct images or add information (Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-075702 A Summary of the Invention [Problem to be solved by the invention]

[0006] Interpretation of medical images requires accurate identification of anatomical landmarks from the images. Therefore, it is necessary to capture medical images accurately and to present information about anatomical landmarks in a way that is easy to understand and visually recognize.

[0007] An object of the present disclosure is to provide a computer program, an information processing method, and an information processing device that can present information on anatomical landmarks in a manner that is easier to understand. [Means for solving the problem]

[0008] The computer program of the present disclosure causes a computer to acquire multiple tomographic images of a tubular organ having a branching portion, generated based on a signal detected by an imaging device provided on a catheter inserted into a first tube of the tubular organ, acquire data for each of the multiple tomographic images divided into different ranges including the lumen and membrane of the tubular organ, determine for each of the multiple tomographic images whether or not the tomographic image is a candidate for an image showing a second tube branching from the first tube, based on an organ boundary corresponding to the outer contour of the membrane of the tubular organ identified from the shape of the divided range, perform a detection process for the tomographic image determined to be a candidate, including determining whether the second tube is present, identifying the boundary of the first tube, and calculating the angle of the second tube, identify the position of the branching portion in the longitudinal direction of the tubular organ based on the result of the detection process, and perform a process of displaying a tomographic image, longitudinal image, or three-dimensional image of the tubular organ and data related to the second tube.

[0009] An information processing method according to the present disclosure includes a computer that acquires multiple tomographic images of a tubular organ generated based on signals detected by an imaging device provided on a catheter inserted into the main trunk of the tubular organ having a branching portion, acquires data for each of the multiple tomographic images divided into different ranges including the lumen and membrane of the tubular organ, determines for each of the multiple tomographic images whether the tomographic image is a candidate for an image depicting a second tube branching from the first tube based on an organ boundary corresponding to the outer contour of the membrane of the tubular organ identified from the shape of the divided range, and performs a detection process for a tomographic image determined to be a candidate, including determining whether the second tube is present, identifying the boundary of the first tube, and calculating the angle of the second tube, identifies the position of the branching portion in the longitudinal direction of the tubular organ based on the results of the detection process, and displays a tomographic image, longitudinal image, or three-dimensional image of the tubular organ and data related to the second tube.

[0010] An information processing device according to the present disclosure is an information processing device that acquires multiple tomographic images of a tubular organ generated based on a signal detected by an imaging device provided on a catheter inserted into a first tube of the tubular organ having a branching portion, and is provided with a processing unit that performs image processing on the multiple tomographic images, and the processing unit acquires data for each of the multiple tomographic images divided into different ranges including the lumen and membrane of the tubular organ, and determines for each of the multiple tomographic images whether or not the tomographic image is a candidate for an image depicting a second tube branching from the first tube based on an organ boundary corresponding to the outer contour of the membrane of the tubular organ identified from the shape of the divided range, and performs a detection process for the tomographic image determined to be a candidate, including determining whether or not the second tube is present, identifying the boundary of the first tube, and calculating the angle of the second tube, and identifies the position of the branching portion in the longitudinal direction of the tubular organ based on the result of the detection process, and displays the tomographic image, longitudinal image, or three-dimensional image of the tubular organ and data related to the second tube on a display device. Effect of the Invention

[0011] According to the present disclosure, it becomes possible to accurately and easily grasp anatomical landmarks, such as whether or not a second tube branching off from a first tube is visible in a cross-sectional image of a tubular organ, and, if so, data such as the boundary of the first tube or the angle of the second tube. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging diagnostic apparatus. [Diagram 2] FIG. 13 is an explanatory diagram showing the operation of the catheter. [Diagram 3] FIG. 1 is a block diagram showing a configuration of an image processing device. [Figure 4] FIG. 1 is an overview of a trained model. [Diagram 5] FIG. 13 is a diagram showing a detected boundary (contour). [Figure 6] 11 is a flowchart illustrating an example of an information processing procedure performed by the image processing device. [Figure 7] 11 is a flowchart illustrating an example of an information processing procedure performed by the image processing device. [Figure 8] 13 is a flowchart showing an example of a detailed processing procedure of a side branch detection process. [Figure 9] FIG. 13 is a schematic diagram of a side branch detection process. [Figure 10] FIG. 13 is a schematic diagram of a side branch detection process. [Figure 11] FIG. 13 is a schematic diagram of a side branch detection process. [Figure 12] FIG. 13 is a schematic diagram of a side branch detection process. [Figure 13] FIG. 13 is a schematic diagram of a side branch detection process. [Figure 14] 13 is a flowchart illustrating an example of a processing procedure for removing noise data in side canal detection. [Figure 15] FIG. 13 is a schematic diagram of noise data removal processing for side canal detection. [Figure 16] 1 shows an example screen containing information presented on a display device. [Figure 17] FIG. 11 is a diagram showing another example of a screen displayed on the display device. [Figure 18]FIG. 11 is a diagram showing another example of a screen displayed on the display device. [Figure 19] FIG. 11 is a diagram showing another example of a screen displayed on the display device. [Figure 20] FIG. 11 is a diagram showing another example of a screen displayed on the display device. [Figure 21] FIG. 11 is a diagram showing another example of a screen displayed on the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Specific examples of a computer program, an information processing method, and an information processing device according to an embodiment of the present invention will be described below with reference to the drawings.

[0014] 1 is a diagram showing an example of the configuration of an image diagnostic apparatus 100. The image diagnostic apparatus 100 is an apparatus for generating medical images including ultrasonic tomographic images of blood vessels (hollow organs) by the IVUS method and for performing ultrasonic examination and diagnosis inside blood vessels.

[0015] The image diagnostic apparatus 100 includes a catheter 1, an MDU (Motor Drive Unit) 2, an image processing device (information processing device) 3, a display device 4, and an input device 5.

[0016] The catheter 1 is a flexible tube for medical use. The catheter 1 is particularly known as an imaging catheter, which has an imaging device 11 at its tip and rotates in a circumferential direction by being driven from its base end. In the case of the IVUS method, the imaging device 11 is an ultrasonic probe including an ultrasonic transducer and an ultrasonic sensor. In the case of OCT, the imaging device 11 is an OCT device including a near-infrared laser and a near-infrared sensor. The imaging device 11 may be another device that uses electromagnetic waves of other wavelengths, such as visible light.

[0017] The MDU 2 is a drive unit attached to the base end of the catheter 1, and controls the operation of the catheter 1 by driving an internal motor in response to the operation of a medical professional.

[0018] The image processing device 3 generates a plurality of medical images, such as tomographic images of blood vessels, based on the signal output from the imaging device 11 of the catheter 1. The configuration of the image processing device 3 will be described in detail later.

[0019] The display device 4 uses a liquid crystal display panel, an organic EL display panel, etc. The display device 4 displays the medical image generated by the image processing device 3 and information related to the medical image.

[0020] The input device 5 is an input interface that accepts operations for the image processing device 3. The input device 5 may be a keyboard, a mouse, or the like, or may be a touch panel, soft keys, hard keys, or the like built into the display device 4.

[0021] Fig. 2 is an explanatory diagram showing the operation of the catheter 1. In Fig. 2, the catheter 1 is inserted into a tubular blood vessel L by a medical professional along a guidewire W inserted into a coronary artery shown in the figure. In the enlarged view of the blood vessel L in Fig. 2, the right side corresponds to the distal side from the insertion point of the catheter 1 and the guidewire W, and the left side corresponds to the proximal side.

[0022] The catheter 1 is driven by the MDU 2 to move from the distal end to the proximal end within the blood vessel L as indicated by the arrow in the figure, and while rotating in the circumferential direction, the imaging device 11 scans the inside of the blood vessel in a spiral manner.

[0023] In the image diagnostic device 100 of this embodiment, the image processing device 3 acquires a signal for each scan output from the imaging device 11 of the catheter 1. In each scan, the imaging device 11 emits a detection wave in the radial direction and detects the reflected wave. The imaging device 11 performs this scan several tens to several thousands of times while rotating 360 degrees, scanning in a spiral shape. The image processing device 3 generates a tomographic image (transverse cross-sectional image) obtained by polar coordinate conversion (inverse conversion) of the signal for each scan for every 360 degrees (I1 in FIG. 2). The tomographic image I1 is also called a frame image. The reference point (center) of the tomographic image I1 corresponds to the range of the catheter 1 (not imaged). The image processing device 3 further generates a long-axis image (longitudinal cross-sectional image) in which the pixel values ​​on a straight line passing through the reference point of the tomographic image I1 are arranged along the length direction (long-axis direction) of the blood vessel by the catheter 1 (I2 in FIG. 2). The image processing device 3 analyzes and processes the branching structure of the blood vessels based on the obtained tomographic image I1 and long-axis image I2, and outputs a two-dimensional or three-dimensional image showing the structure of the blood vessels so that it can be visually recognized by a medical professional. In the image diagnostic device 100 of the present disclosure, information about the side branches, such as the side branch position, outline, etc. of the first tube (e.g., main trunk) into which the guidewire W and catheter 1 are inserted, and the second tube (e.g., side branch) branching from the first tube, is superimposed on both the long-axis image and the tomographic image displayed on the display device 4. The process by which the image processing device 3 creates information about the side branches and superimposes it on the display device 4 will be described in detail below.

[0024] 3 is a block diagram showing the configuration of the image processing device 3. The image processing device 3 is a computer, and includes a processing unit 30, a storage unit 31, and an input / output I / F 32.

[0025] The processing unit 30 includes one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), tensor processing units (TPUs), etc. The processing unit 30 has a built-in non-temporary storage medium such as a random access memory (RAM), and executes calculations based on a computer program 3P stored in the storage unit 31 while storing data generated during processing in the non-temporary storage medium.

[0026] The storage unit 31 is a non-volatile storage medium such as a hard disk, a flash memory, etc. The storage unit 31 stores a computer program 3P, setting data, etc., which are read by the processing unit 30. The storage unit 31 also stores a trained model 3M.

[0027] The computer program 3P and the trained model 3M may be copies of the computer program 9P and the trained model 9M stored in a non-temporary storage medium 9 outside the device read out via the input / output I / F 32. The computer program 3P and the trained model 3M may be distributed by a remote server device, acquired by the image processing device 3 via a communication unit (not shown), and stored in the storage unit 31.

[0028] The input / output I / F 32 is an interface to which the catheter 1, the display device 4, and the input device 5 are connected. The processing unit 30 acquires a signal (digital data) output from the imaging device 11 via the input / output I / F 32. The processing unit 30 outputs screen data of a screen including the generated tomographic image I1 and / or long axis image I2 to the display device 4 via the input / output I / F 32. The processing unit 30 accepts operation information input to the input device 5 via the input / output I / F 32.

[0029] FIG. 4 is a schematic diagram of the trained model 3M. In the present disclosure, the trained model 3M is a model trained to output an image indicating the area of ​​one or more objects appearing in an image when an image is input. The trained model 3M is, for example, a model that performs semantic segmentation. The trained model 3M is designed to output an image tagged with data indicating which object each pixel in the input image is a pixel in, and a probability.

[0030] The trained model 3M uses, for example, a so-called U-net in which a convolution layer, a pooling layer, an upsampling layer, and a softmax layer are symmetrically arranged, as shown in Fig. 4. When a tomographic image I1 created by a signal from the catheter 1 is input, the trained model 3M outputs a tag image IS and accuracy. The tag image IS is obtained by tagging the pixels at the positions of the lumen range of the blood vessel, the membrane range corresponding to the area between the lumen boundary of the blood vessel including the tunica media and the blood vessel boundary, the range in which the guidewire W and its reflection are captured, and the range corresponding to the catheter 1 with different pixel values ​​(shown by different types of hatching and solid color in Fig. 4).

[0031] As described above, the trained model 3M is exemplified by semantic segmentation and U-net, but it is needless to say that the trained model 3M is not limited to these. In addition, the trained model 3M may be a model that realizes individual recognition processing by instance segmentation, etc. The trained model 3M is not limited to being based on U-net, and may be a model based on SegNet, R-CNN, or an integrated model with other edge extraction processing, etc.

[0032] The processing unit 30 can detect edges of the lumen boundary and vascular boundary of the blood vessel shown in the tomographic image I1 by using pixel values ​​in a tag image IS obtained by inputting the image I1 to the trained model 3M and coordinates in the image. Strictly speaking, the vascular boundary is the external elastic membrane (EEM) between the tunica media and tunica adventitia of the blood vessel, and is shown relatively clearly with low brightness in the image I1 by the IVUS method. FIG. 5 is a diagram showing the detected boundary (contour). FIG. 5 shows a state in which a curve B1 indicating the lumen boundary and a curve B2 indicating the vascular boundary obtained based on the output from the trained model 3M are superimposed on the tomographic image I1 shown in FIG. 4.

[0033] The image processing device 3 of the present disclosure further derives and displays anatomical feature points using the tomographic image I1 obtained from the signal from the catheter 1 and information on the lumen boundary and the blood vessel boundary obtained when the tomographic image I1 is input to the trained model 3M. The detailed processing procedure will be described below.

[0034] 6 and 7 are flowcharts showing an example of an information processing procedure by the image processing device 3. When a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing device 3 starts the following process.

[0035] Each time the processing unit 30 acquires a predetermined amount (e.g., 360 degrees) of signals (data) from the imaging device 11 of the catheter 1 (step S301), it performs polar coordinate conversion (inverse conversion) on the signals arranged in a rectangle to generate a tomographic image I1 (step S302). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time on the screen displayed on the display device 4 (step S303). The processing unit 30 stores the signal data acquired in step S301 and the tomographic image I1 in the storage unit 31 in association with the position (position on the long axis, angle) of the imaging device 11 (step S304).

[0036] The processing unit 30 inputs the tomographic image I1 to the trained model 3M (step S305). Based on the tag image IS obtained from the trained model 3M, the processing unit 30 calculates data on the lumen boundary and the vascular boundary in the tomographic image I1 (step S306). In step S306, the processing unit 30 calculates the lumen range output from the trained model 3M, the inner membrane range including the tunica media of the blood vessel, the contour (edge) of the lumen range as the lumen boundary, and the outer contour of the membrane range as the vascular boundary. In step S306, the processing unit 30 may implement high-speed processing, such as reducing the size of the tomographic image I1 before inputting it to the trained model 3M.

[0037] The processing unit 30 calculates a parameter for determining whether or not a side branch is captured based on the data (coordinate data) of the lumen boundary and the vascular boundary obtained in step S306 (step S307). If the tomographic image I1 captures the branching portion of the main trunk and the side branch, the shape of the vascular boundary derived from the tomographic image I1 deviates from a circle or an ellipse. In step S307, the processing unit 30 calculates a parameter corresponding to the degree of deviation from a circle or an ellipse.

[0038] In step S307, the processing unit 30 may basically calculate parameters only for the vascular boundary. However, when the vascular boundary intersects with the outside of the tomographic image I1, that is, when the membrane range extends to the outside of the tomographic image I1, the processing unit 30 may calculate parameters for the inner region of the lumen boundary instead of the vascular boundary.

[0039] In step S307, in the first example, the processing unit 30 calculates the eccentricity by dividing the difference between the maximum diameter and the minimum diameter passing through the center of gravity of the inner region of the blood vessel boundary by the maximum diameter.

[0040] In step S307, in the second example, the processing unit 30 may use a learning model for judgment (not shown) that has been trained to output a degree of accuracy corresponding to the possibility that a side branch is captured when data on the lumen boundary and the vascular boundary are input. Here, the learning model for judgment may be trained to output a degree of accuracy corresponding to the possibility that a side branch is captured when data on the lumen range and the membrane range in the tag image output from the trained model 3M are input. In step S307, the processing unit 30 calculates the output from the learning model for judgment as a parameter.

[0041] In step S307, in the third example, the processing unit 30 may calculate a parameter obtained by comparing the diameter (maximum diameter and minimum diameter) of the target vascular boundary with the diameter of the vascular boundary for the already scanned tomographic image I1. If the diameter changes suddenly by more than a predetermined ratio or by more than a predetermined length, it can be determined that there is a high possibility that a side branch is captured.

[0042] As a fourth example, the processing unit 30 may calculate circularity instead of eccentricity. Circularity is the ratio of the area of ​​the inner region of the blood vessel boundary to the circumference of the blood vessel boundary. The closer the circularity is to the ratio of the area of ​​a circle to the circumference, the higher the circularity is, and it can be determined that the possibility of a side branch being captured is low.

[0043] The processing unit 30 stores the parameters in association with the position of the imaging device 11 (step S308). The processing unit 30 determines whether the target tomographic image I1 is a candidate for an image showing not only the main trunk into which the catheter 1 is inserted but also a side branch based on the calculated parameters (step S309). In step S309, the processing unit 30 may make the determination based on, for example, whether the eccentricity is higher than a predetermined value. If it is determined in step S309 that it is not a candidate (S309: NO), the processing unit 30 advances the process to step S313.

[0044] If it is determined to be a candidate (S309: YES), the processing unit 30 executes a side branch detection process (step S310) including determining whether or not a side branch is present based on the vascular boundary calculated from the target tomographic image I1, identifying the boundary of the main trunk in the tomographic image, and calculating the angle of the side branch.

[0045] The processing unit 30 superimposes an image showing the boundary of the main trunk region and data on the angle of the side branch obtained as a result of the side branch detection process in step S310 on the tomographic image I1 being displayed in step S303 (step S311). Step S311 may be skipped if it is determined that no side branch is present in the tomographic image I1. The processing unit 30 stores the result of the side branch detection process in association with the tomographic image I1 stored in step S304 (step S312). If it is determined that a side branch is present, data such as a flag indicating a side branch image is associated with the tomographic image I1. The position corresponding to the tomographic image I1 associated with the data of the flag indicating a side branch image is the position of the bifurcation of the main trunk and the side branch on the long axis.

[0046] The processing unit 30 may also perform the processes of steps S307-S310 on the lumen boundary calculated in step S306.

[0047] The processing unit 30 determines whether or not scanning by the imaging device 11 of the catheter 1 has been completed (step S313). If it is determined that scanning has not been completed (S313: NO), the processing unit 30 returns the process to step S301 and generates the next tomographic image I1.

[0048] If it is determined that the scan is completed (S313: YES), the processing unit 30 executes a process of removing noise data based on the presence or absence of side branches and the distribution of the angles of the detected side branches relative to the position of the imaging device 11 (step 314). In step S314, the processing unit 30 re-determines that no side branches are present for the tomographic image I1 that is determined to have low accuracy even if the tomographic image I1 is determined to have a side branch by the process of step S310, and stores the result. Details will be described later.

[0049] The processing unit 30 causes the display device 4 to display the long axis image I2 with a mark indicating the presence of a side branch at a location corresponding to the position of the tomographic image I1 where it has been determined that a side branch is shown by the processing in step S314 (step S315), and then terminates the processing.

[0050] 8 is a flowchart showing an example of a detailed process of the side branch detection process, which corresponds to details of step S310 shown in the flowcharts of FIGS.

[0051] The processing unit 30 extracts a circle along the boundary of the target (blood vessel boundary or lumen boundary) by, for example, Hough transform (step S101). When, for example, Hough transform is used in step S101, the processing unit 30 extracts a plurality of large and small circles.

[0052] The processing unit 30 determines a plausible circle corresponding to the main trunk from the circles extracted in step S101 (step S102). In step S102, the processing unit 30 selects a circle having a diameter equal to or greater than a predetermined length corresponding to the size of the blood vessel diameter and having a center closest to the center of gravity of the area within the boundary of the target as the plausible circle corresponding to the main trunk. In step S102, the processing unit 30 may select a circle having a center closest to the center of the image as the plausible circle, or may select a circle closest to the center of gravity of the previous or next frame image as the plausible circle. The processing unit 30 may select a combination of some or all of these selection methods as appropriate.

[0053] The processing unit 30 determines the center of the circle determined to correspond to the trunk as the center of the blood vessel (step S103), and calculates the distances of points on the boundary of interest from the center of the blood vessel along the entire circumference (360 degrees) of the boundary (step S104). The processing unit 30 creates a distribution of the distances from the center of the blood vessel along the entire circumference (step S105), and determines whether there are a predetermined number or more consecutive points on the boundary whose distances from the center of the blood vessel are longer than a predetermined reference value (step S106).

[0054] If it is determined that the number is equal to or greater than the predetermined number (S106: YES), the processing unit 30 determines that a side branch exists (is captured) in the tomographic image I1 (step S107).The processing unit 30 identifies arcs on the main trunk circle determined in step S102 that correspond to points on the boundary whose distance from the center of the blood vessel is continuously longer than a predetermined reference value (step S108).

[0055] The processing unit 30 calculates the side branch angle as the angle of a straight line connecting the center of the tomographic image I1 to the center point of the arc identified in step S108 for the tomographic image I1 from, for example, the 12 o'clock direction (upward) in the tomographic image I1 (step S109).

[0056] The processing unit 30 specifies an interpolation trajectory by connecting two points on the boundary that are closest to the end points of the arc specified in step S108 with a new arc (step S110). The processing unit 30 stores the interpolation trajectory specified in step S110 as an image showing the area boundary of the main trunk (step S111), and returns the process to step S311 in Figs. 6 and 7.

[0057] In step S106, if it is determined that the number is less than the predetermined number (S106: NO), the processing unit 30 determines that no lateral branches are present in the tomographic image I1 (step S112), and returns the process to step S311 in FIGS.

[0058] The processing procedure shown in the flowchart of FIG. 8 will be described with a specific example. FIGS. 9 to 13 are schematic diagrams of side branch detection processing. FIG. 9 is a diagram showing an example of a circle extracted for a vascular boundary. FIG. 9 shows a cross-sectional image I1 superimposed with a curve B2 of the vascular boundary calculated for the cross-sectional image I1, and shows a plurality of circles extracted in bold after a circle extraction process for the curve B2. As shown in FIG. 9, from a cross-sectional image I1 that is a candidate for showing a side branch, circles are extracted for a portion corresponding to the main trunk in the lower left part of the cross-sectional image I1 and a portion corresponding to the side branch. However, in FIG. 9, the circle with a large diameter in the lower left part is determined to be closer to the center of gravity.

[0059] Fig. 10 shows the distribution obtained by the process of step S104 in the flowchart of Fig. 8. The upper part of Fig. 10 shows the calculated vascular boundary curve B2 and the determined circle superimposed on the tomographic image I1. The lower part of Fig. 10 shows the distribution of the distance from the center of the blood vessel to the coordinate points on the curve B2. The arrows in Fig. 10 indicate the order in which the distances are calculated. As shown in Fig. 10, in the image in which the side branch is captured in the tomographic image I1, a clear peak in the distance is present, as shown in the distribution at the bottom.

[0060] FIG. 11 is a diagram showing the determination of the presence or absence of side branches based on the distribution of FIG. 10 (S106). In FIG. 11, a reference value for distance is shown by a thick line on the distribution shown in FIG. 10. In the diagram of FIG. 11, it is determined that there are a predetermined number or more pixels whose distance is equal to or greater than the reference value. In the lower part of FIG. 11, points on the boundary (B2) whose distance from the center of the blood vessel is continuously longer than the predetermined reference value are shown by black dots, and a corresponding arc (black end point circle) is shown (S108). Specifically, the arc of the black end point circle is specified as the range between two straight lines connecting the center of the blood vessel and both ends of a part on the boundary (B2) that is continuously longer than the predetermined reference value among the circles corresponding to the main trunk.

[0061] The determination (S106) of whether or not a side branch exists (is captured) with a high probability is not limited to the content shown in Fig. 11. For example, it is not limited to whether or not a predetermined number or more of consecutive coordinate points exist at a distance equal to or greater than the radius of a circle determined for the vascular boundary. It may be determined that a predetermined number or more of consecutive coordinate points exist on the lumen boundary at a distance equal to or greater than the radius of a circle determined for the lumen boundary, and that the range overlaps with the range specified for the vascular boundary.

[0062] Fig. 12 shows an example of an interpolation trajectory determined based on the arc determined in Fig. 11. As shown in Fig. 12, the interpolation trajectory is determined as an arc passing through two points on the curve B2 of the vascular boundary that are close to the respective end points of the arc determined in step S108. Also, information on the calculated side branch angle (side branch direction) is shown as a hollow arrow in Fig. 12. At this time, the image (arrow) indicating the side branch angle is displayed outside the curve B2 so that the curve B2 and the interpolation trajectory can be easily viewed.

[0063] Fig. 13 shows a display example on the display device 4. In Fig. 13, a tomographic image I1, the lumen boundary, the vascular boundary curves B1 and B2 superimposed on the tomographic image I1, an interpolation trajectory, and information on the side branch angle are superimposed and displayed. In the example shown in Fig. 13, an interpolation trajectory determined in a similar manner is also superimposed and displayed on the lumen boundary. This makes it easy for even medical personnel who are not accustomed to reading images to determine the presence or absence of a side branch and to read the location of the side branch where plaque is likely to accumulate.

[0064] 14 is a flowchart illustrating an example of a process for removing noise data in side canal detection. The flowchart in Fig. 14 corresponds to details of step S314 shown in the flowcharts in Figs.

[0065] The processing unit 30 judges whether or not there are one or more other tomographic images I1 in which a side branch is judged to exist within a predetermined range from the position of the tomographic image I1 in which a side branch is judged to exist within the scanning range of the catheter 1 (step S401). The predetermined range is, for example, a range corresponding to the diameter (1 to 5 mm) of the side branch. In a location where a side branch actually exists, the side branch should be judged to exist in multiple consecutive tomographic images I1. Therefore, if a tomographic image I1 in which a side branch is judged to exist is isolated, this is presumed to be a false detection. In step S401, the processing unit 30 may judge that there are multiple side branches, for example, if the side branch is judged to exist in 80% of the multiple tomographic images I1 generated for each position within a predetermined range from the position of the tomographic image I1 in which a side branch is judged to exist.

[0066] Therefore, when it is determined in step S401 that no other tomographic image I1 exists (S401: NO), the processing unit 30 removes the target tomographic image I1 from the images in which side branches exist (step S402). Specifically, in step S402, the processing unit 30 deletes the flag indicating the side branch image, information on the side branch, etc., that were associated with the tomographic image I1. The processing unit 30 returns the process to step S315 in Figs. 6 and 7.

[0067] If it is determined that another tomographic image I1 exists (S401: YES), it is compared with other tomographic images I1 in which a side branch is determined to exist within a predetermined range, and it is determined whether the side branch angles are similar (step S403). If the same side branch is detected, the calculated side branch angles should be similar.

[0068] Therefore, if the processing unit 30 determines that they are not similar (S403: NO), the processing unit 30 removes the target tomographic image I1 from the image in which the side branch is present (S402), and returns the processing to step S315 in Figures 6 and 7.

[0069] If it is determined that they are similar (S403: YES), the processing unit 30 returns the process to step S315 in Fig. 6 and Fig. 7. In this case, the tomographic image I1 determined to be similar remains associated with data such as a flag indicating that it is a side branch image. The tomographic image I1 that has not been removed according to the flowchart in Fig. 14 becomes an image that has been confirmed to include a side branch from among the candidate images that include a side branch.

[0070] Fig. 15 is a schematic diagram of noise data removal processing for side branch detection. Fig. 15 is a distribution diagram of side branch angles with respect to positions of tomographic images I1 where a side branch is determined to exist within the scanning range of the catheter 1. The horizontal axis of Fig. 15 is the long axis direction of the blood vessel through which the catheter 1 moves, and the vertical axis indicates the intravascular angle. Each point indicates the position in the long axis direction of the tomographic image I1 where a side branch is determined to exist, and the angle range for that side branch and the side branch angle which is the center value of that range. In Fig. 15, the range on the long axis where the side branch actually exists is hatched.

[0071] As shown in Fig. 15, where a side branch actually exists, consecutive points continue in a similar angle range. In contrast, for points corresponding to a tomographic image I1 where a side branch is determined to exist but where a side branch does not actually exist, the calculated side branch angles are also inconsistent and not continuous. Therefore, depending on whether or not there is continuity in the positions on the long axis and the side branch angles, the information of the tomographic image I1 where a side branch is erroneously determined to exist can be removed as noise.

[0072] By clustering the features of each point (position in the long axis direction, side canal range, and side canal angle) shown in Fig. 15, points that do not belong to a cluster may be removed as noise, assuming that the tomographic image I1 is erroneously determined to have a side canal. For the clustering, for example, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) may be adopted, or other known methods may be used.

[0073] In this way, the image diagnostic device 100 can present each tomographic image I1 together with data such as the angle of the side branch in the tomographic image I1 by narrowing down the information of the tomographic image I1 in which it is determined that the side branch is not noise but has a high probability. Specifically, the memory unit 31 of the image processing device 3 stores signal data at each time point from the imaging device 11 and the tomographic image I1 converted into polar coordinates for one examination using the catheter 1. The memory unit 31 also stores coordinate data (curve, center of gravity, center of blood vessel, blood vessel diameter (minimum diameter), etc.) related to the lumen boundary and blood vessel boundary obtained by processing each tomographic image I1, the presence or absence of a side branch, the angle of the side branch, data of an interpolated trajectory, etc. Based on these data, the image processing device 3 can present data of anatomical landmarks in real time during an examination using the catheter 1 or after the fact.

[0074] Fig. 16 shows an example of a screen including information presented on the display device 4. The screen 400 shown in Fig. 16 includes display areas for a long-axis image I2 and a tomographic image I1 of a position being scanned or selected at the time of display. As shown in Fig. 13, the display area for the tomographic image I1 displays superimposed information on the lumen boundary, the vascular boundary curves B1 and B2, the interpolation trajectory, and the side branch angle.

[0075] As shown in FIG. 16, the long axis image I2 on the screen 400 is a longitudinal cross-sectional image with the long axis direction of the blood vessel through which the catheter 1 moves as the horizontal direction. By default, it is displayed as a collection of brightness data along the vertical direction in the tomographic image I1. A cursor 403 indicating a position on the long axis corresponding to the tomographic image I1 is displayed on the long axis image I2. The screen 400 also includes a first button 401 and a second button 402 at a position on the long axis image I2 corresponding to a position in the long axis direction of the tomographic image I1 where it has been determined and confirmed that a side branch exists. The buttons 401 and 402 are displayed for the number of side branches confirmed to exist.

[0076] The first button 401 and the second button 402 are buttons for displaying the tomographic image I1 in which the presence of each side branch has been confirmed, and the data related to the corresponding side branch. When the first button 401 and the second button 402 are selected, the long axis image I2 becomes a longitudinal section image at the side branch angle corresponding to each button in each tomographic image I1.

[0077] 17 and 18 show other examples of screens displayed on the display device 4. FIG. 17 and FIG. 18 show screens displayed when the first button 401 and the second button 402 in the screen shown in FIG. 16 are pressed, respectively. FIG. 17 shows a screen displayed when the first button 401 is selected by the input device 5, and FIG. 18 shows a screen displayed when the second button 402 is selected by the input device 5. Comparing FIG. 16 to FIG. 18, it can be seen that the cross-sectional direction of the long axis image I2 is different between the default, the first button 401, and the second button 402. This makes it easier for medical personnel who interpret the images to observe the entire blood vessel with a focus on the side branches and grasp the anatomical structure from the examination results using the catheter 1.

[0078] By accurately detecting the position of the side branch in the direct axis direction, and detecting the interpolated trajectory of the main trunk and the side branch, as well as the side branch angle, other anatomical features at the branching portion of the blood vessel can be displayed in various forms on the display device 4. Fig. 19 shows another example of a screen displayed on the display device 4. The screen 400 shown in Fig. 19 includes a schematic longitudinal section image I3 in which the blood vessel region identified for each tomographic image I1 is connected in the longitudinal direction with the center of the blood vessel as the axis. In addition to the schematic, a longitudinal section image may be displayed taking into consideration dimensions based on the diameter of the blood vessel boundary and the diameter of the lumen boundary obtained from each tomographic image I1.

[0079] 19, a first button 401, a second button 402, and a cursor 403 are displayed on the long axis image I2. When the first button 401 is selected by the input device 5, the cutting direction of the schematic longitudinal section image I3 is changed to the lateral canal angle of the lateral canal corresponding to the first button 401.

[0080] When the first button 401 or the second button 402 for selecting the location where the side branch is present is selected, a cross-sectional image I4 centered on the center of the side branch may be further generated and displayed on the schematic longitudinal cross-sectional image I3 as shown in Fig. 19. Such an image makes it easier for medical personnel to visually check the display device 4 and understand the structure of the blood vessel.

[0081] As shown in FIG. 4, since the guidewire W and its reflection range can be recognized, a point in each cross-sectional image I1 corresponding to the guidewire W (e.g., a point closer to the catheter 1) can be recognized, and an image connecting the vascular boundaries with that point as the center can be displayed.

[0082] FIG. 20 shows another example of a screen displayed on the display device 4. FIG. 20 shows an example of a three-dimensional image. FIG. 20 shows a screen 400 displayed on the display device 4, similar to FIGS. 16-19. The screen 400 in FIG. 20 includes a three-dimensional image I5 of the main trunk and side branches. The three-dimensional image I5 is created by simplifying both the main trunk and side branches into tubular shapes, so that the position of the side branches in the longitudinal direction and the angle of the side branches can be visually grasped from the data related to the side branches obtained by the above-mentioned processing. This allows medical personnel to easily grasp the anatomical characteristics of the blood vessels (side branches).

[0083] Since the image processing device 3 stores the side branch detection results in association with the positions in the long axis direction (S312), it is possible to use these stored data to display the anatomical characteristics of the blood vessels in a way that makes it easier for medical personnel to understand them. FIG. 21 shows another example of a screen displayed on the display device 4. In FIG. 21, a curve B3 indicating the lumen boundary detected at each position and a curve B4 indicating the region boundary (vascular boundary) of the main trunk are superimposed on the long axis image I2. The curves B3 and B4 may be drawn superimposed on the long axis image I2 in a color that is easy to see. The processing device 30 creates a curved surface in which the curve B1 of the lumen boundary and the curve B2 of the vascular boundary of each tomographic image I1 shown in FIG. 13 are superimposed in the long axis direction, and creates a longitudinal section image by cutting the curved surface in the long axis direction. At a position where a side branch is captured and the region boundary of the main trunk is interpolated, the processing device 30 connects the interpolation trajectory shown by the dashed line to the curve B2 of the vascular boundary. The same applies to the lumen boundary. The processing unit 30 creates a curved surface by connecting points on the curve B1 and points on the curve B2 with a spline curve in the long axis direction, and creates a longitudinal section at a specific cutting angle from the created curved surface. The boundary curve of the lumen region connecting the points on the curve B1, and the curve connecting the interpolated boundary in some places, are shown by symbol B3, and the boundary curve connecting the points on the curve B2 of the blood vessel boundary and the points on the trunk region boundary in some places is shown by symbol B4. The specific cutting angle may be selected by the input device 5. This makes it easier for medical personnel to grasp the anatomical features of the blood vessels over the scanning range.

[0084] In this embodiment, the image processing device 3 connected to the catheter 1 generates a tomographic image I1 in almost real time based on a signal from the imaging device 11, judges whether the image shows a side branch, and displays data showing anatomical features on the display device 4. However, each process including the above-described judgment by the image processing device 3 of whether the image shows a side branch may be performed after the fact on the generated tomographic image I1. That is, the image processing device 3 is not necessarily directly connected to the imaging device 11 of the catheter 1. The image processing device 3 may be a device capable of reading a storage device that stores a signal from the imaging device 11 via a network, such as a server device. That is, the processing procedure of steps S301-S304 shown in the flowcharts of Figs. 6 and 7 may be performed by an existing image diagnostic device, and the processing of steps S305-S315 and the flowchart of Fig. 8 may be performed by the image processing device 3 connected to the processing device, and displayed on the display device 4.

[0085] In the present embodiment, the medical image is an image obtained by IVUS of a coronary artery, but the application is not limited to this, and may be OCT / OFDI, etc., and the luminal organ is not limited to a blood vessel.

[0086] The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0087] 1 Catheter 11 Imaging Devices 3. Image processing device (information processing device) 30 Processing section 31 Storage section 3P Computer Program 3M pre-trained model 4 Display device 400 screens 401 First button 402 Second Button I1,I4 Tomographic images I2 long axis image I3 Schematic longitudinal section image I5 3D Image

Claims

1. a computer that acquires a plurality of tomographic images of a hollow organ based on a signal detected by an imaging device provided in a catheter that is inserted into a first tube of the hollow organ having a branch, For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel; A screen in which an image showing the boundary of the first tube or the second tube specified by the detection process is superimposed on the tomographic image or the longitudinal section image is output. A computer program that executes a process.

2. The computer includes: Calculating a parameter corresponding to the degree of deviation of the shape of the organ boundary from an approximately circular shape; Based on a comparison between the parameter and a predetermined value, it is determined whether the image is a candidate for an image in which the second tube is shown.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

3. The computer includes: When the organ boundary falls outside the detection range of the imaging device, the lumen boundary of the hollow organ is used in the calculation instead of the organ boundary.

3. A computer program according to claim 1 or 2.

4. The computer includes: The detection process includes: determining a circle that corresponds to the boundary of the first vessel of the luminal organ among circles that at least partially follow the organ boundary; Calculating the distance from the centre of the determined circle for each point on the organ boundary; When the number of consecutive points at which the distance is determined to be equal to or greater than the reference value is equal to or greater than a predetermined number, the number of consecutive points is determined to be within the range of the second tube, and the second tube is determined to be captured in the tomographic image of the target. A computer program product according to any one of claims 1 to 3, which is adapted to carry out a process.

5. The computer includes: The detection process includes: The angle between a straight line connecting the center of the determined circle or the center of the catheter in the tomographic image to the center or center of gravity of a range in which the points whose distance is determined to be equal to or greater than the reference value are consecutive, and a predetermined straight line direction in the tomographic image is calculated as the angle of the second tube shown in the tomographic image.

5. A computer program product as claimed in claim 4, which causes a process to be carried out.

6. The computer includes: The detection process includes: Among the points on the organ boundary, a locus obtained by connecting two points that are closest to both end points of the arc of the determined circle corresponding to the range of the second tube with a new arc is identified as a part of the boundary of the first tube.

6. A computer program product according to claim 4 or 5, which causes a process to be executed.

7. The computer includes: A screen in which an image showing the angle of the second pipe identified by the detection process is superimposed on the tomographic image is output. A computer program product according to any one of claims 1 to 6, which is adapted to carry out a process.

8. The computer includes: generating a long axis image by arranging a group of pixel values ​​along a straight line at a predetermined angle in the image, which passes through a reference point in the multiple tomographic images, in a long axis direction of the hollow organ; A screen is displayed on the long-axis image at the position of the branching portion specified in the long-axis direction, on which a tomographic image of the branching portion and a button for displaying data on the second tube at the branching portion are superimposed. A computer program product according to any one of claims 1 to 7, which is adapted to carry out a process.

9. The computer includes: When the button is selected, the long-axis image is switched to a long-axis image in which a group of pixel values ​​along a straight line passing through a reference point in a tomographic image and corresponding to an angle of the second tube in the image is arranged in the long-axis direction of the hollow organ.

9. A computer program product as claimed in claim 8, which causes a process to be carried out.

10. The computer includes: Among the candidates of the image in which the second tube is shown, the candidates are removed except for those that are consecutive within a predetermined range on the long axis and have similar angles of the second tube calculated between the consecutive candidates. A computer program product according to any one of claims 1 to 9 for carrying out a process.

11. A computer that acquires a plurality of tomographic images of a tubular organ having a branch based on a signal detected by an imaging device provided on a catheter that is inserted into a first tube of the tubular organ, comprising: For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel; The detection process includes: determining a circle that corresponds to the boundary of the first vessel of the luminal organ among circles that at least partially follow the organ boundary; Calculating the distance from the centre of the determined circle for each point on the organ boundary; When the number of consecutive points at which the distance is determined to be equal to or greater than the reference value is equal to or greater than a predetermined number, the number of consecutive points is determined to be within the range of the second tube, and the second tube is determined to be captured in the tomographic image of the target. A computer program that executes a process.

12. A computer that acquires a plurality of tomographic images of a tubular organ having a branch based on a signal detected by an imaging device provided on a catheter that is inserted into a first tube of the tubular organ, comprising: For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel; A screen in which an image showing the angle of the second pipe identified by the detection process is superimposed on the tomographic image is output. A computer program that executes a process.

13. A computer that acquires a plurality of tomographic images of a tubular organ having a branch based on a signal detected by an imaging device provided on a catheter that is inserted into a first tube of the tubular organ, comprising: For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel; generating a long axis image by arranging a group of pixel values ​​along a straight line at a predetermined angle in the image, which passes through a reference point in the multiple tomographic images, in a long axis direction of the hollow organ; A screen is displayed on the long-axis image at the position of the branching portion specified in the long-axis direction, on which a tomographic image of the branching portion and a button for displaying data on the second tube at the branching portion are superimposed. A computer program that executes a process.

14. A computer that acquires a plurality of tomographic images of a tubular organ having a branch based on a signal detected by an imaging device provided on a catheter that is inserted into a first tube of the tubular organ, comprising: For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Among the candidates of the image in which the second tube is shown, all the candidates other than those which are consecutive within a predetermined range on the long axis and have similar angles of the second tube calculated between the consecutive candidates are removed; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel. A computer program that executes a process.

15. a computer that acquires a plurality of tomographic images of a tubular organ based on a signal detected by an imaging device provided in a catheter that is inserted into a first tube of the tubular organ having a branch, For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal image, or a three-dimensional image of the luminal organ and data relating to the second vessel; A screen in which an image showing the boundary of the first tube or the second tube specified by the detection process is superimposed on the tomographic image or the longitudinal section image is output. Information processing methods.

16. 1. An information processing device for acquiring a plurality of tomographic images of a tubular organ having a branch, the tomographic images being generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of the tubular organ, the information processing device comprising: a processing unit that performs image processing on the plurality of tomographic images, The processing unit includes: For each of the plurality of tomographic images, data is obtained that is divided into different ranges including the lumen and membrane of the hollow organ; For each of the plurality of tomographic images, it is determined whether or not the tomographic image is a candidate for an image showing a second tube branching off from the first tube, based on an organ boundary corresponding to an outer contour of a membrane of the hollow organ identified from the shape of the divided range; performing a detection process for the tomographic image determined to be a candidate, the detection process including determining whether the second tube is present, identifying a boundary of the first tube, and calculating an angle of the second tube; Identifying a position of the bifurcation in a longitudinal direction of the tubular organ based on a result of the detection process; Displaying a cross-sectional image, a longitudinal cross-sectional image, or a three-dimensional image of the hollow organ and data related to the second tube on a display device; A screen in which an image showing the boundary of the first tube or the second tube identified by the detection process is superimposed on the tomographic image or the longitudinal section image is output to the display device. Information processing device.

Images