Computer program, information processing method, and information processing device

Abstract

To provide a computer program capable of presenting information on anatomical feature points of a hollow organ so as to be grasped more, an information processing method, and an information processing device.SOLUTION: A computer program makes a computer for acquiring a plurality of tomographic images of a hollow organ generated on the basis of a signal detected by an imaging device provided in a catheter inserted into a first tube of the hollow organ with a bifurcation execute detection processing including determination of the presence / absence of a second tube bifurcated from the first tube and angle calculation of the second tube to the plurality of tomographic images, and executes processing for noise removal processing on the basis of distribution information including a position in a longitudinal axis direction of the hollow organ of a tomographic image determined to be an image showing the second tube by the detection processing and an angle of the second tube in the tomographic image calculated by the detection processing.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing apparatus for processing medical images.

Background Art

[0002] In medical examinations, images obtained by directly imaging an image of an examination subject or imaging measurement results using electromagnetic waves are used for diagnosis. In particular, in the examination of luminal organs, various techniques using images obtained by moving an imaging element into the organ are used.

[0003] Among luminal organs, in particular, the image diagnosis of blood vessels is essential for safely and surely performing procedures such as percutaneous coronary intervention (PCI). For this reason, in addition to angiography that captures images from outside the body using a contrast agent, intravascular imaging techniques such as IVUS (Intra Vascular Ultra Sound) and OCT (Optical Coherence Tomography) / OFDI (Optical Frequency Domain Imaging) using a catheter have become widespread.

[0004] In the above-described image diagnosis, it is not easy to accurately obtain information related to diagnosis from the captured medical images. In order to assist in the reading of medical images, various techniques for correcting images or adding information using image analysis or machine learning have been proposed (Patent Document 1, etc.).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] For reading medical images, it is necessary to accurately grasp anatomical feature points from the images. Therefore, it is required to accurately capture medical images and present information regarding anatomical feature points in a more understandable and visible manner.

[0007] An object of the present disclosure is to provide a computer program, an information processing method, and an information processing apparatus that can present information regarding anatomical feature points of a luminal organ in a more understandable manner.

Means for Solving the Problem

[0008] The computer program according to the present disclosure causes a computer that acquires a plurality of tomographic images of the luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of a luminal organ having a branch portion, to execute a detection process including determination of the presence or absence of a second tube branching from the first tube and calculation of the angle of the second tube with respect to the plurality of tomographic images, and to execute a process of performing noise removal processing based on distribution information including a position in the longitudinal axis direction of the luminal organ of the tomographic image determined to be an image in which the second tube appears and the angle of the second tube in the tomographic image calculated in the detection process.

[0009] The information processing method according to the present disclosure includes a computer that acquires a plurality of tomographic images of the luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of a luminal organ having a branch portion, executing a detection process including determination of the presence or absence of a second tube branching from the first tube and calculation of the angle of the second tube with respect to the plurality of tomographic images, and performing noise removal processing based on distribution information including a position in the longitudinal axis direction of the luminal organ of the tomographic image determined to be an image in which the second tube appears and the angle of the second tube in the tomographic image calculated in the detection process.

[0010] An information processing apparatus according to the present disclosure is an information processing apparatus that acquires a plurality of tomographic images of a luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of a luminal organ having a branch portion. The information processing apparatus includes a processing unit that executes image processing on the plurality of tomographic images. The processing unit executes detection processing including determination of the presence or absence of a second tube branching from the first tube and calculation of the angle of the second tube with respect to the plurality of tomographic images. Based on distribution information including the position in the longitudinal axis direction of the luminal organ in the tomographic image determined to be an image in which the second tube appears by the detection processing and the angle of the second tube in the tomographic image calculated by the detection processing, noise removal processing is performed.

Advantages of the Invention

[0011] According to the present disclosure, it becomes possible to more easily grasp and display information regarding anatomical feature points of a luminal organ.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Embodiments for Carrying Out the Invention

[0013] Specific examples of a computer program, an information processing method, and an information processing apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

[0014] FIG. 1 is a diagram showing a configuration example of an imaging diagnostic apparatus 100. The imaging diagnostic apparatus 100 is an apparatus for generating a medical image including an ultrasonic tomographic image of a blood vessel (lumen organ) by the IVUS method and performing an ultrasonic examination and diagnosis inside the blood vessel.

[0015] The imaging diagnostic apparatus 100 includes a catheter 1, an MDU (Motor Drive Unit) 2, an image processing apparatus (information processing apparatus) 3, a display device 4, and an input device 5.

[0016] The catheter 1 is a flexible tube for medical use. The catheter 1 is particularly called an imaging catheter that is provided with an imaging device 11 at the tip and rotates circumferentially by driving from the proximal end. The imaging device 11 is an ultrasonic probe including an ultrasonic vibrator and an ultrasonic sensor in the case of the IVUS method. In the case of OCT, it is an OCT device including a near-infrared laser, a near-infrared sensor, and the like. The imaging device 11 may also use other devices that use electromagnetic waves of other wavelengths such as visible light.

[0017] The MDU 2 is a driving device attached to the proximal end of the catheter 1, and controls the operation of the catheter 1 by driving an internal motor according to the operation of a medical staff.

[0018] The image processing device 3 generates a plurality of medical images such as tomographic images of blood vessels based on the signals output from the imaging device 11 of the catheter 1. Details of the configuration of the image processing device 3 will be described later.

[0019] The display device 4 uses a liquid crystal display panel, an organic EL display panel, or the like. The display device 4 displays the medical images generated by the image processing device 3 and information related to the medical images.

[0020] The input device 5 is an input interface that receives operations on the image processing device 3. The input device 5 may be a keyboard, a mouse, or the like, or may be a touch panel, soft keys, hard keys, or the like built in the display device 4.

[0021] Figure 2 is an explanatory diagram showing the operation of the catheter 1. In Figure 2, the catheter 1 is inserted into the tubular blood vessel L along a guide wire W inserted into the coronary artery shown in the figure by a medical staff within the blood vessel. In the enlarged view of the blood vessel L in Figure 2, the right part corresponds to the distal side from the insertion location of the catheter 1 and the guide wire W, and the left part corresponds to the proximal side.

[0022] The catheter 1 is driven by the MDU 2 and moves from the distal end to the proximal end in the blood vessel L as indicated by the arrow in the figure, and scans the inside of the blood vessel spirally by the imaging device 11 while rotating in the circumferential direction.

[0023] In the imaging diagnostic apparatus 100 of the present embodiment, the image processing apparatus 3 acquires the signal for each scan output from the imaging device 11 of the catheter 1. In one scan, the imaging device 11 emits a detection wave in the radial direction and detects the reflected wave. The imaging device 11 performs this scan dozens to thousands of times while rotating 360 degrees, and scans spirally. The image processing apparatus 3 generates a tomographic image (cross-sectional image) obtained by performing polar coordinate transformation (inverse transformation) for every 360 degrees on the signal for each scan (I1 in FIG. 2). The tomographic image I1 is also referred to as a frame image. The reference point (center) of the tomographic image I1 corresponds to the range of the catheter 1 (not imaged). The image processing apparatus 3 further generates a long-axis image (longitudinal sectional image) in which the pixel values on the straight line passing through the reference point of the tomographic image I1 are arranged along the length direction (long axis direction) of the blood vessel (I2 in FIG. 2). The image processing apparatus 3 analyzes the branch structure of the blood vessel based on the obtained tomographic image I1 and long-axis image I2, and outputs a two-dimensional or three-dimensional image showing the structure of the blood vessel so that it can be visually recognized by medical staff. In the imaging diagnostic apparatus 100 according to the present disclosure, for both the long-axis image and the tomographic image displayed on the display device 4, information regarding the side branch such as the side branch position and contour of the first tube (for example, the main trunk) into which the guide wire W and the catheter 1 are inserted, and the second tube (for example, the collateral branch) branching from the first tube is superimposed and displayed. Hereinafter, the process for the image processing apparatus 3 to create information regarding the side branch and superimpose and display it on the display device 4 will be described in detail.

[0024] FIG. 3 is a block diagram showing the configuration of the image processing apparatus 3. The image processing apparatus 3 is a computer and includes a processing unit 30, a storage unit 31, and an input / output I / F 32.

[0025] The processing unit 30 includes one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), GPGPUs (General-purpose computing on graphics processing units), TPUs (Tensor Processing Units), etc. The processing unit 30 incorporates a non-temporary storage medium such as a RAM (Random Access Memory), stores data generated during processing in the non-temporary storage medium, and executes operations based on the computer program 3P stored in the storage unit 31.

[0026] The storage unit 31 is a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 31 stores the computer program 3P, setting data, etc. that the processing unit 30 reads. The storage unit 31 also stores the learned model 3M.

[0027] The computer program 3P and the learned model 3M may be those read out and replicated via the input / output I / F 32 from the computer program 9P and the learned model 9M stored in the non-temporary storage medium 9 outside the device. The computer program 3P and the learned model 3M may be those acquired by the image processing device 3 via a communication unit (not shown) from what is distributed by a remote server device and stored in the storage unit 31.

[0028] The input / output I / F 32 is an interface to which the catheter 1, the display device 4, and the input device 5 are connected. The processing unit 30 acquires a signal (digital data) output from the imaging device 11 via the input / output I / F 32. The processing unit 30 outputs the screen data of a screen including the generated tomographic image I1 and / or the long-axis image I2 to the display device 4 via the input / output I / F 32. The processing unit 30 receives operation information input to the input device 5 via the input / output I / F 32.

[0029] FIG. 4 is a schematic diagram of the learned model 3M. In the present disclosure, the learned model 3M is a model trained to output an image indicating the region of one or more objects depicted in the image when an image is input. The learned model 3M is, for example, a model that performs semantic segmentation. The learned model 3M is designed to output an image and accuracy in which data indicating which object each pixel in the input image belongs to is tagged for each pixel.

[0030] For example, as shown in FIG. 4, the learned model 3M uses a so-called U-net in which a convolutional layer, a pooling layer, an upsampling layer, and a softmax layer are arranged in a targeted manner. When the tomographic image I1 created from the signal from the catheter 1 is input, the learned model 3M outputs a tag image IS and accuracy. In the tag image IS, the lumen range of the blood vessel, the membrane range corresponding to the boundary between the lumen boundary of the blood vessel including the tunica media of the blood vessel and the blood vessel boundary, the range in which the guide wire W and the echo therefrom are depicted, and the range corresponding to the catheter 1 are each tagged with different pixel values (shown by different types of hatching and plain in FIG. 4) for the pixels at that position.

[0031] As described above, the learned model 3M is exemplified by semantic segmentation and U-net, but it is of course not limited thereto. In addition, the learned model 3M may be a model that realizes individual recognition processing such as instance segmentation. The learned model 3M is not limited to being based on U-net, and may use a model based on SegNet, R-CNN, or an integrated model with other edge extraction processes.

[0032] The processing unit 30 can detect the lumen boundary and the blood vessel boundary of the blood vessels shown in the tomographic image I1 by the pixel values in the tag image IS obtained by inputting the image I1 into the learned model 3M and the coordinates within the image. The blood vessel boundary is strictly the external elastic membrane (EEM) between the tunica media and the tunica adventitia of the blood vessel, which appears with a relatively low luminance clearly in the image I1 by the IVUS method. FIG. 5 is a diagram showing the detected boundaries (contours). FIG. 5 shows a state in which a curve B1 indicating the lumen boundary and a curve B2 indicating the blood vessel boundary obtained based on the output from the learned model 3M are superimposed and displayed with respect to the tomographic image I1 shown in FIG. 4.

[0033] The image processing apparatus 3 of the present disclosure further derives and displays anatomical feature points using the tomographic image I1 obtained from the signal from the catheter 1 and the information on the lumen boundary and the blood vessel boundary obtained when the tomographic image I1 is input into the learned model 3M. Hereinafter, detailed processing procedures will be described.

[0034] FIGS. 6 and 7 are flowcharts showing an example of the information processing procedure by the image processing apparatus 3. When a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing apparatus 3 starts the following processing.

[0035] Each time the processing unit 30 acquires a signal (data) from the imaging device 11 of the catheter 1 in a predetermined amount (for example, for 360 degrees) (step S301), it performs polar coordinate transformation (inverse transformation) on the signals arranged in a rectangle to generate a tomographic image I1 (step S302). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time within the screen displayed on the display device 4 (step S303). The processing unit 30 stores the signal data acquired in step S301 and the tomographic image I1 in the storage unit 31 in association with the position (position on the long axis, angle) of the imaging device 11 (step S304).

[0036] The processing unit 30 inputs the tomographic image I1 into the learned model 3M (step S305). Based on the tag image IS obtained from the learned model 3M, the processing unit 30 calculates data on the lumen boundary and blood vessel boundary in the tomographic image I1 (step S306). In step S306, the processing unit 30 calculates the contour (edge) of the lumen range as the lumen boundary and the outer contour of the membrane range as the blood vessel boundary from among the lumen range and the membrane range including the tunica media of the blood vessel output from the learned model 3M. In step S306, the processing unit 30 may perform high-speed processing, such as reducing the size of the tomographic image I1 and then inputting it into the learned model 3M.

[0037] Based on the data (coordinate data) of the lumen boundary and blood vessel boundary obtained in step S306, the processing unit 30 calculates a parameter for determining whether a collateral branch is imaged (step S307). When the tomographic image I1 images the bifurcation of the main trunk and the collateral branch, the shape of the blood vessel boundary derived from the tomographic image I1 deviates from a circle or an ellipse. In step S307, the processing unit 30 calculates a parameter corresponding to the degree of deviation from a circle or an ellipse.

[0038] In step S307, the processing unit 30 may basically calculate the parameter only for the blood vessel boundary. However, when the blood vessel boundary intersects outside the tomographic image I1, that is, when the membrane range reaches outside the image of the tomographic image I1, the processing unit 30 may calculate the parameter for the inner region of the lumen boundary instead of the blood vessel boundary.

[0039] In step S307, in the first example, the processing unit 30 calculates the eccentricity obtained by dividing the difference between the maximum diameter and the minimum diameter of the diameter passing through the centroid of the inner region of the blood vessel boundary by the maximum diameter.

[0040] In step S307, in the second example, when the processing unit 30 inputs data of the lumen boundary and the blood vessel boundary, it may use a learning model for determination (not shown) that has been learned to output a probability corresponding to the possibility that a collateral branch is shown. Here, the learning model for determination may be learned to output a probability corresponding to the possibility that a collateral branch is shown when data of the lumen range and the membrane range in the tag image output from the learned model 3M is input. The processing unit 30 calculates, in step S307, the output from the learning model for determination as a parameter.

[0041] In step S307, in the third example, the processing unit 30 may calculate, as a parameter, a value obtained by comparing the diameter (maximum diameter and minimum diameter) of the blood vessel boundary of the target with the diameter of the blood vessel boundary for the already scanned tomographic image I1. When the diameter changes suddenly by a predetermined ratio or more and for a predetermined length or more, it can be determined that there is a high possibility that a collateral branch is shown.

[0042] As a fourth example, the processing unit 30 may calculate circularity instead of eccentricity. Circularity is the ratio of the area of the inner region of the blood vessel boundary to the length of the circumference of the blood vessel boundary. The closer the circularity is to the ratio of the area to the circumference in a circle, the higher the circularity, and it can be determined that the possibility that a collateral branch is shown is low.

[0043] The processing unit 30 stores the parameter in association with the position of the imaging device 11 (step S308). The processing unit 30 determines, based on the calculated parameter, whether the target tomographic image I1 is a candidate for an image in which not only the main trunk into which the catheter 1 is inserted but also a collateral branch is shown (step S309). In step S309, the processing unit 30 may determine, for example, based on whether the eccentricity is higher than a predetermined value. If it is determined in step S309 that it is not a candidate (S309: NO), the processing unit 30 advances the process to step S313.

[0044] If it is determined to be a candidate (S309: YES), the processing unit 30 executes a collateral detection process including determination of the presence or absence of collateral with respect to the blood vessel boundary calculated from the target tomographic image I1, identification of the boundary of the main trunk in the tomographic image, and calculation of the angle of the collateral (step S310).

[0045] Based on the result of the collateral detection process in step S310, the processing unit 30 superimposes and displays an image showing the region boundary of the main trunk obtained and the data of the angle of the collateral on the tomographic image I1 being displayed in step S303 (step S311). Step S311 may be skipped if it is determined that there is no collateral in the tomographic image I1. The processing unit 30 stores the result of the collateral detection process in association with the tomographic image I1 stored in step S304 (step S312). When it is determined that there is a collateral, data such as a flag indicating the collateral image is associated with the tomographic image I1. The position corresponding to the tomographic image I1 associated with the data of the flag indicating the collateral image is the position of the branch of the main trunk and the collateral on the long axis.

[0046] The processing unit 30 may also perform the processing of steps S307 - S310 on the lumen boundary calculated in step S306.

[0047] The processing unit 30 determines whether the scanning by the imaging device 11 of the catheter 1 has been completed (step S313). If it is determined that the scanning has not been completed (S313: NO), the processing unit 30 returns the process to step S301 to generate the next tomographic image I1.

[0048] If it is determined that the scanning has been completed (S313: YES), the processing unit 30 executes a process of removing noise data based on the presence or absence of collateral and the distribution of the angles of the detected collateral with respect to the position of the imaging device 11 (step 314). In step S314, the processing unit 30 re - judges and stores as having no collateral even for a tomographic image I1 in which a collateral is determined to be shown by the process of step S310 but is determined to have low accuracy. Details will be described later.

[0049] The processing unit 30 causes the display device 4 to display a long-axis image I2 with a mark indicating the presence of a side branch at a location corresponding to the position of the tomographic image I1 determined to have a side branch in the process of step S314 (step S315), and ends the process.

[0050] FIG. 8 is a flowchart showing an example of the detailed processing procedure of the side branch detection process. The flowchart of FIG. 8 corresponds to the details of step S310 shown in the flowcharts of FIGS. 6 and 7.

[0051] The processing unit 30 extracts a circle along the boundary, for example, by Hough transform, with respect to the target boundary (vascular boundary or lumen boundary) (step S101). When, for example, Hough transform is used in step S101, the processing unit 30 extracts a plurality of circles of different sizes.

[0052] The processing unit 30 determines a likely circle corresponding to the main trunk from the circles extracted in step S101 (step S102). In step S102, the processing unit 30 selects, as the likely circle corresponding to the main trunk, a circle having a diameter equal to or greater than a predetermined length corresponding to the size of the blood vessel diameter and having a center closest to the centroid of the region within the target boundary. In step S102, the processing unit 30 may select, as the likely circle, a circle having a center closest to the center of the image, or may select, as the likely circle, a circle having the closest distance to the centroid of the previous and subsequent frame images. The processing unit 30 may appropriately select some or all of these selection methods in combination.

[0053] The processing unit 30 determines the center of the circle determined as the circle corresponding to the main trunk as the center of the blood vessel (step S103), and calculates the distance from the center of the blood vessel of the points on the target boundary over the entire circumference (360 degrees) along the boundary (step S104). The processing unit 30 creates a distribution of the distances from the center of the blood vessel along the entire circumference (step S105), and determines whether there are a continuous predetermined number or more of points on the boundary where the distance from the center of the blood vessel is longer than a predetermined reference value (step S106).

[0054] When it is determined that there are more than a predetermined number (S106: YES), the processing unit 30 determines that there are branches (shown) in the tomographic image I1 (step S107). The processing unit 30 identifies an arc on the circle of the main trunk determined in step 102 that corresponds to a point on the boundary that is continuously longer than a predetermined reference value from the center of the blood vessel (step S108).

[0055] The processing unit 30 calculates, as the branch angle, the angle from, for example, the 12 o'clock direction (upward direction) in the tomographic image I1 of a straight line connecting the center of the tomographic image I1 and the center point of the arc identified in step S108 with respect to the tomographic image I1 (step S109).

[0056] The processing unit 30 identifies an interpolation locus that reconnects, with an arc, two points on the boundary that are closest to the endpoints of the arc identified in step S108 (step S110). The processing unit 30 stores the interpolation locus identified in step S110 as an image indicating the region boundary of the main trunk (step S111), and returns the process to step S311 in FIGS. 6 and 7.

[0057] In step S106, when it is determined that there are not more than a predetermined number (S106: NO), the processing unit 30 determines that there are no branches in the tomographic image I1 (step S112), and returns the process to step S311 in FIGS. 6 and 7.

[0058] The processing procedure shown in the flowchart of FIG. 8 will be described with a specific example. FIGS. 9-13 are schematic diagrams of the branch detection process. FIG. 9 is a diagram showing an example of a circle extracted with respect to the blood vessel boundary. FIG. 9 shows the tomographic image I1 with the curve B2 of the blood vessel boundary calculated with respect to the tomographic image I1 superimposed, and after the circle extraction process for the curve B2, the extracted circles are shown in bold. As shown in FIG. 9, from the tomographic image I1 that is a candidate for showing a branch, circles are extracted for the part corresponding to the main trunk in the lower left part of the tomographic image I1 and the part corresponding to the branch. However, in FIG. 9, the circle with a larger diameter in the lower left is determined as the one with a shorter distance from the center of gravity.

[0059] FIG. 10 shows the distribution obtained by the process of step S104 in the flowchart of FIG. 8. At the upper part of FIG. 10, the calculated curve B2 of the blood vessel boundary and the determined circle are shown superimposed on the tomographic image I1. At the lower part of FIG. 10, the distribution of the distances from the center of the blood vessel to the coordinate points on the curve B2 is shown. The arrow signs in FIG. 10 indicate the order in which the distances are calculated in sequence. As shown in FIG. 10, in the image where the side branch is shown within the tomographic image I1, as shown in the lower distribution, there is clearly a peak in the distance.

[0060] FIG. 11 is a diagram showing the determination (S106) of the presence or absence of a side branch based on the distribution of FIG. 10. In FIG. 11, on the distribution shown in FIG. 10, the reference value for the distance is shown by the thick line in the figure. In the diagram of FIG. 11, it is determined that there are a predetermined number or more of pixels with a distance greater than or equal to the reference value. At the lower part of FIG. 11, the points on the boundary (B2) where the distance from the center of the blood vessel is continuously longer than a predetermined reference value are shown in black, and the corresponding arc (with black dots at the endpoints) is shown (S108). The arc with black dots at the endpoints is specifically specified as the range between two straight lines connecting the two ends of a part on the boundary (B2) that is continuously longer than a predetermined reference value among the circles corresponding to the main trunk and the center of the blood vessel.

[0061] The determination (S106) of whether a side branch is highly likely to exist (shown) is not limited to the content shown in FIG. 11. For example, it is not limited to only whether there are a continuous predetermined number or more of coordinate points with a distance greater than or equal to the radius of the circle determined for the blood vessel boundary. A determination may be made such that there are a continuous predetermined number or more of coordinate points on the lumen boundary with a distance greater than or equal to the radius of the circle determined for the lumen boundary, and the range overlaps with the range specified for the blood vessel boundary.

[0062] FIG. 12 shows an example of an interpolation trajectory specified based on the arc specified in FIG. 11. As shown in FIG. 12, the interpolation trajectory is specified as an arc passing through two points on the curve B2 of the blood vessel boundary, which are close to each of the end points of the arc specified in step S108. In FIG. 12, information on the calculated branch angle (direction of the branch) is shown as a white arrow. At this time, the image (arrow) showing the branch angle is shown to be displayed outside the curve B2 so that the curve B2 and the interpolation trajectory can be easily visually recognized.

[0063] FIG. 13 shows a display example on the display device 4. In FIG. 13, the tomographic image I1, the inner cavity boundary and the curves B1 and B2 of the blood vessel boundary superimposed on the tomographic image I1, the interpolation trajectory, and the information on the branch angle are superimposed and displayed. In the example shown in FIG. 13, an interpolation trajectory specified in the same manner for the inner cavity boundary is also superimposed and displayed. This makes it easier for medical staff who are not accustomed to reading radiographs to read the presence or absence of branches and the locations where branches where plaques tend to accumulate exist.

[0064] FIG. 14 is a flowchart showing an example of a processing procedure for removing noise data of branch detection. The flowchart of FIG. 14 corresponds to the details of step S314 shown in the flowcharts of FIGS. 6 and 7.

[0065] The processing unit 30 determines whether there is one or more other tomographic images I1 in which a branch is determined to exist within a predetermined range from the position of the tomographic image I1 in which a branch is determined to exist within the scanning range of the catheter 1 (step S401). The predetermined range is, for example, a range corresponding to the diameter of the branch (1 to 5 mm). At an actual location where a branch exists, it should be determined that a branch exists in a plurality of consecutive tomographic images I1. Therefore, if the tomographic image I1 in which a branch is determined to exist is isolated, this is presumed to be a false detection. In step S401, for example, when it is determined that a branch exists in 80% of the plurality of tomographic images I1 generated for each position within the predetermined range from the position of the tomographic image I1 in which a branch is determined to exist, the processing unit 30 may determine that there are a plurality of them.

[0066] Therefore, when the processing unit 30 determines in step S401 that there is no other tomographic image I1 (S401: NO), the processing unit 30 removes the target tomographic image I1 from the images in which side branches exist (step S402). Specifically, in step S402, the processing unit 30 deletes the flag indicating the side branch image associated with the tomographic image I1, the information of the side branch, and the like. The processing unit 30 returns the process to step S315 in FIGS. 6 and 7.

[0067] When it is determined that there is another tomographic image I1 (S401: YES), it is determined whether the side branch angles are similar as compared with another tomographic image I1 in which a side branch is determined to exist within a predetermined range (step S403). When the same side branch is detected, the calculated side branch angles should be similar.

[0068] Therefore, when it is determined that they are not similar (S403: NO), the processing unit 30 removes the target tomographic image I1 from the images in which side branches exist (S402), and returns the process to step S315 in FIGS. 6 and 7.

[0069] When it is determined that they are similar (S403: YES), the processing unit 30 returns the process to step S315 in FIGS. 6 and 7. In this case, for the tomographic image I1 determined to be similar, the data such as the flag indicating that it is a side branch image remains associated. The tomographic image I1 not removed by the flowchart in FIG. 14 becomes an image in which it is confirmed that a side branch is shown from the candidate images in which the side branch is shown.

[0070] FIG. 15 is a schematic diagram of the noise data removal process for side branch detection. FIG. 15 is a distribution diagram of side branch angles with respect to the position of the tomographic image I1 in which a side branch is determined to exist within the scanning range of the catheter 1. The horizontal axis in FIG. 15 is the longitudinal axis direction of the blood vessel in which the catheter 1 moves, and the vertical axis indicates the angle within the blood vessel. Each point indicates the position of the tomographic image I1 in which a side branch is determined to exist in the longitudinal axis direction, the angle range in which a side branch is determined to exist for the side branch, and the side branch angle which is the central value thereof. In FIG. 15, the range on the longitudinal axis where an actual side branch exists is hatched.

[0071] As shown in FIG. 15, at the locations where side branches actually exist, the continuous points continue within the same angular range. In contrast, the points corresponding to the tomographic image I1 where side branches are determined to exist at locations where there are actually no side branches have calculated side branch angles that are also scattered and lack continuity. Therefore, based on whether there is continuity in the position on this major axis and the side branch angle, the information of the tomographic image I1 that has been erroneously determined to have side branches can be removed as noise.

[0072] By clustering the features of each point shown in FIG. 15 (position in the major axis direction, side branch range, and side branch angle), points that do not belong to the cluster may be removed as noise as the tomographic image I1 that has been erroneously determined to have side branches. For clustering, for example, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) may be adopted, or other known methods may be used.

[0073] In this way, the image diagnostic apparatus 100 can narrow down to the information of the tomographic image I1 that is determined to have side branches with a high probability rather than noise, and can present the respective tomographic images I1 together with data such as the angle of the side branches in the tomographic image I1. Specifically, in the storage unit 31 of the image processing apparatus 3, for an examination using the catheter 1 once, the signal data at each time point from the imaging device 11 and the tomographic image I1 after polar coordinate conversion are stored. Also, in the storage unit 31, coordinate data related to the lumen boundary and blood vessel boundary (curves, centroids, centers of blood vessels, blood vessel diameters (minimum diameters), etc.), presence or absence of side branches, side branch angles, data of interpolation trajectories, etc., obtained by processing each tomographic image I1 are stored. Based on these data, the image processing apparatus 3 can present the data of anatomical feature points in real time or retrospectively during the examination using the catheter 1.

[0074] FIG. 16 shows an example of a screen including information presented on the display device 4. The screen 400 shown in FIG. 16 includes respective display areas of the long-axis image I2 and the tomographic image I1 at the position being scanned or selected at the display time. In the display area of the tomographic image I1, as shown in FIG. 13, the curves B1 and B2 of the lumen boundary and the blood vessel boundary, the interpolation locus, and the information on the collateral angle are superimposed and displayed.

[0075] As shown in FIG. 16, the long-axis image I2 of the screen 400 is a longitudinal cross-sectional image with the longitudinal direction of the blood vessel in which the catheter 1 moves as the horizontal direction. By default, it is displayed as a set of luminance data along the vertical direction in the tomographic image I1. A cursor 403 indicating the position on the long axis corresponding to the tomographic image I1 is displayed on the long-axis image I2. Further, the screen 400 includes a first button 401 and a second button 402 at a location corresponding to the position in the longitudinal direction of the tomographic image I1 determined and confirmed to have a collateral on the long-axis image I2. The buttons 401 and 402 are displayed for the number of collaterals determined and confirmed to exist.

[0076] The first button 401 and the second button 402 are buttons for displaying the tomographic image I1 in which each collateral is determined and confirmed to exist and the data related to the corresponding collateral. When the first button 401 and the second button 402 are selected, the long-axis image I2 becomes a longitudinal cross-sectional image at the collateral angle corresponding to each button in each tomographic image I1.

[0077] Figures 17 and 18 show other screen examples displayed on the display device 4. Figures 17 and 18 are the screens displayed when the first button 401 and the second button 402 in the screen shown in Figure 16 are pressed respectively. Figure 17 is the screen displayed when the first button 401 is selected by the input device 5, and Figure 18 is the screen displayed when the second button 402 is selected by the input device 5. Comparing Figures 16 - 18, it can be seen that the cross-sectional direction of the long-axis image I2 is different among the default, the case where the first button 401 is selected, and the case where the second button 402 is selected. Thereby, for medical staff performing fluoroscopy, it becomes easier to observe the entire blood vessel by focusing on the side branch and grasp the structure anatomically from the inspection results using the catheter 1.

[0078] By accurately detecting the position in the straight-axis direction of the side branch, detecting the interpolation locus of the main trunk and the side branch, and further detecting the side branch angle, in addition, anatomical feature points at the bifurcation of the blood vessel can be displayed on the display device 4 in various forms. Figure 19 shows another screen example displayed on the display device 4. The screen 400 shown in Figure 19 includes a schematic longitudinal cross-sectional image I3 in which the blood vessel regions specified for each tomographic image I1 are connected in the long-axis direction with the center of the blood vessel as the axis. Not only schematically, but also a longitudinal cross-sectional image considering the dimensions based on the diameter of the blood vessel boundary and the diameter of the lumen boundary obtained from each tomographic image I1 may be displayed.

[0079] Also in the screen 400 of Figure 19, the first button 401, the second button 402, and the cursor 403 are displayed on the long-axis image I2. When the first button 401 is selected by the input device 5, the cutting direction of the schematic longitudinal cross-sectional image I3 is changed to the side branch angle of the side branch corresponding to the first button 401.

[0080] When the first button 401 or the second button 402 for selecting the location where the side branch exists is selected, as shown in Figure 19, a tomographic image I4 centered on the center of the side branch may be further generated and displayed for the schematic longitudinal cross-sectional image I3. Such an image makes it easier for medical staff to grasp the structure of the blood vessel by visually recognizing the display device 4.

[0081] As shown in FIG. 4, since the guide wire W and its echo range can be recognized, points corresponding to the guide wire W in each tomographic image I1 (for example, points closer to the catheter 1) can be recognized, and an image connecting the blood vessel boundaries centered on those points may be displayed.

[0082] FIG. 20 shows another example of a screen displayed on the display device 4. FIG. 20 shows an example of displaying a three-dimensional image. FIG. 20 shows the screen 400 displayed on the display device 4 similar to FIGS. 16-19. The screen 400 in FIG. 20 includes a three-dimensional image I5 of the main trunk and branches. The three-dimensional image I5 simplifies both the main trunk and the branches into tubes, and is created from the data regarding the branches obtained by the above-described processing so that the position of the branch and the branch angle in the major axis direction can be visually grasped. Thereby, medical staff can easily grasp the anatomical features (branches) of the blood vessels.

[0083] Since the image processing apparatus 3 stores the side branch detection results in association with the positions in the major axis direction (S312), it is also possible to use these stored data to make it easier for medical staff to understand and display the anatomical features of blood vessels. FIG. 21 shows another example of a screen displayed on the display device 4. In FIG. 21, a curve B3 indicating the lumen boundary detected at each position and a curve B4 indicating the region boundary (blood vessel boundary) of the main trunk are superimposed and displayed on the major axis image I2. The curves B3 and B4 may be drawn superimposed on the major axis image I2 in a color that is easy to visually recognize. The processing unit 30 creates a curved surface by overlapping the curve B1 of the lumen boundary and the curve B2 of the blood vessel boundary of each tomographic image I1 shown in FIG. 13 in the major axis direction, and creates a longitudinal sectional image obtained by cutting the curved surface in the major axis direction. At a position where a side branch appears and the region boundary of the main trunk is interpolated, the processing unit 30 connects the interpolation locus indicated by a broken line to the curve B2 of the blood vessel boundary. The same applies to the lumen boundary. The processing unit 30 creates the curved surface by connecting the points on the curve B1 and the points on the curve B2 with spline curves in the major axis direction, respectively, and creates a longitudinal section at a specific cutting angle from the created curved surface. The boundary curve of the lumen region connecting the points on the curve B1 and the curve connecting the interpolated boundary depending on the location are indicated by the symbol B3, and the boundary curve connecting the points on the curve B2 of the blood vessel boundary and the points on the region boundary of the main trunk depending on the location are indicated by the symbol B4. The specific cutting angle may be selectable by the input device 5. As a result, it is easier for medical staff to understand the anatomical features of blood vessels over the scanning range.

[0084] In this embodiment, an image processing device 3 connected to a catheter 1 generates a tomographic image I1 almost in real time based on a signal from an imaging device 11, determines whether an image shows a side branch, and causes a display device 4 to display data indicating anatomical features. However, each process including the determination of whether an image shows a side branch by the image processing device 3 described above may be performed separately and retrospectively on the generated tomographic image I1. That is, the image processing device 3 is not necessarily directly connected to the imaging device 11 of the catheter 1. The image processing device 3 may be, for example, a server device that can read a storage device storing signals from the imaging device 11 via a network. That is, the processing procedures of steps S301 - S304 shown in the flowcharts of FIGS. 6 and 7 are performed by an existing image diagnostic device, and the processing of steps S305 - S315 and the flowchart of FIG. 8 are executed by the image processing device 3 connected to the processing device, and may be displayed on the display device 4.

[0085] In this embodiment, a medical image has been described by taking as an example an image obtained by IVUS for coronary arteries. However, the application target is not limited to this, and it may be OCT / OFDI or the like, and the luminal organ is not limited to blood vessels.

[0086] The embodiments disclosed as above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and includes all modifications within the meaning and scope equivalent to the claims.

Explanation of Reference Numerals

[0087] 1 Catheter 11 Imaging device 3 Image processing device (information processing device) 30 Processing unit 31 Storage unit 3P Computer program 3M Trained model 4 Display device 400 Screen 401 First button 402 Second Button I1, I4 Tomographic Images I2 Long Axis Image I3 Pattern Vertical Cross-Section Image I5 Three-Dimensional Image

Claims

1. A computer that acquires a plurality of tomographic images of the luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of the luminal organ having a bifurcation portion, performs a detection process including determination of the presence or absence of a second tube branching from the first tube and calculation of the angle of the second tube on the plurality of tomographic images, and performs a noise removal process based on distribution information including the position in the longitudinal axis direction of the luminal organ of the tomographic image determined to be an image in which the second tube appears and the angle of the second tube in the tomographic image calculated by the detection process. A computer program for executing the process.

2. The noise removal process determines whether there is one or more other tomographic images determined to be images in which the second tube appears within a predetermined range from the position in the longitudinal axis direction of the luminal organ of the target tomographic image determined to be an image in which the second tube appears, and when it is determined that there are no other tomographic images, removes the result of the detection process corresponding to the target tomographic image as noise. The computer program according to claim 1, including the process.

3. The noise removal process performs a clustering process on the distribution information, and removes the result of the detection process corresponding to a position or angle not belonging to the cluster as noise. The computer program according to claim 1, including the process.

4. A computer that acquires a plurality of tomographic images of the luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of the luminal organ having a bifurcation portion, performs a detection process including determination of the presence or absence of a second tube branching from the first tube and calculation of the angle of the second tube on the plurality of tomographic images, and performs a noise removal process based on distribution information including the position in the longitudinal axis direction of the luminal organ of the tomographic image determined to be an image in which the second tube appears and the angle of the second tube in the tomographic image calculated by the detection process. An information processing method.

5. In an information processing apparatus that acquires a plurality of tomographic images of the luminal organ generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of the luminal organ having a bifurcation portion, a processing unit that performs image processing on the plurality of tomographic images is provided, The processing unit Execute a detection process including determination of the presence or absence of a second tube branched from the first tube and calculation of the angle of the second tube with respect to the plurality of tomographic images. Perform a noise removal process based on distribution information including the position in the major axis direction of the luminal organ of the tomographic image determined to be an image in which the second tube appears by the detection process and the angle of the second tube in the tomographic image calculated by the detection process. Information processing apparatus.

Images