Computer program, information processing method, and information processing apparatus

Abstract

To provide a computer program, an information processing method, and an information processing device capable of three-dimensionally presenting information on an anatomical feature point on the basis of an image acquired using a catheter.SOLUTION: A computer program causes a computer for acquiring a signal detected by an imaging device equipped to a catheter inserted into a hollow organ that moves in a length direction while rotating with the length direction of the hollow organ as an axis to execute processing for, by using a three-dimensional identification model in which learning is executed so as to output three-dimensional data indicating regions of one or a plurality of objects in a three-dimensional image when the three-dimensional image of the hollow organ based on the signal is input, inputting the three-dimensional image based on the signal acquired from the imaging device to the three-dimensional identification model, and creating a three-dimensional image indicating the structure of the hollow organ from the three-dimensional data output from the three-dimensional identification model.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to a computer program, an information processing method, and an information processing apparatus for processing medical images.

Background Art

[0002] In medical examinations, images obtained by directly imaging the image of the examination object or imaging the measurement results using electromagnetic waves are used for diagnosis. In particular, in the examination of luminal organs, various techniques using images obtained by moving an imaging element into the organ are used.

[0003] Among luminal organs, image diagnosis of blood vessels is particularly indispensable for safely and surely performing procedures such as percutaneous coronary intervention (PCI). For this reason, in addition to angiography that takes images from outside the body using a contrast agent, intravascular imaging techniques such as IVUS (Intra Vascular Ultra Sound) and OCT (Optical Coherence Tomography) / OFDI (Optical Frequency Domain Imaging) using a catheter have become widespread.

[0004] In the above-described image diagnosis, it is not easy to accurately obtain information related to diagnosis from the taken medical images. In order to assist in reading medical images, various techniques for correcting images or adding information using image analysis or machine learning have been proposed (Patent Document 1, etc.).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Interpreting medical images requires accurately identifying anatomical features from the images. Therefore, it is necessary to not only accurately capture medical images but also to present information about anatomical features in a more easily understandable and visually apparent manner. Providing more accurate three-dimensional reconstructions of tubular organs from data obtained using imaging devices makes it easier for medical professionals to grasp the features, leading to improved accuracy in examinations and procedures.

[0007] The purpose of this disclosure is to provide a computer program, an information processing method, and an information processing device that present information about anatomical feature points in three dimensions based on images obtained using a catheter. [Means for solving the problem]

[0008] The computer program relating to this disclosure uses a three-dimensional recognition model that is trained to output three-dimensional data indicating the regions of one or more objects depicted in a three-dimensional image when a three-dimensional image of the lumen organ based on the signal is input to a computer that acquires a signal detected by an imaging device provided on a catheter inserted into a lumen organ and moving in the longitudinal direction while rotating around the longitudinal axis of the lumen organ. The computer program inputs the three-dimensional image based on the signal obtained from the imaging device to the three-dimensional recognition model and executes a process to create a three-dimensional image showing the structure of the lumen organ from the three-dimensional data output from the three-dimensional recognition model.

[0009] The information processing method relating to this disclosure involves a computer that acquires signals detected by an imaging device provided on a catheter inserted into a tubular organ and moving in the longitudinal direction while rotating about the longitudinal axis of the tubular organ. The computer uses a three-dimensional identification model that is trained to output three-dimensional data indicating the regions of one or more objects depicted in a three-dimensional image when a three-dimensional image of the tubular organ based on the signals is input to the three-dimensional identification model. The computer inputs the three-dimensional image based on the signals obtained from the imaging device to the three-dimensional identification model, and creates a three-dimensional image showing the structure of the tubular organ from the three-dimensional data output from the three-dimensional identification model.

[0010] The information processing device according to this disclosure is an information processing device that acquires a signal detected by an imaging device provided on a catheter inserted into a tubular organ and moving in the longitudinal direction while rotating about the longitudinal direction of the tubular organ as an axis, and comprises a storage unit that stores a three-dimensional identification model which is trained to output three-dimensional data indicating the regions of one or more objects depicted in the three-dimensional image when a three-dimensional image of the tubular organ based on the signal is input, and a processing unit that performs image processing based on the signal, wherein the processing unit inputs the three-dimensional image based on the signal obtained from the imaging device to the three-dimensional identification model and creates a three-dimensional image showing the structure of the tubular organ from the three-dimensional data output from the three-dimensional identification model. [Effects of the Invention]

[0011] According to this disclosure, it is possible to present a three-dimensional image that reconstructs a tubular organ based on tomographic images of the tubular organ, and further present data on the anatomical features in that image. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of the configuration of an imaging diagnostic device. [Figure 2] This is an explanatory diagram showing the operation of a catheter. [Figure 3] This is a block diagram showing the configuration of an image processing device. [Figure 4] It is a schematic diagram of a learned model. [Figure 5] It is a diagram showing the detected boundary. [Figure 6] It is a flowchart showing an example of an information processing procedure by an image processing apparatus. [Figure 7] It is a flowchart showing an example of an information processing procedure by an image processing apparatus. [Figure 8] It is a diagram showing how to create a three-dimensional image to be input into the learned model 3M. [Figure 9] It is a schematic diagram of an image obtained by overlapping the divided images before correction. [Figure 10] It is a schematic diagram of the correction of the divided image of the three-dimensional map and the three-dimensional image after correction. [Figure 11] It is a diagram showing an example of a screen on a display device. [Figure 12] It is a schematic diagram of a three-dimensional image created by a signal from an imaging device. [Figure 13] It is a flowchart showing an example of an information processing procedure in the second embodiment. [Figure 14] It is a block diagram showing the configuration of an image processing apparatus in the third embodiment. [Figure 15] It is a schematic diagram of the second model. [Figure 16] It is a flowchart showing the information processing procedure in the third embodiment. [Figure 17] It is a flowchart showing the information processing procedure in the third embodiment. [Figure 18] It is a schematic diagram of the processing in the third embodiment. [Figure 19] It is a flowchart showing the information processing procedure in the fourth embodiment. [Figure 20] It is a flowchart showing the information processing procedure in the fourth embodiment. [Figure 21] It is a schematic diagram of the processing in the fourth embodiment.

Embodiments for Carrying Out the Invention

[0013] Specific examples of a computer program, information processing method, and information processing apparatus according to the present invention will be described below with reference to the drawings.

[0014] (First Embodiment) Figure 1 shows an example of the configuration of the diagnostic imaging device 100. The diagnostic imaging device 100 is a device that generates medical images including ultrasound tomographic images of blood vessels (luminal organs) using the IVUS method and is used for ultrasound examination and diagnosis of blood vessels.

[0015] The diagnostic imaging device 100 comprises a catheter 1, an MDU (Motor Drive Unit) 2, an image processing device (information processing device) 3, a display device 4, and an input device 5.

[0016] Catheter 1 is a flexible tube used for medical purposes. Catheter 1 is specifically an imaging catheter, which has an imaging device 11 at its tip and rotates circumferentially by a drive from its proximal end. In the case of IVUS, the imaging device 11 is an ultrasound probe including an ultrasound transducer and an ultrasound sensor. In the case of OCT, it is an OCT device including a near-infrared laser and a near-infrared sensor, etc. The imaging device 11 may also be a device that uses electromagnetic waves of other wavelengths, such as visible light.

[0017] The MDU2 is a drive device attached to the proximal end of the catheter 1, and controls the movement of the catheter 1 by driving an internal motor in response to the actions of a medical professional.

[0018] The image processing device 3 generates multiple medical images, such as tomographic images of blood vessels, based on signals output from the imaging device 11 of the catheter 1. Details of the configuration of the image processing device 3 will be described later.

[0019] The display device 4 uses a liquid crystal display panel, an organic EL display panel, or the like. The display device 4 displays medical images generated by the image processing device 3 and information related to those medical images.

[0020] The input device 5 is an input interface that accepts operations for the image processing device 3. The input device 5 may be a keyboard, mouse, etc., or it may be a touch panel, soft keys, hard keys, etc., built into the display device 4.

[0021] Figure 2 is an explanatory diagram illustrating the operation of catheter 1. In Figure 2, catheter 1 is inserted into a tubular blood vessel L by a medical professional, following a guidewire W inserted into the coronary artery shown in the figure. In the enlarged view of blood vessel L in Figure 2, the right side corresponds to the distal part from the insertion site of catheter 1 and guidewire W, and the left side corresponds to the proximal part.

[0022] Driven by the MDU2, catheter 1 moves from distal to proximal within blood vessel L, as indicated by the arrow in the figure, while rotating around its axis in the longitudinal direction. As a result, imaging device 11 scans within blood vessel L in a spiral pattern.

[0023] In the diagnostic imaging apparatus 100 of this embodiment, the image processing apparatus 3 acquires the signal output from the imaging device 11 of the catheter 1 for each scan. In one scan, the imaging device 11 emits a detection wave radially and detects the reflected wave. The imaging device 11 performs this scan tens to thousands of times while rotating 360 degrees. The image processing apparatus 3 generates a tomographic image (cross-sectional image) obtained by aligning the signals from each scan radially and arranging them in a rectangular shape, and then performing a polar coordinate transformation (inverse transformation) every 360 degrees (I1 in Figure 2). The tomographic image I1 is also called a frame image. The reference point (center) of the tomographic image I1 corresponds to the range of the catheter 1 (not imaged). Based on the obtained tomographic image I1, the image processing apparatus 3 analyzes the branching structure of the blood vessels and can output a two-dimensional or three-dimensional image showing the structure of the blood vessels in a way that can be viewed by medical professionals. In the diagnostic imaging device 100 described herein, a three-dimensional image is created from a tomographic image I1 or from a scanning signal, and a model is used to directly divide the three-dimensional image into areas, thereby displaying anatomical features in three dimensions. The creation of three-dimensional images of blood vessels (luminal organs) and image processing by the image processing device 3 will be described in detail below.

[0024] Figure 3 is a block diagram showing the configuration of the image processing device 3. The image processing device 3 is a computer and comprises a processing unit 30, a storage unit 31, and an input / output interface 32.

[0025] The processing unit 30 includes one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), GPGPUs (General-purpose computing on graphics processing units), TPUs (Tensor Processing Units), etc. The processing unit 30 incorporates a non-temporary storage medium such as RAM (Random Access Memory) and stores data generated during processing in the non-temporary storage medium while executing calculations based on the computer program 3P stored in the storage unit 31.

[0026] The memory unit 31 is a non-volatile storage medium such as a hard disk or flash memory. The memory unit 31 stores the computer program 3P, configuration data, etc., which are read by the processing unit 30. The memory unit 31 also stores the learned model 3M.

[0027] The computer program 3P and the trained model 3M may be copies obtained by reading the computer program 9P and the trained model 9M stored in a non-temporary storage medium 9 outside the device via the input / output I / F 32. The computer program 3P and the trained model 3M may also be obtained by the image processing device 3 via a communication unit (not shown) from content distributed by a remote server device and storing it in the storage unit 31.

[0028] The input / output interface 32 is the interface to which the catheter 1, display device 4, and input device 5 are connected. The processing unit 30 acquires signals (digital data) output from the imaging device 11 via the input / output interface 32. The processing unit 30 outputs screen data, including the generated tomographic image I1 and / or longitudinal image I2, to the display device 4 via the input / output interface 32. The processing unit 30 receives operation information input to the input device 5 via the input / output interface 32.

[0029] The image processing device 3 may not only have the configuration shown in Figure 3, but may also be divided into a first processing device that is directly connected to the imaging device 11 (catheter 1) to generate a tomographic image I1, and a second processing device that acquires the generated tomographic image I1 and signals from the imaging device 11 from the processing device and performs processing using a trained model 3M. In this case, a display device 4 is connected to the second processing device, and the second processing device outputs a three-dimensional image obtained by processing the tomographic image I1 and the trained model 3M to the display device 4.

[0030] Figure 4 is an overview diagram of the trained model 3M. The trained model 3M in this disclosure is a model that, when a three-dimensional image (values ​​for each voxel) is input, is trained to output an image showing the region of one or more objects depicted in the three-dimensional image. The trained model 3M is, for example, a model that performs three-dimensional semantic segmentation. The trained model 3M is designed to output a three-dimensional map P that tags each voxel in the input three-dimensional image, indicating which object region each voxel belongs to.

[0031] The trained model 3M uses a so-called U-net, which symmetrically arranges convolutional layers, pooling layers, upsampling layers, and softmax layers, as shown in Figure 4. The trained model 3M is a model that outputs a three-dimensional map P showing the identified areas within an image when a three-dimensional image is input. The input three-dimensional image is data such as a tomographic image I1 superimposed in the longitudinal direction and with three-dimensional coordinates assigned to each pixel, as shown in Figure 4. The output three-dimensional map P tags the voxels at the locations of the blood vessels with different voxel values ​​(shown as different types of hatching and plain in Figure 4), indicating the luminal area of ​​the blood vessel, the membrane area corresponding to the boundary between the luminal boundary of the blood vessel and the vascular boundary including the tunica media of the blood vessel, the area showing the guidewire W and its reflections, and the area corresponding to the catheter 1. The trained model 3M is preferably trained with the voxel size of the input three-dimensional image fixed, i.e., the length, width, and length (length in the longitudinal direction). Fixing the size maintains the accuracy of the output three-dimensional map P. The trained model 3M may also be able to identify areas of plaque, calcification, etc., that are formed in blood vessels.

[0032] As mentioned above, the pre-trained model 3M is exemplified by semantic segmentation and U-net, but it is not limited to these. The pre-trained model 3M may also be a model that implements individual recognition processing using instance segmentation, etc. The pre-trained model 3M is not limited to U-net; it may also use a model based on SegNet, R-CNN, or an integrated model with other edge extraction processing.

[0033] The processing unit 30 can detect the lumen boundary and vascular boundary of a blood vessel examined using the catheter 1 by inputting a three-dimensional image into the trained model 3M and using the voxel values ​​in the three-dimensional map P obtained and the coordinates of the voxels in that image. The vascular boundary is, strictly speaking, the external elastic membrane (EEM) between the media and adventitia of the blood vessel, and is captured relatively clearly with low brightness in the tomographic image I1 using the IVUS method. Figure 5 shows the detected boundary (contour). Figure 5 shows a state in which the curved surface F1 indicating the lumen boundary and the curved surface F2 indicating the vascular boundary, obtained based on the three-dimensional map P output from the trained model 3M, are superimposed on the three-dimensional image obtained by superimposing the tomographic image I1 shown in Figure 4.

[0034] As described above, by displaying the results of identifying the extent of blood vessels in three dimensions based on the signals obtained from the imaging device 11, along with the tomographic image I1, it becomes easier for medical professionals involved in the examination to understand the inside of the tubular organ. However, tubular organs pulsate during the examination, and in addition, the axis may shift due to the progress of the catheter 1, the structure inside the blood vessel, plaque, etc., so the three-dimensional image obtained by simply connecting the signals obtained from the imaging device 11 is often blurry and does not reproduce the actual blood vessel.

[0035] Therefore, the image processing device 3 in this disclosure applies corrections to either the signal obtained from the imaging device 11, or the three-dimensional map P output from the trained model 3M, or both, to create a three-dimensional image that more accurately reproduces the tubular organ. The correction processing procedure and the display content will be described in detail below.

[0036] Figures 6 and 7 are flowcharts illustrating an example of the information processing procedure by the image processing device 3. When a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing device 3 starts the following processing.

[0037] Each time the processing unit 30 acquires a predetermined amount (for example, 360 degrees) of signal (data) from the imaging device 11 of the catheter 1 (step S101), it transforms (inversely transforms) an image of radial signals arranged in a rectangle into polar coordinates to generate a tomographic image I1 (step S102) (see Figure 2). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time on the screen displayed on the display device 4 (step S103). The processing unit 30 stores the signal data acquired in step S101 and the tomographic image I1 in the storage unit 31, associating them with their positions on the long axis of the blood vessel (step S104). In step S104, the processing unit 30 may also store the scanning angle of the imaging device 11.

[0038] The processing unit 30 determines whether or not a three-dimensional image can be created (step S105). In step S105, the processing unit 30 determines, for example, whether or not a predetermined number of tomographic images I1 generated in step S102 have been collected. For example, the processing unit 30 determines that it can be created when 32 frames of tomographic images I1 have been collected. Alternatively, the processing unit 30 may determine whether or not the length of scanning has reached a predetermined length, or it may determine in step S105 whether or not scanning has been completed.

[0039] If it is determined that a three-dimensional image cannot be created (S105: NO), the processing unit 30 returns to step S101 and continues processing until a three-dimensional image can be created.

[0040] If it is determined that a three-dimensional image can be created (S105: YES), the processing unit 30 creates a three-dimensional image by overlapping the tomographic images I1 in the longitudinal direction (step S106). In step S106, the processing unit 30 creates the image by overlapping a predetermined number of tomographic images I1 (for example, 5, 10, etc.). The processing unit 30 may also perform step S106 by overlapping all tomographic images I1 across the scanning range.

[0041] The processing unit 30 inputs the created three-dimensional image to the trained model 3M (step S107). The processing unit 30 removes noise from the three-dimensional map P output from the trained model 3M and corrects the surface (contour) that indicates the boundary (step S108). In step S108, the processing unit 30 removes points on the boundary that are outliers from the boundary surface (noise). The processing unit 30 may correct areas where data is missing by connecting other points with a spline curve or the like, or by connecting them with intermediate values.

[0042] The processing unit 30 divides the corrected three-dimensional map P into a predetermined number of segments based on the cross-section of the blood vessel to create segmented images (step S109), and determines the lumen boundary and the blood vessel boundary for each segmented image (step S110). The processing unit 30 calculates data indicating anatomical features from the lumen boundary and the blood vessel boundary (step S111). In step S111, the processing unit 30 calculates the centroid of the area inside the lumen boundary. In step S111, the processing unit 30 also calculates numerical values ​​such as the maximum diameter, minimum diameter, and average diameter inside the lumen boundary. The processing unit 30 further calculates the maximum diameter, minimum diameter, and average diameter of the blood vessel boundary. In step S111, the processing unit 30 may calculate other data that can identify the structure of the entire blood vessel.

[0043] The processing unit 30 stores the segmented images created in step S109 and the anatomical feature data calculated in step S111 in the storage unit 31, associating them with their positions on the long axis of the segmented images (step S112).

[0044] The processing unit 30 determines whether the scanning of the catheter 1 by the imaging device 11 has been completed (step S113). If it is determined that the scanning has not been completed (S113: NO), the processing unit 30 returns to step S101 and generates the next tomographic image I1.

[0045] If it is determined that the scan is complete (S113: YES), the processing unit 30 uses the data distribution showing the anatomical features of the entire longitudinal axis of the scanned blood vessel to correct each of the segmented images created for the scan range (step S114). In step S114, the processing unit 30 corrects the image so that the centroid of the lumen range in the segmented image is aligned to the center of the image, and expands the blood vessel boundary and lumen boundary to match their respective maximum diameters (peaks). The processing unit 30 may also correct the image so that each segmented image is aligned at the center of the circle that fits the lumen range in the segmented image, and may expand or contract the blood vessel boundary and lumen boundary to match the average diameter.

[0046] The processing unit 30 creates a three-dimensional image by superimposing the corrected segmented images along the long axis of the scanned blood vessel and stores it in the storage unit 31 (step S115). In step S115, the processing unit 30 smoothly connects the lumen boundaries and blood vessel boundaries of the segmented images to form the lumen boundary surface and the blood vessel boundary surface, thereby creating a three-dimensional image.

[0047] The processing unit 30 outputs the created three-dimensional image to be displayed on the display device 4 (step S116), and then terminates the process. In step S116, the processing unit 30 may display both a graph showing the distribution of data indicating anatomical features along the entire long axis of the scanned blood vessel, and a tomographic image I1 corresponding to any selectable location along the long axis.

[0048] In the flowcharts of Figures 6 and 7, in step 105, it is determined whether or not a three-dimensional image can be created, and after collecting a certain number of tomographic images I1 (frames), they are input into the trained model 3M in step S107. If all the tomographic images I1 from all scans are superimposed and input into the trained model 3M after the scan is completed, the calculation will take a relatively long time. Therefore, as mentioned above, for example, 32 frames are input into the trained model 3M as a three-dimensional image. The number of frames should be determined by the computational resources and the upper limit of the time required for processing. Figure 8 is a diagram showing how to create a three-dimensional image to be input into the trained model 3M. In Figure 8, tomographic images I1 are superimposed in the longitudinal direction, and each tomographic image I1 is represented with thickness, with each rectangle corresponding to a tomographic image I1. In Figure 8, the tomographic images I1 are shown continuously in the longitudinal direction, with the left side being the proximal side and the right side being the distal side. As shown in Figure 8, the processing unit 30 creates a three-dimensional image by stacking multiple consecutive frames and inputs it to the trained model 3M.

[0049] The trained model 3M is trained with a fixed voxel size (height × width × length = number of voxels) for the input image. As shown in Figure 8, the three-dimensional image input in step S107 may have overlapping frames (distal and proximal), and the processing unit 30 may, for example, overlay the tomographic images I1 in overlapping manner for several frames at a time and input them to the trained model 3M. When creating the three-dimensional image to be input at the end of the scan, if the number of frames is less than the fixed number, it is preferable to collect past (distal) tomographic images I1 from the last (most recent) tomographic image I1 to make up the fixed number of frames and create the three-dimensional image.

[0050] The processing procedure shown in the flowcharts of Figures 6 and 7 will be explained with specific examples. Figure 9 is a schematic diagram of an image obtained by superimposing the pre-correction segmented images. The three-dimensional map P obtained by superimposing the tomographic image I1 as is and inputting it into the trained model 3M results in the lumen boundary surface F1 or the vascular boundary surface F2, which have a wavy shape or are axially misaligned, as shown in Figure 9. This is because the blood vessel pulsates while scanning as the catheter 1 moves from distal to proximal, and because the catheter 1 does not always pass through the true center of the blood vessel. However, neither the actual lumen boundary nor the vascular boundary surface is a wavy surface as shown in Figure 9. Therefore, the processing unit 30 of the image processing device 3 performs correction using the lumen boundary and vascular boundary in the segmented images obtained by segmenting the three-dimensional map P.

[0051] Figure 10 is a schematic diagram of the correction of segmented images of a three-dimensional map and the corrected three-dimensional image. In Figure 10, the processing unit 30 enlarges each of the determined blood vessel boundaries for the segmented images, indicated by rectangles, so that the maximum diameter (pulsation peak) is aligned, as indicated by arrows, and aligns the axes at the centroid, as indicated by downward arrows. The luminal boundaries are not shown. As shown in Figure 10, the processing unit 30 connects the blood vessel boundaries in each corrected segmented image. As a result, compared to the image shown in Figure 9, the undulation is suppressed, and a more realistic image of the blood vessels in three dimensions can be provided, as shown in the corrected three-dimensional image in Figure 10.

[0052] Figure 11 shows an example screen of the display device 4. The screen 400 in Figure 11 displays the corrected three-dimensional image of the blood vessel shown in Figure 10, as well as a cursor (reference numeral 401 in Figure 11), which is a movable object on the long axis, and a tomographic image I1 at the position selected by the cursor 401. The screen 400 in Figure 11 also shows a graph 402 that shows the distribution of data indicating anatomical features on the long axis. Graph 402 is, for example, the distribution of the average lumen diameter on the long axis. As shown in Figure 11, it is preferable that graph 402 is also displayed three-dimensionally in accordance with the long axis direction of the three-dimensionally displayed blood vessel. The screen 400 should output whether or not there are side branches analyzed from the tomographic image I1, as shown in Figure 11. The presence or absence of side branches can be detected by the processing unit 30 based on the maximum and minimum diameters in the data indicating anatomical features, as well as the shape of the blood vessel boundary.

[0053] With the image diagnostic device 100 disclosed herein, a screen 400 as shown in Figure 11 is displayed on the display device 4, and by providing a more accurate three-dimensional image of blood vessels within the screen 400, it becomes easier for medical professionals to grasp the characteristics.

[0054] (Second Embodiment) In the second embodiment, a three-dimensional image is created by arranging signals obtained from the imaging device 11, rather than the tomographic image I1, in a spiral pattern, similar to the scanning sequence. The configuration of the diagnostic imaging apparatus 100 in the second embodiment is the same as that of the first embodiment, except for the detailed processing procedures described below. Therefore, the same reference numerals are used for common components, and detailed explanations are omitted.

[0055] Figure 12 is a schematic diagram of a three-dimensional image created by the signal from the imaging device 11. The imaging device 11 moves while rotating in a spiral shape as shown in Figure 2, and acquires signals by detecting the reflection of ultrasonic waves. Therefore, when the detected signals are plotted against the position on the long axis and the scanning angle, a three-dimensional image is obtained as shown in Figure 12.

[0056] The trained model 3M of the second embodiment is trained to output a three-dimensional map P as shown in Figure 4 when a three-dimensional image with signals distributed in a spiral shape as shown in Figure 12 is input.

[0057] Figure 13 is a flowchart showing an example of the information processing procedure in the second embodiment. When a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing device 3 in the second embodiment starts the following processing.

[0058] The processing unit 30 acquires a predetermined amount of signal (data) from the imaging device 11 of the catheter 1 (for example, 3600 degrees, equivalent to 10 rotations) (step S201), and stores the signal data in association with the position on the long axis and the scanning angle of the imaging device 11 (step S202). The processing unit 30 creates a three-dimensional image (see Figure 12) by arranging the radially reflected signals (data) acquired from the imaging device 11 in a spiral pattern according to the rotation angle of the imaging device 11 (step S203).

[0059] In step S203, the processing unit 30 may not only arrange the reflected signals in a spiral pattern, but may also divide the data obtained as a rectangular image before transforming the tomographic image I1 into polar coordinates, and then create a three-dimensional image by assigning pixel values ​​to voxels.

[0060] In step S203, the processing unit 30 may perform interpolation for missing signal data (areas that are not scanned when scanned in a spiral pattern). Specifically, in the process of assigning signal data to voxels, for voxels to which no data is assigned, the values ​​of the surrounding voxels may be extended, or an intermediate value of the surrounding voxel values ​​may be assigned.

[0061] The processing unit 30 inputs the three-dimensional image created in step S203 to the trained model 3M (step S204). Therefore, in the second embodiment, it is desirable that the trained model 3M learns the segmentation of an image in which signal data is arranged in a spiral pattern as an input image, rather than learning the three-dimensional image created by superimposing the tomographic image I1 as the input image.

[0062] The processing unit 30 removes noise from the three-dimensional map P output from the trained model 3M and corrects the surface (contour) that indicates the boundary (step S205). In step S205, the processing unit 30 removes points (noise) that are identified as being on the boundary and are outliers from the boundary surface. Points where data is missing may be corrected by connecting other points with spline curves or the like, or by connecting them with intermediate values.

[0063] The processing unit 30 divides the corrected three-dimensional map P into a predetermined number of segments based on the cross-section of the blood vessel (luminal organ) to create segmented images (step S206), and determines the lumen boundary and blood vessel boundary for each segmented image (step S207). The processing unit 30 calculates data showing anatomical features from the lumen boundary and blood vessel boundary (step S208). In step S208, the processing unit 30 calculates the centroid of the area inside the lumen boundary. In step S208, the processing unit 30 also calculates numerical values ​​such as the maximum diameter, minimum diameter, and average diameter inside the lumen boundary. The processing unit 30 further calculates the maximum diameter, minimum diameter, and average diameter of the blood vessel boundary. In step S208, the processing unit 30 calculates data that can identify the structure of the entire blood vessel.

[0064] The processing unit 30 stores the segmented images created in step S206 and the anatomical feature data calculated in step S208 in the storage unit 31, associating them with their positions on the long axis of the segmented images (step S209).

[0065] The processing unit 30 determines whether the scanning of the catheter 1 by the imaging device 11 has been completed (step S210). If it is determined that the scanning has not been completed (S210: NO), the processing unit 30 returns to step S201 and generates the next three-dimensional image.

[0066] If it is determined that scanning is complete (S210: YES), the processing unit 30 uses the distribution of data showing the anatomical features along the entire longitudinal axis of the scanned blood vessel to correct each of the segmented images created for the scanning range (step S211). In step S211, the processing unit 30 corrects the image so that the centroid of the lumen range in the segmented image is aligned with the center of the image, and expands the blood vessel boundary and lumen boundary to match their respective maximum diameters (peaks) (similar to Figures 9 and 10 of the first embodiment). The processing unit 30 may also correct the image so that each segmented image is aligned at the center of the circle that fits the lumen range in the segmented image, and may expand or contract the blood vessel boundary and lumen boundary to match the average diameter.

[0067] The processing unit 30 creates a three-dimensional image by superimposing the corrected segmented images along the long axis of the scanned blood vessel and stores it in the storage unit 31 (step S212). The processing unit 30 outputs the created three-dimensional image to the display device 4 for display (step S213) and terminates the process. In step S213, the processing unit 30 may display both a graph showing the distribution of data indicating anatomical features along the entire long axis of the scanned blood vessel, and a tomographic image I1 corresponding to any selectable location along the long axis, as shown in Figure 11.

[0068] In the second embodiment, a three-dimensional image is created in which the signal from the imaging device 11 is distributed faithfully to the movement of the imaging device 11, making it possible to obtain anatomical features based on a more realistic three-dimensional image. In the second embodiment, further corrections are applied to the segmented images according to the pulsation and movement of the catheter 1, providing an even more realistic image that reproduces the blood vessels in three dimensions.

[0069] (Third embodiment) In the third embodiment, segmentation of the object region is performed two-dimensionally on each tomographic image I1, and only tomographic images I1 in which the lumen diameter or blood vessel diameter is the peak (pulsation peak) are extracted. The extracted tomographic images are then superimposed to create a three-dimensional image. The created three-dimensional image is then input into the trained model.

[0070] Figure 14 is a block diagram showing the configuration of the image processing apparatus 3 of the third embodiment. In the third embodiment, the image processing apparatus 3 stores and uses a pre-trained first model 31M for three-dimensional images and a pre-trained second model 32M for two-dimensional images. The first model 31M is the same as the pre-trained model 3M described in the first or second embodiment. Except for the use of two models and the processing procedure shown below based thereon, the configuration of the image diagnostic apparatus 100 of the third embodiment is the same as that of the image diagnostic apparatus 100 in the first embodiment, so the same reference numerals are used for common components and detailed descriptions are omitted.

[0071] Both the first model 31M and the second model 32M may be stored in the storage unit 31, which is a non-temporary storage medium, as shown in Figure 14, or the processing unit 30 may read the first model 91M and the second model 92M stored in the non-temporary storage medium 9 and copy them to the storage unit 31. The processing unit 30 may also download and store the first model 31M and the second model 32M from a server device (not shown).

[0072] Figure 15 is a schematic diagram of the second model 32M. The second model 32M is a model trained to output an image showing the region of one or more objects in a tomographic image I1 when it is input. The second model 32M is a model that performs semantic segmentation, similar to the first model 31M, i.e., the trained model 3M in the first embodiment, and uses, for example, U-net.

[0073] The second model 32M outputs a tagged image IS and accuracy when a tomographic image I1 is input. The tagged image IS is created by tagging the pixels at the location of the blood vessel with different pixel values, representing the luminal area of ​​the blood vessel, the membrane area including the tunica media of the blood vessel, the area showing the guidewire W and its reflections, and the area corresponding to the catheter 1.

[0074] The processing using the second model 32M will be explained with reference to a flowchart. Figures 16 and 17 are flowcharts of the information processing procedure in the third embodiment. In the third embodiment, the processing unit 30 of the image processing device 3 starts the following processing when a signal is output from the imaging device 11 of the catheter 1.

[0075] Each time the processing unit 30 acquires a predetermined amount (for example, 360 degrees) of signal (data) from the imaging device 11 of the catheter 1 (step S301), it transforms (inversely transforms) the image of radial signals arranged in a rectangle into polar coordinates to generate a tomographic image I1 (step S302). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time on the screen displayed on the display device 4 (step S303). The processing unit 30 stores the signal data acquired in step S301 and the tomographic image I1 in the storage unit 31, associating them with their positions on the long axis of the blood vessel (step S304).

[0076] The processing unit 30 inputs the tomographic image I1 generated in step S302 to the second model 32M (step S305). Based on the tagged image IS obtained from the second model 32M, the processing unit 30 calculates the data for the lumen boundary and the vascular boundary in the tomographic image I1 (step S306). In step S306, the processing unit 30 calculates the lumen boundary as the contour (edge) of the lumen area and the vascular boundary as the outer contour of the vascular area, from the lumen area and membrane area including the tunica media of the vessel output from the second model 32M.

[0077] The processing unit 30 calculates data indicating anatomical features based on the data of the lumen boundary and vascular boundary obtained in step S306 (step S307). In step S307, the processing unit 30 calculates the maximum diameter, minimum diameter, and average diameter of the medial range of the lumen boundary, or any or all of them. The processing unit 30 further calculates the maximum diameter, minimum diameter, and average diameter of the vascular boundary, or any or all of them.

[0078] The processing unit 30 stores the data showing the anatomical features calculated in step S307 in the memory unit 31, associating it with the position on the long axis of the blood vessel (step S308).

[0079] The processing unit 30 determines whether the scanning of the catheter 1 by the imaging device 11 has been completed (step S309). If it is determined that the scanning has not been completed (S309: NO), the processing unit 30 returns to step S301 and generates the next tomographic image I1.

[0080] If it is determined that the scan is complete (S309: YES), the processing unit 30 uses the distribution of data showing the anatomical features along the entire longitudinal axis of the scanned blood vessel to determine the position in the longitudinal axis corresponding to the peak of the average lumen diameter (average diameter of the lumen) (step S310).

[0081] The processing unit 30 extracts the tomographic image I1 corresponding to the determined position from the storage unit 31 (step S311). The processing unit 30 creates a three-dimensional image by superimposing the extracted tomographic images I1 in line with the position along the long axis (step S312). Step S312 corrects the three-dimensional image in a preliminary stage before the first model 31M.

[0082] In step S312, the processing unit 30 creates a three-dimensional image by reflecting the position in the long axis direction as the depth. In step S312, the processing unit 30 may create a three-dimensional image covering the entire scanning range, or it may create three-dimensional images in predetermined lengths (for example, 10 mm, 20 mm, etc.).

[0083] In step S312, the processing unit 30 may arrange the signal data from the imaging device 11 corresponding to the extracted tomographic image I1 in a spiral pattern, as shown in the second embodiment (Figure 12), to create a three-dimensional image.

[0084] The processing unit 30 inputs the created three-dimensional image to the first model 31M (step S313). The processing unit 30 converts the three-dimensional map P output from the first model 31M into a three-dimensional image and stores it in the storage unit 31 (step S314).

[0085] The processing unit 30 outputs the three-dimensional image obtained in step S314 to be displayed on the display device 4 (step S315), and then terminates the process. In step S316, the processing unit 30 may display both a graph showing the distribution of data indicating anatomical features along the entire long axis of the scanned blood vessel, and a tomographic image I1 corresponding to any selectable location along the long axis, as shown in Figure 11.

[0086] Figure 18 is a schematic diagram of the processing in the third embodiment. A graph showing the distribution of the average lumen diameter is shown in correspondence. As shown in the graph, the distribution of the average lumen diameter has peaks corresponding to pulsation. The processing unit 30 identifies the position on the long axis corresponding to the peak from the distribution and extracts a tomographic image I1 corresponding to the identified position (S311). The processing unit 30 creates a three-dimensional image by superimposing the extracted tomographic images I1 (S312). In other words, the processing in the third embodiment is equivalent to inputting the corrected three-dimensional image into the first model 31M (trained model 3M).

[0087] The corrected three-dimensional image is missing tomographic images I1 corresponding to positions other than the peaks. Therefore, the processing unit 30 may interpolate between the tomographic images I1 corresponding to the peaks by connecting the boundary surfaces (boundaries) of the lumen boundary or vascular boundary. Alternatively, the processing unit 30 may superimpose the tomographic images I1, which are missing except for the peaks, and input them to the first model 31M. The input three-dimensional image connects periodic peaks, so it is almost equivalent to an image scaled down in the long axis direction. Therefore, the processing unit 30 may stretch the three-dimensional map P output from the first model 31M in the long axis direction to obtain a three-dimensional image.

[0088] In the third embodiment, the processing unit 30 analyzed the distribution of data showing anatomical features as described above and determined the position corresponding to the peak. However, there may be other methods for determining the position corresponding to the peak. For example, the processing unit 30 may input signals from the imaging device 11 of the catheter 1 and signals from an instrument that measures pulsation via the input / output I / F 32, and determine the position corresponding to the timing of the pulsation as the position corresponding to the peak. In this case as well, the image processing device 3 can accurately create a three-dimensional image with suppressed wavering.

[0089] In the third embodiment, the obtained three-dimensional image is displayed on the screen 400 as shown in Figure 11. It is also possible to obtain a three-dimensional image with separated areas by removing surface undulations and noise from the corrected three-dimensional image before inputting it into the first model 31M which performs three-dimensional segmentation.

[0090] In the third embodiment, the processing unit 30 calculated the data of the lumen boundary and vascular boundary in the tomographic image I1 using the second model 32M. However, the processing unit 30 may calculate the data of the lumen boundary and vascular boundary from segmented images obtained by dividing a three-dimensional map P obtained by inputting a three-dimensional image into the first model 31M, without using the second model 32M for the two-dimensional image. In this case, the processing unit 30 may calculate the average lumen diameter, maximum diameter, etc., at the position corresponding to each segmented image from the lumen boundary and vascular boundary, identify the peak, and determine the position of the long axis corresponding to the peak.

[0091] (Fourth Embodiment) In the fourth embodiment, the image processing device 3 groups the tomographic images I1 into predetermined units, inputs each group into a trained model 3M, and extends the resulting three-dimensional map P to the scanning range to obtain a three-dimensional image for each group. The image processing device 3 estimates a three-dimensional image by smoothly connecting the obtained three-dimensional maps P for each group, such as by taking the average and centroid.

[0092] The configuration of the image diagnostic apparatus 100 in the fourth embodiment is the same as that of the image diagnostic apparatus 100 in the first embodiment, except for the processing procedure shown below. Therefore, the same reference numerals are used for common components, and detailed explanations are omitted.

[0093] Figures 19 and 20 are flowcharts showing the information processing procedure in the fourth embodiment. In the fourth embodiment, the processing unit 30 of the image processing apparatus 3 starts the following processing when a signal is output from the imaging device 11 of the catheter 1.

[0094] The processing unit 30 initializes the group number of the tomographic image I1 (GN=1) (step S401). The group number is set to the maximum number of groups, and is incremented by one each time a tomographic image I1 is generated and stored. The number of groups is preferably determined based on the pulsation period and the movement speed and rotation period of the catheter 1. It is preferable that images with similar phases belong to the same group according to the pulsation period.

[0095] Each time the processing unit 30 acquires a predetermined amount of signal (for example, 360 degrees) from the imaging device 11 of the catheter 1 (step S402), it transforms the image of the radial signals arranged in a rectangle into polar coordinates (inverse transformation) to generate a tomographic image I1 (step S403). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time on the screen displayed on the display device 4 (step S404).

[0096] The processing unit 30 assigns a group number to the generated tomographic image I1 (step S405). The processing unit 30 stores the signal data acquired in step S402 and the tomographic image I1 with the assigned group number in the storage unit 31, associating them with the position on the long axis of the blood vessel (step S406).

[0097] The processing unit 30 determines whether the group number is the maximum value (step S407). If it determines that it is not the maximum value (S407: NO), it increments the group number (step S408). If it determines that it is the maximum value (S407: YES), the processing unit 30 initializes the group number (step S409).

[0098] The processing unit 30 determines whether or not a three-dimensional image can be created (step S410). In step S410, the processing unit 30 determines, for example, whether or not a predetermined number of tomographic images I1 assigned the same group number have been reached. Alternatively, the processing unit 30 may determine whether or not the length of scanning has reached a predetermined length, or it may determine whether or not scanning has been completed in step S410.

[0099] If it is determined that a three-dimensional image cannot be created (S410: NO), the processing unit 30 returns to step S402 and continues processing until a three-dimensional image can be created.

[0100] If it is determined that a three-dimensional image can be created (S410: YES), the processing unit 30 creates a three-dimensional image for each group of tomographic images I1 assigned the same group number, overlapping them in the longitudinal direction (step S411). In step S411, the processing unit 30 creates the image by overlapping a predetermined number of tomographic images I1 (e.g., 5, 10, etc.). The processing unit 30 may also perform step S411 by overlapping all tomographic images I1 across the scanning range.

[0101] The processing unit 30 inputs the three-dimensional images created for the group into the trained model 3M (step S412). The processing unit 30 removes noise from the three-dimensional map P output from the trained model 3M and corrects the surface (contour) that indicates the boundary (step S413). In step S413, the processing unit 30 removes points on the boundary that are outliers from the boundary surface (noise), and for areas where data is missing, it may correct those areas by connecting other points with a spline curve or the like, or by connecting them with intermediate values. In the fourth embodiment, step S413 may be omitted.

[0102] The processing unit 30 divides the obtained three-dimensional map P into a predetermined number of sections based on the cross-section of the blood vessel (luminal organ) to create segmented images for the target group (step S414), and determines the lumen boundary and blood vessel boundary for each segmented image (step S415). The processing unit 30 calculates data indicating anatomical features from the lumen boundary and blood vessel boundary (step S416). In step S416, the processing unit 30 calculates the centroid of the area inside the lumen boundary. In step S416, the processing unit 30 also calculates numerical values ​​such as the maximum diameter, minimum diameter, and average diameter inside the lumen boundary. The processing unit 30 further calculates the maximum diameter, minimum diameter, and average diameter of the blood vessel boundary. In step S416, the processing unit 30 may calculate other data that can identify the structure of the entire blood vessel.

[0103] The processing unit 30 stores the segmented images created in step S414 and the anatomical feature data calculated in step S416 in the storage unit 31 in groups, associating them with their positions on the long axis of the segmented images (step S417).

[0104] The processing unit 30 determines whether the scanning of the catheter 1 by the imaging device 11 has been completed (step S418). If it is determined that the scanning has not been completed (S418: NO), the processing unit 30 returns to step S402 and generates the next tomographic image I1.

[0105] If it is determined that scanning is complete (S418: YES), the processing unit 30 creates a three-dimensional image by superimposing the segmented images created for each group within the scanning range, and stores them in the storage unit 31 for each group (step S419). In step S419, the processing unit 30 connects the lumen boundary and vascular boundary based on the anatomical feature data (e.g., average lumen diameter) derived for each segmented image to form the lumen boundary surface and vascular boundary surface, and creates a three-dimensional image for each group.

[0106] The processing unit 30 stretches the three-dimensional images of each group to match the length of the long axis of the scanning range, places the three-dimensional images of each group in the same coordinate space, and creates a single three-dimensional image based on them (step S420). In step S420, the processing unit 30 calculates a surface that is the average surface (may be a surface connecting the central points) of the luminal and vascular boundaries that intersect or follow parallel paths with the three-dimensional images placed in the same coordinate space, and creates a surface. In other words, in step S420, the processing unit 30 uses the three-dimensional images of each group as multiple draft surfaces, and creates a three-dimensional image by drawing a single representative surface based on these drafts.

[0107] The processing unit 30 stores the three-dimensional image created in step S420 in the storage unit 31 (step S421), outputs it to be displayed on the display device 4 (step S422), and terminates the process. In step S421, the processing unit 30 may display both a graph showing the distribution of data indicating anatomical features along the entire long axis of the scanned blood vessel, and a tomographic image I1 corresponding to any selectable location along the long axis, as shown in Figure 11.

[0108] Figure 21 is a schematic diagram of the processing in the fourth embodiment. In Figure 21, tomographic images I1 are superimposed in the longitudinal direction, with each tomographic image I1 given thickness, and each rectangle corresponds to a tomographic image I1. In Figure 21A, the tomographic images I1 are shown continuously in the longitudinal direction, with the left side being the proximal side and the right side being the distal side. Each tomographic image I1 can be distinguished by different hatching for each group. In the example of superimposed tomographic images I1 shown in Figure 21, there are, for example, 5 groups. This number of groups should be matched to the peak period of the pulsation. The pulsation period (seconds) can be measured in advance, and the number of tomographic images I1 that can be generated by the signal from the imaging device 11 during the period (number of frames) can be determined.

[0109] In step S411 of the flowcharts in Figures 19 and 20, the processing unit 30 collects tomographic images I1 assigned a GN of "1" for the first group and stacks them along the long axis to create a three-dimensional image, as shown in Figure 21B. For the second group, tomographic images I1 assigned a GN of "2" are similarly stacked to create a three-dimensional image.

[0110] The three-dimensional images created by each group are input into the trained model 3M, and a three-dimensional map P is obtained for each group, as shown in Figure 21C. Figure 21C schematically shows the cross-section of the boundary line of the area identified by each group (in this case, only the vascular boundary). The three-dimensional map P, or the three-dimensional image obtained by stitching together segmented images (data showing anatomical features), in the example of Figure 21, is an image with the sampling reduced to 1 / 5, and is an image compressed to 1 / 5 in the longitudinal direction.

[0111] Therefore, the processing unit 30 stretches the three-dimensional map P or the three-dimensional image formed by connecting the segmented images by a factor of 5 in the long axis direction (S420). Figure 21D schematically shows the three-dimensional images of each group stretched by a factor of 5 and placed in the same coordinate space. In Figure 21D, the cross-sections of the boundaries of the areas identified in the three-dimensional map P (in this case, only the blood vessel boundaries) are shown with different line types for each group. As shown in Figure 21D, the boundaries in the three-dimensional images created for each group intersect in some places or follow parallel paths.

[0112] The processing unit 20 creates a three-dimensional image by determining a single boundary surface based on the average, center, centroid, etc., of the boundary lines (boundary surfaces) for each group. Figure 21E shows an example where the determined boundary surface (in this case, a vascular boundary) is shown with a thick line. As shown in Figure 21E, the processing unit 30 creates a three-dimensional image by determining a single representative surface using the boundary surfaces for each group shown in Figure 21D as a draft. This makes it possible for the diagnostic imaging device 100 to three-dimensionally reproduce the blood vessels in order to estimate the true structure of the actual blood vessels from the observed values ​​by the imaging device 11.

[0113] The display example in the fourth embodiment is similar to the example of screen 400 in Figure 11 shown in the first embodiment, so its illustration and detailed description are omitted.

[0114] In the first to fourth embodiments, the image processing device 3 connected to the catheter 1 was described as generating a tomographic image I1 in near real-time based on signals from the imaging device 11, while simultaneously displaying data showing anatomical features on the display device 4. However, the processing by the image processing device 3 described above may be performed retrospectively based on signal data obtained separately from the imaging device 11. In other words, the image processing device 3 is not necessarily directly connected to the imaging device 11 of the catheter 1; it is sufficient that it can acquire signals from the imaging device 11. The image processing device 3 may be a device capable of reading a storage device that stores signals from the imaging device 11 via a network, such as a server device. That is, for example, the processing procedure of steps S101-S104 shown in the flowcharts of Figures 6 and 7 may be performed on an existing processing device, and the processing of steps S105-S116 may be performed on the image processing device 3 connected to the processing device, and the results may be displayed on the display device 4 via the processing device.

[0115] In the first to fourth embodiments, medical images were described using images obtained by IVUS on the coronary arteries as an example. However, the application is not limited to this, and OCT / OFDI, etc., may also be used, and the tubular organ is not limited to blood vessels.

[0116] The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are included. [Explanation of symbols]

[0117] 1 Catheter 11 Imaging devices 3. Image Processing Device (Information Processing Device) 30 Processing Unit 31 Storage section 3P Computer Program 3M Pre-trained Model (Three-Dimensional Classification Model) 31M Model 1 (Three-Dimensional Discrimination Model) 4 Display device 400 screens 401 Cursor I1 Tomographic image P Three-dimensional map

Claims

1. a computer that acquires signals detected by an imaging device provided in a catheter that is inserted into a hollow organ and moves in the longitudinal direction while rotating around the longitudinal direction of the hollow organ; a three-dimensional identification model that is trained to output three-dimensional data indicating the regions of one or more objects appearing in a three-dimensional image of the hollow organ based on the signal; a three-dimensional image based on a signal obtained from the imaging device is input to the three-dimensional identification model; A three-dimensional image showing the structure of the hollow organ is created from the three-dimensional data output from the three-dimensional identification model. A computer program that executes a process.

2. The computer corrects the three-dimensional image to be input to the three-dimensional identification model or the three-dimensional data output from the three-dimensional identification model. The computer program product of claim 1 , which causes a process to be executed.

3. The computer, Dividing the three-dimensional data output from the three-dimensional identification model into cross sections; correcting the boundary of the region in each of the divided images by expanding or contracting it based on the distribution of the maximum diameter, minimum diameter, or average diameter of the hollow organ in the length direction; The boundaries of the corrected regions are connected in the longitudinal direction to generate a three-dimensional image showing the structure of the hollow organ.

3. A computer program according to claim 1 or 2, which causes a process to be executed.

4. The computer, calculating a distribution of data indicating anatomical features of the hollow organ in the longitudinal direction from a plurality of cross-sectional images of the hollow organ based on the signals or a plurality of divided images obtained by dividing the three-dimensional data into cross sections; determining a plurality of peaks in the data in the distribution; extracting a tomographic image or a segmented image corresponding to a position in the length direction of each of the plurality of peaks; The extracted tomographic images or segmented images are joined together to create a three-dimensional image showing the structure of the hollow organ.

3. A computer program according to claim 1 or 2, which causes a process to be executed.

5. The peak is determined according to the timing of the pulsation of the hollow organ or the moving speed of the catheter.

5. A computer program according to claim 4.

6. The computer, dividing the signals detected by the imaging device or the plurality of tomographic images of the hollow organ generated based on the signals into groups; A three-dimensional image in which the separated signals are arranged three-dimensionally for each group, or a three-dimensional image in which the separated tomographic images are superimposed in the length direction for each group, is input to the three-dimensional identification model; The three-dimensional data for each group output from the three-dimensional identification model is expanded and arranged in the same coordinate space, and a three-dimensional image showing the structure of the hollow organ is created from the data.

3. A computer program according to claim 1 or 2, for executing a process.

7. The number of groups is determined according to the timing of the pulsation of the hollow organ or the moving speed of the catheter.

7. A computer program according to claim 6.

8. The timing of the pulse is determined based on measurements taken by a heart rate monitor as the catheter moves.

8. A computer program according to claim 5 or 7.

9. The computer, A three-dimensional image obtained by superimposing the tomographic images generated based on the signals in the longitudinal direction is input to the three-dimensional identification model without correction or after correction.

9. A computer program product according to claim 1, which causes a process to be executed.

10. The computer, A three-dimensional image in which signals detected by the imaging device are arranged in a spiral in accordance with the movement of the imaging device that scans in a radial direction in accordance with the rotation of the catheter is input to the three-dimensional identification model without or after correction.

10. A computer program product according to claim 1, which causes a process to be executed.

11. a computer that acquires signals detected by an imaging device provided in a catheter that is inserted into a hollow organ and moves in a longitudinal direction while rotating about an axis in the longitudinal direction of the hollow organ; a three-dimensional identification model that is trained to output three-dimensional data indicating the regions of one or more objects appearing in a three-dimensional image of the hollow organ based on the signal; a three-dimensional image based on a signal obtained from the imaging device is input to the three-dimensional identification model; A three-dimensional image showing the structure of the hollow organ is created from the three-dimensional data output from the three-dimensional identification model. Information processing methods.

12. 1. An information processing device for acquiring signals detected by an imaging device provided in a catheter that is inserted into a hollow organ and moves in the longitudinal direction of the hollow organ while rotating about an axis in the longitudinal direction of the hollow organ, a storage unit that stores a three-dimensional identification model that is trained to output three-dimensional data indicating the regions of one or more objects appearing in a three-dimensional image of the hollow organ based on the signal; and a processing unit that performs image processing based on the signal; Equipped with The processing unit a three-dimensional image based on a signal obtained from the imaging device is input to the three-dimensional identification model; A three-dimensional image showing the structure of the hollow organ is created from the three-dimensional data output from the three-dimensional identification model. Information processing device.

Images