Information processing device, information processing method, and program

Abstract

The purpose of the present invention is to provide an information processing device and the like for supporting the understanding of images acquired by an image acquisition catheter.　An information processing device is provided with a classified image data acquisition unit that acquires a plurality of items of classified image data classified into a plurality of regions including a first intraluminal region (511) where an image acquisition catheter (40) for three-dimensional scanning is inserted and a second intraluminal region (512) where the image acquisition catheter (40) is not inserted, a merging determination unit that determines whether or not the second intraluminal region (512) in a first catheter image is to be merged with the first intraluminal region (511) in a second catheter image acquired at a different axial position, and an image output unit that outputs a region image including the first intraluminal region (511) on the basis of the plurality of items of classified image data, the image output unit outputting only the second intraluminal region (512) determined to have been merged by the merging determination unit within the second intraluminal region (512), together with the first intraluminal region (511), as the region image.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] BACKGROUND ART A catheter system is used in which an image acquisition catheter is inserted into a hollow organ such as a blood vessel to acquire images (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 164071 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a site with a complex structure where tubular organs have confluences and branches, it may be difficult for the user to quickly understand the images acquired by the imaging catheter.

[0005] In one aspect, an object of the present invention is to provide an information processing device or the like that supports the understanding of images acquired by an image acquisition catheter. [Means for solving the problem]

[0006] The information processing device includes a classified image data acquisition unit that acquires a plurality of classified image data, the classified image data being generated based on a plurality of catheter images acquired using an image acquisition catheter that acquires images while moving a scanning plane axially, and classified into a plurality of regions including a first lumen region into which the image acquisition catheter is inserted and a second lumen region into which the image acquisition catheter is not inserted; a junction determination unit that determines whether the second lumen region in a first catheter image of the plurality of catheter images merges with the first lumen region in a second catheter image acquired at an axial position different from that of the first catheter image; and an image output unit that outputs a regional image including the first lumen region based on the plurality of classified image data, and the image output unit outputs only the second lumen region in the first catheter image that the junction determination unit determines to merge with the first lumen region, together with the first lumen region, as the regional image. [Effects of the Invention]

[0007] In one aspect, it is possible to provide an information processing device or the like that supports the understanding of images acquired by an image acquisition catheter. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating the configuration of a catheter system. [Figure 2] FIG. 2 is an explanatory diagram illustrating the configuration of a classification model. [Figure 3] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 4] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 5] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 6] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 7] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 8] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 9] FIG. 10 is an explanatory diagram illustrating the operation of the catheter system. [Figure 10] FIG. 2 is an explanatory diagram illustrating a record layout of an image DB. [Figure 11] 10 is a flowchart illustrating the flow of processing of a program. [Figure 12] 10 is a flowchart illustrating the flow of processing in a subroutine for changing past classification. [Figure 13] FIG. 10 is an explanatory diagram illustrating a display example of a three-dimensional image. [Figure 14] FIG. 10 is an explanatory diagram illustrating a display example of a three-dimensional image. [Figure 15] FIG. 10 is an explanatory diagram illustrating a display example of a three-dimensional image. [Figure 16] FIG. 11 is an explanatory diagram illustrating a record layout of an image DB according to the second embodiment. [Figure 17] 10 is a flowchart illustrating the flow of processing of a program according to the second embodiment. [Figure 18] 10 is a flowchart illustrating a process flow of a subroutine for generating past meeting area data. [Figure 19] FIG. 10 is an explanatory diagram illustrating an example of a screen according to the second embodiment. [Figure 20] FIG. 10 is an explanatory diagram illustrating an example of a screen according to the second embodiment. [Figure 21] FIG. 10 is an explanatory diagram illustrating an example of a screen according to the second embodiment. [Figure 22] FIG. 10 is an explanatory diagram illustrating the configuration of a catheter system according to a third embodiment. [Figure 23] FIG. 10 is a functional block diagram of an information processing device according to a fourth embodiment. [Figure 24] FIG. 13 is a functional block diagram of an information processing device according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Embodiment 1] 1 is an explanatory diagram illustrating the configuration of a catheter system 10. The catheter system 10 includes a catheter 40 for acquiring three-dimensional images, an MDU (Motor Driving Unit) 33, and an information processing device 20.

[0010] The three-dimensional image acquisition catheter 40 includes a long sheath 41, and a sensor 42 and a shaft 43 disposed inside the sheath 41. The sensor 42 is attached to the end of the shaft 43. The three-dimensional image acquisition catheter 40 is connected to the information processing device 20 via the MDU 33.

[0011] The sensor 42 is, for example, an ultrasonic transducer that transmits and receives ultrasonic waves, or a transmitter / receiver for OCT (Optical Coherence Tomography) that emits near-infrared light and receives reflected light. In the following explanation, the three-dimensional image acquisition catheter 40 will be described as an example in which it is an ultrasonic catheter used for performing so-called three-dimensional scanning, which successively generates multiple ultrasonic tomographic images from the inside of a hollow organ.

[0012] The information processing device 20 includes a control unit 21, a main memory device 22, an auxiliary memory device 23, a communication unit 24, a display unit 25, an input unit 26, a catheter control unit 271, and a bus. The control unit 21 is an arithmetic and control device that executes the program of this embodiment. The control unit 21 uses one or more central processing units (CPUs), graphics processing units (GPUs), multi-core CPUs, or the like. The control unit 21 is connected to each hardware unit that constitutes the information processing device 20 via the bus.

[0013] The main memory device 22 is a storage device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc. The main memory device 22 temporarily stores information required during processing performed by the control unit 21 and programs currently being executed by the control unit 21.

[0014] The auxiliary storage device 23 is a storage device such as an SRAM, a flash memory, a hard disk, or a magnetic tape. The auxiliary storage device 23 stores an image database (DB) 61, a classification model 62, programs to be executed by the control unit 21, and various data required for executing the programs. The communication unit 24 is an interface for communicating between the information processing device 20 and a network. The image DB 61 may be stored in an external mass storage device connected to the information processing device 20.

[0015] The display unit 25 is, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) panel. The input unit 26 is, for example, a keyboard and a mouse. The input unit 26 may be stacked on the display unit 25 to form a touch panel. The display unit 25 may be a display device connected to the information processing device 20.

[0016] The MDU 33 simultaneously advances and retreats the sensor 42 and the shaft 43 while rotating them. The catheter control unit 271 generates one catheter image 55 (see FIG. 2) for each rotation of the sensor 42. The generated catheter image 55 is a so-called transverse image centered on the sheath 41 and approximately perpendicular to the sheath 41. In the following description, the generation of the catheter image 55 by the catheter control unit 271 may be expressed as "capturing the catheter image 55."

[0017] By rotating the sensor 42 while pulling or pushing it, the catheter control unit 271 continuously captures multiple catheter images 55 that are approximately perpendicular to the sheath 41. The continuously captured catheter images 55 can be used to construct a three-dimensional image.

[0018] The advancement / retraction operation of the sensor 42 may be an operation of advancing / retracting the sensor 42 and the shaft 43 inside the sheath 41, or an operation of advancing / retracting the sheath 41, the sensor 42, and the shaft 43 as a whole. The advancement / retraction operation may be performed automatically at a predetermined speed by the MDU 33, or may be performed manually by the user. In the following description, the direction in which the sensor 42 advances / retracts, i.e., the longitudinal direction of the sheath 41, may be referred to as the axial direction.

[0019] In the following explanation, an example will be described in which the sensor 42 and shaft 43 are automatically retracted at a constant speed toward the MDU 33 while rotating inside the sheath 41. In the following explanation, a series of scans performed while the sensor 42 is retracted once will be referred to as one three-dimensional scan.

[0020] The three-dimensional image acquisition catheter 40 is not limited to a mechanical scanning type that mechanically rotates and moves back and forth. It may be an electronic radial scanning type three-dimensional image acquisition catheter 40 that uses a sensor 42 in which multiple ultrasonic transducers are arranged in a ring. It may also be a three-dimensional image acquisition catheter 40 that mechanically rotates an electronic linear type sensor 42 in which multiple ultrasonic transducers are arranged in a line.

[0021] The information processing device 20 of this embodiment is a dedicated ultrasound diagnostic device, or a personal computer, tablet, smartphone, or the like having the functionality of an ultrasound diagnostic device. The following explanation will be mainly based on an example in which the control unit 21 performs software processing. The processing described using the flowcharts and the various trained models may each be implemented by dedicated hardware.

[0022] FIG. 2 is an explanatory diagram illustrating the configuration of the classification model 62. The classification model 62 is a model that receives a catheter image 55 and outputs classified image data. The classified image data is data that associates each part that makes up the catheter image 55 with a label that classifies each subject depicted in that part. Each part is, for example, each pixel. The classified image data can be used to generate a classified image 51 in which the catheter image 55 is colored differently for each subject depicted.

[0023] In the following description, for convenience, a classified image 51 is used to explain the processing performed by the catheter system 10 of this embodiment. However, the control unit 21 does not need to actually generate the classified image 51, nor does it need to display the classified image 51 on the display unit 25. The control unit 21 uses the classified image data output from the classification model 62 as is to perform the processing described below.

[0024] A specific example will be given below. Classification model 62 classifies each pixel constituting input catheter image 55 into, for example, first lumen region 511, second lumen region 512, biological tissue region 516, and non-lumen region 517, and outputs classified image data in which the pixel position is associated with a label indicating the classification result.

[0025] The first lumen region 511 indicates the lumen of the hollow organ into which the three-dimensional image acquisition catheter 40 is inserted. The second lumen region 512 indicates the lumen of the hollow organ into which the three-dimensional image acquisition catheter 40 is not inserted. The biological tissue region 516 indicates the region that combines the hollow organ walls, such as the blood vessel walls, heart walls, or digestive tract walls that make up the hollow organ, with the muscles, nerves, fat, etc. that are adjacent to or close to the hollow organ.

[0026] The non-lumen region 517 indicates a region that is not classified as any of the first lumen region 511, the second lumen region 512, and the biological tissue region 516. For example, if the hollow organ into which the three-dimensional image acquisition catheter 40 is inserted is the left ventricle, the non-lumen region 517 includes the extracardiac region and the region outside the cardiac structure. If the imaging range of the image acquisition catheter 40 is small and the distal wall of the left atrium cannot be adequately imaged, the interior of the left atrium is also included in the non-lumen region 517. Similarly, if the distal wall of the lumen of the left ventricle, pulmonary artery, pulmonary vein, aortic arch, etc. cannot be adequately imaged, the non-lumen region 517 also includes the non-lumen region 517. Regions in which a sufficiently clear image is not imaged due to acoustic shadowing, ultrasound attenuation, or the like are also included in the non-lumen region 517.

[0027] The classification model 62 may classify medical device regions corresponding to medical devices used simultaneously with the 3D imaging catheter 40, such as a guidewire. The classification model 62 may classify lesion regions, such as plaque, calcification, or tumor. The classification model 62 may classify these lesion regions by lesion type.

[0028] 2 schematically shows a catheter image 55 displayed in a so-called XY format and a classified image 51 in which classified image data is displayed in the XY format. The classification model 62 may receive input of a catheter image 55 in a so-called RT format, which is formed by arranging scan line data formed by transmitting and receiving ultrasound waves by the sensor 42 in parallel in order of scan angle, and output classified image data. The method of converting from the RT format to the XY format is well known, so a description thereof will be omitted. Since the catheter image 55 is not affected by interpolation processing or the like when being converted from the RT format to the XY format, more appropriate classified image data is generated.

[0029] Classification model 62 is a trained model that performs semantic segmentation on, for example, catheter image 55. The trained model that performs semantic segmentation is generated by machine learning using training data that combines catheter image 55 with classification images 51 in which an expert colors each part of catheter image 55 according to the depicted subject.

[0030] The classification model 62 may be a combination of image processing such as edge detection and rule-based classification processing. The classification model 62 may be a combination of a trained model and rule-based classification processing.

[0031] 3 to 9 are explanatory diagrams illustrating the operation of the catheter system 10. In FIGS. 3 to 10, a case where three-dimensional scanning of a site where two hollow organ lumens 58, which are the lumens of two hollow organs, are approximately parallel is illustrated as an example. A three-dimensional image acquisition catheter 40 is inserted into one of the hollow organ lumens 58, a first lumen 581, from the right side in FIG. 3. A confluence lumen 585, which is a closed bag except for a portion continuous with the first lumen 581, communicates with the first lumen 581 at the longitudinal center of the first lumen 581 in FIG. 2. A second lumen 582, which is the other hollow organ lumen 58, does not communicate with the first lumen 581.

[0032] FIG. 3 shows the position of sensor 42 at time t1, which is the start time of three-dimensional scanning. Controller 21 causes catheter controller 271 to start three-dimensional scanning. Catheter controller 271 captures catheter image 55 while moving sensor 42 to the right in FIG. 3. Controller 21 generates classified image data based on catheter image 55. For ease of explanation, the figure shows classified image 51 that can be generated using classified image data. As described above, in the processing of this embodiment, controller 21 does not need to generate or display classified image 51 based on classified image data.

[0033] In the following description, classified image data generated based on a catheter image 55 captured at time tx may be referred to as classified image data tx at time tx. Similarly, a classified image 51 that can be generated using classified image data tx may be referred to as classified image 51tx at time tx.

[0034] 3 shows a classified image 51t1 at time t1 and a classified image 51t2 at time t2. The time at which the catheter image 55 was captured is indicated in the lower right corner of each classified image 51. The classified image 51t1 at time t1 includes a first lumen region 511 and a second lumen region 512 displayed above the first lumen region 511. In the classified image 51t2 at time t2, the second lumen region 512 has been added below the first lumen region 511.

[0035] 4 shows a linear classification image 52 at time t2. The linear classification image 52 is an image showing classification of a subject in a so-called linear scanning plane along the longitudinal direction of the sheath 41. The linear scanning plane is a plane that includes the central axis of the sheath 41 and is approximately perpendicular to the catheter image 55. A method for generating the linear classification image 52 based on multiple radial classification images 51 is well known, so a description thereof will be omitted.

[0036] For ease of explanation, the linear classification image 52 is also shown. In the processing of this embodiment, the control unit 21 does not need to generate or display the linear classification image 52. If the linear classification image 52 is to be displayed on the display unit 25, the control unit 21 can generate the linear classification image 52 based on multiple classification image data sets without generating the classification image 51.

[0037] 5, the sensor 42 has reached the confluence of the first lumen 581 and the confluence lumen 585. In the classification image 51t3 at time t3, the location that was the lower second lumen region 512 in the classification image 51t2 at time t2 has changed to the first lumen region 511, and the first lumen region 511 has a long, thin shape that extends downward.

[0038] When a region where the second lumen region 512 has changed to the first lumen region 511 exists in the classified image data generated based on the adjacent catheter image 55, the control unit 21 determines that the second lumen region 512 has merged with the first lumen region 511. The control unit 21 looks back at classified image data generated in the past and changes the second lumen region 512 that has been determined to have merged to the first lumen region 511.

[0039] 6 shows the state after the control unit 21 has changed the classification. In the classified image 51t2 at time t2, the area that was determined to be the second lumen area 512 on the lower side in FIG. 5 has been changed to the first lumen area 511.

[0040] Fig. 7 shows the linear classification image 52 at time t3. In Fig. 4, the portion classified as the second lumen region 512 has been changed to the first lumen region 511 in Fig. 7. This clearly shows that the first lumen region 511 and the confluent lumen 585 are continuous regions.

[0041] 8, sensor 42 has reached the position where first lumen 581 and joining lumen 585 are separated again. The portion corresponding to joining lumen 585 is classified as first lumen region 511.

[0042] 9 shows the linear classification image 52 at time t5, when one three-dimensional scan is completed. The first lumen region 511 and the second lumen region 512 extend substantially parallel to each other, with a portion of the first lumen region 511 protruding in a bag-like shape. If the linear classification image 52 shown in FIG. 9 were displayed on the display unit 25, the user would easily understand that the first lumen 581 and the second lumen 582 extend substantially parallel to each other, and that a bag-like merged lumen 585 protrudes from the side of the first lumen 581.

[0043] 10 is an explanatory diagram illustrating the record layout of the image DB 61. The image DB 61 is a DB that records catheter images 55 and classified image data in association with each other. The image DB 61 has a three-dimensional scan ID (Identifier) field, a number field, a catheter image field, a classified image data field, and a pre-change classified image data field.

[0044] The three-dimensional scan ID field records a three-dimensional scan ID that is uniquely assigned to each three-dimensional scan. The number field records consecutive numbers indicating the order in which each catheter image 55 taken in one three-dimensional scan was taken. The catheter image field records the file in which the catheter image 55 is recorded, or the location of the file in which the catheter image 55 is recorded.

[0045] The classified image data field records a file in which the classified image data is recorded, or the location of the file in which the classified image data is recorded. As described using Figures 5 and 6, the pre-change classified image data field records the pre-change classified image data, i.e., the classified image data output from the classification model 62, when the classification of an area in the classified image data is changed. The image DB 61 has one record for one catheter image 55 captured by one rotation of the sensor 42.

[0046] 10, catheter images 55 are illustrated schematically in XY format. Similarly, in FIG. 10, classified image data and pre-altered classified image data are illustrated schematically as classified images 51 in XY format. Catheter images 55 in RT format may be recorded in the image DB 61. Classified images 51 generated based on classified image data and pre-altered classified image data may be recorded in the image DB 61.

[0047] Record No. 1 records data corresponding to time t1 described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t1 is recorded in the catheter image field. Classified image data generated by classification model 62 is recorded in the classified image data field.

[0048] As described above, one record is recorded in the image DB 61 for one rotation of the sensor 42. In Fig. 10, for example, records 2 to (X1-1) are omitted, and only the records corresponding to the times described using Figs. 3 to 9 are illustrated.

[0049] Record X1 contains data corresponding to time t2, as described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t2 is recorded in the catheter image field. Classified image data after modification based on classified image 51 at time t3 is recorded in the classified image data field. Classified image data generated by classification model 62 is recorded in the pre-modified classified image data field.

[0050] Record X2 contains data corresponding to time t3, as described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t3 is recorded in the catheter image field. Classified image data at time t3 is recorded in the classified image data field. Since no classification change has been made, no data is recorded in the pre-change classified image data field.

[0051] Record X3 records data corresponding to time t4, which was described using Figures 3 to 9. Specifically, catheter image 55 captured at time t4 is recorded in the catheter image field. The classified image data generated by classification model 62, which was described using Figure 5, is recorded in the pre-change classified image data field. The classified image data after the lower second lumen region 512 was changed to the first lumen region 511 based on the classified image data at time t3, which was described using Figure 6, is recorded in the classified image data field.

[0052] Record X4 contains data corresponding to time t5, as described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t5 is recorded in the catheter image field. Classified image data at time t5 is recorded in the classified image data field. Since no classification change has been made, no data is recorded in the pre-change classified image data field.

[0053] For example, when the sensor 42 is moved forward and backward manually, or when the speed at which the sensor 42 is moved forward and backward is variable, the image DB 61 may have a field for recording the position of the sensor 42. Even when the speed at which the sensor 42 is moved forward and backward varies, it is possible to provide a catheter system 10 that can accurately construct a three-dimensional image using the catheter images 55 and classified image data.

[0054] If the angle of the catheter image 55 can be detected, the image DB 61 may have a field for recording the angle of the catheter image 55. Even when three-dimensional scanning is performed with the sheath 41 curved, a catheter system 10 can be provided that can accurately construct a three-dimensional image using the catheter image 55 and classified image data.

[0055] Fig. 11 is a flowchart explaining the flow of program processing. The program in Fig. 11 is executed when a user such as a doctor instructs execution of three-dimensional scanning. The control unit 21 instructs the catheter control unit 271 to start three-dimensional scanning (step S501). The catheter control unit 271 controls the MDU 33 to perform three-dimensional scanning and sequentially captures catheter images 55.

[0056] The control unit 21 acquires the catheter image 55 from the catheter control unit 271 (step S502). In step S502, the control unit 21 realizes the function of the catheter image acquisition unit of this embodiment. The control unit 21 inputs the acquired catheter image 55 to the classification model 62 to acquire classified image data (step S503). In step S503, the control unit 21 realizes the function of the classified image data generation unit of this embodiment, which sequentially generates classified image data based on the catheter images 55 captured sequentially, and the function of the classified image data acquisition unit, which sequentially acquires the generated classified image data.

[0057] The control unit 21 creates a new record in the image DB 61. The control unit 21 records a consecutive number in the number field. The control unit 21 records the catheter image 55 acquired in step S502 in the catheter field, and records the classified image data acquired in step S503 in the classified image data field (step S504).

[0058] The control unit 21 determines whether or not the first lumen region 511 and the second lumen region 512 merge (step S505). Specifically, the control unit 21 compares first classified image data generated based on the first catheter image, which is the latest catheter image 55, with second classified image data generated based on a second catheter image captured at a position different from that of the first catheter image. Here, the second catheter image is a catheter image 55 captured earlier than the first catheter image.

[0059] If there is an area that is determined to be the second lumen area 512 in the second classification image and that is determined to be the first lumen area 511 in the first classification image, the control unit 21 determines that a confluence has occurred. In step S505, the control unit 21 realizes the function of the confluence determination unit of this embodiment.

[0060] If it is determined that a merger has occurred (YES in step S505), the control unit 21 starts a subroutine for changing a past classification (step S506). The subroutine for changing a past classification is a subroutine for changing the classification of classified image data that has already been recorded in the classified image data field of the image DB 61. The processing flow of the subroutine for changing a past classification will be described later.

[0061] If it is determined that no merging has occurred (NO in step S505), or after step S506 is completed, the control unit 21 determines whether a branch from the first lumen region 511 to the second lumen region 512 has occurred (step S507). Specifically, the control unit 21 compares a predetermined number of newest classified image data recorded in the classified image data field with the latest classified image data. If there is a location where the first lumen region 511 has changed to the second lumen region 512, the control unit 21 determines that a branch has occurred.

[0062] If it is determined that a branch has occurred (YES in step S507), the control unit 21 generates changed classification image data in which the classification corresponding to the branched portion is changed from the second lumen area 512 to the first lumen area 511 (step S508).

[0063] If it is determined that no branching has occurred (NO in step S507), the control unit 21 determines whether or not the classified image data recorded in the previous record includes a location that has been changed from the second lumen area 512 to the first lumen area 511 (step S511). If it is determined that a changed location exists (YES in step S511), the control unit 21 generates changed classified image data in which the classification of the second lumen area 512 corresponding to the changed location in the previous record has been changed to the first lumen area 511 (step S512).

[0064] After step S508 or step S512 is completed, the control unit 21 records the changed classified image data in the image DB 61 (step S513). Specifically, the control unit 21 extracts the latest record recorded in the image DB 61 and moves the data recorded in the classified image data field to the pre-change classified image data field. Thereafter, the control unit 21 records the changed classified image data in the classified image data field.

[0065] If it is determined that no locations have been changed (NO in step S511), or after step S513 is completed, the control unit 21 displays a three-dimensional image based on the classified image data recorded in the classified image data field on the display unit 25 (step S514). Since the method of constructing a three-dimensional image based on multiple classified image data is well known, its explanation will be omitted. In step S514, the control unit 21 realizes the function of the three-dimensional image output unit of this embodiment.

[0066] The control unit 21 may transmit the three-dimensional image to a network in step S514. This provides a catheter system 10 that allows a user at a remote location to check the three-dimensional image via an HIS or the like. The control unit 21 may store the three-dimensional image in the auxiliary storage device 23 or an external mass storage device in step S514.

[0067] The control unit 21 determines whether or not the processing of the catheter image 55 acquired by one three-dimensional scan has been completed (step S515). If it is determined that the processing has not been completed (NO in step S515), the control unit 21 returns to step S502. If it is determined that the processing has been completed (YES in step S515), the control unit 21 ends the processing.

[0068] 12 is a flowchart illustrating the process flow of the past classification change subroutine. The past classification change subroutine is a subroutine for changing the classification of classified image data that has already been recorded in the classified image data field of the image DB 61.

[0069] The control unit 21 acquires previously recorded classified image data from the classified image data field of the record immediately preceding the record currently being processed from the image DB 61 (step S521). The control unit 21 extracts the region classified as the second lumen region 512 from the acquired classified image data (step S522).

[0070] The control unit 21 determines whether the extracted second lumen region 512 is continuous with the confluence of the first lumen region 511 and the second lumen region 512 (step S523). Specifically, the control unit 21 determines that the second lumen region 512 existing in the same position as the second lumen region 512 determined to merge with the first lumen region 511 is continuous with the confluence. Note that, if multiple second lumen regions 512 are extracted in step S522, the control unit 21 determines whether each of the second lumen regions 512 is continuous with the other.

[0071] If it is determined that the area is continuous with the junction (YES in step S523), the control unit 21 generates changed classified image data in which the classification corresponding to the area continuous with the junction is changed from the second lumen area 512 to the first lumen area 511 (step S524). In step S524, the control unit 21 realizes the function of the classification change unit of this embodiment, which sequentially processes the classified image data.

[0072] The control unit 21 records the changed classified image data in the image DB 61 (step S525). Specifically, the control unit 21 moves the data recorded in the classified image data field of the record extracted in step S521 to the pre-change classified image data field. Thereafter, the control unit 21 records the changed classified image data in the classified image data field.

[0073] If data is already recorded in the pre-change classified image data field, the control unit 21 rewrites the data in the classified image data field without changing the data in that field. The image DB 61 may have a field that records a history each time classified image data is changed. This allows the classified image data output from the classification model 62 to remain recorded in the image DB 61 as is.

[0074] The control unit 21 determines whether to end the process (step S526). For example, the control unit 21 determines to end the process when NO determination is made a predetermined number of times in succession in step S523. If it is determined not to end the process (NO in step S526), the control unit 21 returns to step S521 and processes the previous record. If it is determined to end the process (YES in step S526), the control unit 21 ends the process.

[0075] 13 to 15 are explanatory diagrams illustrating examples of displaying a three-dimensional image. Using Fig. 13 to Fig. 15, examples of displaying a three-dimensional image performed by control unit 21 in step S514 of Fig. 11 will be described.

[0076] In the following description, an example will be given in which a user observes the shape of the first lumen 581 into which the three-dimensional image acquisition catheter 40 has been inserted. As described above, the control unit 21 constructs a three-dimensional image based on a series of classified image data. As described above, the method of constructing a three-dimensional image based on a series of classified image data is well known, and therefore, description thereof will be omitted.

[0077] For example, control unit 21 displays the portion corresponding to first lumen region 511 in an opaque state, displays the portion corresponding to second lumen region 512 in a semi-transparent state, and does not display the other portions. The shape of first lumen 581 is expressed by the portion corresponding to first lumen region 511, and the shape of second lumen 582 is expressed by the portion corresponding to second lumen region 512. In Figures 13 to 15, the portions displayed opaque are indicated by solid lines, and the portions displayed semi-transparently are indicated by two-dot chain lines.

[0078] FIG. 13 is an example of a three-dimensional image at time t2 described using FIGS. 3 to 9. FIG. 14 is an example of a three-dimensional image at time t3. The left end portion of the confluent lumen 585, which was displayed semi-transparently in FIG. 13, has been changed to opaque. FIG. 15 is an example of a three-dimensional image at time t5. The first lumen 581 and the confluent lumen 585 are displayed opaquely, and the second lumen 582 is displayed semi-transparently.

[0079] The above display allows the user to easily grasp the three-dimensional shape of the target part in real time. Based on the user's instruction, the control unit 21 may display the first lumen 581 opaquely and the second lumen 582 semi-transparently.

[0080] The control unit 21 may display, together with the three-dimensional image, an XY-format catheter image 55, which is a radial two-dimensional image, on the display unit 25. In this case, the control unit 21 realizes the function of a radial image acquisition unit that outputs the classified image 51 as a radial two-dimensional image. The control unit 21 may superimpose the classified image 51 on the catheter image 55. When superimposing the classified image 51, the control unit 21 may display the classified image 51 in a semi-transparent state.

[0081] The control unit 21 may display an image in the form of a linear two-dimensional image generated based on the catheter image 55 on the display unit 25. In the following description, a linear catheter image may be referred to as a linear catheter image. In this case, the control unit 21 realizes the function of a linear image output unit that outputs a linear catheter image. The control unit 21 may superimpose the linear classification image 52 on the linear catheter image. When superimposing the linear classification image 52, the control unit 21 may display the linear classification image 52 in a semi-transparent state.

[0082] The control unit 21 may receive, for example, an instruction from the user to change the position of the cross section of the linear catheter image. The control unit 21 may also receive, from the user, an instruction to change the orientation in which the three-dimensional image is displayed. Methods for appropriately changing the display format of the constructed three-dimensional image based on a user instruction are well known, and therefore will not be described here.

[0083] The control unit 21 may display a cross section of the constructed three-dimensional image cut along an arbitrary plane. A method for receiving an instruction from a user of a plane for cutting a three-dimensional image and a method for displaying a cross section based on the instruction from the user are well known, and therefore a description thereof will be omitted.

[0084] The catheter system 10 may have a function of capturing a catheter image 55 at a fixed position without moving the sensor 42 back and forth. A catheter system 10 can be provided that can switch between B-mode scanning, which transmits and receives ultrasound while rotating at a fixed position, and three-dimensional scanning.

[0085] According to this embodiment, it is possible to provide a catheter system 10 that displays the structure of confluent and branched lumens in an easy-to-understand manner. Therefore, it is possible to provide a catheter system 10 that supports the understanding of images acquired by the image acquisition catheter 40.

[0086] By processing the catheter images 55 taken by the three-dimensional image acquisition catheter 40 in real time, it is possible to provide a catheter system 10 that supports, for example, IVR (Interventional Radiology) procedures.

[0087] In the above description, the first lumen region 511 has a tubular shape similar to a blood vessel, but the catheter 40 for acquiring three-dimensional images may be inserted into a relatively large area such as the atrium or ventricle. The catheter system 10 of this embodiment can be used when the user wants to observe the shape of the left atrial appendage from the left atrium or left pulmonary vein.

[0088] The control unit 21 may acquire and process catheter images 55 that have been recorded in advance in the auxiliary storage device 23 or an external database, etc., instead of catheter images 55 captured in real time. The control unit 21 realizes the function of a catheter image acquisition unit.

[0089] The control unit 21 may acquire and process classified image data previously recorded in the auxiliary storage device 23 or an external database, etc. The control unit 21 realizes the function of a classified image data acquisition unit that acquires multiple classified image data. In this case, the information processing device 20 may be an information processing device such as a general-purpose personal computer, smartphone, or tablet that does not include the catheter control unit 271.

[0090] [Embodiment 2] This embodiment relates to a catheter system 10 that records a region of the second lumen region 512 that is determined to merge with the first lumen region 511. Descriptions of parts common to the first embodiment will be omitted.

[0091] 16 is an explanatory diagram illustrating the record layout of the image DB 61 according to the second embodiment. The image DB 61 is a DB that stores catheter images 55, classified image data, and confluence area data in association with each other. The image DB 61 has a three-dimensional scan ID field, a number field, a catheter image field, a classified image data field, and a confluence area data field.

[0092] The three-dimensional scan ID field records a three-dimensional scan ID that is uniquely assigned to each three-dimensional scan. The number field records consecutive numbers indicating the order in which each catheter image 55 taken in one three-dimensional scan was taken. The catheter image field records the file in which the catheter image 55 is recorded, or the location of the file in which the catheter image 55 is recorded.

[0093] The classified image data field records the file in which the classified image data is recorded or the location of the file in which the classified image data is recorded. In this embodiment, the classified image data recorded in the classified data field is classified image data output from the classification model 62.

[0094] The confluence area data field records a file in which confluence area data is recorded or the location of the file in which confluence area data is recorded. The confluence area data is data that records only the area that is determined to be the second lumen area 512 in the classified image data output from the classification model 62 and that is to merge with the first lumen area 511 as described in the first embodiment.

[0095] In FIG. 16, the confluence region data is illustrated in a schematic XY format. The confluence region data is data in which, for example, "1" is assigned to pixels included in a region determined to be confluent, and "0" is assigned to pixels not determined to be confluent. The confluence region data may be data in which, among the classified image data, only labels associated with regions determined to be confluent with the first lumen region 511 are retained, and labels associated with other regions are changed to, for example, "0."

[0096] Record No. 1 records data corresponding to time t1 described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t1 is recorded in the catheter image field. Classified image data generated by classification model 62 is recorded in the classified image data field. No data is recorded in the confluence area data field.

[0097] Record X1 contains data corresponding to time t2, as described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t2 is recorded in the catheter image field. Classified image data generated by classification model 62 is recorded in the classified image data field. Data indicating only the region of second lumen region 512 that is determined to merge with first lumen region 511 based on classified image 51 at time t3 is recorded in the merge data field.

[0098] Record X2 contains data corresponding to time t3, as described with reference to Figures 3 to 9. Specifically, catheter image 55 captured at time t3 is recorded in the catheter image field. Classified image data for time t3 is recorded in the classified image data field. Since there is no area determined to be a merger, no data is recorded in the merger area data field.

[0099] Record X3 contains data corresponding to time t4, which was described using Figures 3 to 9. Specifically, catheter image 55 captured at time t4 is recorded in the catheter image field. Classified image data generated by classification model 62 is recorded in the image data field. Data indicating only the region of second lumen region 512 that is determined to merge with first lumen region 511 based on classified image 51 at time t3 is recorded in the merge data field.

[0100] Record X4 contains data corresponding to time t5, as described with reference to Figures 3 to 9. Specifically, the catheter image 55 captured at time t5 is recorded in the catheter image field. The classified image data for time t5 is recorded in the classified image data field. Since there is no area determined to be a merger, no data is recorded in the merger area data field.

[0101] Fig. 17 is a flowchart illustrating the processing flow of the program according to the second embodiment. The program in Fig. 17 is executed in place of the program according to the first embodiment described using Fig. 11. The processing from step S501 to step S505 is the same as the processing flow of the program described using Fig. 11, and therefore a description thereof will be omitted.

[0102] When it is determined that a merger has occurred (YES in step S505), the control unit 21 starts a subroutine for generating past merger area data (step S551). The subroutine for generating past merger area data is a subroutine for generating merger area data corresponding to the portion of the second lumen area 512 that merges with the first lumen area 511, based on the classified image data already recorded in the classified image data field of the image DB 61. The processing flow of the subroutine for generating past merger area data will be described later.

[0103] If it is determined that no confluence has occurred (NO in step S505), or after step S551 is completed, the control unit 21 determines whether or not a branch has occurred from the first lumen region 511 to the second lumen region 512 (step S507). If it is determined that a branch has occurred (YES in step S507), the control unit 21 extracts the second lumen region 512 corresponding to the branched portion and generates confluence region data (step S552).

[0104] If it is determined that no branch has occurred (NO in step S507), the control unit 21 determines whether or not confluence area data is recorded in the confluence area data field of the previous record (step S561).If it is determined that confluence area data is recorded (YES in step S561), the control unit 21 extracts the second lumen area 512 corresponding to the confluence area data in the previous record and generates confluence area data (step S562).

[0105] After step S552 or step S562 is completed, the control unit 21 records the confluence area data in the image DB 61 (step S563). Specifically, the control unit 21 extracts the latest record recorded in the image DB 61, and records the confluence area data in the changed area data field.

[0106] If it is determined that no confluence area data has been recorded (NO in step S561), or after step S563 ends, the control unit 21 displays a three-dimensional image based on the classified image data recorded in the classified image data field and the confluence area data field on the display unit 25 (step S564). Display examples of the three-dimensional image will be described later.

[0107] The control unit 21 determines whether or not the processing of the catheter image 55 acquired by one three-dimensional scan has been completed (step S515). If it is determined that the processing has not been completed (NO in step S515), the control unit 21 returns to step S502. If it is determined that the processing has been completed (YES in step S515), the control unit 21 ends the processing.

[0108] 18 is a flowchart illustrating the processing flow of the subroutine for generating past confluence area data. The subroutine for generating past confluence area data is a subroutine for generating confluence area data corresponding to the portion of the second lumen area 512 that merges with the first lumen area 511, based on the classified image data already recorded in the classified image data field of the image DB 61.

[0109] The control unit 21 acquires previously recorded classified image data from the classified image data field of the record immediately preceding the record currently being processed from the image DB 61 (step S571). The control unit 21 extracts the region classified as the second lumen region 512 from the acquired classified image data (step S572).

[0110] The control unit 21 determines whether the extracted second lumen region 512 is continuous with the confluence of the first lumen region 511 and the second lumen region 512 (step S573). If it is determined that the extracted second lumen region 512 is continuous with the confluence (YES in step S573), the control unit 21 extracts the second lumen region 512 corresponding to the portion continuous with the confluence, and generates confluence region data (step S574).

[0111] The control unit 21 records the confluence area data in the image DB 61 (step S525). Specifically, the control unit 21 records the confluence classification data generated in step S574 in the confluence area data field of the record extracted in step S571.

[0112] Control unit 21 determines whether to end the process (step S576). For example, control unit 21 determines to end the process if NO determination is made a predetermined number of times in succession in step S573. If it is determined not to end the process (NO in step S576), control unit 21 returns to step S571 and processes the previous record. If it is determined to end the process (YES in step S576), control unit 21 ends the process.

[0113] Figures 19 to 21 are explanatory diagrams illustrating example screens in embodiment 2. Figures 19 to 21 are examples of three-dimensional images at time t5 described using Figures 3 to 9. Figure 19 shows an example of a three-dimensional image constructed based on a series of classified image data recorded in the classified image data field.

[0114] 19, the control unit 21 displays the portion corresponding to the first lumen region 511 in an opaque state, displays the portion corresponding to the second lumen region 512 in a semi-transparent state, and does not display the other portions. As described using FIG. 16, both end portions of the confluence lumen 585 are displayed semi-transparently because they are in the second lumen region 512.

[0115] The control unit 21 may receive an instruction to display only the portion of the second lumen region 512 that corresponds to the confluence region data in the same manner as the first lumen region 511. When such an instruction is received, the control unit 21 displays the portion that corresponds to the confluence region data in the same manner as the first lumen region 511. That is, as described using FIG. 15 , the control unit 21 displays the first lumen 581 and the confluence lumen 585 opaquely and the second lumen 582 semitransparently.

[0116] The control unit 21 may receive an instruction to display the region corresponding to the confluence region data in a manner different from that of the first lumen 581 and the second lumen 582. For example, the control unit 21 may display both end portions of the confluence lumen 585 shown in FIG. 19 at a transparency intermediate between that of the first lumen region 511 and the second lumen region 512.

[0117] 20, the control unit 21 displays the portion corresponding to the first lumen region 511 in a semi-transparent state, displays the portion corresponding to the second lumen region 512 in an opaque state, and does not display the other portions. For example, when a user's instruction to display only the second lumen region 512, including the portion joining the first lumen region 511, in an opaque state is received, the control unit 21 performs the display shown in FIG.

[0118] 21, the control unit 21 displays the portions of the first lumen region 511 and the second lumen region 512 that merge with the first lumen region 511 in a semi-transparent state, displays the other portions corresponding to the second lumen region 512 in an opaque state, and does not display the other portions. For example, when a user's instruction to display only the second lumen region 512 that does not merge with the first lumen region 511 in an opaque state is received, the control unit 21 performs the display shown in FIG.

[0119] According to this embodiment, by recording both the classified image data and the confluence area data in the image DB 61, it is possible to provide a catheter system 10 that can perform various displays in response to instructions from the user.

[0120] [Embodiment 3] 22 is an explanatory diagram illustrating the configuration of a catheter system 10 according to a third embodiment. This embodiment relates to a configuration in which the catheter system 10 according to the present embodiment is realized by combining and operating a catheter control device 27, an MDU 33, a catheter 40 for acquiring three-dimensional images, a general-purpose computer 90, and a program 97. Explanations of parts common to the first embodiment will be omitted.

[0121] Catheter control device 27 is an ultrasound diagnostic device for IVUS (Intravascular Ultrasound) that controls MDU 33, controls sensor 42, and generates transverse and longitudinal images based on signals received from sensor 42. The function and configuration of catheter control device 27 are similar to those of conventionally used ultrasound diagnostic devices, and therefore a description thereof will be omitted.

[0122] The catheter system 10 of this embodiment includes a computer 90. The computer 90 includes a control unit 21, a main memory device 22, an auxiliary memory device 23, a communication unit 24, a display unit 25, an input unit 26, a reading unit 29, and a bus. The computer 90 is an information device such as a general-purpose personal computer, a tablet, a smartphone, or a server computer. The computer 90 may be a mainframe computer, a virtual machine running on a mainframe computer, a cloud computing system, a quantum computer, or multiple personal computers performing distributed processing.

[0123] The program 97 is recorded on a portable recording medium 96. The control unit 21 reads the program 97 via the reading unit 29 and stores it in the auxiliary storage device 23. The control unit 21 may also read the program 97 stored in a semiconductor memory 98, such as a flash memory, implemented in the computer 90. Furthermore, the control unit 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown) and store it in the auxiliary storage device 23.

[0124] The program 97 is installed as a control program for the computer 90, and is loaded into and executed by the main storage device 22. As a result, the computer 90 and the catheter control device 27 work together to function as the information processing device 20 described above.

[0125] [Embodiment 4] 23 is a functional block diagram of an information processing device 20 according to the fourth embodiment. The information processing device 20 includes a classified image data acquisition unit 81, a confluence determination unit 82, and an image output unit 84. The classified image data acquisition unit 81 acquires, based on a plurality of catheter images 55 acquired using an image acquisition catheter 40 that acquires images while moving in the axial direction, a plurality of classified image data items classified into a plurality of regions including a first lumen region 511 into which the image acquisition catheter 40 is inserted and a second lumen region 512 into which the image acquisition catheter 40 is not inserted.

[0126] The confluence determination unit 82 determines whether the second lumen region 512 in a first catheter image among the multiple catheter images 55 confluences with the first lumen region 511 in a second catheter image acquired at an axial position different from that of the first catheter image.

[0127] The image output unit 84 outputs an image including the first lumen region 511 based on the plurality of classified image data. The image output unit 84 outputs only the second lumen region 512 in the first catheter image that has been determined to merge by the merging determination unit 82, together with the first lumen region 511, as a region image. [Embodiment 5] 24 is a functional block diagram of an information processing device 20 according to the fifth embodiment. The information processing device 20 includes a classified image data acquisition unit 81, a confluence determination unit 82, and a classification change unit 83. The classified image data acquisition unit 81 acquires a plurality of classified image data in which each of a plurality of catheter images 55 acquired using the imaging catheter 40 is classified into a plurality of regions including a first lumen region 511 into which the imaging catheter 40 is inserted and a second lumen region 512 into which the imaging catheter 40 is not inserted.

[0128] The junction determination unit 82 determines whether the second lumen region 512 in a first catheter image among the multiple catheter images 55 merges with the first lumen region 511 in a second catheter image acquired at a different time from the first catheter image. If the junction determination unit 82 determines that the two regions merge, the classification change unit 83 changes the classification of the second lumen region 512 in the first catheter image to the first lumen region 511.

[0129] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0130] 10 Catheter System 20 Information processing equipment 21 Control section 22 Main storage 23 Auxiliary storage device 24 Communications Department 25 Display section 26 Input section 27 Catheter control device 271 Catheter control unit 29 Reading unit 33 MDU 40 Three-dimensional image acquisition catheter (image acquisition catheter) 41 Sheath 42 sensors 43 Shaft 51 Classification Images 511 1st lumen area 512 2nd lumen area 516 Biological Tissue Area 517 Nonluminal region 52 Linear Classification Images 55 Catheter Images 58 Luminal organ lumen 581 1st lumen 582 Second lumen 585 Confluence lumen 61 Image DB 62 Classification Models 81 Classification image data acquisition unit 82 Merging judgment part 83 Classification Change Department 84 Image output unit 90 Computer 96 Portable recording media 97 Programs 98 Semiconductor Memory

Claims

1. a classified image data acquisition unit that acquires a plurality of classified image data, which are generated based on a plurality of catheter images acquired using an image acquisition catheter that acquires images while moving a scanning plane in an axial direction, and which are classified into a plurality of regions including a first lumen region into which the image acquisition catheter is inserted and a second lumen region into which the image acquisition catheter is not inserted; a confluence determination unit that determines whether the second lumen region in a first catheter image among the plurality of catheter images merges with the first lumen region in a second catheter image acquired at an axial position different from that of the first catheter image; an image output unit that outputs a region image including the first lumen region based on the plurality of classified image data, The image output unit outputs only the second lumen region in the first catheter image, which is determined to be a merged region by the merge determination unit, together with the first lumen region, as the region image. Information processing device.

2. a classification change unit that changes the classification of the second lumen region in the first catheter image, which is determined to be a confluence by the confluence determination unit, to the first lumen region, among the classified image data; The image output unit outputs, as the region image, only the second lumen region whose classification has been changed by the classification change unit, among the second lumen regions acquired by the classified image data acquisition unit, together with the first lumen region. The information processing device according to claim 1 .

3. The image output unit includes a three-dimensional image output unit that outputs a three-dimensional image including the first lumen region as the region image based on the plurality of classified image data.

3. The information processing device according to claim 1.

4. the imaging catheter is a radial scanning type; a radial type image output unit that outputs one of the plurality of catheter images as a radial type two-dimensional image; The image output unit outputs the regional image generated based on the catheter image so as to be superimposed on the radial type two-dimensional image.

4. The information processing device according to claim 1.

5. the imaging catheter is a radial scanning type; a linear image output unit that outputs a linear two-dimensional image along the axial direction, The image output unit outputs the region image so as to be superimposed on the linear two-dimensional image.

5. The information processing device according to claim 1.

6. a catheter image acquisition unit that acquires a plurality of the catheter images; a classified image data generating unit that classifies each of the catheter images into a plurality of regions including the first lumen region and the second lumen region, and generates the classified image data.

6. The information processing device according to claim 1.

7. The classified image data generation unit inputs the acquired catheter image into a trained model that, when a catheter image is input, outputs classified image data in which each region of the catheter image is classified into a predetermined region, and generates the classified image data based on the acquired classified image data. The information processing device according to claim 6 .

8. the catheter image acquisition unit sequentially acquires catheter images acquired using the image acquisition catheter in real time; The classified image data generating unit sequentially generates the classified image data.

8. The information processing device according to claim 6 or 7.

9. a classification change unit that changes the second lumen region in the first catheter image, which has been determined to be a confluence by the confluence determination unit, to the first lumen region; The classification change unit sequentially processes the classified image data generated by the classified image data generation unit. The information processing device according to claim 8 .

10. The classified image data is classified into the first lumen region, the second lumen region, a biological tissue region, and a non-lumen region that is not one of the aforementioned regions.

10. The information processing device according to claim 1.

11. a plurality of classified image data are acquired based on a plurality of catheter images acquired using an image acquisition catheter that acquires images while moving a scanning plane in an axial direction, the classified image data being classified into a plurality of regions including a first lumen region into which the image acquisition catheter is inserted and a second lumen region into which the image acquisition catheter is not inserted; determining whether the second lumen region in a first catheter image among the plurality of catheter images merges with the first lumen region in a second catheter image acquired at an axial position different from that of the first catheter image; Only the second lumen region in the first catheter image that is determined to be merged among the second lumen regions based on the plurality of classified image data is output as a region image together with the first lumen region. An information processing method in which processing is performed by a computer.

12. a plurality of classified image data are acquired based on a plurality of catheter images acquired using an image acquisition catheter that acquires images while moving a scanning plane in an axial direction, the classified image data being classified into a plurality of regions including a first lumen region into which the image acquisition catheter is inserted and a second lumen region into which the image acquisition catheter is not inserted; determining whether the second lumen region in a first catheter image among the plurality of catheter images merges with the first lumen region in a second catheter image acquired at an axial position different from that of the first catheter image; Only the second lumen region in the first catheter image that is determined to be merged among the second lumen regions based on the plurality of classified image data is output as a region image together with the first lumen region. A program that causes a computer to perform a process.

Images