Computer program, information processing method, and information processing device

Abstract

A computer program that: obtains a medical image generated on the basis of a signal detected by a catheter inserted in a luminal organ; derives object information pertaining to the type of object included in the obtained medical image; and executes processing that provides support information to the operator of the catheter, on the basis of the derived object information.

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device. [Background technology]

[0002] Medical images including ultrasonic tomographic images of blood vessels are generated by intravascular ultrasound (IVUS) using a catheter, and intravascular ultrasound examinations are performed. Meanwhile, technologies are being developed to add information to medical images using image processing and machine learning to assist doctors in their diagnoses (see, for example, Patent Document 1). The feature detection method for blood vessel images described in Patent Document 1 detects lumen walls, stents, and the like included in the blood vessel images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2016-525893 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the detection method of Patent Document 1 does not take into consideration the provision of information according to objects included in the blood vessel image.

[0005] The object of the present disclosure is to provide a computer program, etc. that provides useful information to a catheter operator based on a medical image obtained by scanning a tubular organ with a catheter, according to objects contained in the medical image. [Means for solving the problem]

[0006] The computer program according to this aspect acquires medical images generated based on signals detected by a catheter inserted into a tubular organ, derives object information regarding the types of objects contained in the acquired medical images, and executes a process of providing support information to the operator of the catheter based on the derived object information.

[0007] The information processing method of this aspect causes a computer to acquire a medical image generated based on a signal detected by a catheter inserted into a tubular organ, derive object information regarding the type of object contained in the acquired medical image, and perform a process of providing support information to the operator of the catheter based on the derived object information.

[0008] The information processing device of this aspect includes an acquisition unit that acquires medical images generated based on signals detected by a catheter inserted into a tubular organ, a derivation unit that derives object information regarding the types of objects contained in the acquired medical images, and a processing unit that provides support information to the operator of the catheter based on the derived object information. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a computer program or the like that provides useful information to the operator of a catheter based on a medical image obtained by scanning a tubular organ with a catheter, depending on objects contained in the medical image. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic apparatus. [Figure 2] FIG. 1 is an explanatory diagram illustrating an overview of a catheter for diagnostic imaging. [Figure 3] FIG. 2 is an explanatory diagram showing a cross section of a blood vessel through which a sensor portion is inserted. [Figure 4] FIG. 2 is an explanatory diagram illustrating a tomographic image. [Figure 5]FIG. 1 is a block diagram illustrating an example of the configuration of an image processing device. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of a learning model. [Figure 7] FIG. 10 is an explanatory diagram illustrating an example of an association table. [Figure 8] 10 is a flowchart showing an information processing procedure performed by a control unit. [Figure 9] 10 is a flowchart showing a procedure for providing information on stent placement. [Figure 10] FIG. 10 is an explanatory diagram showing an example of display of information relating to identification of a reference portion. [Figure 11] FIG. 10 is an explanatory diagram showing an example of display of information related to stent placement. [Figure 12] 10 is a flowchart showing an information providing procedure for endpoint determination. [Figure 13] 10 is a flowchart showing the procedure for calculating the MSA. [Figure 14] FIG. 10 is an explanatory diagram showing a display example that visualizes the expansion state in the vicinity of the stent placement site. [Figure 15] FIG. 10 is an explanatory diagram showing an example of display of information regarding a desired expansion diameter. [Figure 16] FIG. 10 is an explanatory diagram showing an example of display of information relating to endpoint determination. [Figure 17] FIG. 10 is an explanatory diagram showing an example of an association table in the second embodiment. [Figure 18] FIG. 10 is an explanatory diagram illustrating an example of a combination table. [Figure 19] 10 is a flowchart showing an information processing procedure performed by a control unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] The image processing method, image processing device, and program of the present disclosure will be described in detail below with reference to the drawings illustrating embodiments thereof. In the following embodiments, cardiac catheterization, which is an intravascular treatment, will be described as an example. However, the hollow organs that are the subject of catheterization treatment are not limited to blood vessels, and may be other hollow organs such as the bile duct, pancreatic duct, bronchi, intestines, etc.

[0012] (Embodiment 1) FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic device 100. In this embodiment, an imaging diagnostic device using a dual-type catheter equipped with the functions of both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) will be described. The dual-type catheter is provided with a mode for acquiring ultrasound tomographic images using only IVUS, a mode for acquiring optical coherence tomographic images using only OCT, and a mode for acquiring tomographic images using both IVUS and OCT, and these modes can be switched for use. Hereinafter, ultrasound tomographic images and optical coherence tomographic images will be referred to as IVUS images and OCT images, respectively, as appropriate. IVUS images and OCT images will be collectively referred to as tomographic images, which correspond to medical images.

[0013] The diagnostic imaging apparatus 100 of this embodiment includes an intravascular inspection apparatus 101, an angiography apparatus 102, an image processing device 3, a display device 4, and an input device 5. The intravascular inspection apparatus 101 includes a diagnostic imaging catheter 1 and an MDU (Motor Drive Unit) 2. The diagnostic imaging catheter 1 is connected to the image processing device 3 via the MDU 2. The display device 4 and the input device 5 are connected to the image processing device 3. The display device 4 is, for example, a liquid crystal display or an organic EL display, and the input device 5 is, for example, a keyboard, a mouse, a trackball, or a microphone. The display device 4 and the input device 5 may be stacked together to form a touch panel. Alternatively, the input device 5 and the image processing device 3 may be integrated. Furthermore, the input device 5 may be a sensor that accepts gesture input, gaze input, or the like.

[0014] The angiography device 102 is connected to the image processing device 3. The angiography device 102 is an angiography device that images blood vessels from outside the patient's body using X-rays while injecting a contrast agent into the patient's blood vessels, thereby obtaining an angio image, which is a fluoroscopic image of the blood vessels. The angiography device 102 is equipped with an X-ray source and an X-ray sensor, and images an X-ray fluoroscopic image of the patient by the X-ray sensor receiving X-rays irradiated from the X-ray source. Note that the diagnostic imaging catheter 1 is provided with a marker that is opaque to X-rays, and the position of the diagnostic imaging catheter 1 (marker) is visualized in the angio image. The angiography device 102 outputs the angio image obtained by imaging to the image processing device 3, and the angiography image is displayed on the display device 4 via the image processing device 3. Note that the display device 4 displays the angio image and a tomographic image captured using the diagnostic imaging catheter 1.

[0015] FIG. 2 is an explanatory diagram outlining the diagnostic imaging catheter 1. The upper dashed-dotted area in FIG. 2 is an enlarged version of the lower dashed-dotted area. The diagnostic imaging catheter 1 includes a probe 11 and a connector 15 disposed at the end of the probe 11. The probe 11 is connected to the MDU 2 via the connector 15. In the following description, the side of the diagnostic imaging catheter 1 farther from the connector 15 will be referred to as the distal end, and the connector 15 side will be referred to as the proximal end. The probe 11 includes a catheter sheath 11a, the distal end of which is provided with a guidewire insertion portion 14 through which a guidewire can be inserted. The guidewire insertion portion 14 forms a guidewire lumen and is used to receive a guidewire previously inserted into a blood vessel and guide the probe 11 to the affected area via the guidewire. The catheter sheath 11a forms a continuous tubular portion extending from the connection portion with the guidewire insertion portion 14 to the connection portion with the connector 15. A shaft 13 is inserted inside the catheter sheath 11a, and a sensor unit 12 is connected to the tip side of the shaft 13.

[0016] The sensor unit 12 has a housing 12d, the distal end of which is hemispherically shaped to reduce friction and snagging with the inner surface of the catheter sheath 11a. The housing 12d contains an ultrasonic transceiver 12a (hereinafter referred to as the IVUS sensor 12a) that transmits ultrasonic waves into a blood vessel and receives reflected waves from the blood vessel, and an optical transceiver 12b (hereinafter referred to as the OCT sensor 12b) that transmits near-infrared light into the blood vessel and receives reflected light from the blood vessel. In the example shown in FIG. 2, the IVUS sensor 12a is provided at the distal end of the probe 11, and the OCT sensor 12b is provided at the proximal end. The sensors are disposed on the central axis of the shaft 13 (on the two-dot chain line in FIG. 2) at a distance x along the axial direction. In the diagnostic imaging catheter 1, the IVUS sensor 12a and the OCT sensor 12b are attached such that the direction of transmission and reception of ultrasonic waves or near-infrared light is approximately 90 degrees relative to the axial direction of the shaft 13 (the radial direction of the shaft 13). It is desirable that the IVUS sensor 12a and the OCT sensor 12b are mounted slightly offset from the radial direction so as not to receive waves or light reflected from the inner surface of the catheter sheath 11a. In this embodiment, as shown by the arrow in Fig. 2, for example, the IVUS sensor 12a is mounted so that the direction of ultrasound irradiation is inclined toward the proximal end with respect to the radial direction, and the OCT sensor 12b is mounted so that the direction of near-infrared light irradiation is inclined toward the distal end with respect to the radial direction.

[0017] An electric signal cable (not shown) connected to the IVUS sensor 12a and an optical fiber cable (not shown) connected to the OCT sensor 12b are inserted into the shaft 13. The probe 11 is inserted into a blood vessel from the distal end side. The sensor unit 12 and the shaft 13 are movable forward and backward within the catheter sheath 11a and can also rotate circumferentially. The sensor unit 12 and the shaft 13 rotate around the central axis of the shaft 13 as the axis of rotation. The diagnostic imaging device 100 uses an imaging core formed by the sensor unit 12 and the shaft 13 to measure the state inside the blood vessel using ultrasound tomographic images (IVUS images) taken from inside the blood vessel or optical coherence tomographic images (OCT images) taken from inside the blood vessel.

[0018] The MDU 2 is a drive unit to which the probe 11 (diagnostic imaging catheter 1) is detachably attached via a connector unit 15, and controls the operation of the diagnostic imaging catheter 1 inserted into a blood vessel by driving a built-in motor in response to operation by a medical professional. For example, the MDU 2 performs a pull-back operation to rotate the sensor unit 12 and shaft 13 inserted into the probe 11 in a circumferential direction while pulling them toward the MDU 2 at a constant speed. The sensor unit 12 rotates while moving from the distal end to the proximal end due to the pull-back operation, and thereby scans the inside of the blood vessel continuously at predetermined time intervals, thereby continuously capturing multiple transverse cross-sectional images approximately perpendicular to the probe 11 at predetermined intervals. The MDU 2 outputs reflected wave data of ultrasound received by the IVUS sensor 12a and reflected light data received by the OCT sensor 12b to the image processing device 3.

[0019] The image processing device 3 acquires, via the MDU 2, a signal dataset that is reflected wave data of ultrasound received by the IVUS sensor 12a and a signal dataset that is reflected light data received by the OCT sensor 12b. The image processing device 3 generates ultrasound line data from the ultrasound signal dataset and constructs an ultrasound tomographic image (IVUS image) of a transverse layer of the blood vessel based on the generated ultrasound line data. The image processing device 3 also generates optical line data from the reflected light signal dataset and constructs an optical tomographic image (OCT image) of a transverse layer of the blood vessel based on the generated optical line data. Here, the signal datasets acquired by the IVUS sensor 12a and the OCT sensor 12b and the tomographic image constructed from the signal dataset will be described. FIG. 3 is an explanatory diagram showing a cross section of a blood vessel through which the sensor unit 12 has been inserted, and FIG. 4 is an explanatory diagram showing the tomographic image.

[0020] First, referring to FIG. 3 , the operation of the IVUS sensor 12a and the OCT sensor 12b within a blood vessel and the signal data sets (ultrasound line data and optical line data) acquired by the IVUS sensor 12a and the OCT sensor 12b will be described. When imaging of a tomographic image begins with the imaging core inserted into the blood vessel, the imaging core rotates in the direction indicated by the arrow, with the central axis of the shaft 13 as the center of rotation. At this time, the IVUS sensor 12a transmits and receives ultrasound waves at each rotation angle. Lines 1, 2, ... 512 indicate the transmission and reception directions of ultrasound waves at each rotation angle. In this embodiment, the IVUS sensor 12a intermittently transmits and receives ultrasound waves 512 times while rotating 360 degrees (one rotation) within the blood vessel. The IVUS sensor 12a acquires one line of data in the transmission and reception direction by transmitting and receiving ultrasound waves once, so that 512 ultrasound line data extending radially from the center of rotation can be obtained during one rotation. The 512 ultrasound line data are dense near the center of rotation, but become sparser as they move away from the center of rotation. Therefore, the image processing device 3 generates pixels in the empty spaces of each line by well-known interpolation processing, thereby generating a two-dimensional ultrasound tomographic image (IVUS image) as shown in Figure 4A.

[0021] Similarly, the OCT sensor 12b also transmits and receives measurement light at each rotation angle. Since the OCT sensor 12b also transmits and receives measurement light 512 times while rotating 360 degrees within the blood vessel, 512 optical line data extending radially from the center of rotation can be obtained during one rotation. For the optical line data, the image processing device 3 generates pixels in the empty spaces of each line using a well-known interpolation process, thereby generating a two-dimensional optical coherence tomographic image (OCT image) similar to the IVUS image shown in FIG. 4A. That is, the image processing device 3 generates optical line data based on interference light generated by interfering reflected light with reference light obtained by, for example, separating light from a light source within the image processing device 3, and constructs an optical tomographic image (OCT image) capturing a cross-section of the blood vessel based on the generated optical line data.

[0022] A two-dimensional tomographic image generated from 512 lines of data in this manner is called one frame of an IVUS image or an OCT image. Note that since the sensor unit 12 scans while moving inside the blood vessel, one frame of an IVUS image or an OCT image is acquired at each position of one rotation within the range of movement. That is, one frame of an IVUS image or an OCT image is acquired at each position from the distal end to the proximal end of the probe 11 within the range of movement, so that multiple frames of IVUS images or OCT images are acquired within the range of movement, as shown in FIG. 4B.

[0023] The diagnostic imaging catheter 1 has markers that are opaque to X-rays in order to confirm the positional relationship between an IVUS image obtained by the IVUS sensor 12a or an OCT image obtained by the OCT sensor 12b, and an angio image obtained by the angiography device 102. In the example shown in FIG. 2, a marker 14a is provided at the tip of the catheter sheath 11a, for example, at the guidewire insertion portion 14, and a marker 12c is provided on the shaft 13 side of the sensor unit 12. When the diagnostic imaging catheter 1 configured in this manner is imaged with X-rays, an angio image is obtained in which the markers 14a and 12c are visualized. The positions at which the markers 14a and 12c are provided are merely examples; the marker 12c may be provided on the shaft 13 instead of the sensor unit 12, and the marker 14a may be provided at a location other than the tip of the catheter sheath 11a.

[0024] 5 is a block diagram showing an example of the configuration of the image processing device 3. The image processing device 3 is a computer (information processing device) and includes a control unit 31, a main memory unit 32, an input / output I / F 33, an auxiliary memory unit , and a reading unit .

[0025] The control unit 31 is configured using one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), a tensor processing unit (TPU), etc. The control unit 31 is connected to each hardware component constituting the image processing device 3 via a bus.

[0026] The main memory unit 32 is a temporary storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and temporarily stores data required for the control unit 31 to execute arithmetic processing.

[0027] The input / output I / F 33 is an interface to which the intravascular inspection device 101, the angiography device 102, the display device 4, and the input device 5 are connected. The control unit 31 acquires IVUS images and OCT images from the intravascular inspection device 101 and acquires angio images from the angiography device 102 via the input / output I / F 33. The control unit 31 also displays medical images on the display device 4 by outputting medical image signals of the IVUS images, OCT images, or angio images to the display device 4 via the input / output I / F 33. Furthermore, the control unit 31 receives information input to the input device 5 via the input / output I / F 33.

[0028] A communication unit including a wireless terminal such as 4G, 5G, or WiFi is connected to the input / output I / F 33, and the image processing device 3 may be communicably connected to an external server, such as a cloud server, connected to an external network such as the Internet via the communication unit. The control unit 31 may access the external server via the communication unit and the external network, refer to medical data, research paper information, etc. stored (stored) in a storage device included in the external server, and perform processing related to information provision (provision processing for providing support information). Alternatively, the control unit 31 may cooperate with the external server to perform the processing in this embodiment, for example, by performing inter-process communication.

[0029] The auxiliary storage unit 34 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The auxiliary storage unit 34 stores a computer program P (program product) executed by the control unit 31 and various data required for processing by the control unit 31. The auxiliary storage unit 34 may be an external storage device connected to the image processing device 3. The computer program P (program product) may be written to the auxiliary storage unit 34 during the manufacturing stage of the image processing device 3, or may be distributed by a remote server device and acquired by the image processing device 3 via communication and stored in the auxiliary storage unit 34. The computer program P (program product) may be readably recorded on a recording medium 30 such as a magnetic disk, optical disk, or semiconductor memory, or may be read from the recording medium 30 by the reading unit 35 and stored in the auxiliary storage unit 34.

[0030] The image processing device 3 may be a multi-computer including multiple computers. The image processing device 3 may also be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the image processing device 3 is described as being a single computer. In this embodiment, an angiography device 102 that captures two-dimensional angioimages is connected to the image processing device 3, but the device is not limited to the angiography device 102 as long as it captures images of a patient's hollow organs and the diagnostic imaging catheter 1 from multiple directions outside the body.

[0031] In the image processing device 3 of this embodiment, the control unit 31 reads and executes a computer program P stored in the auxiliary storage unit 34 to construct an IVUS image based on the signal data set received from the IVUS sensor 12a and an OCT image based on the signal data set received from the OCT sensor 12b. Note that, as will be described later, the IVUS sensor 12a and the OCT sensor 12b have different observation positions at the same imaging timing. Therefore, the control unit 31 corrects the difference in the observation positions in the IVUS image and the OCT image. Therefore, the image processing device 3 of this embodiment provides IVUS images and OCT images with the same observation positions, thereby providing images that are easy to interpret.

[0032] In this embodiment, the diagnostic imaging catheter is described as a dual-type catheter having both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions, but is not limited to this. The diagnostic imaging catheter may also be a single-type catheter having either intravascular ultrasound (IVUS) or optical coherence tomography (OCT) functions. Hereinafter, in this embodiment, the diagnostic imaging catheter has an intravascular ultrasound (IVUS) function, and the description will be based on IVUS images generated by the IVUS function. However, in the description of this embodiment, the medical image is not limited to an IVUS image, and the processing of this embodiment may also be performed using an OCT image as the medical image.

[0033] 6 is an explanatory diagram showing an example of the learning model 341. The learning model 341 is a neural network that performs, for example, object detection, semantic segmentation, or instance segmentation. Based on each IVUS image in the input IVUS image group, the learning model 341 outputs whether or not an object such as a stent or plaque is included in the IVUS image (presence / absence), and if an object is included (presence), the type (class) of the object, the area in the IVUS image, and the estimation accuracy (score).

[0034] The learning model 341 is configured, for example, by a convolutional neural network (CNN) trained by deep learning. The learning model 341 includes an input layer 341a to which medical images such as IVUS images are input, an intermediate layer 341b that extracts image features, and an output layer 341c that outputs information indicating the position and type of objects included in the medical image. The input layer 341a of the learning model 341 includes multiple neurons that receive input of pixel values ​​of each pixel included in the medical image and passes the input pixel values ​​to the intermediate layer 341b. The intermediate layer 341b includes an alternating convolutional layer that convolves the pixel values ​​of each pixel input to the input layer 341a and a pooling layer that maps the pixel values ​​convolved in the convolutional layer, and extracts image features while compressing the pixel information of the medical image. The intermediate layer 341b passes the extracted features to the output layer 341c. The output layer 341c has one or more neurons that output the position, range, and type of an image region of an object contained in an image. Although the learning model 341 is assumed to be a CNN, the configuration of the learning model 341 is not limited to a CNN. The learning model 341 may be a trained model configured, for example, as a neural network other than a CNN, an SVM (Support Vector Machine), a Bayesian network, or a regression tree. Alternatively, the learning model 341 may input image features output from the intermediate layer into an SVM (Support Vector Machine) to perform object recognition.

[0035] The learning model 341 can be generated by preparing training data in which medical images containing objects such as epicardium, side branches, veins, guidewires, stents, plaques that have deviated into the stent, lipid plaques, fibrous plaques, calcifications, vascular dissections, thrombi, and hematomas are associated with labels indicating the position (region) and type of each object, and then using the training data to train an untrained neural network. According to the learning model 341 configured in this manner, by inputting a medical image such as an IVUS image into the learning model 341, information indicating the position and type of objects contained in the medical image can be obtained. If the medical image does not contain an object, information indicating the position and type is not output from the learning model 341. Therefore, by using the learning model 341, the control unit 31 can determine whether or not an object is contained in the medical image input to the learning model 341, and if so, obtain the type (class), position (region in the medical image), and estimation accuracy (score) of the object.

[0036] The control unit 31 derives object information regarding the presence or absence and type of object included in the IVUS image based on the information acquired from the learning model 341. Alternatively, the control unit 31 may derive the object information by using the information acquired from the learning model 341 itself as object information.

[0037] 7 is an explanatory diagram showing an example of an association table. Object types and support information are associated with each other and stored as, for example, a table-format association table in the auxiliary storage unit 34 of the image processing device 3. The association table includes, for example, object types, presence / absence determination, and support information (launched applications) as management items (fields) of the table.

[0038] The object type management item (field) stores object types (names), such as stent, calcification, plaque, vascular dissection, and bypass surgery scars. The presence / absence determination management item (field) stores the presence / absence of each object type. The support information (launched application) management item (field) stores the content of support information according to the presence / absence of the object type stored in the same record, or the name of the application (name of the launched application) for providing the support information.

[0039] The control unit 31 can efficiently determine support information (launch application) according to the object information by comparing the association table stored in the storage unit with the object information derived using the learning model 341. For example, when the object information relates to a stent, if the object information indicates the presence of a stent, the control unit 31 performs a provision process (executes the endpoint determination application) to provide support information regarding endpoint determination. If the object information indicates the absence of a stent, the control unit 31 performs a provision process (executes the stent placement application) to provide support information regarding stent placement.

[0040] 8 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 executes the following processing based on input data and the like output from the input device 5 in response to operations by an operator of the diagnostic imaging catheter 1, such as a doctor.

[0041] The control unit 31 acquires IVUS images (S11). The control unit 31 reads the IVUS image group obtained by pulling back, and acquires a medical image made up of these IVUS images.

[0042] The control unit 31 derives object information regarding the presence or absence and type of object included in the IVUS image (S12). For example, the control unit 31 inputs the acquired IVUS image group into the learning model 341 and derives the object information based on the presence or absence and type of object estimated by the learning model 341. The learning model 341 is configured, for example, by a neural network that performs object detection, semantic segmentation, or instance segmentation, and based on each IVUS image in the input IVUS image group, the learning model 341 outputs whether or not an object such as a stent or plaque is included in the IVUS image (presence or absence), and if an object is included (presence), the type (class) of the object, the area in the IVUS image, and the estimation accuracy (score).

[0043] The control unit 31 derives object information in the IVUS image based on the estimation results (presence and type of object) output from the learning model 341. As a result, the object information includes the presence and type of object included in the IVUS image, which is the original data of the object information. The object information may be generated, for example, as an XML file, in which the presence or absence of each object type is added (tagged) to all object types to be estimated by the learning model 341. As a result, the control unit 31 can determine, for example, whether a stent is included in the IVUS image (presence or absence), i.e., whether a stent is placed in a blood vessel. In the present embodiment, the control unit 31 derives object information in the IVUS image using the learning model 341. However, the present invention is not limited to this. The control unit 31 may determine the presence and type of object included in the IVUS image using image analysis means, such as edge detection and pattern matching, and derive object information based on the determination result.

[0044] The control unit 31 receives input regarding situation assessment by the operator (S13). The control unit 31 receives input regarding situation assessment of the progress of surgery, the condition of the patient, etc. by the doctor or other operator of the diagnostic imaging catheter 1. The control unit 31 determines support information to be provided based on the object information, etc., and performs a process to provide the support information (S14). The control unit 31 determines support information to be provided based on the derived object information and the received information regarding situation assessment, and performs a process to provide the support information.

[0045] For example, if the input situation judgment is related to a stent, the control unit 31 determines the presence or absence of the stent, i.e., whether the stent is indwelling before or after placement, in the object information derived based on the IVUS image, and decides to provide support information according to the determination result. The provision of the support information includes a provision form in which the support information itself is displayed by superimposing it on the screen of the display device 4, and execution of an application (launch app) that performs calculation processing etc. for generating and presenting the support information.

[0046] Before a stent is placed, the control unit 31 determines support information related to stent placement as the support information to be provided, and launches, for example, an application (stent placement APP) for assisting in determining a stent size and predicting complications. After a stent is placed, the control unit 31 determines support information related to endpoint determination as the support information to be provided, and launches, for example, an application (endpoint determination APP) for assisting in determining an endpoint and predicting complications. The control unit 31 may refer to an association table stored in the auxiliary storage unit 34 and determine the support information (launch app) depending on whether or not each type of object included in the object information is present.

[0047] In the illustrations of this embodiment, a process flow relating to the provision of support information (launched application) for each case in which the presence or absence and type of object indicated in the object information indicate the presence or absence of a stent (after placement, before placement) is described as an example. This flow is merely an example, and the control unit 31 performs a provision process (execution of the launched application) for providing predefined support information depending on the presence or absence of each type of object exemplified in the association table, such as the presence or absence of calcification, the presence or absence of plaque, the presence or absence of vascular dissection, the presence or absence of bypass surgery scars, etc. In performing this provision process (execution of the launched application) for providing support information depending on the presence or absence and type of object indicated in the object information, a program executed by the control unit 31 may perform branching processing depending on the presence or absence and type of object, for example, by using case statements or the like.

[0048] In the example of the association table, the support information (launched application) defined according to the presence or absence of each type of object is not limited to a single one, and multiple pieces of support information (launched applications) may be defined. In this case, the provision process (execution of the launched application) may be performed for all of the multiple pieces of support information (launched applications), or the names of the multiple pieces of support information (launched applications) may be displayed in a list format on the display device 4, and by accepting the selection of any piece of support information (launched application), the provision process (execution of the launched application) for the selected piece of support information (launched application) may be performed.

[0049] In the present embodiment, the control unit 31 determines the support information based on the object information and the information related to the situation judgment, but this is not limited thereto, and the control unit 31 may determine the support information based only on the object information. In other words, the process of receiving input related to the situation judgment by the operator is not necessary, and the control unit 31 may determine the support information to be provided based only on the object information derived from the IVUS image, and perform the provision process, such as starting an application for providing the support information.

[0050] 9 is a flowchart showing the procedure for providing information on stent placement. In this flow, the provision process (launch application: stent placement APP) for providing support information on stent placement will be described with reference to FIG.

[0051] The control unit 31 acquires an IVUS image before stent placement (S101). The control unit 31 acquires multiple IVUS images before tent placement, which constitute one pullback. The IVUS image may be the IVUS image used when deriving object information. If multiple IVUS images (IVUS image group) are used when deriving object information, any of the IVUS images included in the IVUS image group may be acquired.

[0052] The control unit 31 calculates the plaque burden (S102). The control unit 31 calculates the plaque burden by segmenting the lumen and blood vessels from the acquired IVUS image using, for example, the learning model 341. By segmenting the lumen and blood vessels, their areas (cross-sectional areas in a tomogram) may be calculated, and the plaque burden or the area of ​​the plaque may be calculated by dividing or subtracting the area of ​​the blood vessel from the area of ​​the lumen.

[0053] The control unit 31 determines whether the plaque burden is equal to or greater than a predetermined threshold (S103). By determining whether the plaque burden is equal to or greater than a predetermined threshold, the control unit 31 classifies the plaque burden based on the threshold. The control unit 31 classifies the calculated plaque burden area based on a predetermined threshold, such as 40%, 50%, or 60%. The threshold may be configured to allow multiple settings.

[0054] If the value is equal to or greater than the predetermined threshold (S103: YES), the control unit 31 groups frames (IVUS images) equal to or greater than the threshold (S104). The control unit 31 groups frames (IVUS images) equal to or greater than the plaque burden threshold as lesions. If the lesions are scattered and spaced apart, they may be grouped individually (L1, L2, L3, ...). However, if the spacing (separation distance) between groups is 0.1 to 3 mm or less, they may be grouped together.

[0055] The control unit 31 identifies the group including the maximum value of plaque burden as the lesion (S105). The control unit 31 identifies the group including the maximum value of plaque burden, that is, the site where the lumen diameter is the smallest, as the lesion.

[0056] If the number is not equal to or greater than the predetermined threshold (S103: NO), i.e., if the number is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold (S1031). If the number is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold as references. If the plaque burden that serves as the reference is scattered and spaced apart, they may be grouped together (R1, R2, R3, ...). However, if the spacing (separation distance) between the groups is 0.1 to 3 mm or less, they may be grouped together.

[0057] The control unit 31 identifies each group on the distal side and proximal side of the lesion as a reference portion (S106). For example, the control unit 31 classifies all IVUS images according to the determination result of whether or not the plaque burden is equal to or greater than a threshold, and then identifies each group on the distal side and proximal side of the lesion as a reference portion. Of the multiple grouped reference portions, the control unit 31 identifies each group located on the distal side and proximal side of the identified lesion as a reference portion to be compared with the lesion.

[0058] FIG. 10 is an explanatory diagram showing an example of displaying information related to the identification of a reference portion. In this example, a graph of the average lumen diameter and a graph of the plaque burden (PB) are displayed one above the other. The horizontal axis represents the length (axial length) of the blood vessel. When the plaque burden (PB) threshold is 50%, a region exceeding the threshold is identified as a lesion. The distal and proximal regions within 10 mm of the lesion, where the average lumen diameter is largest, are identified as the distal and proximal reference regions, respectively. Displaying such information can provide assistance (support) to the operator in identifying the reference portion. As shown in the figure in this embodiment, the lesion may be, for example, a region where the plaque burden (PB) is 50% or greater, and may be a group of 3 mm or more in length. The reference portion may be the region with the largest average lumen diameter within 10 mm before and after the lesion. When a large side branch is present in the blood vessel and the blood vessel diameter changes significantly, a reference portion may be identified between the lesion and the side branch. In identifying the reference portion, the image shown in the figure may be displayed on the display device 4 and may be modified by the operator. Furthermore, when displaying the image on the display device 4, the location of the large side branch may be indicated.

[0059] The control unit 31 calculates the blood vessel diameter, lumen diameter, and area of ​​the distal and proximal reference portions (S107). The control unit 31 calculates the blood vessel diameter (EEM), lumen diameter, and area of ​​the distal and proximal reference portions. In this case, the length between the reference portions, i.e., the length from the distal reference portion to the proximal reference portion, may be set to, for example, a maximum of 10 mm.

[0060] The control unit 31 displays the support information (S108). As shown in the figure in this embodiment, the control unit 31 outputs support information related to stent placement to the display device 4 and displays the support information on the display device 4. FIG. 11 is an explanatory diagram showing an example of the display of information related to stent placement. In this display example, a transverse tomogram, which is a cross-sectional view of the blood vessel in the axial direction, and a longitudinal tomogram, which is a cross-sectional view of the blood vessel in the radial direction, are displayed side by side. That is, the support information related to stent placement includes multiple longitudinal tomograms (radial cross-sectional areas of the blood vessel) based on IVUS images and a transverse tomogram (axial cross-sectional area of ​​the blood vessel) obtained by connecting these longitudinal tomograms. In the transverse tomogram, a distal reference portion (Ref. Distal) and a proximal reference portion (Ref. Proximal) are shown, and the MLA (minimum lumen area) located between these reference portions is also shown. Displaying such information allows the operator to receive support (assistance) regarding stent placement.

[0061] Fig. 12 is a flowchart showing the procedure for providing information on endpoint determination. Fig. 13 is a flowchart showing the procedure for calculating the MSA. In this flow, the provision process (launched application: endpoint determination APP) for providing support information on endpoint determination will be described with reference to Fig. 12.

[0062] The control unit 31 acquires an IVUS image after stent placement (S111). The control unit 31 acquires multiple IVUS images after tent placement, which constitute one pullback. The control unit 31 determines the presence or absence of a stent for each of the acquired multiple IVUS images (S112). The control unit 31 determines the presence or absence of a stent for the multiple IVUS images (one pullback), for example, by using a learning model 341 having an object detection function or image analysis processing such as edge detection and pattern patching.

[0063] If a stent is not present (S112: YES), the control unit 31 performs segmentation of the lumen and blood vessels for the IVUS image without a stent (S113). The control unit 31 performs segmentation of the lumen and blood vessels for the IVUS image without a stent, for example, using a learning model 341 having a segmentation function. The control unit 31 calculates a representative value of the diameter or area of ​​the blood vessel or lumen (S114). The control unit 31 calculates the representative value of the diameter or area based on the segmented lumen and blood vessel.

[0064] If a stent is present (S112: NO), the control unit 31 performs stent segmentation on the IVUS image with the stent present (S115). The control unit 31 performs stent segmentation on the IVUS image with the stent present, for example, using a learning model 341 having a segmentation function. The control unit 31 calculates a representative value of the diameter or area of ​​the stent lumen (S116). The control unit 31 calculates the representative value of the diameter or area of ​​the stent lumen based on the segmented stent.

[0065] The control unit 31 derives the expansion state near the stent placement site (S117). The control unit 31 derives the expansion state near the stent placement site based on the calculated representative value of the diameter or area of ​​the blood vessel or lumen and the representative value of the diameter or area of ​​the stent lumen, and visualizes the expansion state by outputting and displaying it on the display device 4. As illustrated in the present embodiment, the expansion state near the stent placement site displayed (visualized) on the display device 4 may be, for example, displayed by coloring the area where the stent is provided in a cross-sectional view.

[0066] FIG. 14 is an explanatory diagram showing a display example that visualizes the expansion state near the stent placement site. In this display example, graphs of the diameters and areas of the vessel, lumen, and stent when the stent is placed are displayed one above the other. The horizontal axis indicates the length of the vessel (axial length). The position of the MAS is indicated in these graphs. Displaying such information provides support (assistance) to the operator in understanding the expansion state near the stent placement site. In deriving the expansion state near the stent placement site, the control unit 31 calculates the MSA (Minimum Stent Area) at the stent placement site. The processing procedure for calculating the MSA will be described later.

[0067] The control unit 31 derives the planned expansion diameter (S118). For example, the control unit 31 refers to a preliminary plan stored in advance in the auxiliary storage unit 34, and derives the planned expansion diameter to be set as desired based on the diameter at the time of stent expansion included in the preliminary plan. The control unit 31 may receive input from the operator when deriving the planned expansion diameter. The control unit 31 may display a graph showing the derived desired (planned) expansion diameter superimposed on an image showing the expansion state near the stent placement site.

[0068] The control unit 31 derives the expansion diameter based on the evidence information (S119). The control unit 31 derives the desirable (requested) expansion diameter by referring to evidence information such as research paper information pre-stored in the auxiliary storage unit 34, for example. The control unit 31 may accept input of a self-directed index by the operator when deriving the desirable (requested) expansion diameter. The control unit 31 may display a graph showing the derived desirable (requested) expansion diameter superimposed on an image showing the expansion state near the stent placement site.

[0069] The control unit 31 presents information according to the calculated expansion diameter (S120). When the calculated expansion diameter is equal to or smaller than the desired diameter or area, the control unit 31 presents the information in different display modes, for example, by changing the display color, depending on whether the diameter or area is equal to or smaller than the desired diameter or area or exceeds the desired diameter or area. FIG. 15 is an explanatory diagram showing an example of the display of information related to the desired expansion diameter. In this example, a graph of the desired stent diameter and area is displayed above and below the blood vessel in which the stent has been placed. The horizontal axis represents the length of the blood vessel (axial length). Displaying such information can provide support (assistance) to the operator in grasping the desired stent diameter and area.

[0070] The control unit 31 receives a determination by the operator as to whether post-dilation is necessary (S121). The control unit 31 receives input regarding the determination by the operator as to whether post-dilation is necessary. The control unit 31 derives a recommended inflation pressure based on the inflation diameter during post-dilation (S122). The control unit 31 derives the recommended inflation pressure by, for example, referring to a compliance chart stored in the auxiliary storage unit 34 and identifying the recommended inflation pressure included in the compliance chart based on the inflation diameter during post-dilation. The control unit 31 displays the derived recommended inflation pressure, for example, by superimposing it on an image showing the inflation state near the stent placement site. FIG. 16 is an explanatory diagram showing an example display of information related to endpoint determination. In this display example, a cross-sectional view, which is a cross-sectional view of the blood vessel in the axial direction, and a longitudinal view, which is a cross-sectional view of the blood vessel in the radial direction, are displayed one above the other. The cross-sectional view indicates the position of the MAS. Displaying such information can provide support (assistance) to the operator regarding endpoint determination.

[0071] The processing procedure for calculating the MSA (Minimum Stent Area) of the stent placement site will be described with reference to Fig. 13. The processing procedure for calculating the MSA may be performed as a subroutine process in the process of S117 (process for deriving the expansion state near the stent placement site), for example.

[0072] The control unit 31 acquires an IVUS image (M001). The control unit 31 acquires the IVUS image by reading one pullback. The control unit 31 determines the presence or absence of a stent (M002). The control unit 31 determines the presence or absence of a stent in each frame (IVUS image), and stores the processing results for each frame (IVUS image) in, for example, an array (array-type variable).

[0073] The control unit 31 accepts a correction regarding the presence or absence of a stent (M003). The control unit 31 accepts a correction regarding the presence or absence of a stent based on, for example, an operation by an operator. The control unit 31 calculates the stent area (M004). The control unit 31 calculates (identifies) the stent area by processing each frame (IVUS image) that includes a stent. In performing this processing, the control unit 31 may calculate the lumen diameter and area using a learning model 341 that has a segmentation function. Alternatively, the control unit 31 may calculate the minor axis and major axis of the lumen diameter and derive the eccentricity (minor axis / major axis) by dividing the minor axis by the major axis.

[0074] The control unit 31 calculates the MSA (M005). The control unit 31 calculates the MSA based on the calculated lumen diameter, area, and eccentricity in the identified stent region. The control unit 31 determines the risk of stent thrombosis (M006). The control unit 31 may function, for example, as an MSA determiner, determining whether the MSA is greater than 5.5 square mm (MSA>5.5 [mm^2]), and if it is greater than 5.5 square mm, determining that there is no risk of stent thrombosis.

[0075] According to this embodiment, the image processing device 3 derives object information regarding the presence or absence and type of object included in a medical image, such as an IVUS image, acquired using the diagnostic imaging catheter 1. The image processing device 3 performs a provision process to provide support information to an operator of the diagnostic imaging catheter 1, such as a doctor, based on the derived object information, and is therefore able to provide the operator with appropriate support information according to the presence or absence and type of object included in the medical image generated using the diagnostic imaging catheter 1.

[0076] According to this embodiment, the image processing device 3 inputs a medical image into the learning model 341, and derives object information using the type of object estimated by the learning model 341. The learning model 341 is trained to estimate an object contained in the medical image by inputting the medical image, and therefore the image processing device 3 can efficiently obtain whether or not an object is contained in the medical image, and if an object is contained, the type of the object.

[0077] According to this embodiment, the types of objects identified as being contained in a medical image include at least one of the epicardium, side branches, veins, guide wires, stents, plaques that have deviated into the stent, lipid plaques, fibrous plaques, calcifications, vascular dissections, thrombi, and hematomas, and therefore appropriate support information can be provided to the operator depending on the objects that may be areas of interest such as lesions in tubular organs.

[0078] According to this embodiment, the support information provision process performed in accordance with object information includes a process for providing support information regarding stent placement and endpoint determination. Therefore, if the type of object in question is a stent, appropriate support information depending on whether or not the stent is present can be provided to the operator.

[0079] (Embodiment 2) Fig. 17 is an explanatory diagram showing an example of an association table in embodiment 2. Fig. 18 is an explanatory diagram showing an example of a combination table in embodiment 2. The association table in embodiment 2 includes, as in embodiment 1, management items (fields) of the table, for example, object type and presence / absence determination, and further includes a determination flag value.

[0080] The object type management item (field) stores the type (name) of an object such as a stent, as in embodiment 1. The presence / absence determination management item (field) stores the presence / absence of each object type, as in embodiment 1.

[0081] The management item (field) for the determination flag value stores a determination flag value corresponding to the presence or absence of an object type stored in the same record. As an example, the determination flag value includes a type flag indicating the object type and a presence / absence flag indicating the presence or absence of the object, and is composed of a value obtained by concatenating the type flag and the presence / absence flag. In this embodiment, letters such as A (stent) and B (calcification) indicate the type of object (type flag), and numbers such as 1 (present) and 0 (absent) indicate the presence or absence of the object (presence / absence flag). By using such determination flag values, it is possible to uniquely determine a value indicating the presence or absence of each object type.

[0082] The combination table includes, as management items (fields) of the table, for example, a combination code, support information (launched applications), and the number of pieces of support information. The management item (field) of the combination code stores, for example, information (combination code) indicating a combination of determination flag values ​​indicated in the related table.

[0083] The combination code stores a character string in which determination flag values ​​indicating presence (1) or absence (0) of each object type are concatenated. For example, a combination code of A0:B0:C0:D0:E0 indicates that all objects indicated by A to E are absent in the IVUS image. A combination code of A1:B0:C0:D0:E0 indicates that only object A (stent) is present. A combination code of A1:B1:C0:D0:E0 indicates that only object A (stent) and object B (calcification) are present. In this way, even when an IVUS image contains multiple types of objects, the combination code can be used to uniquely determine a value indicating the combination of presence / absence for each object type.

[0084] The support information (launched application) management item (field) stores, for example, the content of the support information corresponding to the combination code stored in the same record, or the name of the application (launched application) for providing the support information. The number of stored support information (launched applications) is not limited to one, and may be two or more. Alternatively, depending on the combination code, information indicating that no support information (launched application) is stored may be stored. For example, if the combination code is A0:B0:C0:D0:E0, the IVUS image does not include any type of object, and the blood vessels shown in the IVUS image can be said to be healthy, and the control unit 31 may not perform processing to provide support information (launched application). For example, if the combination code is A0:B0:C0:D1:E1, it indicates that the IVUS image includes multiple objects, and multiple pieces of support information (launched applications) corresponding to these multiple objects may be stored.

[0085] The control unit 31 may perform a provision process (execution of the launched application) for all of the plurality of pieces of support information (launched applications). Alternatively, the control unit 31 may accept selection of any one piece of support information (launched application) from the plurality of pieces of support information (launched applications) and perform a provision process (execution of the launched application) for the selected support information (launched application). For example, the control unit 31 derives object information in accordance with the format of the combination code, and compares the derived object information with a combination table to efficiently determine the support information (launched application).

[0086] The management item (field) for the number of pieces of support information stores, for example, the number (number of types) of support information (launched applications) stored in the same record. The control unit 31 may change the display mode when performing the process of providing support information (launched applications) depending on the number stored in the management item (field) for the number of pieces of support information.

[0087] 19 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 executes the following processing based on input data and the like output from the input device 5 in response to operations by an operator of the diagnostic imaging catheter 1, such as a doctor.

[0088] The control unit 31 acquires an IVUS image (S21). The control unit 31 derives object information regarding the presence or absence and type of object included in the IVUS image (S22). The control unit 31 performs the processes from S21 to S22 in the same manner as S11 to S12 in the first embodiment.

[0089] The control unit 31 determines the support information to be provided based on the object information, etc. (S23). The control unit 31 may derive the object information based on the presence or absence of all types of objects defined in the association table, for example, by referring to the association table stored in the auxiliary storage unit 34. The learning model 341 has already learned about all types of objects, and by inputting IVUS images into the learning model 341, the control unit 31 can obtain the presence or absence of all types of objects defined in the association table. The control unit 31 determines the support information to be provided by comparing the object information derived in this manner with, for example, a combination table stored in the auxiliary storage unit 34, thereby specifying the number of pieces of support information.

[0090] The control unit 31 determines whether there are multiple types of support information to be provided (S24). The control unit 31 determines whether there are multiple types of support information determined according to the object information, for example, by referring to a combination table stored in the auxiliary storage unit 34.

[0091] If there are multiple types of support information to be provided (S24: YES), the control unit 31 causes the display device 4 to display the names of the multiple types of support information (S25). The control unit 31 accepts the selection of any one of the support information (S26). The control unit 31 causes the display device 4 to display the names of the multiple types of support information (launched applications), for example in a list format, and accepts the selection of any one of the support information (launched applications) by the user in response to the user's operation using the touch panel function of the display device 4 or the input device 5.

[0092] The control unit 31 performs a process of providing support information (S27). After the process of S26, or if there are not multiple types of support information to be provided (S24: NO), the control unit 31 performs a process of providing support information. If the process of S26 has been performed, the control unit 31 performs a process of providing support information selected in the process of S26 (executing the launched app). If there are not multiple types of support information to be provided (S24: NO), that is, if there is only one type of support information to be provided, the control unit 31 performs a process of providing support information determined in the process of S23 (executing the launched app). The control unit 31 performs a process of providing support information (executing the launched app) such as a stent placement app or an endpoint determination app for the selected or determined support information (launched app) in the same manner as in the first embodiment.

[0093] According to this embodiment, an association table associating object types with support information is stored in a predetermined storage area accessible by the control unit 31 of the image processing device 3, such as a storage unit. Therefore, the control unit 31 can efficiently determine support information according to the type of object by referring to the association table stored in the storage unit. The association table includes not only support information according to the presence or absence of a specific type of object, but also support information according to a combination of the presence or absence of each of multiple types of objects. Therefore, appropriate support information can be provided to the operator not only according to the presence or absence of a single type of object, but also according to a combination of the presence or absence of each of multiple types of objects.

[0094] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. [Explanation of symbols]

[0095] 1 Diagnostic imaging catheter 11 Probe 11a Catheter sheath 12 Sensor section 12a Ultrasonic transmitter / receiver 12a IVUS sensor 12b Optical transmitter / receiver 12b OCT sensor 12c marker 12d Housing 13 Shaft 14 Guidewire insertion part 14a Marker 15 Connector part 2 MDU 3. Image processing device (information processing device) 30 Recording media 31 Control Unit 32 Main memory 33 Input / Output Interface 34 Auxiliary storage 341 Learning Model 35 Reading unit 4 Display device 5 Input Devices 100 Diagnostic imaging equipment 101 Intravascular inspection device 102 Angiography equipment

Claims

1. On the computer, A medical image is acquired based on a signal detected by a catheter inserted into a hollow organ. Deriving object information regarding the types of objects included in the acquired medical image; providing support information to an operator of the catheter based on the derived object information; determining an application to be launched based on the derived object information; Executing the determined application to provide support information according to the object information; determining support information corresponding to the derived object information by referring to an association table in which types of objects included in the medical image are associated with support information; The association table stores names of applications for providing support information, The application to be launched is determined by referring to the association table. A computer program for executing a process.

2. Identifying the type of object included in a medical image by inputting the acquired medical image into a learning model that estimates objects included in the medical image by inputting the medical image.

2. The computer program of claim 1.

3. The object types include at least one of epicardium, side branch, vein, guidewire, stent, plaque displaced within the stent, lipid plaque, fibrous plaque, calcification, vessel dissection, thrombus, and hematoma.

3. A computer program according to claim 1 or claim 2.

4. If the medical image includes a stent, an application is executed to provide support information for endpoint determination; If the medical image does not contain a stent, run an application that provides assistance with stent placement. A computer program according to any one of claims 1 to 3.

5. The support information for stent placement includes information on the extent of the lesion in the length direction of the blood vessel, the proximal reference portion, and the distal reference portion.

5. A computer program according to claim 4.

6. The endpoint determination support information includes information indicating the expansion state of the stent at a position along the length of the blood vessel.

5. A computer program according to claim 4.

7. The association table associates the combination of the presence or absence of a plurality of objects with the support information. A computer program according to any one of claims 1 to 6.

8. The hollow organ is a blood vessel A computer program according to any one of claims 1 to 7.

9. Accepts input information from the operator regarding the progress of surgery or assessment of the condition of the patient, Providing support information based on the received input information and object information A computer program according to any one of claims 1 to 8.

10. On the computer, A medical image is acquired based on a signal detected by a catheter inserted into a hollow organ. Deriving object information regarding the types of objects included in the acquired medical image; providing support information to an operator of the catheter based on the derived object information; determining an application to be launched based on the derived object information; Executing the determined application to provide support information according to the object information; determining support information corresponding to the derived object information by referring to an association table in which types of objects included in the medical image are associated with support information; The association table stores names of applications for providing support information, The application to be launched is determined by referring to the association table. An information processing method for executing a process.

11. an acquisition unit that acquires a medical image generated based on a signal detected by a catheter inserted into a hollow organ; a derivation unit that derives object information regarding the type of object included in the acquired medical image; a processing unit that provides support information to an operator of the catheter based on the derived object information, The processing unit determining an application to be launched based on the derived object information; By executing the determined application, support information corresponding to the object information is provided; determining support information corresponding to the derived object information by referring to an association table in which types of objects included in the medical image are associated with support information; The association table stores names of applications for providing support information, The application to be launched is determined by referring to the association table. Information processing device.

Images