Computer program, information processing method, and information processing apparatus

Abstract

This invention provides a computer program that provides information about stents contained in medical images. [Solution] The computer program instructs the computer to acquire a medical image generated based on signals detected by a catheter inserted into a tubular organ, to identify the type and region of objects contained in the acquired medical image, to derive information about a stent to be inserted into the tubular organ based on the identified type and region of objects, and to output the derived information about the stent.

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing method, and an information processing apparatus.

Background Art

[0002] Medical images including intravascular ultrasound (IVUS) images of blood vessels are generated by the intravascular ultrasound method using a catheter, and intravascular ultrasound examinations are performed. On the other hand, for the purpose of assisting doctors' diagnoses, technologies for adding information to medical images by image processing and machine learning have been developed (for example, Patent Document 1). The feature detection method in the blood vessel image described in Patent Document 1 detects a lumen wall, a stent, etc. included in the blood vessel image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the detection method of Patent Document 1, when an object included in a blood vessel image is detected, information regarding a stent corresponding to the object is not considered.

[0005] An object of the present disclosure is to provide a computer program or the like that provides information regarding a stent corresponding to an object included in a medical image based on the medical image obtained by scanning a lumen organ with a catheter.

Means for Solving the Problems

[0006] The computer program according to this embodiment causes the computer to acquire a medical image generated based on a signal detected by a catheter inserted into a tubular organ, identify the type and region of an object included in the acquired medical image, derive information about a stent to be inserted into the tubular organ based on the identified type and region of the object, and output the derived information about the stent.

[0007] The information processing method according to this embodiment involves causing a computer to acquire a medical image generated based on a signal detected by a catheter inserted into a tubular organ, to identify the type and region of an object contained in the acquired medical image, to derive information about a stent to be inserted into the tubular organ based on the identified type and region of the object, and to output the derived information about the stent.

[0008] The information processing device according to this embodiment includes an acquisition unit that acquires a medical image generated based on a signal detected by a catheter inserted into a tubular organ; an identification unit that identifies the type and region of an object included in the acquired medical image; an extraction unit that derives information about a stent to be inserted into the tubular organ based on the identified type and region of the object; and an output unit that outputs the extracted information about the stent. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide a computer program, etc., that provides information about stents corresponding to objects contained in medical images obtained by scanning a tubular organ with a catheter. [Brief explanation of the drawing]

[0010] [Figure 1] This is an explanatory diagram showing an example of the configuration of an imaging diagnostic device. [Figure 2] This is an explanatory diagram illustrating the overview of a diagnostic imaging catheter. [Figure 3] This is an explanatory diagram showing a cross-section of a blood vessel through which the sensor part has been inserted. [Figure 4] It is an explanatory diagram for explaining a tomographic image. [Figure 5] It is a block diagram showing a configuration example of an image processing apparatus. [Figure 6] It is an explanatory diagram showing an example of a learning model. [Figure 7] It is a flowchart showing an information processing procedure by a control unit. [Figure 8] It is a flowchart showing a procedure for calculating a stent diameter. [Figure 9] It is an explanatory diagram showing an example of display of information such as an average lumen diameter. [Figure 10] It is an explanatory diagram exemplifying a selection screen for a stent diameter derivation method and the like. [Figure 11] It is a flowchart showing a procedure for calculating a stent length. [Figure 12] It is an explanatory diagram showing an example of display of information such as a Landing Zone. [Figure 13] It is an explanatory diagram showing an example of display of information such as a stent length. [Figure 14] It is an explanatory diagram showing a configuration example of an image diagnostic apparatus and the like according to Embodiment 2. [Figure 15] It is an explanatory diagram for explaining an example of a stent inventory DB. [Figure 16] It is a flowchart showing an information processing procedure by a control unit.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, the image processing method, image processing apparatus, and program of the present disclosure will be described in detail based on the drawings showing their embodiments. In each of the following embodiments, cardiac catheter therapy that is an endovascular treatment will be described as an example, but the luminal organ targeted for catheter therapy is not limited to blood vessels, and may be other luminal organs such as bile ducts, pancreatic ducts, bronchi, intestines, and the like.

[0012] (Embodiment 1) Figure 1 is an explanatory diagram showing an example configuration of the diagnostic imaging device 100. In this embodiment, a diagnostic imaging device using a dual-type catheter equipped with both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions will be described. The dual-type catheter is provided with three modes: a mode in which ultrasound tomography images are acquired using IVUS only, a mode in which optical coherence tomography images are acquired using OCT only, and a mode in which both tomography images are acquired using IVUS and OCT. These modes can be switched between. Hereinafter, ultrasound tomography images and optical coherence tomography images will be referred to as IVUS images and OCT images, respectively. Furthermore, IVUS images and OCT images will be collectively referred to as tomography images and will correspond to medical images.

[0013] The image diagnostic apparatus 100 of this embodiment comprises an intravascular examination device 101, an angiography device 102, an image processing device 3, a display device 4, and an input device 5. The intravascular examination device 101 comprises an image diagnostic catheter 1 and an MDU (Motor Drive Unit) 2. The image diagnostic catheter 1 is connected to the image processing device 3 via the MDU 2. The display device 4 and the input device 5 are connected to the image processing device 3. The display device 4 is, for example, a liquid crystal display or an organic EL display, and the input device 5 is, for example, a keyboard, mouse, trackball, or microphone. The display device 4 and the input device 5 may be stacked together to form a touch panel. The input device 5 and the image processing device 3 may also be configured as a single unit. Furthermore, the input device 5 may be a sensor that accepts gesture input or gaze input, etc.

[0014] The angiography device 102 is connected to the image processing device 3. The angiography device 102 is an angiography device for imaging blood vessels using X-rays from outside the patient's body while injecting a contrast agent into the patient's blood vessels to obtain an angiogram, which is a fluoroscopic image of the blood vessels. The angiography device 102 includes an X-ray source and an X-ray sensor, and the X-ray sensor receives the X-rays irradiated from the X-ray source to image the X-ray fluoroscopic image of the patient. Note that the image diagnostic catheter 1 is provided with a marker that does not transmit X-rays, and the position of the image diagnostic catheter 1 (marker) in the angiogram is visualized. The angiography device 102 outputs the obtained angiogram to the image processing device 3 and is displayed on the display device 4 via the image processing device 3. Note that the display device 4 displays an angiogram and a tomographic image taken using the image diagnostic catheter 1.

[0015] FIG. 2 is an explanatory diagram for explaining the outline of the image diagnostic catheter 1. Note that the upper dashed line region in FIG. 2 is an enlarged view of the lower dashed line region. The image diagnostic catheter 1 includes a probe 11 and a connector portion 15 disposed at the end of the probe 11. The probe 11 is connected to the MDU 2 via the connector portion 15. In the following description, the side far from the connector portion 15 of the image diagnostic catheter 1 is referred to as the distal end side, and the connector portion 15 side is referred to as the proximal end side. The probe 11 includes a catheter sheath 11a, and a guide wire insertion portion 14 through which a guide wire can be inserted is provided at the distal end thereof. The guide wire insertion portion 14 forms a guide wire lumen, receives a guide wire previously inserted into the blood vessel, and is used to guide the probe 11 to the affected area by the guide wire. The catheter sheath 11a forms a continuous tube portion extending from the connection portion with the guide wire insertion portion 14 to the connection portion with the connector portion 15. A shaft 13 is inserted inside the catheter sheath 11a, and a sensor portion 12 is connected to the distal end side of the shaft 13.

[0016] The sensor unit 12 has a housing 12d, and the tip end of the housing 12d is formed in a hemispherical shape to suppress friction and snagging with the inner surface of the catheter sheath 11a. Inside the housing 12d are an ultrasonic transmitting / receiving unit 12a (hereinafter referred to as IVUS sensor 12a) that transmits ultrasonic waves into the blood vessel and receives reflected waves from within the blood vessel, and an optical transmitting / receiving unit 12b (hereinafter referred to as OCT sensor 12b) that transmits near-infrared light into the blood vessel and receives reflected light from within the blood vessel. In the example shown in Figure 2, the IVUS sensor 12a is provided on the tip end of the probe 11, and the OCT sensor 12b is provided on the proximal end, and they are arranged at a distance x along the axial direction on the central axis of the shaft 13 (on the dashed line in Figure 2). In the diagnostic imaging catheter 1, the IVUS sensor 12a and the OCT sensor 12b are mounted so that the direction at which ultrasonic waves or near-infrared light are transmitted and received is approximately 90 degrees with respect to the axial direction of the shaft 13 (the radial direction of the shaft 13). Furthermore, it is desirable that the IVUS sensor 12a and OCT sensor 12b be mounted slightly offset from the radial direction so as not to receive reflected waves or reflected light on the inner surface of the catheter sheath 11a. In this embodiment, for example, as indicated by the arrow in Figure 2, the IVUS sensor 12a is mounted with the direction inclined toward the proximal end with respect to the radial direction as the direction of ultrasonic irradiation, and the OCT sensor 12b is mounted with the direction inclined toward the tip with respect to the radial direction as the direction of near-infrared light irradiation.

[0017] The shaft 13 contains an electrical signal cable (not shown) connected to the IVUS sensor 12a and an optical fiber cable (not shown) connected to the OCT sensor 12b. The probe 11 is inserted into the blood vessel from the tip end. The sensor unit 12 and shaft 13 can move forward and backward inside the catheter sheath 11a and can also rotate circumferentially. The sensor unit 12 and shaft 13 rotate around the central axis of the shaft 13 as the axis of rotation. The imaging diagnostic device 100 uses an imaging core composed of the sensor unit 12 and shaft 13 to measure the condition inside the blood vessel by taking ultrasound images (IVUS images) or optical coherence tomography images (OCT images) taken from inside the blood vessel.

[0018] The MDU2 is a drive device to which the probe 11 (imaging catheter 1) is detachably attached by a connector 15. It controls the operation of the imaging catheter 1 inserted into a blood vessel by driving a built-in motor in response to the operation of a medical professional. For example, the MDU2 performs a pullback operation, rotating the sensor unit 12 and shaft 13 inserted into the probe 11 circumferentially while pulling them toward the MDU2 at a constant speed. The sensor unit 12 rotates while moving from the tip to the proximal end due to the pullback operation, and continuously scans the inside of the blood vessel at predetermined time intervals, thereby continuously acquiring multiple transverse images approximately perpendicular to the probe 11 at predetermined intervals. The MDU2 outputs the reflected ultrasound wave data received by the IVUS sensor 12a and the reflected light data received by the OCT sensor 12b to the image processing device 3.

[0019] The image processing device 3 acquires a signal dataset, which is the reflected ultrasound wave data received by the IVUS sensor 12a via the MDU 2, and a signal dataset, which is the reflected light data received by the OCT sensor 12b. The image processing device 3 generates ultrasound line data from the ultrasound signal dataset and constructs an ultrasound tomographic image (IVUS image) of the transverse layer of the blood vessel based on the generated ultrasound line data. The image processing device 3 also generates optical line data from the reflected light signal dataset and constructs an optical tomographic image (OCT image) of the transverse layer of the blood vessel based on the generated optical line data. Here, the signal datasets acquired by the IVUS sensor 12a and the OCT sensor 12b, and the tomographic images constructed from the signal datasets will be explained. Figure 3 is an explanatory diagram showing a cross-section of a blood vessel through which the sensor unit 12 is inserted, and Figure 4 is an explanatory diagram explaining the tomographic image.

[0020] First, using Figure 3, we will explain the operation of the IVUS sensor 12a and OCT sensor 12b within a blood vessel, and the signal dataset (ultrasound line data and optical line data) acquired by the IVUS sensor 12a and OCT sensor 12b. When tomographic imaging is started with the imaging core inserted into the blood vessel, the imaging core rotates around the central axis of the shaft 13 in the direction indicated by the arrow. At this time, the IVUS sensor 12a transmits and receives ultrasound at each rotation angle. Lines 1, 2, ... 512 indicate the direction of ultrasound transmission and reception at each rotation angle. In this embodiment, the IVUS sensor 12a intermittently transmits and receives ultrasound 512 times while rotating 360 degrees (1 rotation) within the blood vessel. Since the IVUS sensor 12a acquires data for one line in the transmission and reception direction with each ultrasound transmission and reception, 512 ultrasound line data lines extending radially from the center of rotation can be obtained during one rotation. The 512 ultrasound line data are dense near the rotation center, but become sparser as they move away from the rotation center. Therefore, the image processing device 3 can generate a two-dimensional ultrasound tomography image (IVUS image) as shown in Figure 4A by generating pixels in the empty spaces of each line using a well-known interpolation process.

[0021] Similarly, the OCT sensor 12b also transmits and receives measurement light at each rotation angle. Since the OCT sensor 12b also transmits and receives measurement light 512 times while rotating 360 degrees within the blood vessel, 512 optical line data points extending radially from the center of rotation can be obtained during one rotation. With respect to the optical line data, the image processing device 3 can generate a two-dimensional optical coherence tomography (OCT) image similar to the IVUS image shown in Figure 4A by generating pixels in the empty spaces of each line using a well-known interpolation process. That is, the image processing device 3 generates optical line data based on interference light generated by interfering reflected light with, for example, reference light obtained by separating light from a light source within the image processing device 3, and constructs an optical tomography (OCT) image of the transverse layer of the blood vessel based on the generated optical line data.

[0022] The two-dimensional tomographic image generated from these 512 line data points is called one frame of an IVUS image or OCT image. Since the sensor unit 12 scans while moving within the blood vessel, one frame of an IVUS image or OCT image is acquired at each position after one rotation within the movement range. That is, one frame of an IVUS image or OCT image is acquired at each position from the tip to the proximal end of the probe 11 within the movement range, so as shown in Figure 4B, multiple frames of IVUS images or OCT images are acquired within the movement range.

[0023] The diagnostic imaging catheter 1 has X-ray-insensitive markers to confirm the positional relationship between the IVUS image obtained by the IVUS sensor 12a or the OCT image obtained by the OCT sensor 12b and the angiography image obtained by the angiography device 102. In the example shown in Figure 2, marker 14a is provided at the tip of the catheter sheath 11a, for example, at the guidewire insertion section 14, and marker 12c is provided on the shaft 13 side of the sensor section 12. When the diagnostic imaging catheter 1 configured in this way is imaged with X-rays, an angiography image is obtained in which markers 14a and 12c are visualized. The positions of markers 14a and 12c are just examples; marker 12c may be provided on the shaft 13 instead of the sensor section 12, and marker 14a may be provided at a location other than the tip of the catheter sheath 11a.

[0024] Figure 5 is a block diagram showing an example configuration of the image processing device 3. The image processing device 3 is a computer (information processing device) and includes a control unit 31, a main memory unit 32, an input / output interface 33, an auxiliary memory unit 34, and a reading unit 35.

[0025] The control unit 31 is configured using one or more computing devices such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), and TPU (Tensor Processing Unit). The control unit 31 is connected to each hardware component of the image processing device 3 via a bus.

[0026] The main memory unit 32 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and temporarily stores the data necessary for the control unit 31 to perform calculation processing.

[0027] The input / output interface 33 is an interface to which the endovascular examination device 101, the angiography device 102, the display device 4, and the input device 5 are connected. The control unit 31 acquires IVUS images and OCT images from the endovascular examination device 101 and angiography images from the angiography device 102 via the input / output interface 33. The control unit 31 also outputs the medical image signals of the IVUS images, OCT images, or angiography images to the display device 4 via the input / output interface 33, thereby displaying the medical images on the display device 4. Furthermore, the control unit 31 receives information input to the input device 5 via the input / output interface 33.

[0028] A communication unit consisting of, for example, a wireless slave device such as 4G, 5G, or WiFi is connected to the input / output I / F 33, and the image processing device 3 may be connected to an external server such as a cloud server connected to an external network such as the Internet via the communication unit. The control unit 31 may access the external server via the communication unit and the external network, refer to medical data, research paper information, etc. stored (stored) in the storage device included in the external server, and perform processing related to information provision (processing to provide support information). Alternatively, the control unit 31 may perform the processing in this embodiment in cooperation with the external server, for example, by performing inter-process communication.

[0029] The auxiliary storage unit 34 is a storage device such as a hard disk, EEPROM (Electrically Erasable Programmable ROM), or flash memory. The auxiliary storage unit 34 stores the computer program P (program product) executed by the control unit 31 and various data necessary for the processing of the control unit 31. The auxiliary storage unit 34 may also be an external storage device connected to the image processing device 3. The computer program P (program product) may be written to the auxiliary storage unit 34 during the manufacturing stage of the image processing device 3, or it may be distributed by a remote server device and acquired by the image processing device 3 via communication and stored in the auxiliary storage unit 34. The computer program P (program product) may also be recorded in a readable form on a recording medium 30 such as a magnetic disk, optical disk, or semiconductor memory, and the reading unit 35 may read it from the recording medium 30 and store it in the auxiliary storage unit 34.

[0030] The image processing device 3 may be a multicomputer comprising multiple computers. Alternatively, the image processing device 3 may be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the image processing device 3 will be described as a single computer. In this embodiment, an angiography device 102 that acquires two-dimensional angiography images is connected to the image processing device 3, but it is not limited to the angiography device 102 as long as it is a device that images the patient's tubular organs and the diagnostic imaging catheter 1 from multiple directions outside the body.

[0031] In the image processing apparatus 3 of this embodiment, the control unit 31 reads and executes a computer program P stored in the auxiliary storage unit 34 to construct an IVUS image based on the signal dataset received from the IVUS sensor 12a and an OCT image based on the signal dataset received from the OCT sensor 12b. As will be described later, the IVUS sensor 12a and the OCT sensor 12b have a shift in their observation positions at the same imaging timing, so the control unit 31 performs a process to correct the shift in observation positions in the IVUS image and the OCT image. Therefore, the image processing apparatus 3 of this embodiment provides images that are easy to interpret by providing IVUS images and OCT images with aligned observation positions.

[0032] In this embodiment, the imaging catheter is described as a dual-type catheter equipped with both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions, but is not limited to this. The imaging catheter may also be a single-type catheter equipped with either intravascular ultrasound (IVUS) or optical coherence tomography (OCT) functions. Hereafter, in this embodiment, the imaging catheter is assumed to have intravascular ultrasound (IVUS) functionality, and the description will be based on IVUS images generated by this IVUS function. However, in the description of this embodiment, the medical image is not limited to IVUS images, and OCT images may be used as medical images to perform the processing of this embodiment.

[0033] Figure 6 is an explanatory diagram showing an example of the learning model 341. The learning model 341 is a neural network (segmentation NN) such as YOLO or R-CNN that performs object detection, semantic segmentation, or instance segmentation. Based on each IVUS image in the input IVUS image set, the learning model 341 outputs whether or not an object such as a stent or plaque is contained in the IVUS image (presence or absence), and if an object is contained (presence or absence), the type (class) of the object, the region in the IVUS image, and the estimation accuracy (score).

[0034] The learning model 341 is composed of, for example, a pre-trained convolutional neural network (CNN) using deep learning. The learning model 341 has, for example, an input layer 341a into which a medical image such as an IVUS image is input, an intermediate layer 341b that extracts image features, and an output layer 341c that outputs information indicating the position and type of objects contained in the medical image. The input layer 341a of the learning model 341 has multiple neurons that accept input of the pixel value of each pixel contained in the medical image, and passes the input pixel value to the intermediate layer 341b. The intermediate layer 341b has a configuration in which a convolutional layer that convolves the pixel value of each pixel input to the input layer 341a and a pooling layer that maps the pixel value convolved in the convolutional layer are alternately connected, and extracts image features while compressing the pixel information of the medical image. The intermediate layer 341b passes the extracted features to the output layer 341c. The output layer 341c has one or more neurons that output the position, range, and type of the image region of an object contained in the image, for example, as a label image. The label image is, for example, an image where pixels corresponding to the plaque region are of class "1" and pixels corresponding to other images are of class "0". The learning model 341 is assumed to be a CNN, but the configuration of the learning model 341 is not limited to a CNN. The learning model 341 may be a pre-trained model with a configuration such as a neural network other than a CNN, an FCN (fully convolutional network) such as U-net, SegNet, SSD, SPPnet, SVM (Support Vector Machine), Bayesian network, or regression tree. Alternatively, the learning model 341 may perform object recognition by inputting the image features output from the hidden layer into an SVM (support vector machine).

[0035] The learning model 341 can be generated by preparing training data that associates medical images containing objects such as the epicardium, collateral branches, veins, guidewires, stents, plaques prolapsed into stents, lipid plaques, fibrous plaques, calcified areas, vascular dissections, thrombi, and hematomas with labels (label images) indicating the location (region) and type of each object, and then using this training data to machine-learn an untrained neural network. With the learning model 341 configured in this way, by inputting a medical image such as an IVUS image into the learning model 341, information indicating the location and type of objects contained in the medical image can be obtained. If the medical image does not contain objects, the learning model 341 does not output information indicating the location and type. Therefore, by using the learning model 341, the control unit 31 can obtain whether or not objects are contained in the medical image input to the learning model 341, and if so, the type (class), location (region in the medical image), and estimation accuracy (score) of the object. In other words, by using the learned model 341, as shown in the figure in this embodiment, a label image indicating the plaque region on a pixel-by-pixel basis can be obtained by inputting an IVUS image into the learned model 341.

[0036] The control unit 31 may input each IVUS image (frame image) to the learning model 341 one by one for processing, or it may input multiple consecutive frame images simultaneously and detect plaque regions from multiple frame images simultaneously. For example, the control unit 31 uses a 3D-CNN (e.g., 3D U-net) as the learning model 341 to handle 3D input data. The control unit 31 then treats the 2D frame images as 3D data with the coordinates of the 2D frame images as the 2 axes and the time (generation time) t when each frame image was acquired as the 1 axis. The control unit 31 inputs multiple frame images (e.g., 16 frames) for a predetermined unit time as a set to the learning model 341 and simultaneously outputs images with labels indicating the plaque regions for each of the multiple frame images. This makes it possible to detect plaque regions by considering consecutive frame images before and after in the time series, thereby improving detection accuracy.

[0037] Based on this information obtained from the learning model 341, the control unit 31 derives object information relating to the types and regions of objects included in the IVUS image. Alternatively, the control unit 31 may derive the object information by using the information obtained from the learning model 341 itself as object information.

[0038] The control unit 31 functions as a data input unit, a data processing unit, and a data output unit by executing a computer program P stored in the auxiliary storage unit 34. The data input unit acquires all IVUS images generated in a single pullback. The data input unit may also acquire angiography images from the angiography device 102. The IVUS images and angiography images acquired by the data input unit may be displayed on the display device 4.

[0039] The data processing unit includes a learning model 341 with segmentation capabilities, and uses this learning model 341 to segment the lumen and vessels of each acquired IVUS image. Based on the results of this segmentation, the data processing unit identifies the plaque area and calculates the plaque bathen and stenosis rate. The data processing unit further uses the segmentation results to calculate the average lumen diameter and vessel diameter in the tomographic images of the vessels shown in each IVUS image. Based on these calculated plaque bathen, stenosis rate, average lumen diameter, and vessel diameter, the data processing unit identifies a lesion area, for example, that includes a region where the plaque bathen is at its maximum value (average lumen diameter is at its minimum value), and identifies a reference area in the distal and proximal regions of the lesion area. The data processing unit may identify a reference area that includes a region where the plaque bathen is at its minimum value (average lumen diameter is at its maximum value) within a predetermined range, such as 10 mm, from the lesion.

[0040] The data processing unit identifies the area to be covered by the stent (stent cover area) based on the positions of the distal and proximal reference points in the axial direction of the blood vessel that has been pulled back. The data processing unit can execute multiple derivation methods for calculating the stent diameter, such as mean mid-wall reference, and uses one of these derivation methods to calculate the stent diameter based on the lumen diameters of the distal and proximal reference points. The data processing unit then calculates the stent length based on the identified area to be covered by the stent (stent cover area).

[0041] The data output unit outputs various numerical data (output data) calculated by the data processing unit to the display device 4, and displays this information (output data) on the display device 4. This information (output data) may include, for example, IVUS images, plaque burden, mean lumen diameter, lesion area, reference area, stent cover area, stenosis rate, stent diameter, and stent length, and may also include various calculated values ​​(various calculated values) used in calculating this information. Details of the processing related to these functional units will be explained in the flowchart and other documents described later.

[0042] Figure 7 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 performs the following processing based on the input data output from the input device 5 in response to the operation of the operator of the diagnostic imaging catheter 1, such as a physician.

[0043] The control unit 31 acquires IVUS images (S11). The control unit 31 acquires a medical image consisting of these IVUS images by reading the group of IVUS images obtained by pullback (multiple IVUS images that constitute one pullback).

[0044] The control unit 31 calculates the stent diameter (S12). Figure 8 is a flowchart showing the procedure for calculating the stent diameter. The control unit 31 calculates the stent diameter based on the processing in the flowchart.

[0045] The control unit 31 calculates the plaque bathen (S121). The control unit 31, for example, uses a learning model 341 to segment the acquired IVUS image into lumen and vessels and calculates the plaque bathen. Alternatively, the control unit 31 may calculate the area (cross-sectional area in the tomographic diagram) of the lumen and vessels by segmenting them, and then calculate the plaque bathen by dividing the area of ​​the region other than the lumen by the area of ​​the vessels.

[0046] The control unit 31 determines whether the plaque bathen is above a predetermined threshold (S122). By determining whether the plaque bathen is above a predetermined threshold, the control unit 31 classifies the plaque bathen based on that threshold. The control unit 31 classifies all acquired IVUS images based on the calculated plaque bathen, for example, based on a predetermined threshold such as 40%, 50%, or 60%. The threshold may be configured to allow for multiple settings.

[0047] If the plaque bathen is above a predetermined threshold (S122: YES), the control unit 31 groups the frames (IVUS images) that are above the threshold (S123). The control unit 31 groups the frames (IVUS images) in which the plaque bathen is above the threshold as lesions. If the lesions are scattered and spaced apart, they may be grouped individually (L1, L2, L3, ...). However, if the spacing between the groups (spacing distance) is 0.1 to 3 mm or less, they may be grouped together.

[0048] The control unit 31 identifies the group containing the maximum value of the plaque bathen as the lesion (S124). The control unit 31 identifies the group containing the maximum value of the plaque bathen, i.e., the area where the lumen diameter is at its minimum, as the lesion.

[0049] If the value is not above a predetermined threshold (S122:NO), i.e., below the predetermined threshold, the control unit 31 groups the frames (IVUS images) below the threshold (S1221). If the value is below the predetermined threshold, the control unit 31 groups the frames (IVUS images) below the threshold as references. If the plaque bathens that become references are scattered at intervals, they may be grouped individually (R1, R2, R3, ...). However, if the interval (spacing distance) between the groups is 0.1 to 3 mm or less, they may be grouped together.

[0050] The control unit 31 identifies distal and proximal groups as reference units for the lesion (S125). For example, the control unit 31 classifies all IVUS images according to the determination result of whether or not they are above the plaque barden threshold, and then identifies distal and proximal groups as reference units for the lesion. The control unit 31 identifies the groups located distal and proximal to the identified lesion from among the grouped reference units as reference units for comparison with the lesion.

[0051] The control unit 31 calculates the vessel diameter, lumen diameter, and area of ​​the distal and proximal reference sections (S126). The control unit 31 calculates the vessel diameter (EEM), lumen diameter, and area of ​​the distal and proximal reference sections. In this case, the length between the reference sections, i.e., the length from the distal reference section to the proximal reference section, may be set to a maximum of, for example, 10 mm.

[0052] Figure 9 is an explanatory diagram showing an example of how information such as the average lumen diameter is displayed. In this example, a graph of the average lumen diameter and a graph of plaque bathen (PB) are displayed side by side, one above the other. The horizontal axis represents the length of the blood vessel (length in the axial direction). When the threshold for plaque bathen (PB) is 50%, the area exceeding this threshold is identified as a lesion. For this lesion, the areas containing the points where the average lumen diameter is maximum in the distal and proximal areas within 10 mm of each other are identified as the distal and proximal reference areas, respectively. By displaying this information, the operator can be assisted in identifying the reference areas. As shown in the figure in this embodiment, the lesion may be, for example, a portion where the plaque bathen (PB) is 50% or more, and may be a continuous group of, for example, 3 mm or more. The reference area may be the area containing the point with the maximum average lumen diameter within 10 mm before and after the lesion. If there are large side branches within the blood vessel and the vessel diameter changes significantly, a reference point may be identified between the lesion and the side branch. In identifying this reference point, the image shown in the figure may be displayed on the display device 4, and the operator may be allowed to make corrections. Furthermore, when displaying the image on the display device 4, the location of the large side branch may be indicated.

[0053] The control unit 31 may accept modifications to the identified distal or proximal reference unit. For example, on the screen displaying information such as the average lumen diameter described above, the control unit 31 may accept modifications from the operator of the diagnostic imaging catheter (physician, etc.) regarding the position of the identified lesion, distal reference unit, or proximal reference unit in the axial direction of the blood vessel, or the blood vessel diameter (EEM) or lumen diameter at that location. By providing a modification acceptance unit that accepts modifications to identified or derived information on the screen displaying information such as the average lumen diameter, it is possible to provide physicians, etc., with the ability to present and modify the information. When modifications are made to the reference unit (after modification), the control unit 31 recalculates to reflect the modifications, redisplays the information such as the average lumen diameter, and performs the processing described later, such as deriving the stent size.

[0054] The control unit 31 selects one of several derivation methods for deriving the stent diameter (S127). These multiple derivation methods for deriving the stent diameter are included, for example, as submodules, subroutines, or function libraries in the program executed by the control unit 31.

[0055] Figure 10 is an explanatory diagram illustrating a selection screen for stent diameter derivation methods. The multiple derivation methods include, for example, known derivation methods (rules) such as EEL-to-EEL(lesion), Smallest reference EEL, Mean mid-wall reference, Largest reference lumen, Mean reference lumen mm, and Smallest reference lumen. The control unit 31 may output the names (types) of these multiple derivation methods to the display device 4 in a menu format such as a list as shown in this embodiment, and display them on the display device 4 together with function buttons (selection acceptance unit) for accepting selections.

[0056] The control unit 31 calculates (derives) the stent diameter based on the lumen diameter and vessel diameter of the identified distal and proximal reference parts using a selected derivation method, such as Mean mid-wall reference (S128). In this embodiment, the stent diameter is calculated using a selected derivation method (rule), but this is not limited to this. The control unit 31 may calculate the stent diameter using all pre-prepared derivation methods based on the lumen diameter and vessel diameter of the identified reference parts, output the derivation method and the stent diameter for each derivation method in association with each method, and display it on the display device 4 in a menu format, such as a list. This allows for the efficient provision of information to support the decision of which derivation method to select by presenting the stent diameter for each derivation method to the physician or other medical professional.

[0057] The control unit 31 calculates the stent diameter (S13). Figure 11 is a flowchart showing the procedure for calculating the stent length. The control unit 31 calculates the stent length based on the processing in the flowchart.

[0058] The control unit 31 calculates the plaque bathen (S131). The control unit 31 determines whether the plaque bathen is above a predetermined threshold (S132). If it is above the predetermined threshold (S132: YES), the control unit 31 groups the frames (IVUS images) above the threshold (S133). The control unit 31 identifies the group containing the maximum value of the plaque bathen as the lesion (S134). The control unit 31 performs the processing from S131 to S134 in the same way as described above for S121 to S124. The control unit 31 may use the processing results from S121 to S124 as the processing results from S131 to S134. In other words, the control unit 31 may make the processing from S121 to S124 and the processing from S131 to S134 into a common routine.

[0059] If the threshold is not above a predetermined threshold (S132:NO), i.e., below the predetermined threshold, the control unit 31 groups the frames (IVUS images) below the threshold (S1321). If the threshold is below the predetermined threshold, the control unit 31 groups the frames (IVUS images) below the threshold as healthy areas. If the plaque bathens that are healthy areas are scattered and spaced apart, they may be grouped separately (H1, H2, H3, ...). However, if the spacing between the groups (spacing distance) is 0.1 to 3 mm or less, they may be grouped together. The healthy areas and the reference areas identified in processing S1221 and S125 may be the same area.

[0060] The control unit 31 identifies the distal and proximal healthy areas (each H group) as Landing Zones for the identified lesion (S135). The control unit 31 calculates the stent length that can cover the distance between the Landing Zones (S136).

[0061] Figure 12 is an explanatory diagram showing an example of the display of Landing Zone information. In this example, the vertical axis represents plaque burden (PB), and the horizontal axis represents the length of the pulled-back vessel (axial length). When the plaque burden (PB) threshold is 50%, the area exceeding this threshold is identified as the lesion. For the lesion, the areas with the maximum average lumen diameter within, for example, 10 mm on the distal and proximal sides are identified as the distal healthy area and the proximal healthy area, respectively. The Landing Zone is identified based on the distal and proximal healthy areas, and the stent length is calculated based on the distance of the identified Landing Zone.

[0062] The control unit 31 may display the illustrated image in this embodiment on the display device 4 and accept modifications by the operator to the identified distal healthy part or proximal healthy part. If a modification is made to the healthy part (after modification), the control unit 31 recalculates to reflect the modification, redisplays the information, and performs the processing described later, such as deriving the stent length. The control unit 31 may accept selection from a plurality of pre-prepared derivation methods (rules), similar to when deriving the stent diameter, and calculate (derive) the stent length using the selected derivation method (rule).

[0063] The control unit 31 outputs support information such as the stent size (S14). The control unit 31 outputs the support information, including the calculated stent size (stent diameter and stent length), to the display device 4 and displays the support information on the display device 4. In addition to the stent size (stent diameter and stent length), the support information may also include IVUS images (longitudinal and transverse views), plaque bathen, mean lumen diameter, lesion area, reference area, stent cover area, and stenosis rate.

[0064] Figure 13 is an explanatory diagram showing an example of the display of information such as stent length (data display section). This explanatory diagram is an example of the screen display when support information is displayed on the display device 4. In this display example, a transverse view, which is a tomographic view of the blood vessel in the axial direction, and a longitudinal view, which is a tomographic view of the blood vessel in the radial direction, are displayed side by side. That is, support information regarding stent placement includes multiple longitudinal views (radial cross-sectional area of ​​the blood vessel) from IVUS images and a transverse view (axial cross-sectional area of ​​the blood vessel) created by linking these longitudinal views. In the transverse view, the distal reference area (Ref. Distal) and the proximal reference area (Ref. Proximal) are shown, and the lesion area (MLA: minimum lumen area) located between these reference areas is shown. By displaying such information, support (assistance) regarding stent placement can be provided to the operator.

[0065] The control unit 31 may output (display) the calculated stent length by superimposing it on the IVUS image, for example, on a transverse view, as shown in the figure, showing the area covered by the stent (stent cover area: shown as a dotted line). This allows the physician to be provided with the positional relationship between the lesion area (MLA) covered by the stent and the distal and proximal reference areas (Ref. D, Ref. P) when a stent of the calculated length is inserted. Various calculated values ​​such as the stent size (stent diameter and stent length), plaque burden, mean lumen diameter, and stenosis rate may be displayed by superimposing them on the IVUS image (longitudinal view, transverse view) or by annotation.

[0066] In this embodiment, when displaying support information on the display device 4 as shown in the figure, the control unit 31 may display information about multiple candidate stents. The control unit 31 may also display comments for selecting from the multiple candidate stents. The control unit 31 may also display information about the type of stent and the degree of expansion (low pressure / high pressure), etc. When identifying a reference area, the control unit 31 may perform a process to identify a suitable Landing Zone even if there are no areas where the plaque bathen is less than 50%, for example. In this case, the cross-section corresponding to the Landing Zone may be displayed. The control unit 31 may perform a process to determine the position of the stent edge according to the plaque bathen. The control unit 31 may display the size of the selected stent after expansion. The control unit 31 may perform a simulation of placing two stents, and in this case, it may change and display the plaque bathen threshold in the placement area of ​​each stent.

[0067] According to this embodiment, the image processing device 3 identifies (segments) the outer membrane (Vessel) and lumen (Lumen) of a blood vessel as object types using the learning model 341, and calculates the plaque area ratio (plaque bathen) based on the segmented outer membrane (Vessel) and lumen (Lumen) regions of the blood vessel. Based on the plaque bathen of the identified plaque, the image processing device 3 derives information about a stent to be inserted into the blood vessel and applied to the plaque. Since the information about the stent includes stent size, such as the diameter (stent diameter) and length (stent length), useful information can be efficiently provided to physicians and others. In this way, by segmenting IVUS images using the learning model 341 and deriving information such as stent size based on the identified plaque, it is possible to mitigate differences (variations) depending on the subject or operator of the diagnostic imaging catheter and provide physicians and others with appropriate stent information.

[0068] According to this embodiment, the image processing device 3 identifies the lesion area, including the region where the plaque barden is at its maximum value, in all IVUS images generated in a single pullback, and identifies reference areas located distally and proximal to the identified lesion area. Therefore, the diameter and length of the stent (stent size) corresponding to the range including the lesion area where the plaque barden is at its maximum value, in the axial direction of the blood vessel, can be efficiently derived. If the maximum length of the reference area used to determine the stent diameter is constrained to, for example, 10 mm, the reference area used to determine the stent diameter and the reference area used to determine the stent length may be different locations. In this case, healthy areas located distally and proximal to the identified lesion area may be identified separately from the reference area used to determine the stent diameter, and the stent length may be derived based on the identified healthy areas.

[0069] According to this embodiment, the image processing device 3 accepts corrections from the operator of the diagnostic imaging catheter (such as a physician) regarding the identified lesion, distal reference part, or proximal reference part, for example, the position of these parts in the axial direction of the blood vessel, or the blood vessel diameter (EEM) or lumen diameter at said part. When the image processing device 3 accepts such corrections, it derives the diameter and length of the stent based on the corrected lesion, distal reference part, or proximal reference part. By deriving the diameter and length of the stent (stent size) based on the lesion, distal reference part, and proximal reference part that reflect the corrections made by the physician, useful information can be efficiently provided to the physician.

[0070] According to this embodiment, the program executed by the control unit 31 of the image processing device 3 includes a module or subroutine that performs multiple derivation methods for deriving the stent diameter based on the lumen diameter and vessel diameter at either the distal side, the proximal side, or both of the reference sections. When deriving the stent diameter, the image processing device 3 outputs these multiple derivation methods to the display device 4, for example in list format, and displays them on the display device 4 along with a selection section, such as a function button, for accepting the selection of one of the derivation methods. This allows the selection of a derivation method based on the operation of a physician or other person, and enables the efficient deriving of the stent diameter using the selected derivation method.

[0071] (Embodiment 2) Figure 14 is an explanatory diagram showing an example configuration of an image diagnostic device, etc., according to Embodiment 2 (Inventory DB). The image diagnostic device is connected to the inventory DB server S wirelessly or via a wired connection. The image diagnostic device has a communication unit consisting of a wireless slave unit such as 4G, 5G, or WiFi connected to an input / output I / F 33 such as a USB port, and performs data communication with the inventory DB server S, which is connected to the Internet or an intranet, via this communication unit.

[0072] The inventory DB server S is configured as an external server such as a cloud server and is equipped with a storage device S1. The storage device S1 of the inventory DB server S stores a stent inventory DB (Data Base) that stores data on stent varieties and inventory. The medical imaging device can search and refer to (retrieve) the data stored in the stent inventory DB by communicating with (accessing) the inventory DB server S. In this embodiment, the stent inventory DB that stores data on stent varieties and inventory is described as being stored in the storage device S1 of the inventory DB server S, but it is not limited to this, and the stent inventory DB may also be stored in the auxiliary storage unit 34 of the medical imaging device.

[0073] Figure 15 is an explanatory diagram illustrating an example of a stent inventory database. The stent inventory database includes management items (fields) such as stent type, stent diameter, stent length, and inventory quantity.

[0074] The stent variety management field stores the variety name of an existing sales variety. The stent diameter management field stores the diameter of the stent with the same variety name stored in the same record. The stent length management field stores the length of the stent with the same variety name stored in the same record. The inventory quantity management field stores the current inventory quantity of the stent with the same variety name stored in the same record. The diagnostic imaging device can access the inventory DB server S to search for the variety name of a stent that fits a predetermined stent size (stent length and stent diameter) and obtain (confirm) whether the stent with that variety name is in stock.

[0075] Figure 16 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 performs the following processing based on the input data output from the input device 5 in response to the operation of the operator of the diagnostic imaging catheter 1, such as a physician.

[0076] The control unit 31 acquires an IVUS image (S21). The control unit 31 calculates the stent diameter (S22). The control unit 31 calculates the stent length (S23). The control unit 31 performs the processes from S21 to S23 in the same way as the processes from S11 to S13 in Embodiment 1.

[0077] The control unit 31 refers to the stent inventory database (S24). The control unit 31 selects a recommended stent variety (S25). By accessing the inventory database server S, the control unit 31 searches the stent inventory database based on the stent size (stent diameter and stent length) calculated in the preprocessing step, and selects a stent variety (variety name) that matches the stent size and is currently in stock as the recommended stent variety. When selecting a recommended stent variety, the control unit 31 is not limited to selecting a single recommended stent variety, but may also select multiple recommended stent varieties.

[0078] The control unit 31 outputs support information such as stent size (recommended stent type) (S26). In outputting support information in the same manner as in Embodiment 1, the control unit 31 may output support information including information about the selected recommended stent type, and display it on the display device 4.

[0079] According to this embodiment, the image processing device 3 accesses, for example, an inventory DB server S (external server) that manages the inventory quantity for each type of stent, and obtains the inventory quantity for each type of stent. The image processing device 3 selects a stent of a type that matches the derived stent diameter and length (stent size), and for example, a stent of a type that is currently in stock, as a recommended size stent (recommended stent). This makes it possible to efficiently provide physicians and others with information on appropriate stent types based on the derived stent diameter and length (stent size).

[0080] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The technical features described in each embodiment can be combined with each other, and the scope of the invention is intended to include all modifications within the claims and equivalents thereof. [Explanation of Symbols]

[0081] 1. Diagnostic imaging catheter 11 probes 11a Catheter Sheath 12 Sensor section 12a Ultrasonic Transceiver Unit 12a IVUS sensor 12b Optical Transceiver Unit 12b OCT sensor 12c marker 12d Housing 13 shafts 14 Guide wire insertion section 14a Marker 15 Connector section 2 MDU 3. Image Processing Device (Information Processing Device) 30 Recording media 31 Control Unit 32 Main memory 33 Input / Output Interfaces 34 Auxiliary storage 341 Learning Models 35 Reading section 4 Display device 5 Input devices 100 Imaging diagnostic equipment 101 Intravascular Examination Device 102 Angiography equipment S Inventory DB Server (External Server) S1 storage

Claims

1. On the computer, Medical images are obtained based on signals detected by a catheter inserted into a tubular organ. Identify the type and region of objects included in the acquired medical image, Based on the identified type and region of the object, information regarding the stent to be inserted into the tubular organ is derived. Output information about the derived stent. A computer program designed to execute a process.

2. By inputting the aforementioned medical image into a learning model that estimates the objects contained in the medical image, the type and region of the objects contained in the acquired medical image are identified. The computer program according to claim 1.

3. The aforementioned tubular organ is a blood vessel, The identified type of object includes plaque interposed between the outer membrane and lumen of the blood vessel, Based on the identified plaque, information regarding the stent to be inserted into the blood vessel is derived. The computer program according to claim 1 or claim 2.

4. In the identified plaque, the lesion containing the area with the maximum plaque bathen is identified, Identify a reference area located distal to and proximal to the lesion, which includes a region where the plaque barden is less than or equal to the minimum value. The diameter and length of the stent are derived based on the identified distal and proximal reference portions. The computer program according to claim 3.

5. We accept modifications to the identified distal or proximal reference part. The diameter and length of the stent are derived based on the modified distal or proximal reference portion. The computer program according to claim 4.

6. The system accepts the selection of one of several derivation methods for determining the diameter of the stent. Using the selected derivation method, the diameter of the stent is derived based on the identified distal and proximal reference portions. The computer program according to claim 4 or claim 5.

7. The system selects a variety corresponding to the derived diameter and length of the stent by referring to a storage unit that stores variety information associated with the diameter and length of the stent and the variety of the stent. A computer program according to any one of claims 4 to 6.

8. The memory unit further stores the inventory quantity for each type of stent, and by referring to the memory unit, it selects a type of stent that corresponds to the derived diameter and length of the stent and is in stock. The computer program according to claim 7.

9. On the computer, Medical images are obtained based on signals detected by a catheter inserted into a tubular organ. Identify the type and region of objects included in the acquired medical image, Based on the identified type and region of the object, information regarding the stent to be inserted into the tubular organ is derived. Output information about the derived stent. An information processing method that executes a process.

10. An acquisition unit that acquires medical images generated based on signals detected by a catheter inserted into a tubular organ, A unit for identifying the type and region of objects included in the acquired medical image, Based on the identified type and region of the object, the system includes a derivation unit that derives information regarding a stent to be inserted into the tubular organ, An output unit that outputs the derived information about the stent, An information processing device equipped with the following features.

Images