Computer program, information processing method, and information processing device

Abstract

This computer program causes a computer to execute processing for acquiring a medical image generated on the basis of a signal detected by a catheter inserted into a hollow organ, identifying the type and region of an object included in the acquired medical image, deriving, on the basis of the identified type and region of the object, information relating to a stent inserted into the hollow organ, and outputting the derived information relating to the stent.

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device. [Background technology]

[0002] Medical images including ultrasonic tomographic images of blood vessels are generated by intravascular ultrasound (IVUS) using a catheter, and intravascular ultrasound examinations are performed. Meanwhile, technologies are being developed to add information to medical images using image processing and machine learning to assist doctors in their diagnoses (see, for example, Patent Document 1). The feature detection method for blood vessel images described in Patent Document 1 detects lumen walls, stents, and the like included in the blood vessel images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2016-525893 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the detection method of Patent Document 1 does not take into consideration the point that when an object contained in a blood vessel image is detected, information about a stent corresponding to the object is provided.

[0005] An object of the present disclosure is to provide a computer program, etc. that provides information about a stent based on an object contained in a medical image obtained by scanning a tubular organ with a catheter. [Means for solving the problem]

[0006] The computer program of this aspect causes a computer to perform a process of acquiring a medical image generated based on a signal detected by a catheter inserted into a tubular organ, identifying the type and area of ​​an object contained in the acquired medical image, deriving information about a stent to be inserted into the tubular organ based on the identified type and area of ​​the object, and outputting the derived information about the stent.

[0007] The information processing method of this aspect causes a computer to perform a process of acquiring a medical image generated based on a signal detected by a catheter inserted into a tubular organ, identifying the type and area of ​​an object contained in the acquired medical image, deriving information about a stent to be inserted into the tubular organ based on the identified type and area of ​​the object, and outputting the derived information about the stent.

[0008] The information processing device of this aspect includes an acquisition unit that acquires medical images generated based on signals detected by a catheter inserted into a tubular organ, an identification unit that identifies the type and area of ​​an object contained in the acquired medical image, a derivation unit that derives information about a stent to be inserted into the tubular organ based on the identified type and area of ​​the object, and an output unit that outputs the derived information about the stent. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a computer program or the like that provides information about a stent based on an object contained in a medical image obtained by scanning a tubular organ with a catheter. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic apparatus. [Figure 2] FIG. 1 is an explanatory diagram illustrating an overview of a catheter for diagnostic imaging. [Figure 3] FIG. 2 is an explanatory diagram showing a cross section of a blood vessel through which a sensor portion is inserted. [Figure 4] FIG. 2 is an explanatory diagram illustrating a tomographic image. [Figure 5] FIG. 1 is a block diagram illustrating an example of the configuration of an image processing device. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of a learning model. [Figure 7] 10 is a flowchart showing an information processing procedure performed by a control unit. [Figure 8] 10 is a flowchart showing a procedure for calculating a stent diameter. [Figure 9] FIG. 10 is an explanatory diagram showing an example of displaying information such as an average lumen diameter. [Figure 10] FIG. 10 is an explanatory diagram illustrating an example of a selection screen for a stent diameter derivation method. [Figure 11] 10 is a flowchart showing a procedure for calculating a stent length. [Figure 12] FIG. 10 is an explanatory diagram showing an example of displaying information such as landing zones. [Figure 13] FIG. 10 is an explanatory diagram showing an example of displaying information such as stent length. [Figure 14] FIG. 10 is an explanatory diagram showing an example of the configuration of an imaging diagnostic apparatus according to a second embodiment. [Figure 15] FIG. 10 is an explanatory diagram illustrating an example of a stent inventory DB. [Figure 16] 10 is a flowchart showing an information processing procedure performed by a control unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] The image processing method, image processing device, and program of the present disclosure will be described in detail below with reference to the drawings illustrating embodiments thereof. In the following embodiments, cardiac catheterization, which is an intravascular treatment, will be described as an example. However, the hollow organs that are the subject of catheterization treatment are not limited to blood vessels, and may be other hollow organs such as the bile duct, pancreatic duct, bronchi, intestines, etc.

[0012] (Embodiment 1) FIG. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic device 100. In this embodiment, an imaging diagnostic device using a dual-type catheter equipped with the functions of both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) will be described. The dual-type catheter is provided with a mode for acquiring ultrasound tomographic images using only IVUS, a mode for acquiring optical coherence tomographic images using only OCT, and a mode for acquiring tomographic images using both IVUS and OCT, and these modes can be switched for use. Hereinafter, ultrasound tomographic images and optical coherence tomographic images will be referred to as IVUS images and OCT images, respectively, as appropriate. IVUS images and OCT images will be collectively referred to as tomographic images, which correspond to medical images.

[0013] The diagnostic imaging apparatus 100 of this embodiment includes an intravascular inspection apparatus 101, an angiography apparatus 102, an image processing device 3, a display device 4, and an input device 5. The intravascular inspection apparatus 101 includes a diagnostic imaging catheter 1 and an MDU (Motor Drive Unit) 2. The diagnostic imaging catheter 1 is connected to the image processing device 3 via the MDU 2. The display device 4 and the input device 5 are connected to the image processing device 3. The display device 4 is, for example, a liquid crystal display or an organic EL display, and the input device 5 is, for example, a keyboard, a mouse, a trackball, or a microphone. The display device 4 and the input device 5 may be stacked together to form a touch panel. Alternatively, the input device 5 and the image processing device 3 may be integrated. Furthermore, the input device 5 may be a sensor that receives gesture input, gaze input, or the like.

[0014] The angiography device 102 is connected to the image processing device 3. The angiography device 102 is an angiography device that images blood vessels from outside the patient's body using X-rays while injecting a contrast agent into the patient's blood vessels, thereby obtaining an angio image, which is a fluoroscopic image of the blood vessels. The angiography device 102 is equipped with an X-ray source and an X-ray sensor, and images an X-ray fluoroscopic image of the patient by the X-ray sensor receiving X-rays irradiated from the X-ray source. Note that the diagnostic imaging catheter 1 is provided with a marker that is opaque to X-rays, and the position of the diagnostic imaging catheter 1 (marker) is visualized in the angio image. The angiography device 102 outputs the angio image obtained by imaging to the image processing device 3, and the angiography image is displayed on the display device 4 via the image processing device 3. Note that the display device 4 displays the angio image and a tomographic image captured using the diagnostic imaging catheter 1.

[0015] FIG. 2 is an explanatory diagram outlining the diagnostic imaging catheter 1. The upper dashed-dotted area in FIG. 2 is an enlarged version of the lower dashed-dotted area. The diagnostic imaging catheter 1 includes a probe 11 and a connector 15 disposed at the end of the probe 11. The probe 11 is connected to the MDU 2 via the connector 15. In the following description, the side of the diagnostic imaging catheter 1 farther from the connector 15 will be referred to as the distal end, and the connector 15 side will be referred to as the proximal end. The probe 11 includes a catheter sheath 11a, the distal end of which is provided with a guidewire insertion portion 14 through which a guidewire can be inserted. The guidewire insertion portion 14 forms a guidewire lumen and is used to receive a guidewire previously inserted into a blood vessel and guide the probe 11 to the affected area via the guidewire. The catheter sheath 11a forms a continuous tubular portion extending from the connection portion with the guidewire insertion portion 14 to the connection portion with the connector 15. A shaft 13 is inserted inside the catheter sheath 11a, and a sensor unit 12 is connected to the tip side of the shaft 13.

[0016] The sensor unit 12 has a housing 12d, the distal end of which is hemispherically shaped to reduce friction and snagging with the inner surface of the catheter sheath 11a. The housing 12d contains an ultrasonic transceiver 12a (hereinafter referred to as the IVUS sensor 12a) that transmits ultrasonic waves into a blood vessel and receives reflected waves from the blood vessel, and an optical transceiver 12b (hereinafter referred to as the OCT sensor 12b) that transmits near-infrared light into the blood vessel and receives reflected light from the blood vessel. In the example shown in FIG. 2, the IVUS sensor 12a is provided at the distal end of the probe 11, and the OCT sensor 12b is provided at the proximal end. The sensors are disposed on the central axis of the shaft 13 (on the two-dot chain line in FIG. 2) at a distance x along the axial direction. In the diagnostic imaging catheter 1, the IVUS sensor 12a and the OCT sensor 12b are attached such that the direction of transmission and reception of ultrasonic waves or near-infrared light is approximately 90 degrees relative to the axial direction of the shaft 13 (the radial direction of the shaft 13). It is desirable that the IVUS sensor 12a and the OCT sensor 12b are mounted slightly offset from the radial direction so as not to receive waves or light reflected from the inner surface of the catheter sheath 11a. In this embodiment, as shown by the arrow in Fig. 2, for example, the IVUS sensor 12a is mounted so that the direction of ultrasound irradiation is inclined toward the proximal end with respect to the radial direction, and the OCT sensor 12b is mounted so that the direction of near-infrared light irradiation is inclined toward the distal end with respect to the radial direction.

[0017] An electric signal cable (not shown) connected to the IVUS sensor 12a and an optical fiber cable (not shown) connected to the OCT sensor 12b are inserted into the shaft 13. The probe 11 is inserted into a blood vessel from the distal end side. The sensor unit 12 and the shaft 13 are movable forward and backward within the catheter sheath 11a and can also rotate circumferentially. The sensor unit 12 and the shaft 13 rotate around the central axis of the shaft 13 as the axis of rotation. The diagnostic imaging device 100 uses an imaging core formed by the sensor unit 12 and the shaft 13 to measure the state inside the blood vessel using ultrasound tomographic images (IVUS images) taken from inside the blood vessel or optical coherence tomographic images (OCT images) taken from inside the blood vessel.

[0018] The MDU 2 is a drive unit to which the probe 11 (diagnostic imaging catheter 1) is detachably attached via a connector unit 15, and controls the operation of the diagnostic imaging catheter 1 inserted into a blood vessel by driving a built-in motor in response to operation by a medical professional. For example, the MDU 2 performs a pull-back operation to rotate the sensor unit 12 and shaft 13 inserted into the probe 11 in a circumferential direction while pulling them toward the MDU 2 at a constant speed. The sensor unit 12 rotates while moving from the distal end to the proximal end due to the pull-back operation, and thereby scans the inside of the blood vessel continuously at predetermined time intervals, thereby continuously capturing multiple transverse cross-sectional images approximately perpendicular to the probe 11 at predetermined intervals. The MDU 2 outputs reflected wave data of ultrasound received by the IVUS sensor 12a and reflected light data received by the OCT sensor 12b to the image processing device 3.

[0019] The image processing device 3 acquires, via the MDU 2, a signal dataset that is reflected wave data of ultrasound received by the IVUS sensor 12a and a signal dataset that is reflected light data received by the OCT sensor 12b. The image processing device 3 generates ultrasound line data from the ultrasound signal dataset and constructs an ultrasound tomographic image (IVUS image) of a transverse layer of the blood vessel based on the generated ultrasound line data. The image processing device 3 also generates optical line data from the reflected light signal dataset and constructs an optical tomographic image (OCT image) of a transverse layer of the blood vessel based on the generated optical line data. Here, the signal datasets acquired by the IVUS sensor 12a and the OCT sensor 12b and the tomographic image constructed from the signal dataset will be described. FIG. 3 is an explanatory diagram showing a cross section of a blood vessel through which the sensor unit 12 has been inserted, and FIG. 4 is an explanatory diagram showing the tomographic image.

[0020] First, referring to FIG. 3 , the operation of the IVUS sensor 12a and the OCT sensor 12b within a blood vessel and the signal data sets (ultrasound line data and optical line data) acquired by the IVUS sensor 12a and the OCT sensor 12b will be described. When imaging of a tomographic image begins with the imaging core inserted into the blood vessel, the imaging core rotates in the direction indicated by the arrow, with the central axis of the shaft 13 as the center of rotation. At this time, the IVUS sensor 12a transmits and receives ultrasound waves at each rotation angle. Lines 1, 2, ... 512 indicate the transmission and reception directions of ultrasound waves at each rotation angle. In this embodiment, the IVUS sensor 12a intermittently transmits and receives ultrasound waves 512 times while rotating 360 degrees (one rotation) within the blood vessel. The IVUS sensor 12a acquires one line of data in the transmission and reception direction by transmitting and receiving ultrasound waves once, so that 512 ultrasound line data extending radially from the center of rotation can be obtained during one rotation. The 512 ultrasound line data are dense near the center of rotation, but become sparser as they move away from the center of rotation. Therefore, the image processing device 3 generates pixels in the empty spaces of each line by well-known interpolation processing, thereby generating a two-dimensional ultrasound tomographic image (IVUS image) as shown in Figure 4A.

[0021] Similarly, the OCT sensor 12b also transmits and receives measurement light at each rotation angle. Since the OCT sensor 12b also transmits and receives measurement light 512 times while rotating 360 degrees within the blood vessel, 512 optical line data extending radially from the center of rotation can be obtained during one rotation. For the optical line data, the image processing device 3 generates pixels in the empty spaces of each line using a well-known interpolation process, thereby generating a two-dimensional optical coherence tomographic image (OCT image) similar to the IVUS image shown in FIG. 4A. That is, the image processing device 3 generates optical line data based on interference light generated by interfering reflected light with reference light obtained by, for example, separating light from a light source within the image processing device 3, and constructs an optical tomographic image (OCT image) capturing a cross-section of the blood vessel based on the generated optical line data.

[0022] A two-dimensional tomographic image generated from 512 lines of data in this manner is called one frame of an IVUS image or an OCT image. Note that since the sensor unit 12 scans while moving inside the blood vessel, one frame of an IVUS image or an OCT image is acquired at each position of one rotation within the range of movement. That is, one frame of an IVUS image or an OCT image is acquired at each position from the distal end to the proximal end of the probe 11 within the range of movement, so that multiple frames of IVUS images or OCT images are acquired within the range of movement, as shown in FIG. 4B.

[0023] The diagnostic imaging catheter 1 has markers that are opaque to X-rays in order to confirm the positional relationship between an IVUS image obtained by the IVUS sensor 12a or an OCT image obtained by the OCT sensor 12b, and an angio image obtained by the angiography device 102. In the example shown in FIG. 2, a marker 14a is provided at the tip of the catheter sheath 11a, for example, at the guidewire insertion portion 14, and a marker 12c is provided on the shaft 13 side of the sensor unit 12. When the diagnostic imaging catheter 1 configured in this manner is imaged with X-rays, an angio image is obtained in which the markers 14a and 12c are visualized. The positions at which the markers 14a and 12c are provided are merely examples; the marker 12c may be provided on the shaft 13 instead of the sensor unit 12, and the marker 14a may be provided at a location other than the tip of the catheter sheath 11a.

[0024] 5 is a block diagram showing an example of the configuration of the image processing device 3. The image processing device 3 is a computer (information processing device) and includes a control unit 31, a main memory unit 32, an input / output I / F 33, an auxiliary memory unit , and a reading unit .

[0025] The control unit 31 is configured using one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), a tensor processing unit (TPU), etc. The control unit 31 is connected to each hardware component constituting the image processing device 3 via a bus.

[0026] The main memory unit 32 is a temporary storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and temporarily stores data required for the control unit 31 to execute arithmetic processing.

[0027] The input / output I / F 33 is an interface to which the intravascular inspection device 101, the angiography device 102, the display device 4, and the input device 5 are connected. The control unit 31 acquires IVUS images and OCT images from the intravascular inspection device 101 and acquires angio images from the angiography device 102 via the input / output I / F 33. The control unit 31 also displays medical images on the display device 4 by outputting medical image signals of the IVUS images, OCT images, or angio images to the display device 4 via the input / output I / F 33. Furthermore, the control unit 31 receives information input to the input device 5 via the input / output I / F 33.

[0028] A communication unit including a wireless terminal such as 4G, 5G, or WiFi is connected to the input / output I / F 33, and the image processing device 3 may be communicably connected to an external server, such as a cloud server, connected to an external network such as the Internet via the communication unit. The control unit 31 may access the external server via the communication unit and the external network, refer to medical data, research paper information, etc. stored (stored) in a storage device included in the external server, and perform processing related to information provision (provision processing for providing support information). Alternatively, the control unit 31 may cooperate with the external server to perform the processing in this embodiment, for example, by performing inter-process communication.

[0029] The auxiliary storage unit 34 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The auxiliary storage unit 34 stores a computer program P (program product) executed by the control unit 31 and various data required for processing by the control unit 31. The auxiliary storage unit 34 may be an external storage device connected to the image processing device 3. The computer program P (program product) may be written to the auxiliary storage unit 34 during the manufacturing stage of the image processing device 3, or may be distributed by a remote server device and acquired by the image processing device 3 via communication and stored in the auxiliary storage unit 34. The computer program P (program product) may be readably recorded on a recording medium 30 such as a magnetic disk, optical disk, or semiconductor memory, or may be read from the recording medium 30 by the reading unit 35 and stored in the auxiliary storage unit 34.

[0030] The image processing device 3 may be a multi-computer including multiple computers. The image processing device 3 may also be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the image processing device 3 is described as being a single computer. In this embodiment, an angiography device 102 that captures two-dimensional angioimages is connected to the image processing device 3, but the device is not limited to the angiography device 102 as long as it captures images of a patient's hollow organs and the diagnostic imaging catheter 1 from multiple directions outside the body.

[0031] In the image processing device 3 of this embodiment, the control unit 31 reads and executes a computer program P stored in the auxiliary storage unit 34 to construct an IVUS image based on the signal data set received from the IVUS sensor 12a and an OCT image based on the signal data set received from the OCT sensor 12b. Note that, as will be described later, the IVUS sensor 12a and the OCT sensor 12b have different observation positions at the same imaging timing. Therefore, the control unit 31 corrects the difference in the observation positions in the IVUS image and the OCT image. Therefore, the image processing device 3 of this embodiment provides IVUS images and OCT images with the same observation positions, thereby providing images that are easy to interpret.

[0032] In this embodiment, the diagnostic imaging catheter is described as a dual-type catheter having both intravascular ultrasound (IVUS) and optical coherence tomography (OCT) functions, but is not limited to this. The diagnostic imaging catheter may also be a single-type catheter having either intravascular ultrasound (IVUS) or optical coherence tomography (OCT) functions. Hereinafter, in this embodiment, the diagnostic imaging catheter has an intravascular ultrasound (IVUS) function, and the description will be based on IVUS images generated by the IVUS function. However, in the description of this embodiment, the medical image is not limited to an IVUS image, and the processing of this embodiment may also be performed using an OCT image as the medical image.

[0033] 6 is an explanatory diagram showing an example of the learning model 341. The learning model 341 is, for example, a neural network (segmentation NN) such as YOLO or R-CNN that performs object detection, semantic segmentation, or instance segmentation. Based on each IVUS image in the input IVUS image group, the learning model 341 outputs whether or not an object such as a stent or plaque is included in the IVUS image (presence / absence), and if an object is included (presence), the type (class) of the object, the area in the IVUS image, and the estimation accuracy (score).

[0034] The learning model 341 is configured, for example, by a convolutional neural network (CNN) trained by deep learning. The learning model 341 includes an input layer 341a to which medical images such as IVUS images are input, an intermediate layer 341b that extracts image features (image features), and an output layer 341c that outputs information indicating the position and type of objects included in the medical image. The input layer 341a of the learning model 341 includes multiple neurons that receive input of pixel values ​​of each pixel included in the medical image and passes the input pixel values ​​to the intermediate layer 341b. The intermediate layer 341b has a configuration in which a convolutional layer that convolves the pixel values ​​of each pixel input to the input layer 341a and a pooling layer that maps the pixel values ​​convolved in the convolutional layer are alternately connected, extracting image features while compressing the pixel information of the medical image. The intermediate layer 341b passes the extracted features to the output layer 341c. The output layer 341c has one or more neurons that output, for example, the position, range, and type of an image region of an object contained in an image as a labeled image. In the labeled image, for example, pixels corresponding to a plaque region are assigned class "1" and pixels corresponding to other images are assigned class "0." While the learning model 341 is assumed to be a CNN, the configuration of the learning model 341 is not limited to a CNN. The learning model 341 may be, for example, a neural network other than a CNN, a fully convolution network (FCN) such as a U-net, a SegNet, an SSD, an SPPnet, a support vector machine (SVM), a Bayesian network, or a trained model configured with a regression tree. Alternatively, the learning model 341 may input image features output from the intermediate layer into an SVM (support vector machine) to perform object recognition.

[0035] The learning model 341 can be generated by preparing training data in which medical images containing objects such as epicardium, side branches, veins, guidewires, stents, plaques that have deviated into the stent, lipid plaques, fibrous plaques, calcifications, vascular dissections, thrombi, and hematomas are associated with labels (label images) indicating the position (region) and type of each object, and then using the training data to train an untrained neural network. With the learning model 341 configured in this manner, by inputting a medical image such as an IVUS image into the learning model 341, information indicating the position and type of objects contained in the medical image can be obtained. If the medical image does not contain an object, information indicating the position and type is not output from the learning model 341. Therefore, by using the learning model 341, the control unit 31 can determine whether or not an object is contained in the medical image input to the learning model 341, and if so, obtain the type (class), position (region in the medical image), and estimation accuracy (score) of the object. That is, by using the learning model 341 trained in this manner, a labeled image showing the plaque area in pixel units can be obtained by inputting an IVUS image into the learning model 341 as shown in this embodiment.

[0036] The control unit 31 may input each IVUS image (frame image) one by one into the learning model 341 for processing, or may input multiple consecutive frame images simultaneously and simultaneously detect plaque regions from the multiple frame images. For example, the control unit 31 uses a 3D-CNN (e.g., 3D U-net) that handles three-dimensional input data as the learning model 341. The control unit 31 then handles the data as three-dimensional data, with the coordinates of the two-dimensional frame images as the two axes and the time (generation time) t at which each frame image was acquired as the first axis. The control unit 31 inputs multiple frame images (e.g., 16 frames) for a predetermined unit time into the learning model 341 as one set and simultaneously outputs images in which the plaque region is labeled for each of the multiple frame images. This allows plaque regions to be detected taking into account the frame images that are consecutive in time series, thereby improving detection accuracy.

[0037] The control unit 31 derives object information regarding the type and area of ​​the object included in the IVUS image based on the information acquired from the learning model 341. Alternatively, the control unit 31 may derive the object information by using the information acquired from the learning model 341 itself as the object information.

[0038] The control unit 31 functions as a data input unit, a data processing unit, and a data output unit by executing a computer program P stored in the auxiliary storage unit 34. The data input unit acquires all IVUS images generated in one pullback. The data input unit may also acquire angio images (Angio) from the angiography device 102. The IVUS images and angio images acquired by the data input unit may be displayed on the display device 4.

[0039] The data processing unit includes a learning model 341 with a segmentation function, and uses the learning model 341 to perform lumen and vessel segmentation for each acquired IVUS image. Based on the results of the lumen and vessel segmentation, the data processing unit identifies plaque regions and calculates plaque burden and stenosis rate. The data processing unit further uses the segmentation results to calculate the average lumen diameter and vessel diameter in the tomographic image of the blood vessel shown in each IVUS image. Based on the calculated plaque burden, stenosis rate, average lumen diameter, and vessel diameter, the data processing unit identifies, for example, a location where plaque burden is maximum (average lumen diameter is minimum) as the lesion, and identifies reference areas in each of the distal and proximal regions of the lesion. The data processing unit may identify, as the reference portion, a location including a portion where the plaque burden is minimum (the average lumen diameter is maximum) within a predetermined distance, such as 10 mm, from the lesion.

[0040] The data processing unit identifies the area covered by the stent (stent covered area) based on the positions of the distal and proximal reference areas in the axial direction of the blood vessel where the pullback was performed. The data processing unit can execute multiple derivation methods for calculating the stent diameter, such as mean mid-wall reference, and calculates the diameter of the stent (stent diameter) based on the lumen diameters of the distal and proximal reference areas using any of the derivation methods. The data processing unit calculates the length of the stent (stent length) based on the identified area covered by the stent (stent covered area).

[0041] The data output unit outputs information (output data) such as various numerical values ​​calculated by the data processing unit to the display device 4, and this information (output data) is displayed on the display device 4. The information (output data) may include, for example, an IVUS image, plaque burden, mean lumen diameter, lesion area, reference area, stent covered area, stenosis rate, stent diameter, and stent length, and may further include various values ​​(various calculated values) calculated when calculating this information. Details of the processing related to these functional units will be described later using flowcharts and the like.

[0042] 7 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 executes the following processing based on input data and the like output from the input device 5 in response to operations by an operator of the diagnostic imaging catheter 1, such as a doctor.

[0043] The control unit 31 acquires IVUS images (S11). The control unit 31 reads an IVUS image group (a plurality of IVUS images for one pullback) obtained by pullback, and acquires a medical image made up of these IVUS images.

[0044] The control unit 31 calculates the stent diameter (S12). Figure 8 is a flowchart showing the procedure for calculating the stent diameter. The control unit 31 calculates the stent diameter based on the processing of this flowchart.

[0045] The control unit 31 calculates the plaque burden (S121). The control unit 31 calculates the plaque burden by segmenting the lumen and blood vessels from the acquired IVUS image using, for example, the learning model 341. By segmenting the lumen and blood vessels, their areas (cross-sectional areas in a tomogram) are calculated, and the plaque burden may be calculated by dividing the area of ​​the region other than the lumen by the area of ​​the blood vessel.

[0046] The control unit 31 determines whether the plaque burden is equal to or greater than a predetermined threshold (S122). By determining whether the plaque burden is equal to or greater than a predetermined threshold, the control unit 31 classifies the plaque burden based on the threshold. The control unit 31 classifies all acquired IVUS images based on the calculated plaque burden based on a predetermined threshold, such as 40%, 50%, or 60%. The threshold may be configured to allow multiple settings.

[0047] If the plaque burden is equal to or greater than the predetermined threshold (S122: YES), the control unit 31 groups frames (IVUS images) equal to or greater than the threshold (S123). The control unit 31 groups frames (IVUS images) in which the plaque burden is equal to or greater than the threshold as lesions. If the lesions are scattered and spaced apart, they may be grouped individually (L1, L2, L3, ...). However, if the spacing (separation distance) between groups is 0.1 to 3 mm or less, they may be grouped together.

[0048] The control unit 31 identifies the group including the maximum value of plaque burden as the lesion (S124). The control unit 31 identifies the group including the maximum value of plaque burden, that is, the site where the lumen diameter is the smallest value, as the lesion.

[0049] If the number is not equal to or greater than the predetermined threshold (S122: NO), i.e., if the number is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold (S1221). If the number is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold as references. If the plaque burden that serves as the reference is scattered and spaced apart, they may be grouped together (R1, R2, R3, ...). However, if the spacing (separation distance) between the groups is 0.1 to 3 mm or less, they may be grouped together.

[0050] The control unit 31 identifies each group on the distal side and proximal side of the lesion as a reference portion (S125). For example, the control unit 31 classifies all IVUS images according to the determination result of whether or not the plaque burden is equal to or greater than a threshold, and then identifies each group on the distal side and proximal side of the lesion as a reference portion. Of the multiple grouped reference portions, the control unit 31 identifies each group located on the distal side and proximal side of the identified lesion as a reference portion to be compared with the lesion.

[0051] The control unit 31 calculates the blood vessel diameter, lumen diameter, and area of ​​the distal and proximal reference portions (S126). The control unit 31 calculates the blood vessel diameter (EEM), lumen diameter, and area of ​​the distal and proximal reference portions. In this case, the length between the reference portions, i.e., the length from the distal reference portion to the proximal reference portion, may be set to, for example, a maximum of 10 mm.

[0052] FIG. 9 is an explanatory diagram showing an example of displaying information such as the average lumen diameter. In this example, a graph of the average lumen diameter and a graph of the plaque burden (PB) are displayed one above the other. The horizontal axis represents the length (axial length) of the blood vessel. When the plaque burden (PB) threshold is 50%, a region exceeding the threshold is identified as a lesion. Regions distal to and proximal to the lesion, including the locations where the average lumen diameter is largest within 10 mm, are identified as the distal reference region and the proximal reference region, respectively. Displaying such information can provide assistance (support) to the operator in identifying the reference region. As shown in the figure in this embodiment, the lesion may be, for example, a region where the plaque burden (PB) is 50% or more, e.g., a group of 3 mm or more in length. The reference region may be a region including the location where the average lumen diameter is largest within 10 mm before and after the lesion. When a large side branch is present in the blood vessel and the blood vessel diameter changes significantly, a reference portion may be identified between the lesion and the side branch. In identifying the reference portion, the image shown in the figure may be displayed on the display device 4 and may be modified by the operator. Furthermore, when the image is displayed on the display device 4, the location of the large side branch may be indicated.

[0053] The control unit 31 may accept corrections to the identified distal reference portion or proximal reference portion. For example, on a screen displaying information such as the average lumen diameter, the control unit 31 may accept corrections from an operator (doctor, etc.) of the diagnostic imaging catheter regarding the positions of the identified lesion portion, distal reference portion, or proximal reference portion in the axial direction of the blood vessel, or the vessel diameter (EEM) or lumen diameter at the portion. By providing a correction accepting unit that accepts corrections to the identified or derived information on the screen displaying information such as the average lumen diameter, it is possible to provide a doctor, etc. with an operation function for presenting and correcting the information. When a correction is made to the reference portion (after the correction), the control unit 31 recalculates the information by reflecting the corrections, redisplays information such as the average lumen diameter, and performs the processes described below, such as deriving a stent size.

[0054] The control unit 31 selects one of a plurality of derivation methods for deriving the stent diameter (S127). The plurality of derivation methods for deriving the stent diameter are included, for example, as submodules, subroutines, or function libraries in the program executed by the control unit 31.

[0055] 10 is an explanatory diagram illustrating an example of a selection screen for a stent diameter derivation method. The multiple derivation methods include, for example, known derivation methods (rules), such as EEL-to-EEL (lesion), smallest reference EEL, mean mid-wall reference, largest reference lumen, mean reference lumen mm, and smallest reference lumen. The control unit 31 may output the names (types) of these multiple derivation methods to the display device 4 in a menu format such as a list as shown in the present embodiment, and display them on the display device 4 together with function buttons for accepting selection (selection receiving unit).

[0056] The control unit 31 calculates (derives) the stent diameter based on the lumen diameter and vessel diameter of the identified distal reference portion and proximal reference portion using the selected derivation method, such as mean mid-wall reference (S128). In the present embodiment, the stent diameter is calculated using the selected derivation method (rule), but this is not limiting. The control unit 31 may calculate the stent diameter using all of the derivation methods prepared in advance based on the lumen diameter and vessel diameter of the identified reference portion, output the derivation method and the stent diameter for each derivation method in association with each other, and display the results on the display device 4 in a menu format, such as a list. This allows the stent diameters for each derivation method to be presented to a physician or other professional, thereby efficiently providing information to assist in determining which derivation method to select.

[0057] The control unit 31 calculates the stent diameter (S13). Figure 11 is a flowchart showing the procedure for calculating the stent length. The control unit 31 calculates the stent length based on the processing of this flowchart.

[0058] The control unit 31 calculates the plaque burden (S131). The control unit 31 determines whether the plaque burden is equal to or greater than a predetermined threshold (S132). If the plaque burden is equal to or greater than the predetermined threshold (S132: YES), the control unit 31 groups frames (IVUS images) whose plaque burden is equal to or greater than the threshold (S133). The control unit 31 identifies the group containing the maximum plaque burden as a lesion (S134). The control unit 31 performs the processes from S131 to S134 in the same manner as S121 to S124 described above. The control unit 31 may use the results of the processes from S121 to S124 as the results of the processes from S131 to S134. In other words, the control unit 31 may configure the processes from S121 to S124 and the processes from S131 to S134 as a common routine.

[0059] If the result is not equal to or greater than the predetermined threshold (S132: NO), i.e., if the result is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold (S1321). If the result is less than the predetermined threshold, the control unit 31 groups the frames (IVUS images) that are less than the threshold as healthy areas (Healthy). If the plaque burden that constitutes healthy areas (Healthy) is scattered and spaced apart, they may be grouped together (H1, H2, H3, ...). However, if the spacing (separation distance) between the groups is 0.1 to 3 mm or less, they may be grouped together. The healthy area (Healthy) and the reference area (Reference) identified in the processes of S1221 and S125 may be the same site.

[0060] The control unit 31 specifies the distal and proximal healthy areas (each H group) of the specified lesion area as landing zones (S135). The control unit 31 calculates the stent length that can cover the distance between the landing zones (S136).

[0061] FIG. 12 is an explanatory diagram showing an example of displaying information such as the landing zone. In this example, the vertical axis indicates plaque burden (PB), and the horizontal axis indicates the length (axial length) of the blood vessel where the pullback has been performed. When the plaque burden (PB) threshold is 50%, the area exceeding the threshold is identified as a lesion. The locations distal and proximal to the lesion, for example, within 10 mm, where the average lumen diameter is maximum, are identified as the distal healthy area and the proximal healthy area, respectively. A landing zone is identified based on the distal healthy area and the proximal healthy area, and the stent length is calculated based on the distance between the identified landing zones.

[0062] The control unit 31 may display the illustrated image in this embodiment on the display device 4 and accept corrections made by the operator to the identified distal healthy portion or proximal healthy portion. When a correction is made to the healthy portion (after correction), the control unit 31 recalculates by reflecting the correction, redisplays the information, and performs processing described below, such as deriving the stent length. The control unit 31 may accept a selection from a plurality of derivation methods (rules) prepared in advance, as in the case of deriving the stent diameter, and calculate (derive) the stent length using the selected derivation method (rule).

[0063] The control unit 31 outputs support information such as the stent size (S14). The control unit 31 outputs the support information including the calculated stent size (stent diameter and stent length) to the display device 4, and causes the display device 4 to display the support information. In addition to the stent size (stent diameter and stent length), the support information may further include IVUS images (longitudinal and transverse views), plaque burden, average lumen diameter, lesion area, reference area, stent covered area, and stenosis rate.

[0064] FIG. 13 is an explanatory diagram showing an example of a display (data display unit) of information such as stent length. This explanatory diagram is an example of a screen display when support information is displayed on the display device 4. In this display example, a transverse tomogram, which is a cross-sectional view of the blood vessel in the axial direction, and a longitudinal tomogram, which is a cross-sectional view of the blood vessel in the radial direction, are displayed side by side. That is, the support information for stent placement includes multiple longitudinal tomograms (radial cross-sectional area of ​​the blood vessel) from IVUS images and a transverse tomogram (axial cross-sectional area of ​​the blood vessel) obtained by connecting these longitudinal tomograms. In the transverse tomogram, a distal reference portion (Ref. Distal) and a proximal reference portion (Ref. Proximal) are shown, and the lesion portion (MLA: minimum lumen area) located between these reference portions is shown. Displaying such information allows the operator to receive support (assistance) regarding stent placement.

[0065] The controller 31 may output (display) the calculated stent length by superimposing the area covered by the stent (stent covered area: indicated by a dotted line) on, for example, a cross-sectional view of an IVUS image, as shown in the figure in this embodiment. This allows a physician or other person to know the positional relationship between the lesion area (MLA) covered by the stent and the distal and proximal reference areas (Ref. D, Ref. P) when a stent of the calculated length is inserted. The various calculated values, such as the stent size (stent diameter and stent length), plaque burden, mean lumen diameter, and stenosis rate, may be superimposed or annotated on, for example, an IVUS image (longitudinal view or cross-sectional view).

[0066] When displaying support information on the display device 4 as illustrated in the present embodiment, the control unit 31 may display information about multiple candidate stents. The control unit 31 may also display comments for selecting from the multiple candidate stents. The control unit 31 may also display information about the type of stent, the degree of expansion (low pressure / high pressure), and the like. When identifying the reference area, the control unit 31 may perform processing to identify a suitable landing zone even if there is no area where the plaque burden is less than, for example, 50%. In this case, a cross section corresponding to the landing zone may be displayed. The control unit 31 may also perform processing to determine the position of the stent edge according to the plaque burden. The control unit 31 may also display the size of the selected stent after expansion. The control unit 31 may also simulate the placement of two stents and, in this case, display the simulation by changing the plaque burden threshold in the placement area of ​​each stent.

[0067] According to this embodiment, the image processing device 3 identifies (segments) the adventitia (vessel) and lumen (lumen) of a blood vessel as object types using the learning model 341, and calculates the area ratio of plaque (plaque burden) based on the segmented areas of the adventitia (vessel) and lumen (lumen) of the blood vessel. Based on the plaque burden of the identified plaque, the image processing device 3 derives information about a stent to be inserted into the blood vessel and applied to the plaque. The information about the stent includes, for example, stent size, including the diameter (stent diameter) and length (stent length) of the stent, thereby efficiently providing useful information to physicians and the like. In this way, information such as stent size can be derived based on the plaque identified by segmenting the IVUS image using the learning model 341. This reduces differences (variations) depending on the subject or operator of the diagnostic imaging catheter, and allows physicians and the like to be provided with information about appropriate stents.

[0068] According to this embodiment, the image processing device 3 identifies a lesion including a region where the plaque burden is maximum in all IVUS images generated by a single pullback, and identifies reference regions located distal and proximal to the identified lesion. Therefore, the stent diameter and length (stent size) corresponding to the range including the lesion where the plaque burden is maximum in the axial direction of the blood vessel can be efficiently derived. If the maximum length of the reference region used to derive the stent diameter (stent diameter) is restricted to, for example, 10 mm, the reference region used to derive the stent diameter (stent diameter) and the reference region used to derive the stent length (stent length) may be different regions. In this case, healthy regions located distal and proximal to the identified lesion may be identified separately from the reference region used to derive the stent diameter, and the stent length (stent length) may be derived based on the identified healthy regions.

[0069] According to this embodiment, the image processing device 3 accepts corrections from the operator (doctor, etc.) of the diagnostic imaging catheter, for example, regarding the positions of the identified lesion, distal reference portion, or proximal reference portion in the axial direction of the blood vessel, or the vessel diameter (EEM) or lumen diameter at the identified lesion, distal reference portion, or proximal reference portion. When the image processing device 3 accepts such corrections, it derives the diameter and length of the stent based on the corrected lesion, distal reference portion, or proximal reference portion. By deriving the diameter and length of the stent (stent size) based on the lesion, distal reference portion, and proximal reference portion that reflect the corrections made by the doctor, etc., it is possible to efficiently provide useful information to the doctor, etc.

[0070] According to this embodiment, the program executed by the control unit 31 of the image processing device 3 includes modules or subroutines that perform multiple derivation methods for deriving the stent diameter (stent diameter) based on the lumen diameter and blood vessel diameter at either or both of the distal and proximal reference sites. When deriving the stent diameter, the image processing device 3 outputs these multiple derivation methods to the display device 4, for example in a list format, and displays them on the display device 4 together with a selection unit such as a function button for accepting selection of one of the derivation methods. This allows the selection of a derivation method based on an operation by a doctor or the like to be accepted, and the stent diameter can be efficiently derived using the selected derivation method.

[0071] (Embodiment 2) 14 is an explanatory diagram showing a configuration example of an imaging diagnostic device etc. according to embodiment 2 (inventory DB). The imaging diagnostic device is connected wirelessly or wired to be able to communicate with the inventory DB server S. A communication unit consisting of a wireless adapter such as 4G, 5G, or WiFi is connected to an input / output I / F 33 such as a USB port of the imaging diagnostic device, and data communication is performed with the inventory DB server S connected to the Internet or an intranet via the communication unit.

[0072] The inventory DB server S is configured as an external server such as a cloud server, and includes a storage device S1. A stent inventory DB (Data Base) that stores data related to stent types and inventory is stored in the storage device S1 of the inventory DB server S. The imaging diagnostic device can search for and refer to (acquire) the data stored in the stent inventory DB by communicating with (accessing) the inventory DB server S. In this embodiment, the stent inventory DB that stores data related to stent types and inventory is stored in the storage device S1 of the inventory DB server S, but this is not limited to this, and the stent inventory DB may be stored in the auxiliary storage unit 34 of the imaging diagnostic device.

[0073] 15 is an explanatory diagram illustrating an example of a stent inventory DB. The stent inventory DB includes, as management items (fields), for example, stent type, stent diameter, stent length, and inventory quantity.

[0074] The management item (field) for stent variety stores the variety name of an existing commercially available variety. The management item (field) for stent diameter stores the diameter (stent diameter) of the stent with the variety name stored in the same record. The management item (field) for stent length stores the length (stent length) of the stent with the variety name stored in the same record. The management item (field) for stock quantity stores the current stock quantity of stents with the variety name stored in the same record. By accessing the stock DB server S, the imaging diagnostic device can search for the variety name of a stent that matches a specified stent size (stent length and stent diameter) and obtain (confirm) whether or not a stent with that variety name is in stock.

[0075] 16 is a flowchart showing the information processing procedure by the control unit 31. The control unit 31 of the image processing device 3 executes the following processing based on input data and the like output from the input device 5 in response to operations by an operator of the diagnostic imaging catheter 1, such as a doctor.

[0076] The control unit 31 acquires an IVUS image (S21). The control unit 31 calculates the stent diameter (S22). The control unit 31 calculates the stent length (S23). The control unit 31 performs the processes from S21 to S23 in the same manner as the processes from S11 to S13 in the first embodiment.

[0077] The control unit 31 refers to the stent inventory DB (S24). The control unit 31 selects a recommended stent type (S25). By accessing the inventory DB server S, the control unit 31 searches the stent inventory DB based on the stent size (stent diameter and stent length) calculated in preprocessing, and selects, as the recommended stent type, a stent type (type name) that is compatible with the stent size and that is currently in stock. The control unit 31 is not limited to selecting a single recommended stent type when selecting a recommended stent type, and may select multiple recommended stent types.

[0078] The control unit 31 outputs support information (recommended stent type) such as stent size (S26). When outputting the support information, the control unit 31 may output the support information including information about the selected recommended stent type, as in the first embodiment, and display the support information on the display device 4.

[0079] According to this embodiment, the image processing device 3 accesses, for example, an inventory DB server S (external server) that manages the inventory quantity of each stent type, and acquires the inventory quantity of each stent type. The image processing device 3 selects, as a stent of a recommended size (recommended stent), a stent type that is compatible with the derived stent diameter and length (stent size), for example, a stent type that is currently in stock. This makes it possible to efficiently provide doctors and others with information on appropriate stent types based on the derived stent diameter and length (stent size).

[0080] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. [Explanation of symbols]

[0081] 1 Diagnostic imaging catheter 11 Probe 11a Catheter sheath 12 Sensor section 12a Ultrasonic transmitter / receiver 12a IVUS sensor 12b Optical transmitter / receiver 12b OCT sensor 12c marker 12d Housing 13 Shaft 14 Guidewire insertion part 14a Marker 15 Connector part 2 MDU 3. Image processing device (information processing device) 30 Recording media 31 Control Unit 32 Main memory 33 Input / Output Interface 34 Auxiliary storage 341 Learning Model 35 Reading unit 4 Display device 5 Input Devices 100 Diagnostic imaging equipment 101 Intravascular inspection device 102 Angiography equipment S Inventory DB server (external server) S1 storage

Claims

1. On the computer, A medical image is acquired based on a signal detected by a catheter inserted into a hollow organ. Identifying the type and area of ​​an object included in the acquired medical image; deriving information about a stent to be inserted into the luminal organ based on the type and area of ​​the identified object; outputting the derived information about the stent; Identifying the type and area of ​​the object included in the acquired medical image by inputting the medical image into a learning model that estimates the object included in the medical image when the medical image is input; the hollow organ is a blood vessel, the identified object type includes plaque interposed between the adventitia and lumen of the blood vessel; deriving information about the stent to be inserted into the blood vessel based on the identified plaque; In the identified plaque, a lesion including a site where the plaque burden is maximum is identified; Identifying reference regions located distally and proximally to the lesion, each containing a region where the plaque burden is equal to or less than a threshold; Deriving the diameter and length of the stent based on the identified distal reference portion and the identified proximal reference portion. A computer program for executing a process.

2. Accepting a correction to the identified distal reference portion or the identified proximal reference portion; Deriving the diameter and length of the stent based on the corrected distal reference portion or the corrected proximal reference portion.

2. The computer program of claim 1.

3. Using a plurality of derivation methods for deriving the diameter of the stent, the diameter of the stent is derived based on the identified distal reference portion and the identified proximal reference portion, and the derivation method and the stent diameter for each derivation method are associated and output.

3. A computer program according to claim 1 or claim 2.

4. Accepting a selection of one of a plurality of delivery methods for deriving the diameter of the stent; Using the selected derivation technique, derive the diameter of the stent based on the identified distal reference portion and the identified proximal reference portion. A computer program according to any one of claims 1 to 3.

5. A storage unit that stores information relating the diameter and length of the stent to the type of the stent is referenced, and a type corresponding to the derived diameter and length of the stent is selected. A computer program according to any one of claims 1 to 4.

6. The storage unit further stores the number of stocks of each type of stent, and by referring to the storage unit, a type of stent that corresponds to the derived diameter and length of the stent and that is in stock is selected.

6. A computer program according to claim 5.

7. On the computer, A medical image is acquired based on a signal detected by a catheter inserted into a hollow organ. Identifying the type and area of ​​an object included in the acquired medical image; deriving information about a stent to be inserted into the luminal organ based on the type and area of ​​the identified object; outputting the derived information about the stent; Identifying the type and area of ​​the object included in the acquired medical image by inputting the medical image into a learning model that estimates the object included in the medical image when the medical image is input; the hollow organ is a blood vessel, the identified object type includes plaque interposed between the adventitia and lumen of the blood vessel; deriving information about the stent to be inserted into the blood vessel based on the identified plaque; In the identified plaque, a lesion including a site where the plaque burden is maximum is identified; Identifying reference regions located distally and proximally to the lesion, each containing a region where the plaque burden is equal to or less than a threshold; Deriving the diameter and length of the stent based on the identified distal reference portion and the identified proximal reference portion. An information processing method for executing a process.

8. an acquisition unit that acquires a medical image generated based on a signal detected by a catheter inserted into a hollow organ; an identification unit that identifies the type and area of ​​an object included in the acquired medical image; a derivation unit that derives information about a stent to be inserted into the luminal organ based on the type and area of ​​the identified object; an output unit that outputs the derived information about the stent, Identifying the type and area of ​​the object included in the acquired medical image by inputting the medical image into a learning model that estimates the object included in the medical image when the medical image is input; the hollow organ is a blood vessel, the identified object type includes plaque interposed between the adventitia and lumen of the blood vessel; deriving information about the stent to be inserted into the blood vessel based on the identified plaque; In the identified plaque, a lesion including a site where the plaque burden is maximum is identified; Identifying reference regions located distally and proximally to the lesion, each containing a region where the plaque burden is equal to or less than a threshold; Deriving the diameter and length of the stent based on the identified distal reference portion and the identified proximal reference portion. Information processing device.

Images