Information processing device, information processing method, and program

Abstract

To provide an information processing device or the like capable of preferably supporting image diagnosis related to blood vessel treatment.SOLUTION: An information processing device 2 includes: an acquisition part that acquires a transverse tomographic image of a plurality of frames obtained by imaging a blood vessel of a patient; a calculation part that calculates plaque burden in each frame by inputting the acquired transverse tomographic image of each frame to a learned model so as to calculate plaque burden when the transverse tomographic image is input; and a display part that displays a first object representing magnitude of the plaque burden at each position along an axial direction of the blood vessel on the basis of the plaque burden in each frame.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There are technologies that support image diagnosis related to vascular treatments such as PCI (Percutaneous Coronary Intervention). For example, Patent Document 1 discloses a medical image processing device that converts three-dimensional CT (Computer Tomography) data into two-dimensional projection images, extracts plaque regions in blood vessels from the three-dimensional CT data, calculates the stenosis rate of the blood vessels, and displays projection images in which color values of the plaque regions are determined according to the calculated stenosis rate, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-176282 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention of Patent Document 1 extracts plaque regions from three-dimensional CT values on a rule basis, and is not necessarily able to extract plaque regions and calculate stenosis rates with high accuracy. Also, the invention of Patent Document 1 requires three-dimensional CT data, and does not calculate stenosis rates, etc. from two-dimensional data (images).

[0005] In one aspect, an object of the present invention is to provide an information processing device and the like that can suitably support image diagnosis related to vascular treatment. [Means for solving the problem]

[0006] An information processing device according to one aspect includes an acquisition unit that acquires multiple frames of transverse cross-sectional images of a patient's blood vessels, a calculation unit that inputs the acquired transverse cross-sectional images of each frame into a model that has been trained to calculate plaque burden when the transverse cross-sectional images are input, and calculates the plaque burden in each frame, and a display unit that displays a first object that represents the magnitude of the plaque burden at each position along the axial direction of the blood vessel based on the plaque burden in each frame. [Effects of the Invention]

[0007] In one aspect, imaging diagnosis related to vascular treatment can be favorably supported. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an image diagnostic system. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a server. [Figure 3] FIG. 1 is a block diagram showing an example of the configuration of an imaging diagnostic apparatus. [Figure 4] FIG. 10 is an explanatory diagram regarding the calculation process of plaque burden. [Figure 5] FIG. 10 is an explanatory diagram showing an example of displaying plaque burden. [Figure 6] FIG. 10 is an explanatory diagram showing another example of displaying plaque burden. [Figure 7] 10A and 10B are explanatory diagrams relating to a process for determining a treatment range of a blood vessel. [Figure 8] FIG. 10 is an explanatory diagram showing an example of displaying a lesion length. [Figure 9] FIG. 10 is an explanatory diagram showing an example of a superimposed display of a stent. [Figure 10] 10 is a flowchart showing a procedure for generating a calculation model. [Figure 11] 10 is a flowchart showing the procedure of a plaque burden calculation process. [Figure 12] 10 is an explanatory diagram regarding the process of identifying a lesion area according to Modification 1. FIG. [Figure 13]FIG. 10 is a block diagram showing an example of the configuration of an imaging diagnostic apparatus according to a second embodiment. [Figure 14] FIG. 10 is an explanatory diagram relating to the calculation process of plaque burden according to the second embodiment. [Figure 15] 10 is a flowchart showing a procedure for generating an identification model. [Figure 16] 10 is a flowchart showing the procedure of a process for calculating plaque burden according to the second embodiment. [Figure 17] FIG. 10 is an explanatory diagram relating to a generation process of a calculation model according to Modification 2. [Figure 18] 10 is a flowchart showing the procedure of a calculation model generation process according to Modification 2. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below with reference to the drawings showing embodiments thereof. (Embodiment 1) Fig. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic system. In this embodiment, an imaging diagnostic system is described that calculates plaque burden from a cross-sectional image of a patient's blood vessels and supports vascular treatment using a predetermined treatment device. The imaging diagnostic device 2 includes a server 1 and an imaging diagnostic device (information processing device) 2. Each device is communicatively connected via a network N such as the Internet.

[0010] The server 1 is a server computer capable of various information processing and sending and receiving information. Note that the device equivalent to the server 1 may be a personal computer or other device. The server 1 functions as a generating device that performs machine learning to learn predetermined training data and generates a calculation model 51 (see FIG. 4) for calculating plaque burden using a patient's vascular tomographic image as input.

[0011] Plaque burden is the area ratio of plaque to the cross section of a blood vessel, and is calculated by dividing the cross-sectional area of the plaque and tunica media by the cross-sectional area of the blood vessel (the area of the region within the EEM (External Elastic Membrane)) (see Figure 4).

[0012] The diagnostic imaging device 2 is an imaging device that captures medical images of a patient's blood vessels, and is, for example, an IVUS (Intravascular Ultrasound) device that performs ultrasound examinations using a catheter 201. The catheter 201 is a medical instrument that is inserted into a patient's blood vessels, and transmits ultrasound signals from an ultrasound probe attached to the tip of the catheter 201 and receives reflected waves. The diagnostic imaging device 2 generates and displays ultrasound tomographic images based on the reflected waves received by the catheter 201.

[0013] In this embodiment, the image diagnostic device 2 is described as an IVUS device, but it may be an optical imaging device using OCT (Optical Coherence Tomography), etc. Furthermore, the vascular tomographic image may be captured by a method other than IVUS or OCT.

[0014] The image diagnostic device 2 has installed therein the data of the calculation model 51 generated by the server 1, and the image diagnostic device 2 inputs a vascular tomographic image captured using the catheter 201 into the calculation model 51 to calculate the plaque burden. The image diagnostic device 2 then presents the calculated plaque burden to a user (medical professional). The image diagnostic device 2 also identifies a diseased area in the blood vessel based on the calculated plaque burden, and assists in determining the placement position for a stent (treatment device) to be placed in the blood vessel.

[0015] In this embodiment, the diagnostic imaging device 2 calculates the plaque burden using the calculation model 51, but the server 1 on the cloud may execute the processing using the calculation model 51. Alternatively, for example, a general-purpose computer connected to the diagnostic imaging device 2 may execute the processing. In this way, the processing entity that executes the series of processes is not particularly limited.

[0016] Furthermore, in this embodiment, a stent is given as an example of a treatment device, but as will be described later, it may also be a balloon that expands blood vessels, a rotablator that removes stenotic lesions in blood vessels, or the like.

[0017] 2 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit . The control unit 11 has one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc., and performs various information processing, control processing, etc. by reading and executing a program P1 stored in the auxiliary storage unit 14. The main storage unit 12 is a temporary storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or a hard disk, and stores the program P1 and other data required for the control unit 11 to execute processing.

[0018] The server 1 may be a multi-computer consisting of a plurality of computers, or may be a virtual machine virtually constructed by software.

[0019] In this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit for accepting operation input, a display unit for displaying images, etc. The server 1 may also include a reading unit for reading a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage medium 1a. Alternatively, the server 1 may read the program P1 from a semiconductor memory 1b.

[0020] 3 is a block diagram showing an example of the configuration of the diagnostic imaging device 2. The diagnostic imaging device 2 includes a control unit 21, a main memory unit 22, a communication unit 23, a display unit 24, an input unit 25, an image processing unit 26, and an auxiliary memory unit 27. The control unit 21 is an arithmetic processing device such as one or more CPUs, MPUs, GPUs, etc., and performs various information processing, control processing, etc. by reading and executing a program P2 stored in the auxiliary storage unit 27. The main storage unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The display unit 24 is a display screen such as an LCD display, and displays images. The input unit 25 is an operation interface such as a keyboard or mouse, and accepts operation input from the user. The image processing unit 26 is an image processing module that processes signals transmitted and received via the catheter 201 and generates tomographic images.

[0021] The auxiliary storage unit 27 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores the program P2 and other data required for the control unit 21 to execute processing. The auxiliary storage unit 27 also stores a calculation model 51. The calculation model 51 is a machine learning model generated by learning predetermined training data, and is a trained model that calculates plaque burden using as input a cross-sectional image of a blood vessel captured by the image diagnostic device 2. The calculation model 51 is expected to be used as a program module that constitutes part of artificial intelligence software.

[0022] Note that the image diagnostic apparatus 2 may include a reading unit that reads a portable storage medium 2a such as a CD-ROM, and reads and executes the program P2 from the portable storage medium 2a. Alternatively, the image diagnostic apparatus 2 may read the program P2 from the semiconductor memory 2b.

[0023] FIG. 4 is an explanatory diagram regarding the calculation process of the plaque burden. In FIG. 4, each frame of the vascular tomographic image captured by the image diagnostic apparatus 2 is input into the calculation model 51, and the state of calculating the plaque burden is conceptually illustrated.

[0024] The calculation model 51 is a machine learning model that has learned predetermined training data, and is, for example, a CNN (Convolution Neural Network). Note that the calculation model 51 may be any model that can calculate the plaque burden from the vascular tomographic image, and may be a model based on other learning algorithms such as a neural network other than CNN, a decision tree, or an SVM (Support Vector Machine).

[0025] The calculation model 51 includes an input layer that receives an input of an image (tomographic image), an intermediate layer that extracts feature amounts of the input image, and an output layer that calculates (outputs) the plaque burden. The input layer receives an input of an image and passes the data of the input image to the intermediate layer. The intermediate layer includes a convolutional layer that convolves the data of the input image, extracts the feature amounts of the input image, and passes them to the output layer. The output layer calculates the plaque burden based on the feature amounts extracted by the intermediate layer.

[0026] In the present embodiment, the process executed by the calculation model 51 is treated as a regression problem, and the plaque burden is predicted as a continuous value. Note that this process may be treated as a classification problem, and the plaque burden may be predicted (classified) within a certain numerical range, for example, as "0 to X1", "X1 to X2", "X2 to X3"... (0 < X1 < X2 < X3...). Alternatively, the plaque burden may be classified into multiple stages (for example, classified into a red zone, an orange zone, a yellow zone...) without predicting the plaque burden as a numerical value.

[0027] The server 1 uses data to which correct values of plaque burden are assigned for training vascular tomograms (transverse images) as training data (second training data) to generate a calculation model 51. The training tomograms are vascular tomograms captured using an actual patient as the subject, and the correct values are the actual values of plaque burden in the tomograms of that patient. The server 1 inputs the training tomograms into the calculation model 51, calculates the plaque burden, and compares it with the correct value. The server 1 optimizes parameters such as the weights between neurons so that the two approximate each other, and generates the calculation model 51.

[0028] The image diagnostic device 2 calculates the plaque burden in the blood vessels of a patient undergoing catheter treatment, using the calculation model 51 generated by the server 1. Specifically, the image diagnostic device 2 sequentially inputs multiple frames of tomographic images captured in accordance with the pullback operation of the catheter 201 into the calculation model 51, and calculates the plaque burden in each frame. As a result, the image diagnostic device 2 calculates the area ratio obtained by dividing the area of the image region corresponding to the plaque and the tunica media (hereinafter referred to as the "plaque region") by the area of the EEM region (the region obtained by adding the plaque region to the lumen region), i.e., the plaque burden (see the lower right of Figure 4).

[0029] The image diagnostic device 2 presents the plaque burden calculated above in each frame, that is, the plaque burden at each position in the blood vessel along the axial direction, to the user to assist in determining the placement position of the stent.

[0030] Fig. 5 is an explanatory diagram showing an example of displaying plaque burden. In this embodiment, the image diagnostic device 2 displays a predetermined object (first object) representing the magnitude of plaque burden at each position in a blood vessel, corresponding to a longitudinal tomographic image 5 obtained by reconstructing cross-sectional images of multiple frames. Specifically, as shown in Fig. 5, the image diagnostic device 2 displays a gradation bar 6, the display color of which gradually changes depending on the magnitude of plaque burden, above the longitudinal tomographic image 5.

[0031] The longitudinal tomographic image 5 is a longitudinal cross section that reproduces the state of the blood vessel along its axial direction (running direction), and is generated from multiple frames of transverse images captured in response to the pull-back operation. For example, as shown in Fig. 5, the image diagnostic device 2 displays the longitudinal tomographic image 5 with the axial direction of the blood vessel as the horizontal direction.

[0032] The diagnostic imaging device 2 displays a gradation bar 6 above the longitudinal tomographic image 5, the gradation bar 6 having approximately the same length as the longitudinal tomographic image 5 along the axial direction. The gradation bar 6 is a bar-shaped object that represents the magnitude of plaque burden, and is a display bar whose display color changes depending on the magnitude of plaque burden at each position on the corresponding longitudinal tomographic image 5, i.e., in the transverse image of each frame. The diagnostic imaging device 2 changes the display color at each position on the gradation bar 6 depending on the magnitude of plaque burden at each position on the longitudinal tomographic image 5.

[0033] For convenience, regions with high plaque burden (regions above the threshold) are shown hatched in Figure 5. In reality, the gradation of the display color in the gradation bar 6 is set in detail according to the plaque burden value, as described below, but for convenience, regions with high plaque burden are shown with the same hatching in Figure 5.

[0034] Furthermore, in the present embodiment, the object representing the magnitude of plaque burden is described as the gradation bar 6, but the object is not limited to the gradation bar 6. For example, the diagnostic imaging device 2 may display a line graph representing the magnitude of plaque burden above the longitudinal tomographic image 5. In this way, the gradation bar 6 is an example of the object, and any object that appropriately represents the magnitude of plaque burden may be used.

[0035] For example, the image diagnostic device 2 sets the display color of a region with a large amount of plaque by referring to a predetermined threshold value (e.g., 50%) that is set as a default. Specifically, the image diagnostic device 2 sets the display color of a region where the plaque burden is equal to or greater than the threshold to a warm color (e.g., orange), and sets the display color of a region where the plaque burden is less than the threshold to a default color (e.g., white). Then, for a region where the plaque burden is equal to or greater than the threshold, the image diagnostic device 2 changes the display color (e.g., increases the brightness) so that the greater the plaque burden, the more emphasized it is, thereby performing a gradation display.

[0036] As a method for displaying the gradation, it is assumed that the brightness is changed, for example, but the hue, saturation, etc. may also be changed, and the specific display method is not particularly limited.

[0037] When displaying the gradation bar 6 as described above, it is preferable that the imaging diagnostic device 2 hides regions where the reliability of the plaque burden calculation result in the calculation model 51 is low (hereinafter referred to as "low reliability region"). In FIG. 5, the low reliability region is illustrated in black. When calculating the plaque burden in each frame using the calculation model 51, the imaging diagnostic device 2 acquires the reliability of the calculation value (e.g., a probability value between 0 and 1) from the calculation model 51 along with the plaque burden in each frame. When the reliability at each position (frame) of the blood vessel is equal to or less than a predetermined value, i.e., when the reliability is low, the imaging diagnostic device 2 blanks out the corresponding region on the gradation bar 6 as a low reliability region. This makes it possible to present the likelihood of the plaque burden calculation result to the user.

[0038] In this way, the diagnostic imaging device 2 displays the gradation bar 6 together with the longitudinal tomographic image 5, quantitatively presenting the state of the blood vessel at each position. Furthermore, as shown in Fig. 5, the diagnostic imaging device 2 displays the maximum, minimum, and average values of the plaque burden. This allows the user (healthcare professional) to understand the state of the blood vessel and determine the treatment area to be treated with a treatment device such as a stent.

[0039] Fig. 6 is an explanatory diagram showing another example of the display of plaque burden. In the display example of Fig. 5, warm colors are displayed in areas where the plaque burden is equal to or greater than a threshold value, but gradation display may also be used in areas where the plaque burden is less than the threshold value, i.e., areas with little plaque.

[0040] For example, the diagnostic imaging device 2 sets the display color of an area where the plaque burden is equal to or greater than a threshold to a warm color, and sets the display color of an area where the plaque burden is less than the threshold to a cool color. Note that in FIG. 6, the area where the plaque burden is equal to or greater than the threshold and the area where the plaque burden is less than the threshold are illustrated with different hatching. As with the area where the plaque burden is equal to or greater than the threshold, the diagnostic imaging device 2 changes the display color (for example, increases the brightness) as the plaque burden decreases, thereby providing a gradation display. In this way, the diagnostic imaging device 2 may present areas where there is a lot of plaque and where treatment may be required, as well as areas where there is little plaque and where treatment is less necessary.

[0041] In the example of FIG. 6, the area where the gradation is displayed is divided into two, but it is also possible to set a threshold and divide the area into three or more areas for gradation display.

[0042] Furthermore, when performing the above-described gradation display, the user may be allowed to set a threshold value used as a display standard. Specifically, the image diagnostic device 2 receives a setting input from the user to change the threshold value from the default value to an arbitrary value, and determines whether the plaque burden is in an area above or below the threshold value according to the set threshold value, and changes the display color accordingly. This allows the user to visualize the area of interest themselves, thereby providing more appropriate support to the user.

[0043] 7 is an explanatory diagram relating to the process of determining the treatment range of a blood vessel. In this embodiment, the treatment range to be treated with a predetermined treatment device is further presented to the user based on the calculation result of the plaque burden.

[0044] Specifically, the diagnostic imaging device 2 determines the positions of both ends of a stent to be placed in a blood vessel. When performing vascular treatment such as PCI, it is preferable to place a stent so that it covers the blood vessel without leaving any lesions behind. However, in reality, because plaque is continuously present, it is difficult to cover the blood vessel without leaving any lesions behind. For example, the stent is basically placed so that both ends are located in normal areas where no plaque is present, but if this is difficult, the stent is placed so that both ends are located in areas where the plaque burden is a predetermined value (e.g., 50%) or less. If this is also difficult, the stent is placed so that both ends are located in areas with the least plaque burden.

[0045] Currently, the plaque burden is not calculated for all frames as described above, but rather the stenosis is identified visually from the image and the plaque burden is measured for several frames as needed. Therefore, in this embodiment, the range where both ends of the stent should be located is determined based on the plaque burden calculated by the calculation model 51.

[0046] In this embodiment, a stent is used as an example of a treatment device, but the treatment device is not limited to a stent. For example, the imaging diagnostic device 2 may determine both end positions of an expansion range to be expanded by a balloon used to expand a blood vessel. Also, for example, the imaging diagnostic device 2 may determine both end positions of an ablation range to be ablated by a rotablator used to ablate a hard lesion (e.g., calcified tissue). In this way, the imaging diagnostic device 2 is only required to be able to determine both end positions of a treatment range to be treated by a treatment device inserted into a blood vessel, and the treatment device is not limited to a stent.

[0047] First, the image diagnostic device 2 divides the longitudinal tomographic image 5 (longitudinal cross section) of the blood vessel into a first region where the plaque burden is equal to or greater than a predetermined threshold, and a second region where the plaque burden is less than the threshold. The threshold may be the same as the threshold used as the reference for the gradation display, for example, but may be a different value. Based on the plaque burden in each frame calculated by the calculation model 51, the image diagnostic device 2 divides the longitudinal tomographic image 5 into first regions L1, L2, L3... (L: Lesion) where the plaque burden is equal to or greater than the threshold, and second regions H1, H2, H3... (H: Healthy) where the plaque burden is less than the threshold.

[0048] In this embodiment, the imaging diagnostic device 2 identifies a first region LX (X=1, 2, 3...) including the position (frame) where the plaque burden is maximum as the lesion region LX where a stent should be placed. Then, the imaging diagnostic device 2 determines two second regions HY, HZ (Y, Z=1, 2, 3...) located on both sides of the identified lesion region LX as regions where both ends of the stent should be located.

[0049] 7 shows a longitudinal image 5 in which one first region L1 and two second regions HP1 and HD1 (P: Proximal, D: Distal) exist. When the plaque burden is at its maximum in the first region L1, the diagnostic imaging device 2 determines that the second regions HP1 and HD1, which sandwich the first region L1, are regions where both ends of the stent should be located.

[0050] For example, when there is a low-reliability region where the reliability of the plaque burden calculation result is low, the imaging diagnostic device 2 preferably groups multiple first or second regions that straddle the low-reliability region as the same region. For example, in FIG. 7, there are two low-reliability regions in the first region L1. In this case, the imaging diagnostic device 2 determines whether the length of each low-reliability region in the axial direction of the blood vessel is equal to or less than a predetermined length (e.g., 1 mm). If the low-reliability region is equal to or less than the predetermined length, the imaging diagnostic device 2 divides the multiple first regions that straddle the low-reliability region as the same region. In the example of FIG. 7, both of the two low-reliability regions are equal to or less than the predetermined length, so the imaging diagnostic device 2 divides the three regions that straddle the low-reliability region into the same first region L1.

[0051] Similarly, for the second region, if there are multiple second regions that straddle low-reliability regions of a predetermined length or less, the diagnostic imaging device 2 divides the multiple second regions as the same region. In this way, if the length of a low-reliability region in the axial direction is equal to or less than a predetermined length, the diagnostic imaging device 2 divides multiple first regions or multiple second regions that straddle the low-reliability region as the same region. This makes it possible to appropriately determine the placement position of the stent while ensuring reliability.

[0052] The predetermined length may be variable by the user. For example, the imaging diagnostic device 2 accepts a setting input to change the default value of the predetermined length to any value (e.g., a length in predetermined increments such as 0.6 mm, 0.8 mm, 1.2 mm, etc.). The imaging diagnostic device 2 divides each region by determining whether the length is equal to or less than the set predetermined length. This allows the user to more appropriately determine the placement position of the stent.

[0053] Furthermore, the diagnostic imaging device 2 may allow the user to change the threshold used as the basis for region division, as in the case of gradation display. In this case, the diagnostic imaging device 2 may allow different thresholds to be set for the proximal and distal sides of the blood vessel. For example, if 50% is set as the threshold for the proximal side and 60% is set as the threshold for the distal side, the diagnostic imaging device 2 divides the blood vessel by setting the position where the plaque burden is 50% or more as the start point of the first region, as viewed from the left side of FIG. 7, and the position where the plaque burden is less than 60% as the end point of the first region. This allows the user to finely adjust the first and second regions themselves.

[0054] FIG. 8 is an explanatory diagram showing an example of displaying the lesion length. As described above, the imaging diagnostic device 2 divides the longitudinal tomographic image 5 into first regions L1, L2, L3, ... and second regions H1, H2, H3, ... to identify the lesion region L1. When the lesion region L1 is identified, the imaging diagnostic device 2 measures the length of the lesion region L1 in the axial direction of the blood vessel, i.e., the lesion length, and displays it in correspondence with the lesion region L1 in the longitudinal tomographic image 5. Specifically, as shown in FIG. 8, the imaging diagnostic device 2 displays the portion corresponding to the lesion region L1 with an arrow on the gradation bar 6, and also displays the lesion length. This makes it possible to visualize the length of the lesion site.

[0055] 9 is an explanatory diagram showing an example of a stent superimposed display. In this embodiment, the diagnostic imaging device 2 further presents to the user the treatment area to be treated with the treatment device, i.e., the area where the stent should be placed. Specifically, the diagnostic imaging device 2 superimposes a stent area 7 (second object) representing the size of the stent (treatment device) on the longitudinal tomographic image 5.

[0056] The stent region 7 is an object that simulates a stent placed in a blood vessel, and is a rectangular object that represents the area where the stent will be placed. Note that the length of the stent region 7 in at least the axial direction only needs to match the length of the actual stent, and the length in the radial direction (the vertical length in FIG. 8) does not need to match.

[0057] In this embodiment, the treatment device is a stent, so the object is rectangular, but if the treatment device is a balloon, the object may be sandbag-shaped. In this way, the diagnostic imaging device 2 only needs to be able to display an object of a shape corresponding to the treatment device.

[0058] For example, the diagnostic imaging device 2 displays the stent region 7 having a length equal to or greater than the measured lesion length so that both ends of the stent region 7 are located in the second regions HP1, HD1 determined as the regions where both ends should be located. For example, the diagnostic imaging device 2 stores information on each stent product in predetermined length increments (e.g., 9 mm, 12 mm, 15 mm, etc.) that can be used in the procedure in the auxiliary storage unit 27. The diagnostic imaging device 2 selects from these stents a stent that exceeds the lesion length and has the shortest length. The diagnostic imaging device 2 then displays the stent region 7 having the length of the selected stent so that both ends are located in the second regions HP1, HD1. Note that the selection of the stent and the determination of the superimposition position of the stent region 7 may be performed manually by the user.

[0059] For example, the diagnostic imaging device 2 displays the stent region 7 in a superimposed manner so that the center position (midpoint) of the lesion region L1 in the axial direction of the blood vessel coincides with the center position of the stent region 7. In this case, as shown in Fig. 9, downward arrows are displayed at positions corresponding to both ends of the stent, and a double-headed arrow is displayed in the portion exceeding the first region L1. The diagnostic imaging device 2 moves the stent region 7 according to operation input from the user (e.g., mouse operation), and allows the user to determine the optimal stent placement position.

[0060] In this case, it is preferable that the image diagnostic device 2 displays cross-sectional images at positions corresponding to both ends of the stent region 7, as shown in Fig. 9. This allows the user to search for the optimal stent placement position while moving the stent region 7.

[0061] Although the stent region 7 is superimposed on the longitudinal tomographic image 5 in the above example, the diagnostic imaging device 2 may display the stent region 7 in a location other than the longitudinal tomographic image 5 (for example, above the gradation bar 6 or within the gradation bar 6). In this way, the diagnostic imaging device 2 only needs to be able to display an object simulating a treatment device, and the display position is not limited to on the longitudinal tomographic image 5.

[0062] 10 is a flowchart showing the procedure of the process of generating the calculation model 51. The process of generating the calculation model 51 by machine learning will be described with reference to FIG. The control unit 11 of the server 1 acquires training data (second training data) for generating the calculation model 51 (step S11). The training data is data in which the correct value of plaque burden is associated with a cross-sectional image of a training blood vessel.

[0063] Based on the training data, the control unit 11 generates a calculation model 51 that calculates plaque burden when a cross-sectional image of a blood vessel is input (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the calculation model 51. The control unit 11 inputs a training blood vessel tomographic image to the calculation model 51 to calculate plaque burden and compares it with a correct value. The control unit 11 optimizes parameters such as the weights between neurons so that the calculated plaque burden approximates the correct value, and generates the calculation model 51. The control unit 11 ends the series of processes.

[0064] 11 is a flowchart showing the procedure of the plaque burden calculation process. The process executed by the diagnostic imaging apparatus 2 will be described with reference to FIG. The control unit 21 of the image diagnostic apparatus 2 acquires a plurality of frames of transverse images of blood vessels of a patient (step S31). The control unit 21 inputs the transverse image of each frame to the calculation model 51 and calculates the plaque burden in each frame (step S32). Specifically, the control unit 21 acquires the plaque burden in each frame and the reliability of the calculated value from the calculation model 51.

[0065] The control unit 21 identifies the blood vessel position (frame) where the plaque burden is at its maximum value (step S33). The control unit 21 also identifies a low reliability region in the axial direction of the blood vessel where the reliability of the plaque burden is equal to or less than a predetermined value (step S34). The control unit 21 also receives a threshold setting input (step S35). Note that the control unit 21 may skip step S35 and set a default threshold.

[0066] The control unit 21 generates (reconstructs) a longitudinal tomographic image of the blood vessel based on the transverse tomographic images of the multiple frames acquired in step S31 (step S36). The control unit 21 displays the generated longitudinal tomographic image and an object (first object) representing the magnitude of plaque burden at each position on the longitudinal tomographic image along the axial direction of the blood vessel (step S37). Specifically, as described above, the control unit 21 displays a gradation bar whose display color gradually changes according to the magnitude of plaque burden. In this case, the control unit 21 displays a gradation bar in which low-reliability regions are not displayed.

[0067] Based on the plaque burden at each position (frame) of the blood vessel, the control unit 21 divides the longitudinal tomographic image (longitudinal cross section) of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold and a second region where the plaque burden is less than the threshold (step S38). Specifically, the control unit 21 divides the longitudinal tomographic image into a plurality of regions depending on whether the plaque burden is equal to or greater than a threshold, and when the length of a low reliability region is equal to or less than a predetermined length, divides a plurality of first regions or second regions across the low reliability region as the same region.

[0068] The control unit 21 identifies the first region including the maximum value of plaque burden as the lesion region (step S39). Then, the control unit 21 determines the regions where both ends of the treatment range to be treated by the treatment device should be located, according to the identified lesion region (step S40). Specifically, the control unit 21 determines the second regions located on both sides of the lesion region as the regions where both ends of the stent to be placed in the blood vessel should be located.

[0069] The control unit 21 measures the length in the axial direction of the lesion region identified in step S39, i.e., the lesion length, and displays it in correspondence with the lesion region on the longitudinal tomographic image (step S41). The control unit 21 also displays an object (second object) representing the size of the treatment device in correspondence with the longitudinal tomographic image (step S42). For example, as described above, the control unit 21 superimposes and displays the stent region representing the length of the stent in the axial direction of the blood vessel on the longitudinal tomographic image so that both ends are located in the region determined in step S40. The control unit 21 then terminates the series of processes.

[0070] As described above, according to the first embodiment, the plaque burden at each position of the blood vessel is calculated from multiple frames of transverse cross-sectional images of the patient's blood vessel, and an object such as a gradation bar indicating the magnitude of the calculated plaque burden is displayed in correspondence with the longitudinal cross-sectional image. This makes it possible to favorably support image diagnosis related to vascular treatment, such as determining the placement position of a stent.

[0071] Furthermore, according to the first embodiment, by hiding low reliability regions where the reliability of the plaque burden calculation results is low, it is possible to present the likelihood of the plaque burden calculation results to the user.

[0072] Furthermore, according to the first embodiment, the length of the lesion area (lesion length) and the range of stent placement (stent area) can also be presented, thereby more preferably supporting image diagnosis.

[0073] Furthermore, according to the first embodiment, the longitudinal section (longitudinal tomographic image) of the blood vessel is divided into multiple regions according to the plaque burden, and the regions where both ends of the treatment range of the blood vessel by a treatment device such as a stent should be located are determined. This makes it possible to more appropriately support image diagnosis related to blood vessel treatment, such as determining the placement position of a stent.

[0074] Furthermore, according to the first embodiment, when the length of a low reliability region is equal to or shorter than a predetermined length, multiple regions (first region or second region) that straddle the low reliability region are regarded as the same region and divided. This allows the placement position of a stent to be suitably determined while ensuring reliability.

[0075] (Variation 1) In the first embodiment, the first region where the plaque burden is at its maximum value is identified as the lesion region. However, the lesion region does not have to be the region where the plaque burden is at its maximum value.

[0076] 12 is an explanatory diagram regarding the process of identifying a lesion area according to Modification 1. In Embodiment 1, the lesion area is identified first, and then the areas where both ends of the stent should be located are determined, but it is also possible to first determine the areas where both ends of the stent should be located, and then identify the lesion area.

[0077] Specifically, the diagnostic imaging device 2 divides the longitudinal tomographic image 5 into first regions L1, L2, L3... and second regions H1, H2, H3..., and then searches for two second regions Hy, Hz that sandwich the first region Lx. When the two second regions Hy, Hz that sandwich the first region Lx have been found, the diagnostic imaging device 2 identifies the first region Lx on both sides of which the two second regions Hy, Hz are located as the lesion region Lx. The diagnostic imaging device 2 then measures the length of the lesion region Lx (lesion length) and displays the stent region 7 in a superimposed manner.

[0078] In this way, the diagnostic imaging device 2 may determine the region where the stent should be placed based on the positional relationship between the first region and the second region. The determination method described in the first embodiment is an example, and various other methods are conceivable.

[0079] (Embodiment 2) In the first embodiment, a configuration in which the calculation model 51 directly calculates the plaque burden has been described. In the present embodiment, a configuration in which a plaque region is identified from a vascular tomographic image (transverse image) using another machine learning model and the plaque burden is indirectly calculated will be described. Note that the same reference numerals are used to designate the same parts as in the first embodiment, and the description thereof will be omitted.

[0080] 13 is a block diagram showing an example of the configuration of an imaging diagnostic apparatus 2 according to embodiment 2. The auxiliary storage unit 27 of the imaging diagnostic apparatus 2 according to this embodiment stores a discrimination model 52. Like the calculation model 51 according to embodiment 1, the discrimination model 52 is a machine learning model generated by learning predetermined training data, and is a trained model that uses a vascular tomographic image as input and identifies a plaque region in the tomographic image. The discrimination model 52 is expected to be used as a program module that constitutes part of artificial intelligence software.

[0081] Fig. 14 is an explanatory diagram of the calculation process of plaque burden according to the second embodiment. Fig. 14 conceptually illustrates how an image region corresponding to plaque and tunica media (hereinafter referred to as "plaque region") is identified from a cross-sectional image of a blood vessel using a discrimination model 52, and the plaque burden is calculated from the area of the identified plaque region. An outline of this embodiment will be described based on Fig. 14.

[0082] The discriminant model 52 is a machine learning model that has learned predetermined training data, and is, for example, a neural network generated by deep learning. In this embodiment, the discriminant model 52 is described as a semantic segmentation model, which is a type of CNN.

[0083] A semantic segmentation model is a neural network that identifies objects in an image on a pixel-by-pixel basis, and includes a convolutional layer (encoder) that convolves the input image, and a deconvolutional layer (decoder) that maps the convolved features back to the original image size. The deconvolutional layer identifies which object is located in the image based on the features extracted by the convolutional layer, and generates binarized data indicating which object each pixel corresponds to.

[0084] The server 1 generates a discrimination model 52 using training data (first training data) in which data indicating correct plaque regions are associated with training tomographic images. The server 1 inputs the training tomographic images into the discrimination model 52 to discriminate plaque regions, and optimizes parameters such as weights between neurons so that the identified plaque regions approximate the correct plaque regions. The image diagnostic device 2 discriminates plaque regions using the discrimination model 52 generated by the server 1.

[0085] In this embodiment, the identification model 52 is described as a semantic segmentation model, but it may also be a model based on a neural network other than the semantic segmentation model, a GAN (Generative Adversarial Network), or other learning algorithm.

[0086] Furthermore, the server 1 may generate a classification model 52 that can classify regions other than plaque regions by learning training data that also includes correct answer data for regions other than plaque regions (e.g., EEM regions, lumen regions, etc.). That is, the classification model 52 only needs to be able to classify at least plaque regions, and may also be able to classify other image regions.

[0087] When imaging a patient's blood vessels, the image diagnostic device 2 sequentially inputs the tomographic image of each frame into the identification model 52 and identifies the plaque region in each frame. Then, the image diagnostic device 2 calculates the plaque burden based on the identification result of the plaque region. That is, the image diagnostic device 2 calculates the area of the plaque region and the area of the entire blood vessel including the plaque region, and calculates the plaque burden.

[0088] In this way, it is not essential that the machine learning model directly calculates the plaque burden, and the plaque burden may be calculated indirectly from the prediction results of the machine learning model.

[0089] 15 is a flowchart showing the procedure of the process of generating the identification model 52. The process of generating the identification model 52 by machine learning will be described with reference to FIG. The control unit 11 of the server 1 acquires training data (first training data) for generating the discrimination model 52 (step S201). The training data is data in which data indicating the correct plaque region is associated with a cross-sectional image of a training blood vessel.

[0090] Based on the training data, the control unit 11 generates a discrimination model 52 that can discriminate plaque regions when a cross-sectional image of a blood vessel is input (step S202). Specifically, as described above, the control unit 11 generates a CNN related to semantic segmentation as the discrimination model 52. The control unit 11 inputs a training cross-sectional image to the discrimination model 52 to discriminate plaque regions, and compares the discriminated regions with the correct plaque regions. The control unit 11 optimizes parameters such as the weights between neurons so that the two are similar to each other, and generates the discrimination model 52. The control unit 11 ends the series of processes.

[0091] 16 is a flowchart showing the procedure of the plaque burden calculation process according to Embodiment 2. After acquiring multiple frames of transverse cross-sectional images of a patient's blood vessels (step S31), the diagnostic imaging apparatus 2 executes the following process. The control unit 21 of the image diagnostic apparatus 2 inputs the cross-sectional image of each frame into the discrimination model 52 and discriminates the plaque region in each frame (step S221). Then, the control unit 21 calculates the plaque burden in each frame based on the discrimination result (step S222). The control unit 21 shifts the process to step S31.

[0092] As described above, according to the second embodiment, it is also possible to calculate the plaque burden by utilizing the discrimination result of the discrimination model 52 that discriminates the plaque region from the cross-sectional image of the blood vessel.

[0093] (Variation 2) In the second embodiment, the discrimination model 52 capable of discriminating a plaque region in a tomographic image has been described. On the other hand, parameters such as weights obtained by learning the discrimination model 52 may be applied to the calculation model 51, and transfer learning may be performed to generate the calculation model 51 from the learning result of the discrimination model 52.

[0094] Fig. 17 is an explanatory diagram regarding the generation process of the calculation model 51 according to Modification 2. Fig. 17 conceptually illustrates a state in which transfer learning is performed by applying a part of the network structure and parameters of the discriminative model 52 (semantic segmentation model) to the calculation model 51 (CNN). Note that in this modification, the server 1 will be described assuming that the discriminative model 52 has already been generated.

[0095] The server 1 performs learning by setting parameters such as weights obtained by generating (learning) the discriminative model 52 as initial values for a calculation model 51 having a network structure similar to that of a portion of the discriminative model 52. For example, the calculation model 51 has a structure in which an output layer for calculating plaque burden is connected to an input layer similar to that of the discriminative model 52 and a portion of an intermediate layer (convolutional layer) corresponding to the decoder portion of the discriminative model 52. The server 1 sets the parameters obtained by learning the discriminative model 52 as initial values for the parameters of the intermediate layer and provides training data (first training data) for learning. The server 1 inputs training tomographic images, calculates plaque burden, compares it with the correct value, and optimizes the parameters so that the two approximate each other. By performing transfer learning in this manner, the amount of data learned by the calculation model 51 and the learning time can be reduced.

[0096] 18 is a flowchart showing the procedure of the process of generating the calculation model 51 according to Modification 2. After acquiring training data (first training data) for generating the calculation model 51 (step S11), the server 1 executes the following process. Note that the description will be given assuming that the server 1 has already generated the identification model 52.

[0097] The control unit 11 of the server 1 generates the calculation model 51 based on the parameters obtained by generating the discrimination model 52 and the training data (step S301). Specifically, as described above, the control unit 11 sets the parameters such as weights obtained by learning the discrimination model 52 as the initial values of the parameters of the calculation model 51, and then performs learning. The control unit 11 inputs a training tomographic image into the calculation model 51 to calculate plaque burden, compares it with the correct value, and optimizes the parameters so that the two are close to each other. The control unit 11 then ends the series of processes.

[0098] As described above, according to the first modification, by diverting the learning results of the discrimination model 52, the amount of data to be learned by the calculation model 51 and the learning time can be reduced.

[0099] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0100] 1 server 11 Control section 12 Main memory 13 Communications Department 14 Auxiliary storage P1 Program 2. Diagnostic imaging equipment 21 Control section 22 Main memory 23 Communications Department 24 Display 25 Input section 26 Image processing section 27 Auxiliary storage P2 Program 51 Calculation Model 52 Identification Model 5 Longitudinal tomography 6 Gradient Bar 7 Stent area

Claims

1. an acquisition unit that acquires a plurality of frames of cross-sectional images of a patient's blood vessels; a calculation unit that inputs the acquired cross-sectional image of each frame into a model that has been trained to calculate plaque burden when the cross-sectional image is input, and calculates plaque burden in each frame; a display unit that displays a first object representing the magnitude of the plaque burden at each position along the axial direction of the blood vessel based on the plaque burden in each frame; An information processing device comprising:

2. The first object has a different display mode in an area where the plaque burden is equal to or greater than a threshold value and in an area where the plaque burden is less than the threshold value. The information processing device according to claim 1 .

3. a receiving unit that receives a setting input for changing the threshold value; The display unit displays the first object with the display mode changed according to the set threshold value. The information processing device according to claim 2 .

4. The first object is a gradation bar whose display color gradually changes depending on the size of the plaque burden. The information processing device according to any one of claims 1 to 3.

5. the calculation unit calculates the plaque burden in each frame and a reliability indicating the likelihood of the calculation result of the plaque burden in each frame; an identifying unit that identifies a low-reliability region in an axial direction of the blood vessel, where the reliability is equal to or less than a predetermined value; The display unit hides a part of the first object corresponding to the low-reliability region. The information processing device according to any one of claims 1 to 4.

6. The calculation unit The cross-sectional image of each acquired frame is input to a model that has been trained to identify a plaque region and an EEM region when the cross-sectional image is input, and the plaque region and the EEM region in each frame are identified; The plaque burden in each frame is calculated by dividing the area of the plaque region by the area of the EEM region for each frame. The information processing device according to any one of claims 1 to 5.

7. Acquire multiple cross-sectional images of the patient's blood vessels; The cross-sectional image of each acquired frame is input into a model that has been trained to calculate plaque burden when the cross-sectional image is input, and the plaque burden in each frame is calculated; A first object representing the magnitude of the plaque burden at each position along the axial direction of the blood vessel is displayed on a display unit based on the plaque burden in each frame. An information processing method in which processing is performed by a computer.

8. Acquire multiple cross-sectional images of the patient's blood vessels; The cross-sectional image of each acquired frame is input into a model that has been trained to calculate plaque burden when the cross-sectional image is input, and the plaque burden in each frame is calculated; A first object representing the magnitude of the plaque burden at each position along the axial direction of the blood vessel is displayed on a display unit based on the plaque burden in each frame. A program that causes a computer to perform a process.

Images