Information processing apparatus, information processing method, program, and model generation method

Abstract

This information processing device comprises: an acquisition unit which acquires a transverse tomographic image of a plurality of frames obtained by imaging a blood vessel of a patient; a calculation unit which calculates, on the basis of the transverse tomographic image of the plurality of frames, a plaque burden at each position of the vessel along an axial direction of the blood vessel; a division unit which divides a vertical section of the blood vessel into a first area in which the plaque burden is equal to or greater than a threshold and a second area in which the plaque burden is smaller than the threshold; and a determination unit which determines, from each of the divided areas, an area in which both ends of a treatment range of the blood vessel by a prescribed treatment device should be located.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and a model generation method.

Background Art

[0002] There is a technique for assisting image diagnosis related to vascular treatment such as PCI (Percutaneous Coronary Intervention). For example, in Patent Document 1, there is disclosed a medical image processing apparatus that converts three-dimensional CT (Computer Tomography) data into two-dimensional projection images, extracts a plaque region of a blood vessel from the three-dimensional CT data, calculates a stenosis rate of the blood vessel, and displays a projection image in which a color value of the plaque region is determined according to the calculated stenosis rate or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the invention according to Patent Document 1 extracts a plaque region based on a rule from three-dimensional CT values, and does not necessarily extract the plaque region with high accuracy to calculate the stenosis rate. Further, the invention according to Patent Document 1 requires three-dimensional CT data and does not calculate the stenosis rate or the like from two-dimensional data (images).

[0005] In one aspect, an object is to provide an information processing apparatus or the like that can suitably assist image diagnosis related to vascular treatment.

Means for Solving the Problems

[0006] An information processing apparatus according to one aspect includes an acquisition unit that acquires tomographic images of a plurality of frames obtained by imaging a patient's blood vessel, a calculation unit that calculates a plaque burden at each position of the blood vessel along the axial direction of the blood vessel based on the tomographic images of the plurality of frames, a division unit that divides a longitudinal section of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold value and a second region where the plaque burden is less than the threshold value, and a determination unit that determines, from each of the divided regions, a region where both ends of a treatment range of the blood vessel by a predetermined treatment device should be located.

Effect of the Invention

[0007] On one aspect, it is possible to suitably support image diagnosis related to blood vessel treatment.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0009] Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments. (Embodiment 1) FIG. 1 is an explanatory diagram showing a configuration example of an image diagnostic system. In the present embodiment, an image diagnostic system that calculates the plaque burden from a tomographic image of a patient's blood vessel and supports blood vessel treatment using a predetermined treatment device will be described. The image diagnostic apparatus 2 includes a server 1 and an image diagnostic apparatus (information processing apparatus) 2. Each apparatus is communicatively connected via a network N such as the Internet.

[0010] The server 1 is a server computer capable of various information processing and information transmission and reception. Note that the apparatus corresponding to the server 1 may be an apparatus such as a personal computer. The server 1 performs machine learning for learning predetermined training data, and functions as a generation device that generates a calculation model 51 (see FIG. 4) for calculating the plaque burden with a tomographic image of a patient's blood vessel as an input.

[0011] Note that the plaque burden is the area ratio of the plaque in the cross-section of the blood vessel, and is the value obtained by dividing the cross-sectional area of the plaque and the tunica media by the cross-sectional area of the blood vessel (the area of the region within the EEM (External Elastic Membrane)) (see Fig. 4).

[0012] The imaging device 2 is an imaging device that captures a medical image obtained by imaging a patient's blood vessel, and is, for example, an IVUS (Intravascular Ultrasound) device that performs an ultrasonic examination using a catheter 201. The catheter 201 is a medical instrument inserted into a patient's blood vessel, and transmits an ultrasonic signal from an ultrasonic probe attached to the tip of the catheter 201 and receives a reflected wave. The imaging device 2 generates and displays an ultrasonic tomographic image based on the reflected wave received by the catheter 201.

[0013] In the present embodiment, the imaging device 2 is described as an IVUS device, but it may be an optical imaging device such as an OCT (Optical Coherence Tomography) device. Further, the blood vessel tomographic image may be captured by a method other than IVUS and OCT.

[0014] The data of the calculation model 51 generated by the server 1 is installed in the imaging device 2. The imaging device 2 inputs the blood vessel tomographic image captured using the catheter 201 into the calculation model 51 and calculates the plaque burden. Then, the imaging device 2 presents the calculated plaque burden to the user (medical staff). Further, the imaging device 2 identifies the lesion region of the blood vessel based on the calculated plaque burden and supports the determination of the placement position of a stent (treatment device) to be placed in the blood vessel.

[0015] In addition, in this embodiment, the image diagnostic apparatus 2 calculates the plaque burden using the calculation model 51. However, the server 1 on the cloud may execute the process using the calculation model 51. Further, for example, a general-purpose computer connected to the image diagnostic apparatus 2 may perform the process. Thus, the processing entity that executes a series of processes is not particularly limited.

[0016] In addition, in this embodiment, a stent is cited as an example of the treatment device. However, as will be described later, a balloon that dilates blood vessels, a rotablator that shaves stenotic lesions of blood vessels, or the like may also be used.

[0017] FIG. 2 is a block diagram showing a configuration example of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary storage unit 14. The control unit 11 has an arithmetic processing device such as one or more CPUs (Central Processing Unit), MPUs (Micro-Processing Unit), GPUs (Graphics Processing Unit), etc. By reading and executing the program P1 stored in the auxiliary storage unit 14, it performs various information processes, control processes, and the like. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores the data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processes, and transmits and receives information to and from the outside. The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory, a hard disk, etc., and stores the program P1 and other data necessary for the control unit 11 to execute the process.

[0018] Note that the server 1 may be a multi-computer composed of a plurality of computers, or may be a virtual machine virtually constructed by software.

[0019] In addition, in this embodiment, the server 1 is not limited to the above configuration. For example, it may include an input unit that receives operation inputs, a display unit that displays images, and the like. Further, the server 1 may be provided with a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and reads and executes the program P1 from the portable storage medium 1a. Alternatively, the server 1 may read the program P1 from the semiconductor memory 1b.

[0020] FIG. 3 is a block diagram showing a configuration example of the image diagnostic apparatus 2. The image diagnostic apparatus 2 includes a control unit 21, a main storage unit 22, a communication unit 23, a display unit 24, an input unit 25, an image processing unit 26, and an auxiliary storage unit 27. The control unit 21 is an arithmetic processing device such as one or more CPUs, MPUs, GPUs, etc., and performs various information processing, control processing, etc. by reading and executing the program P2 stored in the auxiliary storage unit 27. The main storage unit 22 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard or a mouse, and receives operation inputs from the user. The image processing unit 26 is an image processing module that processes signals transmitted and received via the catheter 201 and generates tomographic images.

[0021] The auxiliary storage unit 27 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. Further, the auxiliary storage unit 27 stores the calculation model 51. The calculation model 51 is a machine learning model generated by learning predetermined training data, and is a learned model that calculates the plaque burden using the cross-sectional image of the blood vessel imaged by the image diagnostic apparatus 2 as an input. The calculation model 51 is assumed to be used as a program module constituting a part of artificial intelligence software.

[0022] Note that the image diagnostic apparatus 2 may include a reading unit that reads a portable storage medium 2a such as a CD-ROM, and reads and executes the program P2 from the portable storage medium 2a. Alternatively, the image diagnostic apparatus 2 may read the program P2 from the semiconductor memory 2b.

[0023] FIG. 4 is an explanatory diagram regarding the calculation process of the plaque burden. In FIG. 4, the state of calculating the plaque burden by inputting each frame of the vascular tomographic image captured by the image diagnostic apparatus 2 into the calculation model 51 is conceptually illustrated.

[0024] The calculation model 51 is a machine learning model that has learned predetermined training data, and is, for example, a CNN (Convolution Neural Network). Note that the calculation model 51 may be any model that can calculate the plaque burden from the vascular tomographic image, and may be a model based on other learning algorithms such as neural networks other than CNN, decision trees, and SVM (Support Vector Machine).

[0025] The calculation model 51 includes an input layer that receives an input of an image (tomographic image), an intermediate layer that extracts feature amounts of the input image, and an output layer that calculates (outputs) the plaque burden. The input layer receives the input of the image and passes the data of the input image to the intermediate layer. The intermediate layer includes a convolutional layer that convolves the data of the input image, extracts the feature amounts of the input image, and passes them to the output layer. The output layer calculates the plaque burden based on the feature amounts extracted by the intermediate layer.

[0026] In the present embodiment, the process executed by the calculation model 51 is treated as a regression problem, and the plaque burden is predicted as a continuous value. Note that this process may be treated as a classification problem, and the plaque burden may be predicted (classified) within a certain numerical range, for example, as "0 to X1", "X1 to X2", "X2 to X3"... (0 < X1 < X2 < X3...). Alternatively, without predicting the plaque burden numerically, it may be classified into multiple stages (for example, classified into a red zone, an orange zone, a yellow zone...).

[0027] Server 1 uses the data with the correct plaque burden values assigned to the training vascular tomographic images (tomographic images) as training data (second training data) to generate the calculation model 51. The training tomographic images are vascular tomographic images taken with actual patients as subjects, and the correct values are the actual performance values of the plaque burden in the tomographic images of the patients. Server 1 inputs the training tomographic images into the calculation model 51 to calculate the plaque burden and compares it with the correct value. Server 1 optimizes parameters such as the weights between neurons so that the two are approximated, and generates the calculation model 51.

[0028] The image diagnostic apparatus 2 calculates the plaque burden in the blood vessels of the patient undergoing catheter treatment by using the calculation model 51 generated by Server 1. Specifically, the image diagnostic apparatus 2 sequentially inputs a plurality of frames of tomographic images taken in response to the pull-back operation of the catheter 201 into the calculation model 51, and calculates the plaque burden in each frame. Thereby, the image diagnostic apparatus 2 calculates the area ratio obtained by dividing the area of the image region corresponding to the plaque and the tunica media (hereinafter referred to as the "plaque region") by the area of the EEM region (the region obtained by adding the lumen region to the plaque region), that is, the plaque burden (see the lower right of FIG. 4).

[0029] The image diagnostic apparatus 2 presents to the user the plaque burden in each frame calculated above, that is, the plaque burden at each position of the blood vessel along the axial direction, and supports the determination of the stent placement position.

[0030] FIG. 5 is an explanatory diagram showing an example of displaying the plaque burden. In the present embodiment, the image diagnostic apparatus 2 displays a predetermined object (first object) representing the magnitude of the plaque burden at each position of the blood vessel corresponding to the longitudinal tomographic image 5 reconstructed from a plurality of frames of tomographic images. Specifically, as shown in FIG. 5, the image diagnostic apparatus 2 displays a gradation bar 6 with the display color gradually changed according to the magnitude of the plaque burden above the longitudinal tomographic image 5.

[0031] The longitudinal sectional image 5 is a longitudinal section that reproduces the state of the blood vessel along the axial direction (running direction) of the blood vessel, and is generated from a plurality of frames of tomographic images captured according to the pull-back operation. For example, as shown in FIG. 5, the imaging diagnostic apparatus 2 displays the longitudinal sectional image 5 with the axial direction of the blood vessel as the horizontal direction.

[0032] The imaging diagnostic apparatus 2 displays a gradation bar 6 having substantially the same length as the length of the longitudinal sectional image 5 along the axial direction above the longitudinal sectional image 5. The gradation bar 6 is a bar-shaped object representing the magnitude of the plaque burden, and is a display bar whose display color changes according to the magnitude of the plaque burden at each position of the corresponding longitudinal sectional image 5, that is, in the tomographic image of each frame. The imaging diagnostic apparatus 2 changes the display color at each position of the gradation bar 6 according to the magnitude of the plaque burden at each position on the longitudinal sectional image 5.

[0033] In FIG. 5, for the sake of convenience, a region where the plaque burden is large (a region equal to or greater than the threshold value) is illustrated by hatching. Also, actually, as described below, the gradation of the display color in the gradation bar 6 is finely set according to the numerical value of the plaque burden, but in FIG. 5, for the sake of convenience, a region where the plaque burden is large is illustrated by the same hatching.

[0034] In the present embodiment, the object representing the magnitude of the plaque burden is described as the gradation bar 6, but the object is not limited to the gradation bar 6. For example, the imaging diagnostic apparatus 2 may display a line graph representing the magnitude of the plaque burden above the longitudinal sectional image 5. Thus, the gradation bar 6 is an example of the object, and any object that appropriately represents the magnitude of the plaque burden may be used.

[0035] For example, the image diagnostic apparatus 2 refers to a predetermined threshold value (e.g., 50%) set by default to set the display color of the area with many plaques. Specifically, the image diagnostic apparatus 2 sets the display color of the area where the plaque burden is equal to or higher than the threshold value to a warm color (e.g., orange), and sets the area where the plaque burden is less than the threshold value to the default color (e.g., white). Then, for the area where the plaque burden is equal to or higher than the threshold value, the image diagnostic apparatus 2 changes the display color (e.g., increases the brightness) so that the larger the plaque burden, the more emphasized it is, and performs a gradation display.

[0036] Note that as a method of gradation display, for example, it is assumed that the brightness is changed, but the hue, saturation, etc. may be changed, and the specific display method is not particularly limited.

[0037] When the gradation bar 6 is displayed as described above, it is preferable that the image diagnostic apparatus 2 does not display the area where the reliability of the calculated result of the plaque burden in the calculation model 51 is low (hereinafter referred to as the "low-reliability area"). In FIG. 5, the low-reliability area is illustrated in black. When the image diagnostic apparatus 2 calculates the plaque burden in each frame using the calculation model 51, together with the plaque burden in each frame, the reliability of the calculated value (e.g., a probability value from 0 to 1) is acquired from the calculation model 51. When the reliability at each position (frame) of the blood vessel is equal to or lower than a predetermined value, that is, when the reliability is low, the area on the corresponding gradation bar 6 is made blank as the low-reliability area. Thereby, the certainty of the calculated result of the plaque burden can be presented to the user.

[0038] In this way, the image diagnostic apparatus 2 displays the gradation bar 6 together with the cross-sectional image 5, and quantitatively presents the state of the blood vessel at each position. Further, as shown in FIG. 5, the image diagnostic apparatus 2 displays the maximum value, the minimum value, and the average value of the plaque burden. Thereby, the user (medical staff) can grasp the state of the blood vessel and determine the treatment range to be treated with a treatment device such as a stent.

[0039] FIG. 6 is an explanatory diagram showing another example of the display of plaque burden. In the display example of FIG. 5, it was described that warm colors are displayed for regions where the plaque burden is equal to or greater than the threshold value. However, gradation display may also be performed for regions where the plaque burden is less than the threshold value, that is, regions with less plaque.

[0040] For example, the image diagnostic apparatus 2 sets the display color of regions where the plaque burden is equal to or greater than the threshold value to warm colors, and sets the display color of regions where the plaque burden is less than the threshold value to cool colors. In FIG. 6, regions where the plaque burden is equal to or greater than the threshold value and regions where the plaque burden is less than the threshold value are illustrated with different hatchings. Similar to regions where the plaque burden is equal to or greater than the threshold value, the image diagnostic apparatus 2 changes the display color (for example, increases the brightness) as the plaque burden decreases, and performs gradation display. In this way, the image diagnostic apparatus 2 may present regions with a large amount of plaque that may require treatment, as well as regions with a small amount of plaque and a low necessity for treatment.

[0041] In the example of FIG. 6, the number of regions for gradation display is two. However, further thresholds may be provided and gradation display may be performed by dividing into three or more regions.

[0042] Also, when performing the above-described gradation display, the threshold value used as the display standard may be settable by the user. Specifically, the image diagnostic apparatus 2 accepts a setting input from the user to change the threshold value from the default value to an arbitrary value, determines regions where the plaque burden is equal to or greater than the threshold value and regions where the plaque burden is less than the threshold value according to the set threshold value, and changes the display color. Thereby, the user can visualize the region of interest by himself / herself, and can more suitably support the user.

[0043] FIG. 7 is an explanatory diagram regarding the determination process of the treatment range of blood vessels. In the present embodiment, further, based on the calculation result of the plaque burden, the treatment range to be treated with a predetermined treatment device is presented to the user.

[0044] Specifically, the imaging diagnostic apparatus 2 determines the positions of both ends of a stent to be indwelled in a blood vessel. When performing a blood vessel treatment such as PCI, it is preferable to indwell the stent so as to cover the lesion without leaving any residue. However, in reality, since plaques continuously exist, it is difficult to cover the lesion without leaving any residue. For example, basically, the stent is indwelled so that both ends of the stent are located at normal sites where there are no plaques. When this is difficult, the stent is indwelled so that both ends of the stent are located at sites where the plaque burden is equal to or less than a predetermined value (for example, 50%). When this is also difficult, the stent is indwelled so that both ends of the stent are located at the site with the least plaque burden.

[0045] Currently, as described above, the plaque burden of all frames is not calculated. Instead, the stenosis part is visually identified from the image, and the plaque burden for several frames is measured as necessary. Therefore, in the present embodiment, based on the plaque burden calculated by the calculation model 51, the range where both ends of the stent should be located is determined.

[0046] In the present embodiment, a stent is cited as an example of the treatment device. However, the treatment device is not limited to a stent. For example, the imaging diagnostic apparatus 2 may determine the positions of both ends of the expansion range to be expanded by a balloon used for expanding a blood vessel. Further, for example, the imaging diagnostic apparatus 2 may determine the positions of both ends of the cutting range to be cut by a rotabrator used for cutting a hard lesion (for example, calcified tissue). Thus, the imaging diagnostic apparatus 2 only needs to be able to determine the positions of both ends of the treatment range to be treated by a treatment device inserted into the blood vessel, and the treatment device is not limited to a stent.

[0047] First, the image diagnostic apparatus 2 divides the longitudinal sectional image 5 (longitudinal section) of the blood vessel into a first region where the plaque burden is equal to or greater than a predetermined threshold value and a second region where the plaque burden is less than the threshold value. The threshold value is, for example, the same value as the threshold value used as the criterion for the above-described gradation display, but may be a different value. The image diagnostic apparatus 2 divides the longitudinal sectional image 5 into first regions L1, L2, L3... (L: Lesion) where the plaque burden is equal to or greater than the threshold value and second regions H1, H2, H3... (H: Healthy) where the plaque burden is less than the threshold value based on the plaque burden in each frame calculated by the calculation model 51.

[0048] In the present embodiment, the image diagnostic apparatus 2 identifies a first region LX (X = 1, 2, 3...) including the position (frame) where the plaque burden takes the maximum value as the lesion region LX where the stent should be implanted. Then, the image diagnostic apparatus 2 determines two second regions HY, HZ (Y, Z = 1, 2, 3...) located on both sides of the identified lesion region LX as the regions where both ends of the stent should be located.

[0049] FIG. 7 illustrates a longitudinal sectional image 5 in which there is one first region L1 and two second regions HP1, HD1 (P: Proximal, D: Distal). When the plaque burden takes the maximum value in the first region L1, the image diagnostic apparatus 2 determines the second regions HP1, HD1 sandwiching the first region L1 as the regions where both ends of the stent should be located.

[0050] For example, when there is a low-reliability region where the reliability of the calculation result of the plaque burden is low in the image diagnostic apparatus 2, it is preferable to group a plurality of first regions or second regions straddling the low-reliability region as the same region. For example, in FIG. 7, there are two low-reliability regions in the first region L1. In this case, the image diagnostic apparatus 2 determines whether the length of each low-reliability region in the axial direction of the blood vessel is equal to or less than a predetermined length (for example, 1 mm). When the low-reliability region is equal to or less than the predetermined length, the image diagnostic apparatus 2 divides a plurality of first regions straddling the low-reliability region as the same region. In the example of FIG. 7, since both of the two low-reliability regions are equal to or less than the predetermined length, the image diagnostic apparatus 2 divides the three regions straddling the low-reliability region into the same first region L1.

[0051] Similarly for the second region, when there are a plurality of second regions straddling a low-reliability region with a length equal to or less than a predetermined length, the image diagnostic apparatus 2 divides the plurality of second regions as the same region. In this way, when the length of the low-reliability region in the axial direction is equal to or less than a predetermined length, the image diagnostic apparatus 2 divides a plurality of first regions or a plurality of second regions straddling the low-reliability region as the same region. Thereby, while ensuring reliability, the stent placement position can be suitably determined.

[0052] Note that the above-mentioned predetermined length may be variable by the user. For example, the image diagnostic apparatus 2 accepts a setting input to change the default value of the predetermined length to an arbitrary value (for example, a length at a predetermined interval such as 0.6 mm, 0.8 mm, 1.2 mm...). The image diagnostic apparatus 2 divides each region by determining whether it is equal to or less than the set predetermined length. Thereby, the user can more suitably determine the stent placement position.

[0053] Also, the image diagnostic apparatus 2 may make the threshold value as a reference for region division variable by the user, similar to during the gradation display. In this case, the image diagnostic apparatus 2 may be able to set different threshold values for the proximal side and the distal side of the blood vessel. For example, when 50% is set as the threshold value for the proximal side and 60% is set as the threshold value for the distal side, the image diagnostic apparatus 2 sets the position where the plaque burden becomes 50% or more as the starting point of the first region and the position where the plaque burden becomes less than 60% as the ending point of the first region and divides them when viewed from the left side of FIG. 7. Thereby, the user can finely adjust the first region and the second region by themselves.

[0054] FIG. 8 is an explanatory diagram showing an example of the display of the lesion length. As described above, the image diagnostic apparatus 2 divides the tomographic image 5 into first regions L1, L2, L3,... and second regions H1, H2, H3,... and identifies the lesion region L1. When the lesion region L1 is identified, the image diagnostic apparatus 2 measures the length of the lesion region L1 in the axial direction of the blood vessel, that is, the lesion length, and displays it corresponding to the lesion region L1 in the tomographic image 5. Specifically, as shown in FIG. 8, the image diagnostic apparatus 2 displays the portion corresponding to the lesion region L1 above the gradation bar 6 with an arrow and also displays the lesion length. Thereby, the length of the lesion site can be visualized.

[0055] FIG. 9 is an explanatory diagram showing an example of the superimposed display of the stent. In the present embodiment, further, the image diagnostic apparatus 2 presents to the user the treatment range to be treated with the treatment device, that is, the range where the stent should be placed. Specifically, the image diagnostic apparatus 2 superimposes and displays a stent region 7 (second object) representing the size of the stent (treatment device) on the tomographic image 5.

[0056] The stent region 7 is an object simulating a stent to be placed in a blood vessel and is a rectangular object representing the placement range of the stent. Note that the stent region 7 only needs to have at least the length in the axial direction match the actual length of the stent, and the length in the radial direction (the length in the vertical direction in FIG. 8) does not need to match.

[0057] Also, in the present embodiment, since the treatment device is a stent, the shape of the object is rectangular. However, for example, when the treatment device is a balloon, the shape of the object may be sandbag-like. Thus, it is sufficient that the image diagnostic apparatus 2 can display an object having a shape corresponding to the treatment device.

[0058] For example, the imaging diagnostic apparatus 2 displays the stent region 7 having a length equal to or greater than the lesion length measured above such that both ends of the stent region 7 are located in the second regions HP1 and HD1 determined as the regions where both ends should be located above. For example, the imaging diagnostic apparatus 2 stores information on each stent product in a predetermined length increment (for example, 9 mm, 12 mm, 15 mm,...) that can be used in the procedure in the auxiliary storage unit 27. The imaging diagnostic apparatus 2 selects a stent that exceeds the lesion length and has the minimum length from these stents. Then, the imaging diagnostic apparatus 2 superimposes and displays the stent region 7 having the length of the selected stent such that both ends are located in the second regions HP1 and HD1. Note that the selection of the stent, the determination of the superimposition position of the stent region 7, etc. may be manually performed by the user.

[0059] For example, the imaging diagnostic apparatus 2 superimposes and displays the stent region 7 such that the center position (midpoint) of the lesion region L1 in the axial direction of the blood vessel coincides with the center position of the stent region 7. In this case, as shown in FIG. 9, downward arrows are displayed at positions corresponding to both ends of the stent, and two-way arrows are displayed in portions exceeding the first region L1. The imaging diagnostic apparatus 2 moves the stent region 7 according to an operation input (for example, mouse operation) from the user, and allows the user to determine the optimal placement position of the stent.

[0060] In this case, it is preferable that the imaging diagnostic apparatus 2 displays cross-sectional images at positions corresponding to both ends of the stent region 7 as shown in FIG. 9. Thereby, the user can search for the optimal placement position of the stent while moving the stent region 7.

[0061] Note that in the above, the stent region 7 is superimposed and displayed on the cross-sectional image 5, but the imaging diagnostic apparatus 2 may display the stent region 7 at a location other than the cross-sectional image 5 (for example, above the gradation bar 6 or within the gradation bar 6). Thus, the imaging diagnostic apparatus 2 only needs to be able to display an object simulating the treatment device, and the display position is not limited to above the cross-sectional image 5.

[0062] FIG. 10 is a flowchart showing the procedure of the generation process of the calculation model 51. Based on FIG. 10, the processing content when generating the calculation model 51 by machine learning will be described. The control unit 11 of the server 1 acquires training data (second training data) for generating the calculation model 51 (step S11). The training data is data in which the correct value of the plaque burden is associated with the cross-sectional image of the blood vessel for training.

[0063] Based on the training data, the control unit 11 generates a calculation model 51 for calculating the plaque burden when a cross-sectional image of a blood vessel is input (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the calculation model 51. The control unit 11 inputs the training cross-sectional image of the blood vessel into the calculation model 51 to calculate the plaque burden and compares it with the correct value. The control unit 11 optimizes parameters such as the weights between neurons so that the calculated plaque burden approximates the correct value, and generates the calculation model 51. The control unit 11 ends a series of processes.

[0064] FIG. 11 is a flowchart showing the procedure of the plaque burden calculation process. Based on FIG. 11, the processing content executed by the image diagnostic apparatus 2 will be described. The control unit 21 of the image diagnostic apparatus 2 acquires cross-sectional images of a plurality of frames obtained by imaging the blood vessel of the patient (step S31). The control unit 21 inputs the cross-sectional image of each frame into the calculation model 51 and calculates the plaque burden in each frame (step S32). Specifically, the control unit 21 acquires the plaque burden in each frame and the reliability of the calculated value from the calculation model 51.

[0065] The control unit 21 specifies the blood vessel position (frame) where the plaque burden takes the maximum value (step S33). Further, the control unit 21 specifies a low-reliability region where the reliability of the plaque burden is equal to or lower than a predetermined value in the axial direction of the blood vessel (step S34). Further, the control unit 21 receives a setting input of a threshold value (step S35). Note that the control unit 21 may skip step S35 and set a default threshold value.

[0066] Based on the cross-sectional images of multiple frames acquired in step S31, the control unit 21 generates (reconstructs) a longitudinal cross-sectional image of the blood vessel (step S36). The control unit 21 displays the generated longitudinal cross-sectional image and an object (first object) representing the magnitude of the plaque burden at each position of the longitudinal cross-sectional image along the axial direction of the blood vessel (step S37). Specifically, as described above, the control unit 21 displays a gradation bar whose display color gradually changes according to the magnitude of the plaque burden. In this case, the control unit 21 displays a gradation bar with the low-reliability region hidden.

[0067] Based on the plaque burden at each position (frame) of the blood vessel, the control unit 21 divides the longitudinal cross-sectional image (longitudinal section) of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold value and a second region where the plaque burden is less than the threshold value (step S38). Specifically, the control unit 21 divides the longitudinal cross-sectional image into a plurality of regions according to whether the plaque burden is equal to or greater than the threshold value, and when the length of the low-reliability region is equal to or less than a predetermined length, divides a plurality of first regions or second regions straddling the low-reliability region into the same region.

[0068] The control unit 21 identifies the first region including the maximum value of the plaque burden as the lesion region (step S39). Then, the control unit 21 determines the regions where both ends of the treatment range to be treated with the treatment device should be located according to the identified lesion region (step S40). Specifically, the control unit 21 determines the second regions located on both sides of the lesion region as the regions where both ends of the stent to be implanted in the blood vessel should be located.

[0069] The control unit 21 measures the length in the axial direction of the lesion area identified in step S39, that is, the lesion length, and displays it corresponding to the lesion area in the longitudinal tomogram (step S41). Further, the control unit 21 displays an object (second object) representing the size of the treatment device corresponding to the longitudinal tomogram (step S42). For example, as described above, the control unit 21 superimposes and displays a stent area representing the length of the stent in the axial direction of the blood vessel on the longitudinal tomogram so that both ends are located in the area determined in step S40. The control unit 21 ends a series of processes.

[0070] As described above, according to the first embodiment, the plaque burden at each position of the blood vessel is calculated from a plurality of frames of transverse tomograms of the patient's blood vessel, and an object such as a gradation bar representing the magnitude of the calculated plaque burden is displayed corresponding to the longitudinal tomogram. Thereby, it is possible to suitably support image diagnosis related to blood vessel treatment, such as determination of the stent placement position.

[0071] Further, according to the first embodiment, by making the low-reliability region where the reliability of the calculation result of the plaque burden is low non-displayed, it is possible to present the certainty of the calculation result of the plaque burden to the user.

[0072] Further, according to the first embodiment, by presenting also the length of the lesion area (lesion length), the placement range of the stent (stent area), etc., it is possible to more suitably support image diagnosis.

[0073] Further, according to the first embodiment, the longitudinal section (longitudinal tomogram) of the blood vessel is divided into a plurality of regions according to the plaque burden, and the regions where both ends of the treatment range of the blood vessel by a treatment device such as a stent should be located are determined. Thereby, it is possible to more suitably support image diagnosis related to blood vessel treatment, such as determination of the stent placement position.

[0074] Further, according to the first embodiment, when the length of the low-reliability region is equal to or less than a predetermined length, a plurality of regions (the first region or the second region) straddling the low-reliability region are regarded as the same region and divided. Thereby, while ensuring reliability, the stent placement position and the like can be suitably determined.

[0075] (Modification 1) In the first embodiment, the first region where the plaque burden takes the maximum value is specified as the lesion region. However, the lesion region does not necessarily have to be the region where the plaque burden takes the maximum value.

[0076] FIG. 12 is an explanatory diagram regarding the specific processing of the lesion region according to Modification 1. In the first embodiment, the lesion region was specified first, and then the regions where both ends of the stent should be located were determined. However, the regions where both ends of the stent should be located may be determined first, and then the lesion region may be specified.

[0077] Specifically, after the tomographic image 5 is divided into the first regions L1, L2, L3... and the second regions H1, H2, H3... by the imaging diagnostic apparatus 2, the imaging diagnostic apparatus 2 searches for two second regions Hy, Hz sandwiching the first region Lx. When two second regions Hy, Hz sandwiching the first region Lx are found, the imaging diagnostic apparatus 2 specifies the first region Lx where the two second regions Hy, Hz are located on both sides as the lesion region Lx. Then, the imaging diagnostic apparatus 2 measures the length (lesion length) of the lesion region Lx and superimposingly displays the stent region 7.

[0078] In this way, the imaging diagnostic apparatus 2 may determine the region where the stent should be placed from the positional relationship between the first region and the second region. The determination method described in the first embodiment is an example, and various methods are assumed.

[0079] (Second Embodiment) In the first embodiment, the form in which the calculation model 51 directly calculates the plaque burden has been described. In the present embodiment, a form in which a plaque region is identified from a vascular tomographic image (tomographic image) using another machine learning model and the plaque burden is indirectly calculated will be described. Note that the same reference numerals are given to the contents overlapping with those of the first embodiment, and the description thereof is omitted.

[0080] FIG. 13 is a block diagram showing a configuration example of the image diagnostic apparatus 2 according to the second embodiment. The auxiliary storage unit 27 of the image diagnostic apparatus 2 according to the present embodiment stores the identification model 52. The identification model 52 is a machine learning model generated by learning predetermined training data, similar to the calculation model 51 according to the first embodiment, and is a learned model that takes a tomographic image of a blood vessel as an input and identifies a plaque region in the tomographic image. The identification model 52 is assumed to be used as a program module that constitutes a part of artificial intelligence software.

[0081] FIG. 14 is an explanatory diagram regarding the calculation process of the plaque burden according to the second embodiment. In FIG. 14, an image region corresponding to a plaque and a tunica media (hereinafter referred to as a "plaque region") is identified from a tomographic image of a blood vessel using the identification model 52, and a state of calculating the plaque burden from the area of the identified plaque region is conceptually illustrated. Based on FIG. 14, the outline of the present embodiment will be described.

[0082] The identification model 52 is a machine learning model that has learned predetermined training data, and is, for example, a neural network generated by deep learning. In the present embodiment, the identification model 52 will be described as being a semantic segmentation model which is a type of CNN.

[0083] The semantic segmentation model is a neural network that identifies objects in an image in pixel units, and includes a convolutional layer (encoder) that convolves the input image, and a transposed convolutional layer (decoder) that maps the convolved feature amounts to the original image size. In the transposed convolutional layer, based on the feature amounts extracted by the convolutional layer, it is identified which object exists at which position in the image, and data in which each pixel corresponds to which object is binarized is generated.

[0084] Server 1 generates the identification model 52 using training data (first training data) in which data indicating the correct plaque region is associated with the tomographic images for training. Server 1 inputs the tomographic images for training into the identification model 52 to identify the plaque region, and optimizes parameters such as the weights between neurons so that the identified plaque region approximates the correct plaque region. The image diagnostic apparatus 2 identifies the plaque region using the identification model 52 generated by Server 1.

[0085] In this embodiment, the identification model 52 is described as being a semantic segmentation model. However, it may be a neural network other than the semantic segmentation model, a model based on GAN (Generative Adversarial Network), or other learning algorithms.

[0086] Further, Server 1 may generate an identification model 52 that can also identify regions other than the plaque region by learning training data in which correct data for regions other than the plaque region (for example, EEM region, lumen region, etc.) is also given. That is, the identification model 52 only needs to be able to identify at least the plaque region, and may also be able to identify other image regions.

[0087] When imaging the blood vessels of a patient, the image diagnostic apparatus 2 sequentially inputs the tomographic images of each frame into the identification model 52 to identify the plaque region in each frame. Then, the image diagnostic apparatus 2 calculates the plaque burden based on the identification result of the plaque region. That is, the image diagnostic apparatus 2 calculates the area of the plaque region and the area of the entire blood vessel including the plaque region, and calculates the plaque burden.

[0088] Thus, a configuration in which the machine learning model directly calculates the plaque burden is not essential, and the plaque burden may be calculated indirectly from the prediction result of the machine learning model.

[0089] FIG. 15 is a flowchart showing the procedure of the generation process of the identification model 52. Based on FIG. 15, the processing content when generating the identification model 52 by machine learning will be described. The control unit 11 of the server 1 acquires training data (first training data) for generating the identification model 52 (step S201). The training data is data in which data indicating the correct plaque region is associated with the cross-sectional image of the blood vessel for training.

[0090] Based on the training data, the control unit 11 generates an identification model 52 that identifies the plaque region when the cross-sectional image of the blood vessel is input (step S202). Specifically, as described above, the control unit 11 generates a CNN related to semantic segmentation as the identification model 52. The control unit 11 inputs the training cross-sectional image into the identification model 52 to identify the plaque region, and compares it with the correct plaque region. The control unit 11 optimizes parameters such as the weights between neurons so that the two approximate each other, and generates the identification model 52. The control unit 11 ends a series of processes.

[0091] FIG. 16 is a flowchart showing the procedure of the plaque burden calculation process according to Embodiment 2. After acquiring cross-sectional images of a plurality of frames obtained by imaging the blood vessel of the patient (step S31), the image diagnostic apparatus 2 executes the following processes. The control unit 21 of the image diagnostic apparatus 2 inputs the cross-sectional image of each frame into the identification model 52, and identifies the plaque region in each frame (step S221). Then, based on the identification result, the control unit 21 calculates the plaque burden in each frame (step S222). The control unit 21 transfers the process to step S31.

[0092] As described above, according to the second embodiment, the plaque burden can also be calculated using the identification result of the identification model 52 that identifies the plaque region from the cross-sectional image of the blood vessel.

[0093] (Modification 2) In Embodiment 2, the identification model 52 capable of identifying plaque regions in tomographic images was described. On the other hand, parameters such as weights obtained by learning the identification model 52 may be applied to the calculation model 51, and transfer learning may be performed to generate the calculation model 51 from the learning results of the identification model 52.

[0094] FIG. 17 is an explanatory diagram regarding the generation process of the calculation model 51 according to Modification 2. In FIG. 17, a part of the network structure and parameters of the identification model 52 (semantic segmentation model) are applied to the calculation model 51 (CNN), and the state of performing transfer learning is conceptually illustrated. In this modification, the server 1 will be described assuming that the identification model 52 has already been generated.

[0095] The server 1 performs learning by setting, as initial values, parameters such as weights obtained by generating (learning) the identification model 52 for a calculation model 51 having a network structure similar to a part of the network structure of the identification model 52. For example, the calculation model 51 has a structure in which an output layer for calculating plaque burden is connected to an input layer similar to the identification model 52 and a part of an intermediate layer (convolutional layer) corresponding to the decoder part of the identification model 52. The server 1 sets the parameters obtained by learning the identification model 52 as the initial values of the parameters of the intermediate layer, gives training data (first training data), and performs learning. The server 1 inputs a tomographic image for training, calculates the plaque burden, compares it with the correct value, and optimizes the parameters so that the two are approximated. By performing transfer learning in this way, the amount of data and the learning time for which the calculation model 51 learns can be suppressed.

[0096] FIG. 18 is a flowchart showing the procedure of the generation process of the calculation model 51 according to Modification 2. After acquiring training data (first training data) for generating the calculation model 51 (step S11), the server 1 executes the following processing. The server 1 will be described assuming that the identification model 52 has already been generated.

[0097] The control unit 11 of server 1 generates the calculation model 51 based on the parameters obtained by generating the identification model 52 and the training data (step S301). Specifically, as described above, the control unit 11 sets the parameters such as the weights obtained by the learning of the identification model 52 as the initial values of the parameters of the calculation model 51 and then performs learning. The control unit 11 inputs the tomographic images for training into the calculation model 51 to calculate the plaque burden, compares it with the correct value, and optimizes the parameters so that the two are approximated. The control unit 11 ends a series of processes.

[0098] As described above, according to the first modification example, by diverting the learning result of the identification model 52, it is possible to suppress the amount of data and the learning time that the calculation model 51 learns.

[0099] The embodiments disclosed this time should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above meaning but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included.

Explanation of Reference Numerals

[0100] 1 Server 11 Control unit 12 Main memory unit 13 Communication unit 14 Auxiliary storage unit P1 Program 2 Image diagnostic apparatus 21 Control unit 22 Main memory unit 23 Communication unit 24 Display unit 25 Input unit 26 Image processing unit 27 Auxiliary storage unit P2 Program 51 Calculation model 52 Identification model 5 Sagittal tomographic image 6 Gradation bar 7 Stent region

Claims

1. An acquisition unit that acquires cross-sectional images of a plurality of frames obtained by imaging a patient's blood vessel; A calculation unit that calculates the plaque burden at each position of the blood vessel along the axial direction of the blood vessel based on the cross-sectional images of the plurality of frames; A division unit that divides the longitudinal section of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold value and a second region where the plaque burden is less than the threshold value; A determination unit that determines, from each of the divided regions, a region where both ends of the treatment range of the blood vessel by a predetermined treatment device should be located An information processing apparatus comprising the same.

2. The determination unit determines a region where both ends of a stent to be indwelled in the blood vessel should be located The information processing apparatus according to claim 1.

3. The determination unit determines the two second regions sandwiching the first region as the regions where both ends should be located The information processing apparatus according to claim 1.

4. The determination unit Identifies the first region where the plaque burden takes the maximum value from each of the regions obtained by dividing the longitudinal section, And determines the two second regions sandwiching the identified first region as the regions where both ends should be located The information processing apparatus according to claim 3.

5. It comprises a reception unit that receives a setting input for setting the threshold value from a user, The division unit divides the longitudinal section according to the set threshold value The information processing apparatus according to any one of claims 1 to 4.

6. The calculation unit inputs the cross-sectional images of the plurality of frames into a learned calculation model that calculates the plaque burden when the cross-sectional images are input, and calculates the plaque burden in each frame The information processing apparatus according to any one of claims 1 to 5.

7. The calculation unit inputs the plurality of tomographic images into a learned discrimination model that discriminates an image region corresponding to a plaque when the tomographic image is input, and discriminates the image region in each frame, and calculates the plaque burden in each frame based on the area of the discriminated image region The information processing apparatus according to any one of claims 1 to 5.

8. The calculation unit calculates the plaque burden in each frame and a reliability indicating the certainty of the calculation result of the plaque burden in each frame, includes a specifying unit that specifies a low-reliability region where the reliability is equal to or lower than a predetermined value in the longitudinal section of the blood vessel, When the length of the low-reliability region in the axial direction is equal to or shorter than a predetermined length, the dividing unit divides a plurality of the first regions or second regions straddling the low-reliability region as the same region The information processing apparatus according to claim 6 or 7.

9. includes a second reception unit that receives a setting input for setting the predetermined length from a user, The dividing unit divides the longitudinal section according to the set predetermined length The information processing apparatus according to claim 8.

10. includes a display unit that displays a longitudinal tomographic image of the blood vessel corresponding to the longitudinal section and a second object that is a predetermined-shaped object displayed corresponding to the longitudinal section and has both ends located in regions where both ends should be located The information processing apparatus according to any one of claims 1 to 9.

11. The display unit further displays a first object representing the magnitude of the plaque burden at each position of the longitudinal tomographic image along the axial direction The information processing apparatus according to claim 10.

12. includes a length measuring unit that measures the length of the first region in the axial direction The display unit displays the measured length in association with the first region in the longitudinal tomographic image. The information processing apparatus according to claim 10 or 11.

13. Obtain tomographic images of a plurality of frames of a patient's blood vessel, Calculate the plaque burden at each position of the blood vessel along the axial direction of the blood vessel based on the tomographic images of the plurality of frames, Divide the longitudinal section of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold value and a second region where the plaque burden is less than the threshold value, Determine, from each of the divided regions, a region where both ends of the treatment range of the blood vessel by a predetermined treatment device should be located. An information processing method in which a computer executes processing.

14. Obtain tomographic images of a plurality of frames of a patient's blood vessel, Calculate the plaque burden at each position of the blood vessel along the axial direction of the blood vessel based on the tomographic images of the plurality of frames, Divide the longitudinal section of the blood vessel into a first region where the plaque burden is equal to or greater than a threshold value and a second region where the plaque burden is less than the threshold value, Determine, from each of the divided regions, a region where both ends of the treatment range of the blood vessel by a predetermined treatment device should be located. A program for causing a computer to execute processing.

15. Obtain first training data in which data indicating an image region corresponding to a plaque is given to a tomographic image of a patient's blood vessel, Generate an identification model that identifies an image region corresponding to the plaque when the tomographic image is input based on the first training data, Obtain second training data in which a correct value of the plaque burden is given to the tomographic image, Generate a calculation model that calculates the plaque burden when the tomographic image is input based on the learned parameters obtained by generating the identification model and the second training data. A method for generating a model for a computer to execute a process.

Images