Information processing device, information processing method, and program

Abstract

To provide an information processing device or the like capable of preferably supporting image diagnosis related to blood vessel treatment.SOLUTION: An information processing device 2 includes: an acquisition part that acquires a transverse tomographic image of a plurality of frames obtained by imaging a blood vessel of a patient; a calculation part that calculates plaque burden in each frame by inputting the acquired transverse tomographic image of each frame to a learned model so as to calculate plaque burden when the transverse tomographic image is input; and a display part that displays a first object representing magnitude of the plaque burden at each position along an axial direction of the blood vessel on the basis of the plaque burden in each frame.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] There is a technique for assisting image diagnosis related to vascular treatment such as PCI (Percutaneous Coronary Intervention). For example, in Patent Document 1, there is disclosed a medical image processing apparatus that converts three-dimensional CT (Computer Tomography) data into two-dimensional projection images, extracts a plaque region of a blood vessel from the three-dimensional CT data, calculates a stenosis rate of the blood vessel, and displays a projection image in which a color value of the plaque region is determined according to the calculated stenosis rate or the like.

Prior Art Documents

[0007] In one respect, it can effectively support image-based diagnostics related to vascular treatment. [Brief explanation of the drawing]

[0008] [Figure 1] This is an explanatory diagram showing an example configuration of an image diagnostic system. [Figure 2] This is a block diagram showing an example server configuration. [Figure 3] This is a block diagram showing an example of the configuration of a medical imaging device. [Figure 4] This is an explanatory diagram regarding the calculation process of plaque baden. [Figure 5] This is an explanatory diagram showing an example of a plaque. [Figure 6] This is an explanatory diagram showing other examples of plaques. [Figure 7] This is an explanatory diagram regarding the process of determining the treatment area for blood vessels. [Figure 8] This is an explanatory diagram showing an example of how lesion length is displayed. [Figure 9] This is an explanatory diagram showing an example of superimposed stent display. [Figure 10] This is a flowchart showing the steps involved in generating the calculation model. [Figure 11] This is a flowchart showing the procedure for calculating plaque badens. [Figure 12] This is an explanatory diagram regarding the process for identifying the lesion area related to Modification Example 1. [Figure 13] It is a block diagram showing a configuration example of an image diagnostic apparatus according to Embodiment 2. [Figure 14] It is an explanatory diagram regarding the calculation process of plaque burden according to Embodiment 2. [Figure 15] It is a flowchart showing the procedure of the generation process of the identification model. [Figure 16] It is a flowchart showing the procedure of the calculation process of plaque burden according to Embodiment 2. [Figure 17] It is an explanatory diagram regarding the generation process of the calculation model according to Modified Example 2. [Figure 18] It is a flowchart showing the procedure of the generation process of the calculation model according to Modified Example 2.

Mode for Carrying Out the Invention

[0009] Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments. (Embodiment 1) FIG. 1 is an explanatory diagram showing a configuration example of an image diagnostic system. In the present embodiment, an image diagnostic system that calculates the plaque burden from a tomographic image of a patient's blood vessel and supports blood vessel treatment using a predetermined treatment device will be described. The image diagnostic apparatus 2 includes a server 1 and an image diagnostic apparatus (information processing apparatus) 2. Each apparatus is communicatively connected via a network N such as the Internet.

[0010] The server 1 is a server computer capable of various information processing and information transmission and reception. Note that the apparatus corresponding to the server 1 may be an apparatus such as a personal computer. The server 1 performs machine learning for learning predetermined training data and functions as a generation device that generates a calculation model 51 (see FIG. 4) for calculating the plaque burden with a tomographic image of a patient's blood vessel as an input.

[0011] Note that the plaque burden is the area ratio of the plaque in the cross-section of the blood vessel, and is the value obtained by dividing the cross-sectional area of the plaque and the tunica media by the cross-sectional area of the blood vessel (the area of the region within the EEM (External Elastic Membrane)) (see Fig. 4).

[0012] The imaging device 2 is an imaging device that captures a medical image of a patient's blood vessel, and is, for example, an IVUS (Intravascular Ultrasound) device that performs an ultrasonic examination using a catheter 201. The catheter 201 is a medical instrument inserted into a patient's blood vessel, and transmits an ultrasonic signal from an ultrasonic probe attached to the tip of the catheter 201 and receives a reflected wave. The imaging device 2 generates and displays an ultrasonic tomogram based on the reflected wave received by the catheter 201.

[0013] In this embodiment, the imaging device 2 is described as an IVUS device, but it may be an optical imaging device such as an OCT (Optical Coherence Tomography) device. Also, the blood vessel tomogram may be captured by a method other than IVUS and OCT.

[0014] The data of the calculation model 51 generated by the server 1 is installed in the imaging device 2. The imaging device 2 inputs the blood vessel tomogram captured using the catheter 201 into the calculation model 51 to calculate the plaque burden. Then, the imaging device 2 presents the calculated plaque burden to the user (medical staff). Also, the imaging device 2 identifies the lesion region of the blood vessel based on the calculated plaque burden and supports the determination of the placement position of a stent (treatment device) to be placed in the blood vessel.

[0015] In this embodiment, the medical imaging device 2 calculates plaque baden using the calculation model 51, but the server 1 on the cloud may perform the processing using the calculation model 51. Alternatively, for example, a general-purpose computer connected to the medical imaging device 2 may perform the processing. Thus, the processing entity that performs the series of processes is not particularly limited.

[0016] Furthermore, while a stent is given as an example of a treatment device in this embodiment, as will be described later, a balloon for dilating blood vessels, a rotablator for removing narrowed lesions in blood vessels, etc. may also be used.

[0017] Figure 2 is a block diagram showing an example configuration of Server 1. Server 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, etc. by reading and executing the program P1 stored in the auxiliary storage unit 14. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and temporarily stores the data necessary for the control unit 11 to perform arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and sends and receives information with the outside. The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or hard disk, and stores the program P1 and other data necessary for the control unit 11 to perform processing.

[0018] Server 1 may be a multi-computer system consisting of multiple computers, or it may be a virtual machine created virtually by software.

[0019] Furthermore, in this embodiment, the server 1 is not limited to the above configuration and may include, for example, an input unit for receiving operation input, a display unit for displaying images, etc. The server 1 may also include a read unit for reading portable storage media 1a such as a CD (Compact Disk)-ROM or DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage media 1a. Alternatively, the server 1 may read the program P1 from a semiconductor memory 1b.

[0020] Figure 3 is a block diagram showing an example configuration of the medical imaging device 2. The medical imaging device 2 comprises a control unit 21, a main memory unit 22, a communication unit 23, a display unit 24, an input unit 25, an image processing unit 26, and an auxiliary memory unit 27. The control unit 21 is one or more arithmetic processing units such as CPUs, MPUs, and GPUs, and performs various information processing and control processing by reading and executing the program P2 stored in the auxiliary storage unit 27. The main memory unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to perform arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information with the outside. The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard or mouse, and accepts operation input from the user. The image processing unit 26 is an image processing module that processes signals transmitted and received via the catheter 201 and generates tomographic images.

[0021] The auxiliary storage unit 27 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. The auxiliary storage unit 27 also stores the calculation model 51. The calculation model 51 is a machine learning model generated by learning predetermined training data, and is a trained model that calculates plaque bathens using transverse images of blood vessels captured by the image diagnostic device 2 as input. The calculation model 51 is intended to be used as a program module that constitutes part of the artificial intelligence software.

[0022] Note that the image diagnostic apparatus 2 may include a reading unit that reads a portable storage medium 2a such as a CD-ROM, and reads and executes the program P2 from the portable storage medium 2a. Alternatively, the image diagnostic apparatus 2 may read the program P2 from the semiconductor memory 2b.

[0023] FIG. 4 is an explanatory diagram regarding the calculation process of the plaque burden. In FIG. 4, each frame of the tomographic image of blood vessels captured by the image diagnostic apparatus 2 is input into the calculation model 51, and the state of calculating the plaque burden is conceptually illustrated.

[0024] The calculation model 51 is a machine learning model that has learned predetermined training data, and is, for example, a CNN (Convolution Neural Network). Note that the calculation model 51 may be any model that can calculate the plaque burden from the tomographic image, and may be a model based on other learning algorithms such as a neural network other than CNN, a decision tree, or a SVM (Support Vector Machine).

[0025] The calculation model 51 includes an input layer that receives an input of an image (tomographic image), an intermediate layer that extracts feature amounts of the input image, and an output layer that calculates (outputs) the plaque burden. The input layer receives an input of an image and passes the data of the input image to the intermediate layer. The intermediate layer includes a convolutional layer that convolves the data of the input image, extracts the feature amounts of the input image, and passes them to the output layer. The output layer calculates the plaque burden based on the feature amounts extracted by the intermediate layer.

[0026] In the present embodiment, the process executed by the calculation model 51 is treated as a regression problem, and the plaque burden is predicted as a continuous value. Note that this process may be treated as a classification problem, and the plaque burden may be predicted (classified) within a certain numerical range, for example, as “0 to X1”, “X1 to X2”, “X2 to X3”... (0 < X1 < X2 < X3...). Alternatively, without predicting the plaque burden numerically, it may be classified into multiple stages (for example, classified into a red zone, an orange zone, a yellow zone...).

[0027] Server 1 generates a computation model 51 using training data (second training data) that has been assigned ground truth values ​​for plaque bathen to training vascular tomography images (transverse images). The training tomography images are vascular tomography images taken of actual patients, and the ground truth values ​​are the actual values ​​of plaque bathen in the tomography images of those patients. Server 1 inputs the training tomography images into the computation model 51 to calculate plaque bathen and compares it with the ground truth values. Server 1 optimizes parameters such as the weights between neurons so that the two approximate each other, and generates the computation model 51.

[0028] The imaging diagnostic device 2 uses the calculation model 51 generated by the server 1 to calculate the plaque bathden in the blood vessels of patients undergoing catheter treatment. Specifically, the imaging diagnostic device 2 sequentially inputs tomographic images of multiple frames acquired in response to the pullback movement of the catheter 201 into the calculation model 51 and calculates the plaque bathden in each frame. As a result, the imaging diagnostic device 2 calculates the area ratio, i.e., the plaque bathden, by dividing the area of ​​the image region corresponding to the plaque and tunica media (referred to as the "plaque region" in the following explanation) by the area of ​​the EEM region (the region that includes the plaque region and the lumen region) (see Figure 4, bottom right).

[0029] The diagnostic imaging device 2 presents the user with the plaque bardens in each frame calculated above, i.e., the plaque bardens at each position in the blood vessel along the axial direction, to assist in determining the placement position of the stent.

[0030] Figure 5 is an explanatory diagram showing an example of plaque bathen display. In this embodiment, the diagnostic imaging device 2 displays predetermined objects (first objects) representing the size of plaque bathen at various locations in the blood vessel, corresponding to the longitudinal tomographic image 5 reconstructed from multiple frames of transverse images. Specifically, as shown in Figure 5, the diagnostic imaging device 2 displays a gradient bar 6 above the longitudinal tomographic image 5, in which the display color gradually changes according to the size of the plaque bathen.

[0031] The longitudinal tomographic image 5 is a longitudinal section that reproduces the appearance of the blood vessel along its axial direction (direction of course), and is generated from transverse tomographic images of multiple frames acquired in response to a pullback motion. For example, as shown in Figure 5, the diagnostic imaging device 2 displays the longitudinal tomographic image 5 with the axial direction of the blood vessel as the transverse direction.

[0032] The imaging diagnostic device 2 displays a gradient bar 6 above the longitudinal tomographic image 5, with a length approximately equal to the length of the longitudinal tomographic image 5 along the axial direction. The gradient bar 6 is a bar-shaped object representing the size of plaque badens, and its display color changes according to the size of the plaque badens at each corresponding position in the longitudinal tomographic image 5, i.e., in the transverse tomographic image of each frame. The imaging diagnostic device 2 changes the display color at each position of the gradient bar 6 according to the size of the plaque badens at each position on the longitudinal tomographic image 5.

[0033] For convenience, in Figure 5, areas with large plaque bathens (areas above the threshold) are shown using hatching. Furthermore, although the gradient of the display color within gradient bar 6 is actually finely adjusted according to the plaque bathen value as described below, for convenience, areas with large plaque bathens are shown using the same hatching in Figure 5.

[0034] Furthermore, in this embodiment, the object representing the size of the plaque baden is described as a gradient bar 6, but the object is not limited to a gradient bar 6. For example, the diagnostic imaging device 2 may display a line graph representing the size of the plaque baden above the longitudinal tomographic image 5. Thus, the gradient bar 6 is just one example of the object, and any object that appropriately represents the size of the plaque baden is acceptable.

[0035] For example, the imaging diagnostic device 2 sets the display color of areas with a high concentration of plaque by referring to a predetermined threshold (e.g., 50%) that is set by default. Specifically, the imaging diagnostic device 2 sets the display color of areas where the plaque bathen is above the threshold to a warm color (e.g., orange), and sets the display color of areas below the threshold to the default color (e.g., white). Then, for areas where the plaque bathen is above the threshold, the imaging diagnostic device 2 changes the display color (e.g., increases the brightness) so that the larger the plaque bathen, the more it is emphasized, and performs a gradient display.

[0036] While gradient display methods are expected to include, for example, changing brightness, hue, saturation, etc., are also acceptable, and the specific display method is not restricted.

[0037] As described above, when displaying the gradient bar 6, it is preferable for the image diagnostic device 2 to hide areas where the reliability of the plaque barden calculation result in the calculation model 51 is low (hereinafter referred to as the "low-reliability area"). In Figure 5, the low-reliability area is shown as a blacked-out area. When the image diagnostic device 2 calculates the plaque barden in each frame using the calculation model 51, it obtains the reliability of the calculated value (for example, a probability value between 0 and 1) along with the plaque barden in each frame from the calculation model 51. When the reliability at each position (frame) of the blood vessel is below a predetermined value, i.e., the reliability is low, the image diagnostic device 2 makes the corresponding area on the gradient bar 6 blank as the low-reliability area. This allows the user to see the accuracy of the plaque barden calculation result.

[0038] In this way, the imaging diagnostic device 2 displays a gradient bar 6 along with the longitudinal tomographic image 5, quantitatively presenting the condition of the blood vessels at each location. Furthermore, as shown in Figure 5, the imaging diagnostic device 2 displays the maximum, minimum, and average values ​​of plaque bathen. This allows the user (healthcare professional) to understand the condition of the blood vessels and determine the treatment area to be treated with a treatment device such as a stent.

[0039] Figure 6 is an explanatory diagram showing another example of plaque bathen display. In the example in Figure 5, warm colors were used to indicate areas where the plaque bathen was above a threshold, but gradient display may also be used for areas where the plaque bathen was below a threshold, i.e., areas with little plaque.

[0040] For example, the diagnostic imaging device 2 sets the display color of areas where the plaque bathen is above a threshold to a warm color, and the display color of areas where the plaque bathen is below a threshold to a cool color. In Figure 6, areas where the plaque bathen is above a threshold and areas where it is below a threshold are illustrated with different hatching. The diagnostic imaging device 2 also changes the display color (for example, increases the brightness) as the plaque bathen decreases, similar to areas where the plaque bathen is above a threshold, and displays a gradient. In this way, the diagnostic imaging device 2 may present areas with a lot of plaque that may require treatment, as well as areas with little plaque that do not require treatment.

[0041] In the example in Figure 6, the gradient display area is divided into two sections, but it is also possible to add a threshold and divide the area into three or more sections for gradient display.

[0042] Furthermore, when performing the gradient display described above, the user may be allowed to set the threshold value used as the display criterion. Specifically, the image diagnostic device 2 receives a setting input from the user to change the threshold value from the default value to an arbitrary value, and determines whether the plaque bathen is above or below the threshold value according to the set threshold, and changes the display color accordingly. This allows the user to visualize their area of ​​interest themselves, and provides more appropriate support to the user.

[0043] Figure 7 is an explanatory diagram regarding the process of determining the treatment area of ​​blood vessels. In this embodiment, the treatment area to be treated with a predetermined treatment device is further presented to the user based on the calculation result of plaque barden.

[0044] Specifically, the imaging diagnostic device 2 determines the positions of both ends of the stent to be placed in the blood vessel. When performing vascular treatment such as PCI, it is preferable to place the stent so that the lesion is completely covered, but in reality, it is difficult to cover the lesion completely because plaque is continuously present. For example, the stent is basically placed so that both ends are located in a normal area where no plaque is present, but if this is difficult, the stent is placed so that both ends are located in an area where the plaque bathen is below a predetermined value (e.g., 50%). If that is also difficult, the stent is placed so that both ends are located in the area with the least amount of plaque bathen.

[0045] Currently, as mentioned above, plaque bathen is not calculated for all frames. Instead, narrowed areas are identified visually from images, and plaque bathen for a few frames is measured as needed. Therefore, in this embodiment, the range in which both ends of the stent should be located is determined based on the plaque bathen calculated using calculation model 51.

[0046] In this embodiment, a stent is given as an example of a treatment device, but the treatment device is not limited to a stent. For example, the imaging diagnostic device 2 may determine the positions of both ends of the dilation range to be dilated using a balloon used to dilate a blood vessel. Alternatively, the imaging diagnostic device 2 may determine the positions of both ends of the cutting range to be cut using a rotablator used to cut a hard lesion (e.g., calcified tissue). Thus, the imaging diagnostic device 2 only needs to be able to determine the positions of both ends of the treatment range to be treated with a treatment device inserted into a blood vessel, and the treatment device is not limited to a stent.

[0047] First, the imaging diagnostic device 2 divides the longitudinal section 5 of the blood vessel into a first region where the plaque bathen is above a predetermined threshold and a second region where the plaque bathen is below a threshold. This threshold is, for example, the same value as the threshold used as the basis for the gradient display described above, but it may be a different value. Based on the plaque bathen in each frame calculated by the calculation model 51, the imaging diagnostic device 2 divides the longitudinal section 5 into a first region L1, L2, L3… (L: Lesion) where the plaque bathen is above a threshold and a second region H1, H2, H3… (H: Healthy) where the plaque bathen is below a threshold.

[0048] In this embodiment, the imaging diagnostic device 2 identifies a first region LX (X=1, 2, 3…) that includes the position (frame) where the plaque bathen is at its maximum value as the lesion region LX where the stent should be placed. The imaging diagnostic device 2 then determines two second regions HY and HZ (Y, Z=1, 2, 3…) located on either side of the identified lesion region LX as the regions where both ends of the stent should be located.

[0049] Figure 7 shows a longitudinal tomographic image 5 containing one first region L1 and two second regions HP1 and HD1 (P: Proximal, D: Distal). When the plaque bathen reaches its maximum value in the first region L1, the imaging device 2 determines that the second regions HP1 and HD1, which sandwich the first region L1, are the regions where both ends of the stent should be located.

[0050] Furthermore, if the diagnostic imaging device 2 has low-confidence regions in the calculation of plaque barden, it is preferable to group multiple first or second regions that straddle the low-confidence regions as the same region. For example, in Figure 7, there are two low-confidence regions in the first region L1. In this case, the diagnostic imaging device 2 determines whether the length of each low-confidence region in the axial direction of the blood vessel is less than or equal to a predetermined length (e.g., 1 mm). If the low-confidence region is less than or equal to the predetermined length, the diagnostic imaging device 2 divides the multiple first regions that straddle the low-confidence regions into the same region. In the example in Figure 7, since both low-confidence regions are less than or equal to the predetermined length, the diagnostic imaging device 2 divides the three regions that straddle the low-confidence regions into the same first region L1.

[0051] Similarly, with respect to the second region, if there are multiple second regions that span a low-reliability region of a predetermined length or less, the imaging diagnostic device 2 divides these multiple second regions into a single region. In this way, when the length of the low-reliability region in the axial direction is less than or equal to a predetermined length, the imaging diagnostic device 2 divides multiple first regions or multiple second regions that span the low-reliability region into a single region. This allows for the appropriate determination of the stent placement position while ensuring reliability.

[0052] The predetermined length may be made variable by the user. For example, the diagnostic imaging device 2 accepts a setting input to change the default value of the predetermined length to an arbitrary value (for example, lengths in predetermined increments of 0.6 mm, 0.8 mm, 1.2 mm, etc.). The diagnostic imaging device 2 divides each region by determining whether or not it is less than or equal to the set predetermined length. This allows the user to more favorably determine the stent placement position.

[0053] Furthermore, the diagnostic imaging device 2 may allow the user to vary the threshold used as the basis for region division, similar to when displaying gradients. In this case, the diagnostic imaging device 2 may allow different thresholds to be set for the proximal and distal sides of the blood vessel. For example, if the threshold for the proximal side is set to 50% and the threshold for the distal side is set to 60%, the diagnostic imaging device 2 will divide the area, with the position where the plaque barden is 50% or more (viewed from the left side of Figure 7) as the starting point of the first region, and the position where the plaque barden is less than 60% as the ending point of the first region. This allows the user to finely adjust the first and second regions themselves.

[0054] Figure 8 is an explanatory diagram showing an example of how lesion length is displayed. As described above, the imaging diagnostic device 2 divides the longitudinal tomographic image 5 into first regions L1, L2, L3... and second regions H1, H2, H3..., and identifies lesion region L1. Once lesion region L1 is identified, the imaging diagnostic device 2 measures the length of lesion region L1 in the axial direction of the blood vessel, i.e., the lesion length, and displays it in correspondence with lesion region L1 in the longitudinal tomographic image 5. Specifically, as shown in Figure 8, the imaging diagnostic device 2 displays the portion corresponding to lesion region L1 on the gradient bar 6 with an arrow, and also displays the lesion length. This makes it possible to visualize the length of the lesion site.

[0055] Figure 9 is an explanatory diagram showing an example of superimposed display of a stent. In this embodiment, the diagnostic imaging device 2 further presents the user with the treatment area to be treated with the treatment device, that is, the area where the stent should be placed. Specifically, the diagnostic imaging device 2 superimposes a stent region 7 (second object) representing the size of the stent (treatment device) onto the longitudinal tomographic image 5.

[0056] The stent region 7 is an object that simulates a stent placed in a blood vessel, and is a rectangular object that represents the stent placement area. Note that the stent region 7 only needs to have a length in the axial direction that matches the length of the actual stent, but its radial length (vertical length in Figure 8) does not need to match.

[0057] Furthermore, in this embodiment, the treatment device is a stent, so the object is rectangular in shape. However, if the treatment device is a balloon, for example, the object could be sandbag-shaped. In this way, the diagnostic imaging device 2 only needs to be able to display an object with a shape appropriate to the treatment device.

[0058] For example, the diagnostic imaging device 2 displays a stent region 7 having a length greater than or equal to the lesion length measured above, such that both ends of the stent region 7 are located in the second regions HP1 and HD1, which were determined above as the regions where both ends should be located. For example, the diagnostic imaging device 2 stores information on each stent product available for use in the procedure in predetermined length increments (e.g., 9 mm, 12 mm, 15 mm, etc.) in the auxiliary storage unit 27. The diagnostic imaging device 2 selects a stent from these stents that exceeds the lesion length and has the minimum length. The diagnostic imaging device 2 then superimposes and displays the stent region 7 having the length of the selected stent so that both ends are located in the second regions HP1 and HD1. Note that the selection of the stent and the determination of the superimposed position of the stent region 7 may be performed manually by the user.

[0059] For example, the diagnostic imaging device 2 overlays the stent region 7 so that its center coincides with the center of the lesion region L1 in the axial direction of the blood vessel. In this case, as shown in Figure 9, downward arrows are displayed at positions corresponding to both ends of the stent, and double-headed arrows are displayed for the portion that exceeds the first region L1. The diagnostic imaging device 2 moves the stent region 7 according to user input (e.g., mouse operation), allowing the user to determine the optimal stent placement position.

[0060] In this case, the imaging diagnostic device 2 preferably displays transverse layer images at positions corresponding to both ends of the stent region 7, as shown in Figure 9. This allows the user to search for the optimal stent placement position while moving the stent region 7.

[0061] In the above example, the stent region 7 was superimposed on the longitudinal section image 5. However, the imaging diagnostic device 2 may display the stent region 7 in a location other than the longitudinal section image 5 (for example, above or within the gradient bar 6). Thus, the imaging diagnostic device 2 only needs to be capable of displaying an object simulating a treatment device, and its display position is not limited to the longitudinal section image 5.

[0062] Figure 10 is a flowchart showing the procedure for generating the computational model 51. Based on Figure 10, the process for generating the computational model 51 using machine learning will be explained. The control unit 11 of server 1 acquires training data (second training data) for generating the calculation model 51 (step S11). The training data is data in which the correct values ​​of plaque bathens are associated with transverse images of training blood vessels.

[0063] The control unit 11 generates a calculation model 51 that calculates plaque barden when a transverse image of a blood vessel is input, based on the training data (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the calculation model 51. The control unit 11 inputs the training transverse image of a blood vessel into the calculation model 51 to calculate the plaque barden and compares it with the correct value. The control unit 11 optimizes parameters such as the weights between neurons so that the calculated plaque barden approximates the correct value, and generates the calculation model 51. The control unit 11 then completes the series of processes.

[0064] Figure 11 is a flowchart showing the procedure for calculating plaque bathen. Based on Figure 11, the processing performed by the imaging diagnostic device 2 will be explained. The control unit 21 of the diagnostic imaging device 2 acquires transverse images of multiple frames of the patient's blood vessels (step S31). The control unit 21 inputs the transverse images of each frame into the calculation model 51 and calculates the plaque bathen for each frame (step S32). Specifically, the control unit 21 obtains the plaque bathen for each frame and the confidence level of the calculated value from the calculation model 51.

[0065] The control unit 21 identifies the vessel location (frame) where the plaque bathen is at its maximum value (step S33). The control unit 21 also identifies a low-confidence region in the axial direction of the vessel where the confidence level of the plaque bathen is below a predetermined value (step S34). The control unit 21 also accepts a threshold setting input (step S35). The control unit 21 may skip step S35 and set a default threshold.

[0066] The control unit 21 generates (reconstructs) a longitudinal section of the blood vessel based on the transverse section images of multiple frames acquired in step S31 (step S36). The control unit 21 displays the generated longitudinal section and objects (first objects) representing the size of plaque bathens at each position in the longitudinal section along the axial direction of the blood vessel (step S37). Specifically, as described above, the control unit 21 displays a gradient bar in which the display color gradually changes according to the size of the plaque bathens. In this case, the control unit 21 displays a gradient bar in which the low-confidence region is hidden.

[0067] The control unit 21 divides the longitudinal section of the blood vessel into a first region where the plaque bathen is above a threshold and a second region where the plaque bathen is below a threshold, based on the plaque bathen at each position (frame) of the blood vessel (step S38). Specifically, the control unit 21 divides the longitudinal section into multiple regions depending on whether the plaque bathen is above a threshold or not, and if the length of the low-reliability region is less than or equal to a predetermined length, it divides multiple first or second regions that span the low-reliability region into a single region.

[0068] The control unit 21 identifies a first region containing the maximum value of the plaque bathin as the lesion region (step S39). Then, according to the identified lesion region, the control unit 21 determines the regions where both ends of the treatment area to be treated with the treatment device should be located (step S40). Specifically, the control unit 21 determines that the second regions located on both sides of the lesion region should be the regions where both ends of the stent to be placed in the blood vessel should be located.

[0069] The control unit 21 measures the axial length of the lesion region identified in step S39, i.e., the lesion length, and displays it in correspondence with the lesion region in the longitudinal tomographic image (step S41). The control unit 21 also displays an object representing the size of the treatment device (second object) in correspondence with the longitudinal tomographic image (step S42). For example, as described above, the control unit 21 superimposes a stent region representing the length of the stent in the axial direction of the blood vessel onto the longitudinal tomographic image so that both ends are located in the region determined in step S40. The control unit 21 then completes the series of processes.

[0070] As described above, according to this embodiment 1, plaque bardens at each position in the blood vessel are calculated from transverse images of multiple frames of the patient's blood vessels, and objects such as gradient bars representing the magnitude of the calculated plaque bardens are displayed corresponding to the longitudinal images. This effectively supports image diagnosis related to vascular treatment, such as determining the placement position of a stent.

[0071] Furthermore, according to this embodiment 1, by hiding the low-confidence region where the reliability of the plaque-baden calculation result is low, the reliability of the plaque-baden calculation result can be presented to the user.

[0072] Furthermore, according to this embodiment 1, by also presenting the length of the lesion area (lesion length) and the stent placement area (stent area), image diagnosis can be more effectively supported.

[0073] Furthermore, according to this embodiment 1, the longitudinal section (longitudinal tomographic image) of the blood vessel is divided into multiple regions according to the plaque bathen, and the regions where both ends of the treatment area of ​​the blood vessel by a treatment device such as a stent should be located are determined. This makes it possible to more effectively support image diagnosis related to vascular treatment, such as determining the placement position of a stent.

[0074] Furthermore, according to this embodiment 1, if the length of the low-reliability region is less than or equal to a predetermined length, multiple regions (first region or second region) that span the low-reliability region are considered as the same region and divided accordingly. This allows for the appropriate determination of the stent placement position, etc., while ensuring reliability.

[0075] (Variation 1) In Embodiment 1, the first region where the plaque bathen is at its maximum value was identified as the lesion region. However, the lesion region does not necessarily have to be the region where the plaque bathen is at its maximum value.

[0076] Figure 12 is an explanatory diagram regarding the process of identifying the lesion area according to the modified example 1. In Embodiment 1, the lesion area was identified first, and then the areas where both ends of the stent should be located were determined. However, it is also possible to determine the areas where both ends of the stent should be located first, and then identify the lesion area.

[0077] Specifically, the imaging diagnostic device 2 divides the longitudinal tomographic image 5 into first regions L1, L2, L3… and second regions H1, H2, H3…, and then searches for two second regions Hy and Hz flanking the first region Lx. If two second regions Hy and Hz flanking the first region Lx are found, the imaging diagnostic device 2 identifies the first region Lx where the two second regions Hy and Hz are located on both sides as the lesion region Lx. The imaging diagnostic device 2 then measures the length of the lesion region Lx (lesion length) and displays the stent region 7 superimposed on it.

[0078] Thus, the diagnostic imaging device 2 may determine the area where the stent should be placed based on the positional relationship between the first and second regions. The determination method described in Embodiment 1 is just one example, and various other methods are conceivable.

[0079] (Embodiment 2) Embodiment 1 described a configuration in which the calculation model 51 directly calculates plaque barden. This embodiment describes a configuration in which a different machine learning model is used to identify plaque regions from vascular tomographic images (transverse images) and indirectly calculate plaque barden. Note that components that overlap with Embodiment 1 are denoted by the same reference numerals and their descriptions are omitted.

[0080] Figure 13 is a block diagram showing an example configuration of the medical imaging device 2 according to Embodiment 2. The auxiliary storage unit 27 of the medical imaging device 2 according to this embodiment stores the identification model 52. The identification model 52 is a machine learning model generated by learning predetermined training data, similar to the calculation model 51 according to Embodiment 1, and is a trained model that identifies plaque regions within a tomographic image using a vascular tomographic image as input. The identification model 52 is intended to be used as a program module that constitutes part of artificial intelligence software.

[0081] Figure 14 is an explanatory diagram of the plaque bathen calculation process according to Embodiment 2. Figure 14 conceptually illustrates how the identification model 52 is used to identify image regions corresponding to plaque and tunica media (hereinafter referred to as "plaque regions") from a transverse image of a blood vessel, and how the plaque bathen is calculated from the area of ​​the identified plaque regions. Based on Figure 14, the outline of this embodiment will be explained.

[0082] The discrimination model 52 is a machine learning model that has been trained on predetermined training data, and is, for example, a neural network generated by deep learning. In this embodiment, the discrimination model 52 is described as a semantic segmentation model, which is a type of CNN.

[0083] A semantic segmentation model is a neural network that identifies objects in an image on a pixel-by-pixel basis. It comprises a convolutional layer (encoder) that convolves the input image and a deconvolutional layer (decoder) that maps the convolved features back to the original image size. The deconvolutional layer identifies which objects are located where in the image based on the features extracted by the convolutional layer, and generates binarized data indicating which object each pixel corresponds to.

[0084] Server 1 generates a discrimination model 52 using training data (first training data) in which data indicating the correct plaque region is associated with training tomographic images. Server 1 inputs the training tomographic images into the discrimination model 52 to identify plaque regions and optimizes parameters such as interneuronal weights so that the identified plaque regions approximate the correct plaque regions. The image diagnostic device 2 identifies plaque regions using the discrimination model 52 generated by Server 1.

[0085] In this embodiment, the identification model 52 is described as a semantic segmentation model, but it may also be a neural network other than a semantic segmentation model, a GAN (Generative Adversarial Network), or a model based on other learning algorithms.

[0086] Furthermore, Server 1 may generate a discrimination model 52 capable of identifying regions other than the plaque region by learning from training data that also includes ground truth data for regions other than the plaque region (e.g., the EEM region, the lumen region, etc.). In other words, the discrimination model 52 only needs to be able to identify the plaque region, and may also be able to identify other image regions.

[0087] When imaging the patient's blood vessels, the imaging diagnostic device 2 sequentially inputs the tomographic images of each frame into the identification model 52 and identifies the plaque region in each frame. Based on the identification results of the plaque region, the imaging diagnostic device 2 calculates the plaque bathen. That is, the imaging diagnostic device 2 calculates the area of ​​the plaque region and the area of ​​the entire blood vessel including the plaque region, and calculates the plaque bathen.

[0088] Thus, it is not essential for the machine learning model to directly calculate plaque bathen; it is also acceptable to calculate plaque bathen indirectly from the prediction results of the machine learning model.

[0089] Figure 15 is a flowchart showing the procedure for generating the discrimination model 52. Based on Figure 15, the process for generating the discrimination model 52 using machine learning will be explained. The control unit 11 of server 1 acquires training data (first training data) for generating the identification model 52 (step S201). The training data is data in which data indicating the correct plaque region is associated with a transverse image of a blood vessel used for training.

[0090] The control unit 11 generates a recognition model 52 that identifies plaque regions when a transverse image of a blood vessel is input, based on the training data (step S202). Specifically, as described above, the control unit 11 generates a CNN related to semantic segmentation as the recognition model 52. The control unit 11 inputs the training transverse image into the recognition model 52 to identify plaque regions and compares it with the correct plaque regions. The control unit 11 optimizes parameters such as the weights between neurons so that the two approximate each other, and generates the recognition model 52. The control unit 11 then completes the series of processes.

[0091] Figure 16 is a flowchart showing the procedure for calculating plaque bathen according to Embodiment 2. After acquiring transverse images of multiple frames of the patient's blood vessels (step S31), the diagnostic imaging device 2 performs the following processing. The control unit 21 of the diagnostic imaging device 2 inputs the transverse layer images of each frame into the identification model 52 and identifies the plaque region in each frame (step S221). Then, the control unit 21 calculates the plaque bathen in each frame based on the identification result (step S222). The control unit 21 then proceeds to step S31.

[0092] Based on the above, according to this embodiment 2, it is also possible to calculate plaque dirt using the identification results of the identification model 52, which identifies plaque regions from transverse images of blood vessels.

[0093] (Modification 2) In Embodiment 2, a discrimination model 52 capable of identifying plaque regions within a tomographic image was described. Alternatively, transfer learning may be performed in which parameters such as weights obtained from training the discrimination model 52 are applied to a computation model 51, and the computation model 51 is generated from the learning results of the discrimination model 52.

[0094] Figure 17 is an explanatory diagram regarding the generation process of the computational model 51 in the modified example 2. Figure 17 conceptually illustrates how transfer learning is performed by applying some of the network structure and parameters of the discrimination model 52 (semantic segmentation model) to the computational model 51 (CNN). In this modified example, it is assumed that Server 1 has already generated the discrimination model 52.

[0095] Server 1 performs training on a computation model 51, which has a network structure similar to that of a part of the discrimination model 52, by setting parameters such as weights obtained from generating (learning) the discrimination model 52 as initial values. For example, the computation model 51 has a structure in which an output layer for calculating plaque-baden is connected to an input layer similar to that of the discrimination model 52 and a part of the intermediate layer (convolutional layer) corresponding to the decoder part of the discrimination model 52. Server 1 sets the parameters obtained from learning the discrimination model 52 as the initial values ​​of the parameters of the intermediate layer and performs training by providing training data (first training data). Server 1 inputs training tomographic images, calculates plaque-baden, compares it with the correct value, and optimizes the parameters so that the two approximate each other. By performing transfer learning in this way, the amount of data that the computation model 51 learns and the training time can be suppressed.

[0096] Figure 18 is a flowchart showing the procedure for generating the calculation model 51 according to the modified example 2. After acquiring the training data (first training data) for generating the calculation model 51 (step S11), Server 1 executes the following process. Note that this explanation assumes that Server 1 has already generated the identification model 52.

[0097] The control unit 11 of server 1 generates a computation model 51 based on the parameters obtained from generating the discrimination model 52 and the training data (step S301). Specifically, as described above, the control unit 11 sets the parameters such as weights obtained from training the discrimination model 52 as the initial values ​​of the parameters of the computation model 51 and then performs training. The control unit 11 inputs the training tomographic images into the computation model 51 to calculate the plaque baden, compares it with the correct value, and optimizes the parameters so that the two approximate each other. The control unit 11 then completes the series of processes.

[0098] Based on the above, according to this modified example 1, by reusing the learning results of the discrimination model 52, the amount of data and learning time required for the calculation model 51 to learn can be reduced.

[0099] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims and not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0100] 1 server 11 Control Unit 12 Main memory 13 Communications Department 14 Auxiliary storage P1 Program 2. Diagnostic imaging equipment 21 Control Unit 22 Main memory 23 Communications Department 24 Display 25 Input section 26 Image Processing Unit 27 Auxiliary storage P2 Program 51 Calculation Model 52 Discriminant Models 5. Longitudinal Fault Images 6. Gradient Bar 7 Stent Area

Claims

1. An acquisition unit that acquires transverse layer images of multiple frames capturing the patient's blood vessels, A calculation unit that inputs the transverse layer images of each acquired frame into a model that has been trained to calculate plaque bathen when the aforementioned transverse layer images are input, and calculates the plaque bathen in each frame, The system includes a display unit that displays a first object representing the size of the plaque barden at each position along the axial direction of the blood vessel, based on the plaque barden in each frame. The calculation unit calculates the plaque baden for each frame and a confidence level indicating the certainty of the calculation result of the plaque baden for each frame. The vessel is provided with a specific unit that identifies a low-reliability region in the axial direction of the blood vessel where the reliability is less than or equal to a predetermined value. The display unit hides a portion of the first object corresponding to the low-confidence region. Information processing device.

2. The first object has different display characteristics in the region where the plaque is above a threshold and in the region where it is below a threshold. The information processing apparatus according to claim 1.

3. The system includes a receiving unit that accepts setting inputs to change the threshold, The display unit displays the first object, with the display mode changed according to the set threshold. The information processing apparatus according to claim 2.

4. The first object is a gradient bar whose display color gradually changes according to the size of the plaque baden. The information processing apparatus according to any one of claims 1 to 3.

5. The calculation unit described above, The model, which has been trained to identify plaque regions and EEM regions when the aforementioned transverse layer image is input, is then given the transverse layer image of each acquired frame to identify the plaque region and EEM region in each frame. The plaque bathen for each frame is calculated by dividing the area of ​​the plaque region by the area of ​​the EEM region for each frame. The information processing apparatus according to any one of claims 1 to 4.

6. By acquiring transverse layer images of multiple frames capturing the patient's blood vessels, The model, which has been trained to calculate plaque bathen when the aforementioned transverse layer image is input, is then given the transverse layer image of each acquired frame, and the plaque bathen in each frame is calculated. Based on the plaque barden in each frame, a first object representing the size of the plaque barden at each position along the axial direction of the blood vessel is displayed on the display unit. The plaque bathen in each frame and the confidence level indicating the certainty of the calculation result of the plaque bathen in each frame are calculated. In the axial direction of the blood vessel, a low-reliability region is identified where the reliability is less than or equal to a predetermined value. A portion of the first object corresponding to the low confidence region is hidden. An information processing method in which a computer performs the processing.

7. By acquiring transverse layer images of multiple frames capturing the patient's blood vessels, The model, which has been trained to calculate plaque bathen when the aforementioned transverse layer image is input, is then given the transverse layer image of each acquired frame, and the plaque bathen in each frame is calculated. Based on the plaque barden in each frame, a first object representing the size of the plaque barden at each position along the axial direction of the blood vessel is displayed on the display unit. The plaque bathen in each frame and the confidence level indicating the certainty of the calculation result of the plaque bathen in each frame are calculated. In the axial direction of the blood vessel, a low-reliability region is identified where the reliability is less than or equal to a predetermined value. A portion of the first object corresponding to the low confidence region is hidden. A program that instructs a computer to perform a process.

Images