Information processing method, program, information processing device, and model generation method

Abstract

In this information processing method, a computer executes a process for: acquiring data indicating a vascular tomographic image; inputting the acquired data indicating the vascular tomographic image into a model that has been trained so that, once data indicating a vascular tomographic image has been inputted, a plurality of designated detection regions including indistinct regions will be detected, and the plurality of designated detection regions including the indistinct regions are thereby detected; and correcting the boundaries of other designated detection regions on the basis of the regions detected as the indistinct regions.

Description

Information processing method, program, information processing device, and model generation method

[0001] The present invention relates to an information processing method, a program, an information processing device, and a model generation method.

[0002] There is a technology that uses machine learning models to detect desired objects from vascular tomographic images to support image diagnosis related to vascular treatments such as PCI (Percutaneous Coronary Intervention). However, vascular tomographic images may contain unclear image regions called artifacts, which may make it difficult to properly detect objects.

[0003] For example, Patent Document 1 discloses a program that uses a machine learning model to detect an object area from a medical image, calculates the reliability of each boundary point located on the boundary of the object area, and corrects the boundary of the object area using boundary points whose reliability meets a predetermined standard.

[0004] International Publication No. 2022 / 071265

[0005] In one aspect, an object is to provide an information processing method and the like that can suitably correct the boundary of a detection target area.

[0006] In one aspect, (1) an information processing method includes a computer that acquires data representing a vascular tomographic image, inputs the acquired data representing the vascular tomographic image into a model that has been trained to detect multiple detection target areas including unclear areas when the data representing the vascular tomographic image is input, thereby detecting the multiple detection target areas including the unclear areas, and correcting the boundaries of the other detection target areas based on the areas detected as the unclear areas.

[0007] (2) In the information processing method of (1) above, the detection target region includes at least one of a region representing the lumen of a blood vessel, a region representing the inside of a stent, and a region representing the inside of the external elastic lamina.

[0008] (3) In the information processing method of (1) or (2) above, a boundary portion of the other detection target region that is included in the unclear region is corrected based on a boundary portion that is not included in the unclear region.

[0009] (4) In the information processing method described in any one of (1) to (3) above, the model is a trained model that, when data representing a vascular tomographic image is input, assigns multiple labels indicating each of the multiple detection target areas to overlapping areas where multiple detection target areas overlap, and assigns a single label indicating the detection target area to non-overlapping areas where a detection target area does not overlap with another detection target area; and by inputting data representing the acquired vascular tomographic image into the model, the multiple labels are assigned to the overlapping areas and a single label is assigned to the non-overlapping areas.

[0010] (5) In the information processing method of (4) above, the detection target region further includes a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque, and if the unclear region detected by the model includes a region overlapping with any of a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque, the overlapping region is detected as being a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque.

[0011] (6) The information processing method described in any one of (1) to (5) above outputs the vascular tomographic image in which the corrected boundary portion of the other detection target region is displayed in a different display mode from the uncorrected boundary portion.

[0012] In one aspect, (7) the program acquires data representing a vascular tomographic image, inputs the acquired data representing the vascular tomographic image into a model that has been trained to detect multiple detection target areas including unclear areas when the data representing the vascular tomographic image is input, thereby detecting the multiple detection target areas including the unclear areas, and causes a computer to perform a process of correcting the boundaries of the other detection target areas based on the areas detected as the unclear areas.

[0013] In one aspect, (8) an information processing device is an information processing device that includes a control unit, wherein the control unit acquires data representing a vascular tomographic image, inputs the acquired data representing the vascular tomographic image into a model that has been trained to detect multiple detection target areas including unclear areas when the data representing the vascular tomographic image is input, thereby detecting the multiple detection target areas including the unclear areas, and corrects the boundaries of the other detection target areas based on the areas detected as the unclear areas.

[0014] In one aspect, (9) a model generation method acquires training data in which labels indicating multiple detection target regions, including a region indicating the lumen of a blood vessel, a region indicating the inside of a stent, or a region indicating the external elastic lamina, are associated with data indicating a vascular tomographic image; if the boundary of the region indicating the lumen of the blood vessel, the region indicating the inside of a stent, or the region indicating the external elastic lamina, does not satisfy a predetermined condition, the computer assigns a label indicating an unclear region to the data indicating the vascular tomographic image; and, based on the training data to which the label indicating the unclear region has been assigned, a computer executes a process to generate a trained model to detect the multiple detection target regions, including the unclear region, when data indicating the vascular tomographic image is input.

[0015] In one aspect, the boundary of the detection target area can be suitably corrected.

[0016] FIG. 1 is an explanatory diagram showing an example of the configuration of an image diagnostic system. FIG. 2 is a block diagram showing an example of the configuration of a server. FIG. 3 is a block diagram showing an example of the configuration of an image diagnostic device. FIG. 4 is an explanatory diagram showing an example of detection of a detection target region. FIG. 5 is an explanatory diagram relating to a detection model. FIG. 6 is an explanatory diagram relating to annotation processing of an unclear region. FIG. 7 is an explanatory diagram relating to boundary correction processing. FIG. 8 is an explanatory diagram showing an example of display of a vascular tomographic image. FIG. 9 is a flowchart showing the procedure of generation processing of a detection model. FIG. 10 is a flowchart showing the procedure of detection processing of a detection target region. FIG. 11 is an explanatory diagram relating to a detection model according to embodiment 2. FIG. 12 is an explanatory diagram relating to label assignment processing to an unclear region. FIG. 13 is a flowchart showing the procedure of detection processing of a detection target region according to embodiment 2.

[0017] The present invention will be described in detail below with reference to the drawings showing embodiments thereof. (Embodiment 1) Fig. 1 is an explanatory diagram showing an example of the configuration of an imaging diagnostic system. In this embodiment, an imaging diagnostic system will be described that detects a predetermined detection target region (e.g., the lumen of a blood vessel, the inside of a stent, the inside of the external elastic lamina, etc.) from a vascular tomographic image obtained by capturing a cross section of a patient's blood vessel, and corrects the boundary of the detection target region. The imaging diagnostic system includes a server 1 and an imaging diagnostic device 2. Each device is communicatively connected via a network N.

[0018] The server 1 is a server computer capable of various information processing and transmitting and receiving information. Note that the device equivalent to the server 1 may be a device such as a personal computer. The server 1 performs machine learning to learn predetermined training data, thereby generating a detection model 50 (see FIG. 5 ) that detects multiple detection target regions, including unclear regions, using a vascular tomographic image as input.

[0019] The diagnostic imaging device 2 is an imaging device that captures medical images of a patient's blood vessels, and is, for example, an IVUS (Intravascular Ultrasound) device that performs ultrasound examinations using a catheter 201. The catheter 201 is a medical instrument that is inserted into a patient's blood vessels, and transmits ultrasound signals from an ultrasound probe attached to the tip of the catheter 201 and receives reflected waves. The diagnostic imaging device 2 generates and displays ultrasound tomographic images based on the reflected waves received by the catheter 201.

[0020] In this embodiment, the diagnostic imaging device 2 is described as an IVUS device, but it may also be an optical imaging device using OCT (Optical Coherence Tomography). The diagnostic imaging device 2 may also be a dual sensor system having both IVUS and OCT functions. The vascular tomographic image may also be captured by a method other than IVUS or OCT.

[0021] Data of the detection model 50 generated by the server 1 is installed in the image diagnostic device 2, and the image diagnostic device 2 inputs a vascular tomographic image captured using the catheter 201 into the detection model 50 to detect a detection target region. As will be described later, the image diagnostic device 2 detects an unclear region, which is an artifact, when detecting the detection target region, and corrects the boundary of the detection target region based on the region detected as an unclear region.

[0022] In this embodiment, the diagnostic imaging device 2 detects the detection target region using the detection model 50, but the server 1 on the cloud may execute the processing using the detection model 50. Alternatively, for example, a general-purpose computer connected to the diagnostic imaging device 2 may execute the processing. In this way, the processing entity that executes the series of processes is not particularly limited.

[0023] FIG. 2 is a block diagram showing an example configuration of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 has one or more arithmetic processing devices, such as a central processing unit (CPU), a microprocessing unit (MPU), or a graphics processing unit (GPU), and performs various information processing, control processing, and the like, by reading and executing a program P1 stored in the auxiliary memory unit 14. The main memory unit 12 is a temporary storage area, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), and temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing and transmits and receives information to and from the outside. The auxiliary memory unit 14 is a non-volatile storage area, such as a large-capacity memory or a hard disk, and stores the program P1 (program product) and other data required for the control unit 11 to execute processing.

[0024] The auxiliary storage unit 14 may be an external storage device connected to the server 1. The server 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0025] Furthermore, in this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit that accepts operation input, a display unit that displays images, etc. Furthermore, the server 1 may be provided with a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage medium 1a.

[0026] FIG. 3 is a block diagram showing an example configuration of the imaging diagnostic device 2. The imaging diagnostic device 2 includes a control unit 21, a main memory unit 22, a communication unit 23, a display unit 24, an input unit 25, an image processing unit 26, and an auxiliary memory unit 27. The control unit 21 has one or more arithmetic processing units such as CPUs, and performs various information processing, control processing, etc. by reading and executing a program P2 stored in the auxiliary memory unit 27. The main memory unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing and transmits and receives information to and from the outside. The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard or mouse, and accepts operation inputs from the user. The image processing unit 26 is an image processing module that processes signals transmitted and received via the catheter 201 and generates a vascular tomographic image.

[0027] The auxiliary storage unit 27 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. The auxiliary storage unit 27 also stores a detection model 50. The detection model 50 is a machine learning model generated by learning predetermined training data, and is a trained model that detects a detection target region, etc., using a vascular tomographic image captured by the image diagnostic device 2 as input.

[0028] The diagnostic imaging device 2 may be provided with a reading unit for reading a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a.

[0029] Fig. 4 is an explanatory diagram showing an example of detection of a detection target region. Fig. 4 illustrates how a machine learning model is used to detect detection target regions such as a blood vessel lumen, a stent, and an external elastic membrane (EEM or EEL) from a vascular tomographic image (left side). An unclear region that appears in a vascular tomographic image will be described with reference to Fig. 4 .

[0030] In this embodiment, the illustrations and explanations will be given assuming that "vascular tomographic image" basically means a transverse cross-sectional image (an image of a cross section intersecting the axial direction of the blood vessel), but this embodiment is not limited to this, and the detection target may also be detected from a longitudinal cross-sectional image of the blood vessel (an image of a cross section along the axial direction of the blood vessel).

[0031] When detecting a desired detection target from a vascular tomographic image, it is possible to perform processing related to semantic segmentation using, for example, a machine learning model. Semantic segmentation is a method for identifying targets from an image on a pixel-by-pixel basis, and is performed using, for example, a convolutional neural network (CNN) such as U-net.

[0032] By training a machine learning model for semantic segmentation to learn the features of various objects, it is possible to detect multiple objects from a single image. For example, as shown on the right side of Figure 4, it is possible to identify, on a pixel-by-pixel basis, areas representing the lumen of a blood vessel, the inside of a stent, and the inside of the external elastic lamina from a vascular tomographic image.

[0033] On the other hand, in the detection results on the right side of Figure 4, the area of ​​the blood vessel lumen, etc., is not circular, and the contours change abruptly. This is because there is an unclear area in the image, and the machine learning model is unable to identify that area. An unclear area refers to an area where the boundaries of the blood vessel lumen, external elastic lamina, etc. cannot be clearly identified (annotated). In the example of Figure 4, the guidewire and its shadow area (acoustic shadow) exist as unclear areas in the image, so the model is unable to identify the boundaries of the lumen, etc. in that unclear area, resulting in a sector-shaped missing boundary of the lumen, etc.

[0034] When an unclear region exists in an image like this, it is conceivable to simply correct (interpolate) the contour using spline interpolation or the like, but this presupposes that there are no sudden changes in the contour, and correction is difficult when the contour changes suddenly as in Figure 4. Therefore, in this embodiment, by detecting not only the detection object such as a lumen but also the unclear region, it is possible to correct the contour (boundary) of the detection object.

[0035] Fig. 5 is an explanatory diagram of the detection model 50. Fig. 5 illustrates how, when a vascular tomographic image is input to the detection model 50, multiple detection target regions (two in Fig. 5) including an unclear region are detected.

[0036] The detection model 50 is a machine learning model that has been trained using predetermined training data, and is a model that has been trained to detect multiple detection target regions, including unclear regions, when a vascular tomographic image is input. For example, the detection model 50 is a model (CNN) related to semantic segmentation such as U-net, and detects the detection target region on a pixel-by-pixel basis.

[0037] The detection model 50 may be a neural network other than a CNN. The detection model 50 may also be a machine learning model other than a neural network, such as a decision tree or a support vector machine (SVM).

[0038] Furthermore, in this embodiment, a vascular tomographic image generated (constructed) from data on reflected ultrasound waves is input to the detection model 50, but this embodiment is not limited to this, and raw data of reflected waves that form the basis of the vascular tomographic image may also be used as input to the detection model 50. In other words, the detection model 50 only needs to be able to receive data representing a vascular tomographic image as input, and this data is not limited to image data.

[0039] The detection target region includes a region showing the lumen of a blood vessel (main trunk), a region showing the inside of a stent, a region showing the inside of the external elastic lamina of a blood vessel (main trunk), a region showing a side branch of the blood vessel, a region showing plaque, a thrombus region, a hematoma region, a calcification region, a dissection region, a crack region, an extravascular tissue region, a device region, etc. Note that "device" refers to a device such as a guide wire inserted into a patient's blood vessel.

[0040] Note that the above is an example of a detection target region, and other detection targets may be detected. The detection model 50 according to this embodiment detects a detection target region representing each of these multiple detection targets. For simplicity, the following description will be illustrated and explained mainly using an example in which the external elastic lamina (EEM) of a blood vessel is detected.

[0041] The server 1 generates the detection model 50 by learning training data for generating the detection model 50. The training data is data in which a label indicating the position of the correct detection target region (region indicating the inside of the external elastic lamina) is added to a training vascular tomographic image.

[0042] 6 is an explanatory diagram of the annotation process for unclear regions, illustrating how labels indicating the positions of unclear regions are assigned (annotated) to the above-mentioned training vascular tomographic image.

[0043] When the server 1 is given the training data, it assigns a label for identifying an unclear region to the training vascular tomographic image. Specifically, the server 1 assigns a label indicating an unclear region when the boundary of the detection target region (the region indicating the inside of the external elastic lamina) to which a label has been assigned in advance is discontinuous or when the boundary of the detection target region does not have a specific shape (for example, a circle).

[0044] For example, the server 1 first converts a vascular tomographic image in a Cartesian coordinate system (XY coordinates) into an image in a polar coordinate system (rθ coordinates) in which the vertical axis represents the distance from the ultrasound probe and the horizontal axis represents the angle. Based on the converted polar coordinate image, the server 1 determines whether the detection target region (the white region in Figure 6) is discontinuous.

[0045] If it is determined that there is discontinuity, the server 1 determines that the discontinuous portion is an unclear region. In this case, the server 1 assigns a label indicating an unclear region (shown by hatching in FIG. 6 ) to the portion. The server 1 converts the polar coordinate image with the label assigned to it into a Cartesian coordinate image, and uses the converted image as a vascular tomographic image for training.

[0046] Although the above describes the case where an unclear region is determined from a polar coordinate image, the server 1 may determine whether the detection target region is circular based on a Cartesian coordinate image, and if it is determined that the detection target region is not circular, determine that the non-circular portion (the missing fan-shaped portion) is an unclear region. In this way, various methods can be considered for assigning labels to unclear regions.

[0047] Returning to FIG. 5 , the explanation continues. After assigning labels indicating unclear regions to the training vascular tomographic images, the server 1 generates a detection model 50 based on the training data. That is, the server 1 inputs the training vascular tomographic images into the detection model 50 to detect each detection target region, including unclear regions. The server 1 compares the detected detection target region with the correct label and updates parameters such as the weights between neurons so that the two approximate each other. The server 1 sequentially provides a group of training vascular tomographic images to the detection model 50 to update the parameters, and finally generates (constructs) a detection model 50 with optimized parameters. The data of the generated detection model 50 is installed in the image diagnostic device 2.

[0048] When actually capturing a vascular tomographic image of a patient and detecting a detection target region from the vascular tomographic image, the image diagnostic device 2 detects a plurality of detection target regions, including an unclear region, by inputting the captured vascular tomographic image into the detection model 50. Specifically, as shown by the white areas (inside the external elastic lamina) or hatched areas (obscure regions) on the right side of Fig. 5, the image diagnostic device 2 assigns a label indicating the class of the detection target region to which each pixel corresponds.

[0049] Fig. 7 is an explanatory diagram of the boundary correction process. Fig. 7 illustrates how the boundaries of other detection target regions are corrected (interpolated) based on an unclear region. Note that for the sake of convenience, hatching and blacking out are not used in Fig. 7.

[0050] When a detection target region including an unclear region is detected, the image diagnostic device 2 corrects the boundaries of the detection target region other than the unclear region based on the region detected as the unclear region. For example, as shown on the left side of Fig. 7, the image diagnostic device 2 first converts the Cartesian coordinate image in which the detection target region is detected into a polar coordinate image. Note that in the example of Fig. 7, the boundary of the detection target region is corrected based on the polar coordinate image, but this embodiment is not limited to this, and the boundary correction may also be performed based on the Cartesian coordinate image.

[0051] The diagnostic imaging device 2 identifies, from the vascular tomographic image (polar coordinate image), boundary portions that are not included in the unclear region at the boundaries of detection target regions (within the EEM) other than the unclear region. Specifically, the diagnostic imaging device 2 identifies boundary portions that do not contact the unclear region. In Figure 7, the boundary portions are illustrated with thick lines.

[0052] Based on the boundary portions identified above, the diagnostic imaging device 2 corrects (interpolates) the boundary portions included in the unclear regions for the detection target regions other than the unclear regions. Specifically, the diagnostic imaging device 2 interpolates new boundary lines using spline interpolation based on the boundary lines shown by the thick lines in FIG. 7. In FIG. 7, the interpolated boundary lines are illustrated by thick dotted lines. In this way, by detecting the unclear regions, it is possible to identify the boundary portions that should be the basis for correction (interpolation) (boundary portions that are not included in the unclear regions), and it is possible to appropriately correct the boundary portions included in the unclear regions. The diagnostic imaging device 2 converts the polar coordinate image into a Cartesian coordinate image and displays a vascular tomographic image in which the boundary of the detection target region has been corrected.

[0053] Fig. 8 is an explanatory diagram showing a display example of a vascular tomographic image. For example, the image diagnostic device 2 displays a vascular tomographic image on which boundary lines indicating the boundaries of each detection target region (in Fig. 8, the vascular lumen, stent, and external elastic lamina) are superimposed. In this case, the image diagnostic device 2 displays the corrected boundary portions in a different display mode from the uncorrected boundary portions. Specifically, as shown in Fig. 8, the image diagnostic device 2 displays the corrected boundary portions with dotted lines. This makes it possible to indicate to the user (medical professional) which parts have been corrected.

[0054] In the example of FIG. 8, the boundary lines are superimposed only on the detection target regions other than the unclear region, but the unclear region may also be clearly indicated to the user by superimposing the boundary line on the unclear region.

[0055] 9 is a flowchart showing the steps of the process for generating the detection model 50. The process for generating the detection model 50 by machine learning will be described with reference to FIG. 9. The control unit 11 of the server 1 acquires training data for generating the detection model 50 (step S11). The training data is data in which a label indicating the position of the correct detection target region is assigned to a training vascular tomographic image. The detection target region includes a region indicating the lumen of the blood vessel, a region indicating the inside of a stent, a region indicating the inside of the external elastic lamina, etc.

[0056] If the boundary of the detection target region indicated by the correct label does not satisfy a predetermined condition, the control unit 11 assigns a label indicating an unclear region to the training vascular tomographic image (step S12). Specifically, the control unit 11 assigns a label indicating an unclear region when the boundary of the detection target region in the vascular tomographic image (polar coordinate image) is discontinuous or when the boundary of the detection target region in the vascular tomographic image (cartesian coordinate image) is not circular.

[0057] The control unit 11 generates a trained detection model 50 based on the training data so as to detect multiple detection target regions, including unclear regions, when a vascular tomographic image is input (step S13). For example, the control unit 11 generates a CNN related to semantic segmentation as the detection model 50. The control unit 11 inputs a training vascular tomographic image to the detection model 50 to detect detection target regions, including unclear regions. The control unit 11 compares the detected detection target regions with correct labels and optimizes parameters such as weights between neurons so that the two are similar to each other, thereby generating the detection model 50. The control unit 11 ends the series of processes.

[0058] Fig. 10 is a flowchart showing the procedure of the detection process for the detection target region. The process for detecting the detection target region using the detection model 50 and correcting the boundary of the detection target region will be described with reference to Fig. 10 . The control unit 21 of the image diagnostic device 2 acquires a vascular tomographic image of a cross section of a patient's blood vessel (step S31). For example, the control unit 21 acquires a transverse image of a cross section intersecting (orthogonal to) the axial direction of the patient's blood vessel. The control unit 21 inputs the acquired vascular tomographic image into the detection model 50 to detect a plurality of detection target regions, including unclear regions (step S32).

[0059] The control unit 21 identifies boundary portions not included in the unclear region from the boundaries of other detection target regions other than the unclear region (step S33). That is, the control unit 21 identifies boundary portions of the detection target region that do not contact the unclear region. The control unit 21 corrects the boundaries included in the unclear region in the boundaries of the other detection target regions based on the identified boundary portions (step S34). For example, the control unit 21 corrects (interpolates) the boundary portions not included in the unclear region to the boundary portions included in the unclear region by spline interpolation or the like, as described above.

[0060] The control unit 21 displays a vascular tomographic image showing the detection target region with the corrected boundary (step S35). For example, the control unit 21 displays a vascular tomographic image with a boundary line superimposed on each detection target region. In this case, the control unit 21 displays the corrected boundary portion in a different display mode from the uncorrected boundary portion. The control unit 21 then ends the series of processes.

[0061] As described above, according to the first embodiment, the boundary of the detection target region can be suitably corrected.

[0062] (Embodiment 2) In the first embodiment, a configuration was described in which each pixel in a vascular tomographic image is classified (detected) into which detection target region it belongs, i.e., multi-class classification (detection). In the present embodiment, a configuration will be described in which multiple labels indicating the multiple detection target regions are assigned to pixels where multiple detection target regions overlap (overlapping region), i.e., multi-label assignment. Note that the same reference numerals are used for the same content as in the first embodiment, and description thereof will be omitted.

[0063] Fig. 11 is an explanatory diagram of the detection model 50 according to embodiment 2. Fig. 11 illustrates how label data is output from the detection model 50 for each type of detection target region (lumen, stent, external elastic lamina (EEM), and unclear region), and how the plurality of label data are superimposed to output a vascular tomographic image on which the boundary lines of the detection target regions are superimposed.

[0064] The detection model 50 according to the first embodiment performs multi-class classification to determine which type of detection target region each pixel in a vascular tomographic image belongs to. On the other hand, the detection model 50 according to the present embodiment performs two-class classification for each type of detection target region, and makes it possible to identify the boundaries of each detection target region by superimposing the classification results of each type.

[0065] Specifically, the detection model 50 according to the present embodiment is a trained model that, when a vascular tomographic image is input, assigns multiple labels indicating each of the multiple detection target regions to overlapping regions where multiple detection target regions overlap, and assigns a single label indicating the detection target region to non-overlapping regions where a detection target region does not overlap with other detection target regions. By inputting the vascular tomographic image into the detection model 50, the image diagnostic device 2 assigns multiple labels to overlapping regions and a single label to non-overlapping regions.

[0066] That is, as shown on the left side of Fig. 11 , the detection model 50 according to this embodiment generates, for each type of detection target region, multiple types of label data (label images) that indicate, with a binary value of "1" or "0," whether or not each pixel belongs to the detection target region. In the example of Fig. 11 , the detection model 50 generates four types of label data: label data representing a region indicating the lumen of a blood vessel, label data representing a region indicating the inside of a stent, label data representing a region indicating the inside of the external elastic lamina, and label data representing an unclear region. The image diagnostic device 2 inputs a vascular tomographic image to the detection model 50, thereby generating different label data according to the type of detection target region.

[0067] The image diagnostic device 2 identifies each region in the vascular tomographic image by overlaying the respective label data output from the detection model 50. This will be explained using the vascular tomographic image on the right side of FIG. 11 and the table on the bottom right of FIG. 11 . For example, when considering a point (pixel) in "Region 1" in the vascular tomographic image, by referring to the respective label data, it can be seen that the value (label) indicating whether or not it is in the lumen is "1," the value indicating whether or not it is inside the stent is "1," the value indicating whether or not it is inside the external elastic lamina is "1," the value indicating whether or not it is an unclear region is "0," and the value indicating whether or not it is in any other region ("Background") is "0." In other words, it can be seen that "Region 1" is inside the lumen of the blood vessel, inside the stent, and inside the external elastic lamina, but is not an unclear region.

[0068] Similarly, for "Area 2," the value of "lumen" is "1," the value of "inside the stent" is "0," the value of "inside the EEM" is "1," the value of "obscure area" is "0," and the value of "Background" is "0." In this case, it can be seen that "Area 2" is within the lumen of the blood vessel and inside the external elastic lamina, but is not within the stent and is not an obscure area.

[0069] The detection model 50 may separately detect the "Background" according to each detection target, such as the "Background" of the "lumen," the "Background" of the "inside the stent," and the "Background" of the "inside the EEM."

[0070] In this way, the detection model 50 according to the present embodiment generates a plurality of label data in which each pixel is classified into two classes according to whether it belongs to a detection target region or not, depending on the type of detection target. On the other hand, the detection model 50 according to the first embodiment differs from this and generates label data in which each pixel is simply classified into multiple classes according to which type of detection target region it belongs to. Explaining this in accordance with the example on the right side of Figure 11, the detection model 50 according to the first embodiment assigns multiple labels in the form of a label "1" to "region 1" indicating that it is the lumen, a label "2" to "region 2" indicating that it is inside the stent, a label "3" to "region 3" indicating that it is inside the external elastic lamina, and so on.

[0071] When multi-class semantic segmentation as in the first embodiment is performed, the task to be performed by the detection model 50 tends to become complicated due to the need for multi-class classification. Therefore, the more detection target regions that must be detected increase and the more complex the region division within the image becomes, the more likely it is that the accuracy of individual detection will decrease, and the learning cost (learning amount) of the detection model 50 will also increase for each divided segment.

[0072] In contrast, the detection model 50 according to this embodiment only needs to perform two-class classification for each type of detection target region, thereby preventing a decrease in detection accuracy while also suppressing an increase in the learning cost of the detection model 50.

[0073] 12 is an explanatory diagram of the labeling process for unclear regions, which illustrates how labels are added to regions such as the guide wire in addition to the unclear regions.

[0074] In this embodiment, the detection target regions further include a region showing the guidewire and its shadow region (acoustic shadow) (hereinafter referred to as the "guidewire region" for simplicity), a region showing the stent strut and its shadow region (hereinafter referred to as the "stent strut region"), and a region showing plaque (hereinafter referred to as the "plaque region"). The guidewire region, stent strut region, and plaque region are regions observed as shadowed portions in a vascular tomographic image and are representative of unclear regions.

[0075] Note that an unclear region due to acoustic shadowing, such as a guidewire region, is just one example, and unclear regions may also be caused by reasons other than acoustic shadowing.

[0076] The detection model 50 according to this embodiment detects guidewire regions, stent strut regions, and plaque regions as detection target regions from the vascular tomographic image, in addition to unclear regions. Here, if the unclear region detected by the detection model 50 includes a region overlapping with any of the guidewire region, stent strut region, or plaque region, the diagnostic imaging device 2 identifies the overlapping region by prioritizing the label of the guidewire region, stent strut region, or plaque region. That is, the diagnostic imaging device 2 detects a region where the unclear region overlaps with the guidewire region, stent strut region, or plaque region as a guidewire region, stent strut region, or plaque region. This allows for more clear identification of unclear regions.

[0077] 13 is a flowchart showing the steps of the detection process for the detection target region according to the second embodiment. After acquiring a vascular tomographic image (step S31), the image diagnostic device 2 executes the following process. The control unit 21 of the image diagnostic device 2 inputs the acquired vascular tomographic image into the detection model 50, thereby detecting a plurality of detection target regions including an unclear region (step S201). Here, the detection model 50 according to the present embodiment is a trained model that, when a vascular tomographic image is input, assigns a plurality of labels indicating each of the plurality of detection target regions to an overlapping region where the plurality of detection target regions overlap, and assigns a single label indicating the detection target region to a non-overlapping region where a detection target region does not overlap with another detection target region.

[0078] The control unit 21 inputs the acquired vascular tomographic image into the detection model 50, thereby assigning multiple labels to overlapping regions and a single label to non-overlapping regions. That is, the control unit 21 generates different label data (label images) depending on the type of detection target region. In this case, if the unclear region detected by the detection model 50 includes a region overlapping with any of the guidewire region, stent strut region, or plaque region, the control unit 21 assigns a label preferentially to the guidewire region, stent strut region, or plaque region for the overlapping region. The control unit 21 proceeds to step S33.

[0079] As described above, according to the second embodiment, it is possible to prevent a decrease in the detection accuracy of the detection target region, while suppressing an increase in the learning cost of the detection model 50.

[0080] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0081] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0082] REFERENCE SIGNS LIST 1 Server 11 Control unit 12 Main memory unit 13 Communication unit 14 Auxiliary memory unit P1 Program 2 Image diagnostic device 21 Control unit 22 Main memory unit 23 Communication unit 24 Display unit 25 Input unit 26 Image processing unit 27 Auxiliary memory unit P2 Program 50 Detection model

Claims

1. An information processing method in which a computer executes the following processes: acquiring data representing a vascular tomographic image; detecting a plurality of detection target regions including an unclear region by inputting the acquired data representing a vascular tomographic image into a model that has been trained to detect a plurality of detection target regions including an unclear region when the data representing a vascular tomographic image is input; and correcting the boundaries of the other detection target regions based on the region detected as an unclear region.

2. The information processing method according to claim 1, wherein the detection target area includes at least one of an area showing the lumen of a blood vessel, an area showing the inside of a stent, and an area showing the inside of the external elastic lamina.

3. The information processing method according to claim 1, further comprising correcting a boundary portion of the other detection target region that is included in the unclear region based on a boundary portion that is not included in the unclear region.

4. The information processing method according to claim 1, wherein the model is trained so that, when data representing a vascular tomographic image is input, an overlapping region in which a plurality of the detection target regions overlap is assigned a plurality of labels indicating each of the plurality of detection target regions, and a single label indicating the detection target region is assigned to a non-overlapping region in which a detection target region does not overlap with another detection target region, and by inputting data representing the acquired vascular tomographic image into the model, the plurality of labels are assigned to the overlapping region, and a single label is assigned to the non-overlapping region.

5. The information processing method according to claim 4, wherein the detection target region further includes a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque, and when the unclear region detected by the model includes a region overlapping with any of a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque, the overlapping region is detected as being a region indicating a guidewire and a shadow region caused by the guidewire, a region indicating a stent strut and a shadow region caused by the stent strut, or a region indicating plaque.

6. The information processing method according to claim 1, further comprising the step of outputting the vascular tomographic image, in which the corrected boundary portion of the other detection target region is displayed in a different display mode from the uncorrected boundary portion.

7. A program that causes a computer to execute the following process: acquire data representing a vascular tomographic image; detect a plurality of detection target regions including an unclear region by inputting the acquired data representing a vascular tomographic image into a model that has been trained to detect a plurality of detection target regions including an unclear region when the data representing a vascular tomographic image is input; and correct the boundaries of the other detection target regions based on the region detected as an unclear region.

8. An information processing device having a control unit, wherein the control unit: acquires data representing a vascular tomographic image; detects the multiple detection target areas including the unclear areas by inputting the acquired data representing the vascular tomographic image into a model that has been trained to detect multiple detection target areas including the unclear areas when data representing the vascular tomographic image is input; and corrects boundaries of other detection target areas based on the areas detected as the unclear areas.

9. A model generation method in which a computer executes the following processes: acquiring training data in which labels indicating multiple detection target regions including a region indicating the blood vessel lumen, a region indicating the inside of a stent, or a region indicating the external elastic lamina are associated with data indicating a vascular tomographic image; if the boundary of the region indicating the blood vessel lumen, the region indicating the inside of a stent, or the region indicating the external elastic lamina does not satisfy a predetermined condition, assigning a label indicating an unclear region to the data indicating the vascular tomographic image; and generating a model trained to detect the multiple detection target regions including the unclear region when data indicating the vascular tomographic image is input based on the training data to which the label indicating the unclear region has been assigned.

Images