Diagnosis support method, diagnosis support program, and diagnosis support apparatus

Abstract

To detect an abnormal part of a blood vessel from an original image including the blood vessel as a subject without using a complicated algorithm.SOLUTION: A diagnosis support method (S1) comprises: an image acquisition process (S10) which acquires an image extracted from an original image including as a subject a blood vessel to be examined, so as to include a fraction of the blood vessel; a determination process (S11) which inputs the extracted image into a learned model and thereby determines whether the blood vessel to be examined includes an abnormal part; and a display control process (S13) which displays to a user a result of the determination in the determination process.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a diagnostic support method, a diagnostic support program, and a diagnostic support apparatus. [Background technology]

[0002] Ischemic heart diseases such as myocardial infarction and angina pectoris are diseases caused by the accumulation of fatty plaque on the inner wall of the coronary artery, which causes the coronary artery to become blocked or narrowed. In general, doctors visually diagnose whether an abnormality such as blockage or narrowing has occurred in the coronary artery based on CT (Computed Tomography) images or MRI (Magnetic Resonance Imaging) images that include the coronary artery as a subject. Doctors also diagnose the type of plaque that causes the blockage or narrowing based on the CT images or MRI images, and determine a treatment plan.

[0003] In recent years, with the development of image analysis technology using artificial intelligence (AI), diagnostic support technology has been developed that uses AI to detect abnormal parts of blood vessels from images that include blood vessels as subjects. One such technology is known, for example, as described in Non-Patent Document 1, which uses AI (neural network) to detect stenosis in a coronary artery from an X-ray contrast image of the coronary artery. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] [online], Automated Detection of Coronary Artery Stenosis in X-ray Angiography using Deep Neural Networks[Retrieved June 1, 2023], Internet〈URL:https: / / arxiv.org / abs / 2103.02969〉 Summary of the Invention [Problem to be solved by the invention]

[0005] However, complex algorithms are required to detect small abnormalities such as plaques in coronary arteries from large medical images such as CT images and MRI images. AI (Artificial Intelligence) models that use such complex algorithms to detect abnormalities from medical images tend to have large data sizes and require a significant amount of time for training.

[0006] One aspect of the present invention has been made in consideration of the above-mentioned problems, and aims to realize a technology that can detect abnormal parts of blood vessels from original medical images that contain blood vessels as subjects, without using complex algorithms. [Means for solving the problem]

[0007] In order to solve the above problems, a diagnostic support method according to one embodiment of the present invention includes an image acquisition step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as a subject, a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model trained to detect abnormal portions of the blood vessel, using an image including the fragment of the blood vessel acquired from an image capturing the blood vessel as input information, and a display control step of displaying the determination result determined in the determination step to a user.

[0008] In order to solve the above problems, a diagnostic support device according to one aspect of the present invention includes an image acquisition unit that acquires an extracted image including a fragment of a blood vessel from an original image that includes the blood vessel as a subject, a determination unit that determines whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model that is trained to detect abnormal portions of the blood vessel using an image including the fragment of the blood vessel acquired from an image of the blood vessel as input information, and a display control unit that displays the determination result determined in the determination process to a user.

[0009] Each aspect of the present invention may be realized by a computer. In this case, the scope of the present invention also includes a program for causing a computer to perform each step of the above-mentioned method, a control program for a diagnostic support device for causing a computer to operate as each part (software element) of the diagnostic support device, thereby realizing the diagnostic support device on a computer, and a computer-readable recording medium having these recorded thereon. Effect of the Invention

[0010] According to one aspect of the present invention, it is possible to detect an abnormal part of a blood vessel from an original medical image that contains the blood vessel as a subject, without using a complex algorithm. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a diagnosis support system according to a first embodiment. [Diagram 2] 3 is a flowchart showing the flow of a diagnosis support method implemented by the diagnosis support system according to the first embodiment. [Diagram 3] 3 is a flow chart showing the flow of processing performed in the image acquiring step shown in FIG. 2. FIG. [Figure 4] FIG. 13 is a schematic diagram showing an example of an image acquiring process. [Diagram 5] 3 is a flowchart showing another example of the flow of the process performed in the image acquiring step shown in FIG. 2. FIG. [Figure 6] 3 is a flowchart showing another example of the flow of the process performed in the image acquiring step shown in FIG. 2. FIG. [Figure 7] FIG. 11 is a schematic diagram showing another example of the image acquiring process. [Figure 8] FIG. 11 is a block diagram showing the configuration of a diagnosis support system according to a second embodiment. [Figure 9] 10 is a flowchart showing the flow of a diagnosis support method implemented by a diagnosis support system according to embodiment 2. FIG. [Figure 10]FIG. 2 is a block diagram showing an example of a configuration in which each unit of the diagnosis support device according to the first and second embodiments is configured using a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [Embodiment 1] Hereinafter, one embodiment of the present invention will be described in detail with reference to FIGS.

[0013] (Overview of diagnostic support system 100) A diagnostic support system 100 according to one embodiment of the present disclosure is a diagnostic support system that uses a trained model to determine whether or not an original image containing a blood vessel to be examined as a subject contains an abnormal area such as an obstruction or stenosis in the blood vessel, and displays the determination result to a user (e.g., a doctor).

[0014] In this embodiment, the "blood vessel to be inspected" refers to a coronary artery distributed on the surface of the heart. The "abnormal portion" refers to a narrowed portion of the blood vessel caused by unstable plaque. That is, the diagnosis support system 100 is a system that determines whether or not the coronary artery includes a narrowed portion caused by unstable plaque. However, the diagnosis support system according to one aspect of the present disclosure is not limited thereto. For example, in order to diagnose diseases occurring in blood vessels, such as aneurysm, thrombosis, arteriosclerosis, cerebral infarction, cerebral hemorrhage, cerebral arterial malformation, vascular trauma, peripheral arterial disease, vasculitis, renal artery stenosis, and aortic dissection, a diagnosis support system that detects abnormal portions of blood vessels corresponding to each of the diseases listed above is also included in the scope of this embodiment.

[0015] In conventional diagnostic support technology, large-sized medical images such as CT images and MRI images that contain the blood vessels to be examined as subjects are directly input into a trained model, and the trained model determines whether the blood vessels contain abnormalities based on the obtained output. Therefore, the trained model takes time to learn using large-sized medical images as training data. In addition, the trained model needs to process large medical images using complex algorithms, which causes the problem of large data size.

[0016] In contrast, the diagnosis support system 100 inputs an extracted image, which is extracted from an original image including the blood vessel to be inspected as a subject, so as to include a fragment of the blood vessel into the trained model, and determines whether or not the blood vessel to be inspected includes an abnormal part based on the obtained output. That is, since a small abnormal part is detected from a small extracted image extracted from the original image, there is no need to use a complex algorithm. Therefore, there is no need to prepare high-performance hardware, etc. In addition, the training data used for training the trained model may be a small image including a fragment of the blood vessel. Therefore, it is possible to realize a diagnosis support system that reduces the time cost required for training the trained model and the introduction cost required for introducing the system.

[0017] (Configuration of diagnostic support system 100) The schematic configuration of the diagnosis support system 100 will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of the configuration of the diagnosis support system.

[0018] As shown in FIG. 1, the diagnosis support system 100 includes an imaging device 1, a diagnosis support device 2, and an image display device 3. The diagnosis support device 2 is configured to be able to communicate with the imaging device 1 and the image display device 3. The communication form between the diagnosis support device 2, the imaging device 1, and the image display device 3 is not particularly limited as long as they are able to transmit and receive data. In this embodiment, it is assumed that the imaging device 1, the diagnosis support device 2, and the image display device 3 are installed in a facility (e.g., a medical institution) that uses the diagnosis support system 100, so that short-distance wireless communication such as WiFi (registered trademark) or Bluetooth (registered trademark) or wired communication is adopted as the communication form.

[0019] In addition, when the imaging device 1 and the image display device 3 are placed within a facility where the diagnostic support system 100 is used, and the diagnostic support device 2 is placed outside the facility where the diagnostic support system 100 is used, the diagnostic support device 2 is configured to be able to communicate with the imaging device 1 and the image display device 3 via long-distance wireless communication such as wireless LAN.

[0020] [Imaging device 1] The imaging device 1 is a device for imaging blood vessels of an inspection target. As an example, the imaging device 1 images a coronary artery. The imaging device 1 transmits imaging data of the blood vessels of the inspection target to the diagnosis support device 2. In this embodiment, a known CT device is adopted as the imaging device 1.

[0021] The CT device is a device that rotates an X-ray tube and an X-ray detector around a subject (e.g., a patient), detects X-rays that have passed through the subject, and generates a cross-sectional image of the subject's living body based on the detected X-ray dose, etc. When a known CT device is used as the imaging device 1, data representing the detection results of X-rays that have passed through the subject, which are used to generate a cross-sectional image of the living body, or image data representing a cross-sectional image of the living body of the subject, is transmitted as imaging data to the diagnosis support device 2. Here, the cross-sectional image of the living body is an image that includes the blood vessels of the subject as a subject.

[0022] In this embodiment, a known CT device is used as the imaging device 1, but the present invention is not limited to this. For example, a known MRI device may be used. In the following, a case where the imaging device 1 is a CT device will be described as an example.

[0023] [Diagnosis support device 2] The diagnosis support device 2 is a device for implementing a diagnosis support method according to an embodiment of the present invention. The diagnosis support method implemented by the diagnosis support device 2 will be described later with reference to the accompanying drawings.

[0024] As shown in FIG. 1, the diagnosis support device 2 includes a control unit 20, a communication unit 21, and a memory 22.

[0025] <Control unit 20> The control unit 20 controls each component included in the diagnosis support device 2. The control unit 20 also includes an image acquisition unit 200, a determination unit 201, an image generation unit 202, and a display control unit 203.

[0026] The image acquiring unit 200 acquires an extracted image extracted to include a fragment of the blood vessel from an original image including the blood vessel as a subject. As an example, the image acquiring unit 200 generates an original image including the blood vessel as a subject based on the imaging data received from the imaging device 1, and acquires an extracted image extracted to include a fragment of the blood vessel from the generated original image.

[0027] The determination unit 201 determines whether or not an abnormal part is present in the blood vessel to be inspected, which is included as a subject in the original image acquired by the image acquisition unit 200. One example of a determination method implemented by the determination unit 201 is a determination method using a trained model. Specifically, the determination unit 201 inputs an image including a fragment of a blood vessel extracted from an image including the blood vessel as a subject as input information, and inputs the extracted image to a trained model that has been trained to detect an abnormal part in the fragment of the blood vessel. Then, the determination unit 201 determines whether or not an abnormal part is present in the blood vessel to be inspected, referring to the detection result by the trained model.

[0028] The determination unit 201 may have a function of the trained model, or a device other than the diagnosis support device 2 may have the function of the trained model, and the determination unit 201 may output the extracted image to the device and obtain the detection result from the device. Details of the trained model will be described later.

[0029] The image generating unit 202 generates a display image for displaying to a user the determination result obtained by the determining unit 201. As an example, the image generating unit 202 generates a display image in which a mark indicating an abnormal part is added to an original image including the blood vessel to be inspected as a subject.

[0030] The display control unit 203 displays various images to the user. As an example, the display control unit 203 displays to the user a display image for displaying to the user the determination result determined by the determination unit 201, which is generated in the image generation unit 202. As an example of a method for displaying a display image to the user, a method of displaying the display image on the display unit 31 of the image display device 3 owned by the user can be given. Specifically, the display control unit 203 transmits image data representing the display image generated by the image generation unit 202 to the image display device 3 via the communication unit 21 of the diagnosis support device 2, and outputs the image on the display unit 31 built in the image display device 3. In this embodiment, a configuration is adopted in which the display image is displayed on the display unit 31 of the image display device 3, which is an external device of the diagnosis support device 2, but the display control unit 203 may display the display image on a display unit (not shown) built in the diagnosis support device 2.

[0031] The control unit 20 of the diagnosis support device 2 may include at least one processor, and may function as the image acquisition unit 200, the determination unit 201, the image generation unit 202, and the display control unit 203 by reading and executing a diagnosis support program recorded in the memory 22 described later. Furthermore, the program for functioning as the image acquisition unit 200, the program for functioning as the determination unit 201, the program for functioning as the image generation unit 202, and the program for functioning as the display control unit 203 may each be separate programs, and may be recorded in the memory of separate devices. Such a configuration will be described later.

[0032] <Communications Division 21> The communication unit 21 communicates with the imaging device 1 and the image display device 3. As an example, the communication unit 21 receives imaging data from the imaging device 1 and transmits a display image for displaying a determination result to a user to the image display device 3. In this embodiment, the communication unit 21 is a wireless communication interface such as WiFi or Bluetooth.

[0033] <Memory 22> Various information is recorded in the memory 22. As an example, the memory 22 records various programs executed by the control unit 20, various images acquired by the image acquisition unit 200, the determination result determined by the determination unit 201, a display image generated by the image generation unit 202, and the like.

[0034] [Image display device 3] The image display device 3 is a device for displaying various images to a user. Devices that can be used as the image display device 3 include a desktop PC, a tablet PC, a tablet terminal, a smartphone, and the like. In this embodiment, a tablet terminal carried by a user is assumed as the image display device 3. The image display device 3 includes a communication unit 30 and a display unit 31, as shown in FIG. 1.

[0035] <Communication Section 30> The communication unit 30 communicates with the diagnosis support device 2. As an example, the communication unit 30 receives a display image for displaying the determination result to the user from the diagnosis support device 2. The communication unit 30 may also accept a request to view the display image from the user at an input unit (not shown) of the image display device 3, and transmit a signal indicating the request to the diagnosis support device 2.

[0036] <Display section 31> The display unit 31 displays various images. As an example, the display unit 31 displays a display image received from the diagnosis support device 2. A specific example of the display image displayed on the display unit 31 will be described later with reference to another drawing.

[0037] (Pre-trained model) In this embodiment, the trained model used to determine whether or not an abnormality exists in the blood vessel to be inspected will be described in detail. The trained model is a model trained to recognize abnormalities in blood vessel fragments. Examples of training data for the trained model include training data including one or more pairs of an image including a fragment of a blood vessel acquired from an image including a blood vessel as a subject, and an image in which an abnormality in the image is labeled. For example, in the case where the trained model has been trained to recognize a stenosis caused by unstable plaque in a coronary artery, examples of the training data include one or more pairs of an image including a fragment of the coronary artery acquired from an image including a coronary artery as a subject, and an image in which a stenosis caused by unstable plaque in the image is labeled. In this way, the training model can be used to determine various diseases by adjusting the training data to match the abnormality to be recognized.

[0038] The trained model uses an image including a fragment of a blood vessel acquired from an image including a blood vessel as a subject as input information, and recognizes an abnormal part included in the input image. Then, the trained model outputs an image in which a bounding box is attached to each area of ​​the recognized abnormal part as an example of a detection result. Note that the trained model may be a model generated by deep learning, or may be a model generated by machine learning other than deep learning.

[0039] (Flow of diagnostic support method S1) The diagnostic support method S1 executed by the diagnostic support device 2 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the diagnostic support method S1.

[0040] 2, the diagnostic support method S1 includes an image acquisition step S10, a determination step S11, an image generation step S12, and a display control step S13. The timing at which the diagnostic support device 2 performs the diagnostic support method S1 can be set arbitrarily. For example, the diagnostic support device 2 may perform the method every time imaging data is received from the imaging device 1, or may perform the method based on a user's instruction.

[0041] [Image acquisition step S10] The image acquiring step S10 is a step for acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as a subject. An example of the process performed by the image acquiring unit 200 in the image acquiring step S10 will be described with reference to Fig. 3. Fig. 3 is a flow diagram showing the flow of each process performed in the image acquiring step.

[0042] First, as shown in FIG. 3, the image acquiring section 200 acquires imaging data of an image of a blood vessel to be inspected from the imaging device 1 (step S100).

[0043] Next, the image acquisition unit 200 generates an original image including the blood vessel to be inspected as a subject based on the imaging data acquired from the imaging device 1 (step S101). Examples of the original image to be generated here include a CPR (Curve Planar Reconstruction) image and an MPR (Multi Planar Reconstruction) image. A CPR image is an image reconstructed as a curved surface along an arbitrary curve in a three-dimensional space, and is suitable for observing a long, meandering tubular structure such as a blood vessel or a site along a curved surface. An MPR image is an image cut out from an arbitrary cross section, and allows the positional relationship and shape of a blood vessel or an organ, which are difficult to grasp only from a cross-sectional image, to be viewed from any direction. Note that the original image is not limited to the images listed above as long as it includes the blood vessel to be inspected as a subject and includes information regarding a center line indicating the center of the blood vessel. Here, the information regarding the center line is a straight line connecting the opposing inner walls or outer walls of the blood vessel, and may be information derived from a width line perpendicular to the center line.

[0044] In this embodiment, a configuration is adopted in which an original image including the blood vessels of the inspection target as a subject is generated based on imaging data acquired from the imaging device 1, but a configuration may also be adopted in which the imaging device 1 generates an original image and the image acquisition unit 200 acquires the original image from the imaging device 1. Also, a configuration may be adopted in which a device other than the imaging device 1 has a function of generating an original image and the image acquisition unit 200 acquires the original image from the device. In these cases, step S101 of generating an original image from imaging data is omitted.

[0045] Finally, the image acquiring unit 200 acquires an extracted image from the generated original image, the extracted image being extracted so as to include a fragment of the blood vessel to be inspected (step S102). A method for the image acquiring unit 200 to extract an extracted image from the original image so as to include a fragment of the blood vessel to be inspected includes a method of detecting the center line of the blood vessel to be inspected and extracting the image based on the center line. A specific example of the method will be described with reference to FIG. 4. FIG. 4 is a schematic diagram showing an example of an image acquiring process. In FIG. 4, a CPR image including a coronary artery as a subject is taken as the original image. In the upper diagram of FIG. 4, blood vessel X indicates a coronary artery, and a rectangular solid line frame indicates the area of ​​the extracted image. In addition, a solid line along blood vessel X indicates the center line of blood vessel X. As an example, as shown in the upper diagram of FIG. 4, the image acquiring unit 200 extracts an extracted image including a fragment of blood vessel X so that the center of the rectangular solid line frame is located on the center line of blood vessel X. An example of the extracted image is shown in the lower diagram of FIG. 4.

[0046] The size of the extracted image, the shape of the extracted image, the extraction frequency, the extraction range, the orientation of the extracted image, and the like can be set arbitrarily. For example, the size of the extracted image may be determined according to the standard width of the blood vessel to be inspected, and the shape of the extracted image may be a circle, a diamond, or the like. The extraction frequency may be determined as the number of times per image. The extraction range may be limited to a range where the occurrence frequency of abnormal parts is high. The orientation of the extracted image may be the same as that of the original image, or may be parallel to the blood vessel. Furthermore, each of the above-mentioned settings may be dynamically changed, and may be set according to the part, for example, by setting the extraction frequency high in a place where the occurrence frequency of abnormal parts is high and setting the extraction frequency low in a place where the occurrence frequency of abnormal parts is low.

[0047] [Judgment step S11] The determination step S11 is a step for determining whether or not the blood vessel to be inspected includes an abnormal part based on the extracted image extracted from the original image in the image acquisition step S10. As an example, in the determination step S11, the determination unit 201 inputs the extracted image acquired in the image acquisition step S10 to a trained model, and acquires an output image in which a bounding box is attached to an abnormal part of a fragment of a blood vessel included in the extracted image recognized by the trained model as a detection result. The determination unit 201 determines whether or not the blood vessel to be inspected includes an abnormal part based on the detection result of the trained model. The determination unit 201 makes a determination based on whether or not the output image acquired as the detection result includes an image to which a bounding box is attached. If the output image includes an image to which a bounding box is attached, it is determined that "an abnormal part is present," and if the output image does not include an image to which a bounding box is attached, it is determined that "no abnormal part is present."

[0048] In the determination step S11, when it is determined that the blood vessel to be inspected does not have an abnormal portion (NO in the determination step S11), the diagnosis support device 2 performs a display control step S13 shown in FIG.

[0049] [Image generation step S12] In the determination step S11, when it is determined that the blood vessel to be inspected has an abnormal part (YES in the determination step S11), the diagnosis support device 2 carries out the image generating step S12.

[0050] The image generating step S12 is a step for generating a display image that displays the determination result determined by the image generating unit 202 in the determination step S11 to the user. As an example, in the image generating step S12, the image generating unit 202 generates a display image in which a mark indicating an abnormal part recognized by the trained model in the determination step S11 is added along the blood vessel inside and outside the blood vessel near the abnormal part of the original image including the blood vessel to be inspected as a subject. In the image generating step S12, an example of a method for generating a display image performed by the image generating unit 202 is a method for generating a display image based on an extracted image to which a bounding box is added, among the output images output by the trained model in the determination unit 201. This method can be realized by linking in advance which part of the original image the extracted image is an image extracted from, using position information on the original image or the like. For example, the position of the abnormal part on the original image is specified based on position information of the extracted image to which the bounding box is added, and a mark is added along the blood vessel inside and outside the blood vessel near the specified abnormal part.

[0051] [Display control process S13] The display control step S13 is a step in which the display control unit 203 displays various images to the user. As an example, in the display control step S13, the display control unit 203 displays the display image generated in the image generation step S12 on the display unit 31 of the image display device 3. Furthermore, when it is determined in the determination step S11 that the blood vessel to be inspected does not have an abnormal part (NO in the determination step S11), the display control unit 203 displays an original image including the blood vessel to be inspected as a subject as the display image.

[0052] (Modification 1 of image acquisition step S10) A modified example of the image acquiring step S10 (hereinafter, referred to as "image acquiring step S20") will be described with reference to Fig. 5. Fig. 5 is a flow diagram showing the flow of each process performed by the modified example of the image acquiring step.

[0053] The image acquiring step S20 differs from the image acquiring step S10 in that an extracted image including a fragment of a blood vessel to be inspected is extracted from a plurality of original images expressed by different pixel values. First, the image acquiring unit 200 acquires imaging data of the blood vessel to be inspected from the imaging device 1 as shown in Fig. 5 (step S200).

[0054] Next, the image acquisition unit 200 generates a plurality of original images including the blood vessels to be inspected as subjects, which are expressed with different pixel values ​​based on the imaging data received from the imaging device 1 (step S201). Here, the pixel value is a value that represents the shade of color in each pixel of the image. As an example, the image acquisition unit 200 generates an image of the same subject that differs only in pixel values ​​as an original image.

[0055] Hereinafter, step S201 will be specifically described by taking the case where the imaging device 1 is a CT device as an example. The CT image generated based on the imaging data captured by the CT device is an image in which the CT value is calculated by quantifying the degree of X-rays absorbed by each organ and each tissue when X-rays are irradiated to the human body, and the difference in the CT value is expressed by shading. That is, the CT value is an example of a pixel value. In a general CT image, the range ww of the CT value expressed by shading on the image and the median wl of the CT value expressed by shading on the image (hereinafter, referred to as "ww / wl") are set according to the standard CT value of the organ or tissue to be inspected. This makes it possible to more clearly express only the necessary information by shading. In this embodiment, multiple images with different values ​​of ww / wl are used as original images including the blood vessels to be inspected as subjects.

[0056] Next, the image acquisition unit 200 extracts an extracted image including a fragment of the blood vessel to be inspected from each of the acquired original images (step S202). The method of extracting the extracted image from the original image is the same as step S102 of the image acquisition process S10, and therefore the explanation is omitted here.

[0057] Finally, the image acquisition unit 200 integrates the extracted images extracted in step S202 (step S203). As an example, the image acquisition unit 200 integrates extracted images that have the same subject and differ only in pixel values ​​among the extracted images extracted in step S202. The method of integrating the images here is not particularly limited. In this embodiment, the images are integrated using a known method.

[0058] In the image acquisition step S20, an extracted image including a fragment of a blood vessel to be inspected is acquired by executing the processes of steps S200 to S202.

[0059] (Effects of diagnostic support system 100) As described above, the diagnosis support system 100 is configured to extract an extracted image including a fragment of a blood vessel to be inspected from a large-sized medical image, and determine whether or not an abnormal part is included in each of the extracted images based on a trained model, thereby determining whether or not an abnormal part is included in the blood vessel to be inspected. As a result, the trained model detects abnormal parts from small-sized extracted images, and therefore does not require the use of a complex algorithm and can shorten the learning time. As a result, there is no need to introduce high-performance hardware equipped with a GPU specialized for image processing, and therefore the introduction cost to a general medical institution can be reduced. In addition, the widespread use of the diagnosis support system 100 can make diagnoses more efficient and reduce the burden on doctors.

[0060] (Effect of Modification 1 of Image Acquisition Step S10) In general CT images, the ww / wl value is set according to the organ or tissue to be examined, and the image is generated according to that value. Therefore, anything having a CT value outside the set ww / wl value is not displayed on the image. Therefore, even if an abnormality actually occurs, it may not appear on the CT image. On the other hand, the image acquisition step S20 adopts a configuration in which an extracted image is acquired from each of a plurality of original images represented by different pixel values ​​and integrated, so that abnormal parts can be displayed on the image with high accuracy. This can contribute to improving the accuracy of doctors' diagnoses.

[0061] (Modification 2 of Image Acquisition Step S10) Another modified example of the image acquiring step S10 (hereinafter, referred to as "image acquiring step S30") will be described with reference to Fig. 6. Fig. 6 is a flow chart showing the flow of each process performed by the modified example of the image acquiring step.

[0062] The image acquiring step S30 differs from the image acquiring step S10 in that an extracted image including a fragment of a blood vessel to be inspected is extracted from an original image including the blood vessel as a subject projected from a different direction. First, the image acquiring unit 200 acquires imaging data of the blood vessel to be inspected from the imaging device 1 (step S300), as shown in FIG.

[0063] Next, the image acquiring unit 200 generates an original image including the blood vessel of the inspection target projected from different directions as a subject based on the imaging data received from the imaging device 1 (step S301). As an example of an original image including the blood vessel of the inspection target projected from different directions as a subject, a CPR image that two-dimensionally represents the blood vessel of the inspection target within one image and an MPR image that represents a cross section of the lumen of the blood vessel of the inspection target are generated.

[0064] Next, the image acquiring unit 200 extracts an extracted image including a fragment of the blood vessel from each of the original images including the blood vessel of the inspection target projected from different directions as a subject (step S302). An example of a method for extracting the extracted image in step S302 will be described with reference to FIG. 7. FIG. 7 is a schematic diagram showing a modified example of the image acquiring step. In FIG. 7, the extraction method will be described using a diagram in which a part of the blood vessel of the inspection target included as a subject in the CPR image and the MPR image is extracted.

[0065] First, as shown in the upper left diagram of FIG. 7, the image acquiring unit 200 extracts an extracted image including a fragment of the blood vessel X along the center line of the blood vessel X to be inspected in the CPR image (lower left diagram of FIG. 7). Then, the image acquiring unit 200 extracts an extracted image including a lumen cross section at the same site as the fragment of the blood vessel X included in each of the extracted images extracted from the CPR image from the MPR image. Here, the "lumen cross section at the same site" may be a lumen cross section corresponding to any position of the fragment of the blood vessel X included in the extracted image. For example, in the extracted image A in the lower left diagram of FIG. 7, the lumen cross sections corresponding to the positions of X1 and X2 are both lumen cross sections at the same site as the fragment of the blood vessel X included in the extracted image A. Note that, as a method for extracting an extracted image including a corresponding lumen cross section from the MPR image, a method of linking position information in the CPR image to position information in the MPR image based on the characteristics (e.g., center line or shape) of the blood vessel X to be inspected can be given.

[0066] In the image acquisition step S30, the processes of steps S300 to S302 are performed to acquire extracted images in which fragments of the blood vessel to be inspected are projected from different directions.

[0067] (Effect of Modification 2 of Image Acquisition Step S10) In the image acquisition step S30, by adopting a configuration for acquiring extracted images in which fragments of the blood vessel to be inspected are projected from different directions, the blood vessel to be inspected can be confirmed from multiple directions. This allows the condition of the blood vessel to be grasped more accurately, and enables the doctor to make a more accurate diagnosis.

[0068] (Modification of diagnosis support system 100) In the diagnosis support system 100 according to this embodiment, the determination unit 201 uses a learned model to determine whether or not the blood vessel to be inspected contains an abnormal part, but the present invention is not limited to this. For example, an artificial intelligence algorithm such as pattern recognition or rule-based may be used to determine whether or not the blood vessel to be inspected contains an abnormal part. In this case, the determination unit 201 executes an artificial intelligence algorithm such as pattern recognition or rule-based to detect an abnormal part in the extracted image acquired by the image acquisition unit 200 based on a pattern or rule set in advance, and outputs an image in which a bounding box is attached to each region of the abnormal part as an example of the detection result. If the output image includes an image with a bounding box attached, it is determined that "an abnormal part is present," and if the output image does not include an image with a bounding box attached, it is determined that "no abnormal part is present."

[0069] Here, examples of the preset patterns and rules include: when an area having a brightness or RGB value equal to or greater than a predetermined value is detected in an extracted image, the area is detected as an abnormal area; when a predetermined shape is identified from an image obtained by Hough transforming an extracted image, the identified shape is detected as an abnormal area; when a predetermined texture is extracted by texture analysis of an extracted image based on a gray level co-occurrence matrix (GLCM), an area having the predetermined texture is detected as an abnormal area; etc. The preset patterns and rules can be arbitrarily determined, and the same applies to a method of outputting a detection result in which an abnormal area is detected based on the patterns and rules. [Embodiment 2] Another embodiment of the present invention will be described below with reference to Figures 8 and 9. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0070] (Outline of diagnostic support system 100a) The diagnostic support system 100a according to this embodiment is a system that determines whether or not the blood vessel to be examined contains an abnormal part, and if it is determined that the blood vessel to be examined contains an abnormal part, analyzes the disease risk of the abnormal part.

[0071] (Configuration of diagnosis support system 100a) The configuration of the diagnosis support system 100a will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the configuration of the diagnosis support system 100a.

[0072] As shown in FIG. 8, the diagnosis support system 100a differs from the diagnosis support system 100 in that it includes a diagnosis support device 2a instead of the diagnosis support device 2, but the rest of the configuration is the same.

[0073] [Diagnosis support device 2a] 8, the diagnosis support device 2a differs from the diagnosis support device 2 in that it includes a control unit 12a instead of the control unit 12. The control unit 20a differs from the control unit 20 in that it includes an image generation unit 202a instead of the image generation unit 202 and a display control unit 203a instead of the display control unit 203, and further includes an analysis unit 204a.

[0074] When the determination unit 201 determines that the blood vessel to be inspected includes an abnormal portion, the analysis unit 204a analyzes the disease risk of the abnormal portion based on an extracted image in which a bounding box is attached to the abnormal portion of the blood vessel fragment to be inspected, which is output by the trained model in the determination unit 201. One example of a method for analyzing the disease risk from an extracted image to which a bounding box is attached is a method using a trained model. Specifically, the analysis unit 204a inputs an extracted image in which a bounding box is attached to the abnormal portion of the blood vessel fragment to be inspected, into a trained model that is trained to determine the type of the abnormal portion, using an image including a blood vessel fragment including an abnormal portion as input information. Then, the analysis unit 204a determines the disease risk of the abnormal portion by referring to the determination result by the trained model.

[0075] The image generating unit 202a generates a display image for displaying to a user the determination result obtained by the determining unit 201 and the analysis result obtained by the analyzing unit 204a. As an example, the image generating unit 202a generates a display image in which an original image including a blood vessel to be inspected as a subject is provided with an abnormal part in the blood vessel and a mark indicating a disease risk of the abnormal part.

[0076] The display control unit 203a displays various images to the user. As an example, the display control unit 203a displays the display image generated in the image generation unit 202a to the user. One example of a method of displaying the display image to the user is a method of displaying the display image on the display unit 31 of the image display device 3 owned by the user. Specifically, the display control unit 203a transmits image data representing the display image generated by the image generation unit 202a to the image display device 3 via the communication unit 21 of the diagnosis support device 2a, and outputs the image on the display unit 31 built in the image display device 3. In this embodiment, a configuration is adopted in which the display image is displayed on the display unit 31 of the image display device 3, which is an external device of the diagnosis support device 2a, but the display control unit 203a may display the display image on a display unit (not shown) built in the diagnosis support device 2a.

[0077] (Pre-trained model) In this embodiment, the trained model used to analyze disease risk will be described in detail. The trained model is a model trained to determine the type of abnormal part of a blood vessel fragment. Examples of training data for the trained model include training data including one or more pairs of an image including an abnormal part in a blood vessel fragment and an image in which the type of the abnormal part in the image is labeled. For example, when the abnormal part determined by the determination unit 201 is a stenosis due to unstable plaque and the model has been trained to determine the type of unstable plaque that forms the stenosis, examples of training data include one or more pairs of an image including a blood vessel fragment including a stenosis due to unstable plaque as a subject and an image in which the type of unstable plaque that forms the stenosis in the image is labeled.

[0078] The trained model uses an extracted image containing an abnormal part in a fragment of a blood vessel as input information and determines the type of the abnormal part contained in the input image. The trained model then outputs, as an example of a detection result, an image in which a bounding box and information indicating the type of the abnormal part are attached to each region of the abnormal part. Note that the trained model may be a model generated by deep learning, or may be a model generated by machine learning other than deep learning.

[0079] (Flow of diagnostic support method S1a) A diagnostic support method S1a executed by the diagnostic support device 2a will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing the flow of the diagnostic support method S1a.

[0080] As shown in Fig. 9, the diagnostic support method S1a includes an image acquisition step S10, a determination step S11, an analysis step S14a, an image generation step S12a, and a display control step S13a. The diagnostic support method S1a differs from the diagnostic support method S1 in that it includes an image generation step S12a instead of the image generation step S12 and a display control step S13a instead of the display control step S13, and further includes an analysis step S14a. The image acquisition step S10 and the determination step S11 are similar to the image acquisition step S10, and therefore will not be described here.

[0081] [Analysis step S14a] The analysis step S14a is a step for analyzing the disease risk of the abnormal part when it is determined in the determination step S11 that the blood vessel to be inspected has an "abnormal part". As an example, in the analysis step S14a, the analysis unit 204a inputs an extracted image with a bounding box representing the abnormal part to the trained model, and obtains an output image in which a character representing the type of the abnormal part determined by the trained model is displayed near the bounding box as a determination result. The analysis unit 204a refers to the output image to analyze the disease risk of the abnormal part. For example, if the abnormal part with the bounding box is a stenosis caused by unstable plaque, the trained model determines the type of unstable plaque that forms the stenosis. The types of unstable plaque determined here are five types: SC, ST, LHU, and NRS. Based on the appearance patterns of these five types of unstable plaque, the disease risk is evaluated in three stages: normal, dangerous, and very dangerous. For example, for each extracted image having a bounding box representing a stenosis caused by unstable plaque, the determination results of the type of unstable plaque that forms each stenosis are integrated, and the disease risk is evaluated as normal, high risk, or very high risk based on the combination of unstable plaque types.

[0082] [Image generation step S12a] The image generating step S12a is a step for generating a display image for displaying to a user the judgment result judged in the judgment step S11 and the analysis result analyzed in the analysis step S14a. As an example, in the image generating step S12a, the image generating unit 202a generates a display image in which a mark representing an abnormal part recognized by the trained model in the judgment step S11 and an analysis result of a disease risk of the abnormal part analyzed in the analysis step S14a are added inside or outside the blood vessel near the abnormal part of an original image including the blood vessel to be inspected as a subject, along the blood vessel. In the image generating step S12a, an example of a method for generating a display image performed by the image generating unit 202a is a method for generating a display image based on an extracted image to which a bounding box is added, among the output images output by the trained model in the judgment unit 201a. This method can be realized by linking in advance which part of the original image the extracted image is an image extracted from, using position information on the original image, or the like. For example, the position of the abnormal part on the original image is specified based on the position information of the extracted image with the bounding box, and the abnormal part is marked along the blood vessel inside or outside the blood vessel near the blood vessel. The color or shade of the marking is then changed to show the analysis result of the disease risk of the abnormal part.

[0083] [Display control process S13a] The display control step S13a is a step for displaying various images to a user. As an example, in the display control step S13a, the display control unit 203a displays the display image generated in the image generation step S12a on the display unit 31 of the image display device 3. Furthermore, when it is determined in the determination step S11 that the blood vessel to be inspected does not have an abnormal part (NO in the determination step S11), the display control unit 203a displays an original image including the blood vessel to be inspected as a subject as a display image.

[0084] (Effects of diagnostic support system 100a) The diagnosis support system 100a not only determines whether or not the blood vessel to be inspected contains an abnormal part, but also determines the disease risk of the abnormal part, thereby contributing to more efficient diagnosis. [Hardware configuration and software implementation examples] The diagnosis support device 2 (mainly each unit included in the control unit 20) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit). In the latter case, the diagnosis support device 2 may be configured using a computer (electronic calculator) as shown in FIG.

[0085] FIG. 10 is a block diagram showing an example of the configuration of a computer 4 that can be used as the diagnostic support device 2. The computer 4 includes a calculation device 40, a main storage device 41, an auxiliary storage device 42, an input / output interface 43, and a communication interface 44, which are connected to each other via a bus. The calculation device 40, the main storage device 41, and the auxiliary storage device 42 may each be, for example, a CPU, a RAM (Random Access Memory), a solid-state drive, or a hard disk drive. The input / output interface 43 is connected to an input device 45 for a user to input various information to the computer 4, and an output device 46 for the computer 4 to output various information to the user. The input device 45 and the output device 46 may be built into the computer 4, or may be connected (externally) to the computer 4. For example, the input device 45 may be a button, a keyboard, a mouse, a touch sensor, or the like, and the output device 46 may be a lamp, a display, a printer, a speaker, or the like. In addition, a device having the functions of both the input device 45 and the output device 46, such as a touch panel in which a touch sensor and a display are integrated, may be applied. The communication interface 44 is an interface for the computer 4 to communicate with external devices.

[0086] The auxiliary storage device 42 stores a diagnostic support program for operating the computer 4 as the diagnostic support device 2. The arithmetic device 40 deploys the information processing program stored in the auxiliary storage device 42 on the main storage device 41 and executes instructions included in the information processing program, thereby causing the computer 4 to function as each unit of the diagnostic support device 2. Note that the recording medium used by the auxiliary storage device 42 to record information such as the information processing program may be any computer-readable "non-transient tangible medium," and may be, for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like.

[0087] Also, a configuration may be adopted in which the computer 4 is made to function using a program recorded on a recording medium external to the computer 4, or a program supplied to the computer 4 via any transmission medium (such as a communication network or broadcast waves).The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0088] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0089] 〔summary〕 The diagnostic support method according to aspect 1 of this embodiment includes an image acquisition step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as a subject, a determination step of determining whether or not the blood vessel includes an abnormal portion by inputting the extracted image into a trained model trained to detect abnormal portions of the fragment of the blood vessel, using as input information an image including the fragment of the blood vessel acquired from an image capturing the blood vessel, and a display control step of displaying the determination result obtained in the determination step to a user.

[0090] According to the above configuration, a doctor can make a diagnosis while referring to the determination result as to whether or not the blood vessel being examined contains an abnormal part, which contributes to improving the efficiency of diagnosis and reduces the possibility of differences in diagnostic results depending on the doctor.

[0091] The diagnostic support method according to aspect 2 of the present embodiment, in the above-mentioned aspect 1, further includes an analysis step of analyzing the disease risk of the abnormal part if it is determined in the judgment step that the blood vessel to be examined contains an abnormal part, and the display control step further displays the analysis results analyzed in the analysis step.

[0092] According to the above configuration, a doctor can make a diagnosis while referring to the analysis results of the disease risk of abnormal areas contained in the blood vessel being examined, which contributes to improving the efficiency of diagnosis and reduces the possibility of differences in diagnostic results depending on the doctor.

[0093] A diagnosis support method according to aspect 3 of the present embodiment is in accordance with aspect 1 or 2 above, wherein the original images include images of the blood vessels of the inspection target projected from different directions.

[0094] According to the above configuration, it is possible to improve the accuracy of determining whether or not the blood vessel to be inspected contains an abnormal part and of analyzing the risk of disease.

[0095] In a diagnostic support method according to aspect 4 of the present embodiment, in aspect 3 above, the extracted images extracted from each of the images projecting the blood vessel to be inspected from different directions include the same portion of the blood vessel as a subject.

[0096] According to the above configuration, it is possible to improve the accuracy of determining whether or not the blood vessel to be inspected contains an abnormal part and of analyzing the risk of disease.

[0097] A diagnostic support method according to aspect 5 of the present embodiment is any one of aspects 1 to 4 above, in which the display control process displays to a user an image of the original image with a marking indicating the abnormal area determined in the determination process, and the marking is placed along the blood vessel outside the blood vessel near the abnormal area.

[0098] According to the above configuration, when the determination result and analysis result are displayed to the doctor, the doctor's diagnosis is not hindered.

[0099] A diagnosis support method according to aspect 6 of the present embodiment is any one of aspects 1 to 5 above, in which the extracted image is extracted based on a center line of the blood vessel of the inspection target included as a subject in the original image.

[0100] According to the above configuration, the position of a blood vessel can be efficiently detected from an image that includes the blood vessel as a subject, thereby making it possible to efficiently extract an extracted image that includes a fragment of the blood vessel.

[0101] A diagnostic support method according to aspect 7 of the present embodiment is any one of aspects 1 to 6 above, wherein the trained model is constructed by machine learning using an image including a fragment of a blood vessel obtained from an image capturing the blood vessel, in which an abnormal part of the fragment of the blood vessel is labeled.

[0102] According to the above configuration, the trained model can suitably detect abnormal parts of blood vessels.

[0103] The diagnosis support method according to an eighth aspect of the present embodiment is any one of the first to seventh aspects, wherein the abnormal part is a stenosis.

[0104] According to the above configuration, it is possible to determine whether or not a stenosis exists in the blood vessel to be inspected as an abnormal part of the blood vessel.

[0105] The diagnostic assistance program according to aspect 9 of this embodiment causes a computer to perform diagnostic assistance processing including an image acquisition step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as a subject, a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model trained to detect abnormal portions in the fragment of the blood vessel using an image including the fragment of the blood vessel acquired from an image of the blood vessel as input information, and a display control step of displaying the determination result determined in the determination step to a user.

[0106] According to the above configuration, a doctor can make a diagnosis while referring to the determination result as to whether or not the blood vessel being examined contains an abnormal part, which contributes to improving the efficiency of diagnosis and reduces the possibility of differences in diagnostic results depending on the doctor.

[0107] A diagnostic support device according to aspect 10 of this embodiment includes an image acquisition unit that acquires an extracted image including a fragment of a blood vessel from an original image that includes the blood vessel as a subject, a judgment unit that judges whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model that is trained to detect abnormal portions of the fragment of the blood vessel using an image including the fragment of the blood vessel acquired from an image of the blood vessel as input information, and a display control unit that displays the judgment result judged in the judgment process to a user.

[0108] According to the above configuration, a doctor can make a diagnosis while referring to the determination result as to whether or not the blood vessel being examined contains an abnormal part, which contributes to improving the efficiency of diagnosis and reduces the possibility of differences in diagnostic results depending on the doctor.

[0109] A diagnosis support device according to an eleventh aspect of the present embodiment is the same as in the tenth aspect, in which the display control unit outputs the determination result to an image display device.

[0110] According to the above configuration, the diagnosis support device can display the display image showing the judgment result on the image display device, which is an external device. In this way, if the image display device 3 is a portable device, the display image can be viewed at any location.

[0111] A diagnostic support method according to aspect 12 of this embodiment includes an image acquisition step of acquiring an extracted image including a fragment of a blood vessel to be inspected from an original image including the blood vessel as a subject, a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by using an artificial intelligence algorithm that detects an abnormal portion of the fragment of the blood vessel from an image including the fragment of the blood vessel acquired from an image capturing the blood vessel, and a display control step of displaying the determination result obtained in the determination step to a user.

[0112] According to the above configuration, the same effects as those of the first aspect are achieved. [Explanation of symbols]

[0113] 1. Imaging device 2. Diagnostic support devices 3 Image display device 4. Computer

Claims

1. an image acquiring step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as an object of the inspection; a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model trained to detect an abnormal portion of the fragment using an image including the fragment of the blood vessel obtained from an image of the blood vessel as input information; A display control step of displaying the determination result determined in the determination step to a user. A diagnostic support method comprising:

2. The method further includes an analysis step of analyzing a disease risk of the abnormal part when it is determined in the determination step that the blood vessel to be inspected contains an abnormal part, The display control step further displays the analysis results analyzed in the analysis step. The diagnostic support method according to claim 1 .

3. The original images include images of the blood vessel of the inspection target projected from different directions. The diagnostic support method according to claim 1 .

4. the extracted images extracted from the images of the blood vessel of the inspection target projected from different directions each include the same portion of the blood vessel as a subject; 4. The diagnostic support method according to claim 3.

5. The display control step displays to a user an image obtained by adding a mark representing the abnormal part determined in the determination step to the original image, The marking is provided along the blood vessel near the abnormality. The diagnostic support method according to claim 1 .

6. the extracted image is extracted based on a center line of the blood vessel of the inspection target included as an object in the original image; The diagnostic support method according to claim 1 .

7. The trained model is constructed by machine learning using an image in which an abnormal part of a fragment of a blood vessel is labeled in an image including the fragment of the blood vessel obtained from an image capturing the blood vessel, The diagnostic support method according to claim 1 .

8. The abnormal portion is a narrowed portion. The diagnostic support method according to claim 1 .

9. On the computer, an image acquiring step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as an object of the inspection; a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model trained to detect an abnormal portion of the fragment using an image including the fragment of the blood vessel obtained from an image of the blood vessel as input information; and a display control step of displaying the result of the judgment made in the judgment step to a user. A diagnostic support program comprising:

10. an image acquisition unit that acquires an extracted image including a fragment of a blood vessel from an original image including the blood vessel as an object of the inspection; a determination unit that determines whether or not the blood vessel to be inspected includes an abnormal portion by inputting the extracted image into a trained model trained to detect an abnormal portion of the fragment of the blood vessel, using an image including the fragment of the blood vessel obtained from an image of the blood vessel as input information; A display control unit that displays the determination result determined by the determination unit to a user. A diagnostic support device comprising:

11. The display control unit outputs the determination result to an image display device. The diagnosis support device according to claim 10 .

12. an image acquiring step of acquiring an extracted image including a fragment of a blood vessel from an original image including the blood vessel as an object of the inspection; a determination step of determining whether or not the blood vessel to be inspected includes an abnormal portion by using an artificial intelligence algorithm to detect an abnormal portion of a fragment of the blood vessel from an image including the fragment of the blood vessel obtained from an image of the blood vessel; A display control step of displaying the determination result determined in the determination step to a user. A diagnostic support method comprising:

Images