Medical image processing device, operating method for medical image processing device, and operating program for medical image processing device

Abstract

Provided are a medical image processing device, a method for operating a medical image processing device, and an operation program for a medical image processing device that employs a method conforming with the thoughts of a physician, whereby it is possible to contribute towards discovery of a lesion that is mostly unrevealed. A CPU is provided with an image acquisition unit, a first derivation unit, and a second derivation unit. The image acquisition unit acquires a medical image. The first derivation unit analyzes the medical image and thereby derives indirect finding information relating to indirect findings representing the characteristics of the condition and / or the shape of tissue surrounding a lesion associated with development of the lesion. The second derivation unit derives, on the basis of the indirect findings information, lesion presence probability information indicating the probability of the presence of a lesion.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a medical image processing device, an operating method for a medical image processing device, and an operating program for a medical image processing device. [Background technology]

[0002] In the medical field, computer-aided diagnosis (CAD) has been put to practical use, which derives the probability of existence and location information of a lesion by analyzing medical images and presents this to doctors such as radiologists. For example, Patent Document 1 describes a technology that calculates the probability of existence of lesions related to lung cancer, such as honeycomb lung and ground-glass opacity, by analyzing chest tomography images taken with a CT (Computed Tomography) device, and presents a probability image in which the existence probability is represented by pixel values ​​to doctors. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-102916 Summary of the Invention [Problem to be solved by the invention]

[0004] Depending on the type and size of the lesion or the imaging technique, the lesion may not be clearly visualized in the medical image. For example, tumors related to pancreatic cancer are relatively clearly visualized in contrast-enhanced tomographic images of the abdomen, but are hardly visualized in non-contrast-enhanced tomographic images.

[0005] Doctors can sometimes discover these barely visible lesions through indirect findings in medical images. Indirect findings are characteristics of the shape and / or properties of the tissue surrounding the lesion that appear as the lesion develops. Examples of indirect findings include atrophy, swelling, and calcification.

[0006] Conventional CAD systems were developed on the assumption that lesions were clearly visualized in medical images to a certain extent, making it difficult to detect lesions that were barely visualized as described above. For this reason, there was a demand for the development of a CAD system that would be able to detect lesions that were barely visualized using indirect findings as clues, in line with the physician's thinking described above.

[0007] One embodiment of the technology disclosed herein provides a medical image processing device, an operating method for a medical image processing device, and an operating program for a medical image processing device that can contribute to the discovery of lesions that are barely visible by using techniques that are in line with a doctor's thinking. [Means for solving the problem]

[0008] The medical image processing device of the present disclosure includes a processor and a memory connected to or built into the processor, and the processor acquires medical images and analyzes the medical images to derive indirect finding information regarding indirect findings that represent at least one of the shape and properties of the tissue surrounding the lesion associated with the occurrence of the lesion, and derives lesion presence possibility information that indicates the possibility of the presence of the lesion based on the indirect finding information.

[0009] The indirect finding information is preferably at least one of the probability of presence of an indirect finding, position information of an indirect finding, shape feature amount of an indirect finding, property feature amount of an indirect finding, and malignancy of a lesion based on an indirect finding.

[0010] The lesion presence possibility information is preferably at least one of the lesion presence probability and lesion position information.

[0011] The processor preferably presents lesion likelihood information.

[0012] Preferably, the processor presents indirect finding information.

[0013] The processor preferably presents indirect finding information that contributes relatively highly to the derivation of lesion presence possibility information in a manner that distinguishes it from indirect finding information that contributes relatively less.

[0014] It is preferable that the processor derives tentative lesion existence possibility information indicating the tentative possibility of the presence of a lesion, which represents at least one of the shape and properties of the lesion, by analyzing the medical image, and derives the lesion existence possibility information based on the indirect finding information and the tentative lesion existence possibility information.

[0015] The provisional lesion existence possibility information is preferably at least one of a provisional lesion existence probability, provisional lesion position information, provisional lesion shape feature amount, provisional lesion property feature amount, and provisional lesion malignancy level.

[0016] The processor preferably presents indirect findings information and tentative lesion likelihood information.

[0017] It is preferable that the processor presents indirect finding information and tentative lesion presence possibility information that have a relatively high contribution to the derivation of lesion presence possibility information separately from indirect finding information and tentative lesion presence possibility information that have a relatively low contribution.

[0018] Preferably, the lesion comprises at least one of a tumor and a cyst.

[0019] Preferably, the indirect findings representing shape characteristics include at least one of atrophy, swelling, stenosis, and dilation, and the indirect findings representing property characteristics include at least one of fatty substitution and calcification.

[0020] The medical image is a non-contrast tomographic image of the abdomen showing the pancreas, and is preferably used for diagnosing pancreatic cancer.

[0021] The operating method of the medical image processing device disclosed herein includes acquiring a medical image, analyzing the medical image to derive indirect finding information regarding indirect findings that represent at least one of the shape and properties of the tissue surrounding the lesion associated with the occurrence of the lesion, and deriving lesion presence possibility information that indicates the possibility of the presence of the lesion based on the indirect finding information.

[0022] The operating program of the medical image processing device disclosed herein causes a computer to perform processes including acquiring medical images, analyzing the medical images to derive indirect finding information regarding indirect findings that represent at least one of the shape and properties of the tissue surrounding the lesion associated with the occurrence of the lesion, and deriving lesion presence possibility information that indicates the possibility of the presence of the lesion based on the indirect finding information. [Effects of the Invention]

[0023] According to the technology disclosed herein, it is possible to provide a medical image processing device, an operating method for a medical image processing device, and an operating program for a medical image processing device that can contribute to the discovery of lesions that are barely visible by using techniques that conform to the way doctors think. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 illustrates a medical system. [Figure 2] FIG. 2 is a block diagram showing a computer constituting a doctor terminal. [Figure 3] FIG. 2 is a block diagram showing a processing unit of a CPU of the doctor terminal. [Figure 4] FIG. 10 is a diagram illustrating a first derived model. [Figure 5] FIG. 10 is a diagram illustrating processing by the SS model. [Figure 6] FIG. 2 is a diagram showing a detailed configuration of an SS model. [Figure 7] FIG. 10 is a diagram illustrating elements of a probability distribution map. [Figure 8] FIG. 10 is a diagram illustrating processing by a shape feature derivation unit. [Figure 9]FIG. 10 is a diagram illustrating processing by a property feature amount derivation unit. [Figure 10] FIG. 10 is a diagram illustrating processing by a cut-out image generation unit and a malignancy derivation model. [Figure 11] FIG. 10 is a diagram illustrating processing according to a second derived model. [Figure 12] FIG. 10 is a diagram showing contribution degree information. [Figure 13] FIG. 10 is a diagram showing a first screen. [Figure 14] FIG. 10 is a diagram showing a second screen. [Figure 15] 10 is a flowchart showing a processing procedure of a doctor terminal. [Figure 16] 10 is a flowchart showing a processing procedure of a doctor terminal. [Figure 17] FIG. 10 is a diagram illustrating another example of processing using the second derived model. [Figure 18] FIG. 10 is a diagram illustrating a processing unit of a second embodiment. [Figure 19] FIG. 10 is a diagram illustrating processing by the SS model of the second embodiment. [Figure 20] FIG. 10 is a diagram illustrating processing by a shape feature derivation unit according to the second embodiment. [Figure 21] FIG. 10 is a diagram illustrating a process performed by a property feature amount derivation unit according to the second embodiment. [Figure 22] FIG. 10 is a diagram illustrating processing by a cut-out image generating unit and a malignancy derivation model according to the second embodiment. [Figure 23] FIG. 10 is a diagram illustrating a process according to a second derived model of the second embodiment. [Figure 24] FIG. 10 is a diagram showing contribution degree information according to the second embodiment. [Figure 25] FIG. 10 is a diagram showing a first screen in the second embodiment. [Figure 26] FIG. 10 is a diagram showing a second screen in the second embodiment. [Figure 27] 10 is a flowchart showing a processing procedure of the doctor terminal of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] [First embodiment] 1, the medical system 2 includes a CT device 10, a PACS (Picture Archiving and Communication System) server 11, and a doctor terminal 12. The CT device 10, the PACS server 11, and the doctor terminal 12 are connected to a LAN (Local Area Network) 13 installed in the medical facility, and are capable of communicating with each other via the LAN 13.

[0026] As is well known, the CT device 10 radiographs the patient P at different projection angles to acquire multiple frames of projection data, and then reconstructs the acquired multiple frames of projection data to output a tomographic image 15 of the patient P. The tomographic image 15 is voxel data representing the three-dimensional shape of the internal structure of the patient P. In this example, the patient P is suspected of having pancreatic cancer. Therefore, the tomographic image 15 is an image of the patient P's abdomen, primarily the pancreas 16. In this example, the tomographic image 15 is a non-contrast image captured without using a contrast agent. The tomographic image 15 is an example of a "medical image" and a "non-contrast tomographic image of the abdomen" according to the technology of the present disclosure. Note that FIG. 1 shows the tomographic image 15 of an axial section.

[0027] The CT device 10 transmits the tomographic images 15 to the PACS server 11. The PACS server 11 stores and manages the tomographic images 15 from the CT device 10. Note that the projection data may be reconstructed by the PACS server 11 or the doctor's terminal 12, instead of by the CT device 10.

[0028] The doctor terminal 12 is, for example, a desktop personal computer that is operated by a doctor. The doctor terminal 12 is an example of a "medical image processing device" according to the technology of the present disclosure.

[0029] The doctor terminal 12 includes a display 17 and an input device 18. The input device 18 is a keyboard, a mouse, a touch panel, a microphone, etc. Cross-sectional images 15 are delivered to the doctor terminal 12 from the PACS server 11. The doctor terminal 12 displays the cross-sectional images 15 delivered from the PACS server 11 on the display 17. The doctor observes the pancreas 16 of the patient P shown in the cross-sectional images 15 and diagnoses pancreatic cancer. Note that although only one CT device 10 and one doctor terminal 12 are shown in FIG. 1, there may be multiple CT devices 10 and multiple doctor terminals 12.

[0030] 2, the computer constituting the doctor terminal 12 includes, in addition to the display 17 and input device 18, a storage 20, a memory 21, a CPU (Central Processing Unit) 22, and a communication unit 23. These are interconnected via a bus line 24.

[0031] The storage 20 is a hard disk drive built into the computer that constitutes the doctor terminal 12 or connected via a cable or network. Alternatively, the storage 20 is a disk array consisting of multiple hard disk drives. The storage 20 stores control programs such as an operating system, various application programs, and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0032] The memory 21 is a work memory for the CPU 22 to execute processing. The CPU 22 loads programs stored in the storage 20 into the memory 21 and executes processing in accordance with the programs. In this way, the CPU 22 comprehensively controls each part of the computer. The CPU 22 is an example of a "processor" according to the technology of the present disclosure. The communication unit 23 controls the transmission of various information to and from external devices such as the PACS server 11. The memory 21 may be built into the CPU 22.

[0033] As an example, as shown in FIG. 3 , an operating program 30 is stored in the storage 20 of the doctor terminal 12. The operating program 30 is an application program for causing the computer constituting the doctor terminal 12 to function as a “medical image processing device” according to the technology of the present disclosure. In other words, the operating program 30 is an example of an “operating program for a medical image processing device” according to the technology of the present disclosure. The operating program 30 may be recorded and distributed on an external recording medium (not shown) and installed by the CPU 22 from the recording medium. Alternatively, the operating program 30 may be stored in an externally accessible state on a server connected to a network, downloaded to the storage 20 by the CPU 22 upon request, installed, and executed. The storage 20 also stores a tomographic image 15, a first derived model 32, and a second derived model 33. The first derived model 32 and the second derived model 33 are machine learning models. The storage 20 also stores data for various screens to be displayed on the display 17. Although only one tomographic image 15 is stored in the storage 20 in FIG. 3, a plurality of tomographic images 15 may be stored in the storage 20.

[0034] When the operating program 30 is started, the CPU 22 of the computer constituting the doctor's terminal 12 works in cooperation with the memory 21, etc. to function as an instruction receiving unit 40, an image acquisition unit 41, a read / write (hereinafter abbreviated as RW (Read Write)) control unit 42, a first derivation unit 43, a second derivation unit 44, and a display control unit 45.

[0035] The instruction receiving unit 40 receives various operation instructions from the input device 18. The operation instructions include an image delivery instruction to instruct the PACS server 11 to deliver a tomographic image 15 of the patient P to be diagnosed, an instruction to analyze the tomographic image 15, and the like. The image delivery instruction includes search keywords such as the patient ID (Identification Data) of the patient P to be diagnosed and the date and time of imaging. The instruction receiving unit 40 outputs the search keywords of the image delivery instruction to the image acquisition unit 41. The instruction receiving unit 40 also outputs to the first derivation unit 43 and the second derivation unit 44 that an instruction to analyze the tomographic image 15 has been received.

[0036] The image acquisition unit 41 transmits an image distribution request 50, to which the search keyword of the image distribution instruction from the instruction receiving unit 40 has been copied, to the PACS server 11. The PACS server 11 searches for the tomographic image 15 requested in the image distribution request 50. The PACS server 11 distributes the searched tomographic image 15 to the image acquisition unit 41. The image acquisition unit 41 acquires the tomographic image 15 distributed from the PACS server 11. The image acquisition unit 41 outputs the acquired tomographic image 15 to the RW control unit 42.

[0037] The RW control unit 42 controls the storage of various data in the storage 20 and the reading of various data from the storage 20. For example, the RW control unit 42 stores the tomographic image 15 from the image acquisition unit 41 in the storage 20. The RW control unit 42 also reads the tomographic image 15 from the storage 20 and outputs the read tomographic image 15 to the first derivation unit 43 and the display control unit 45.

[0038] The RW control unit 42 reads the first derived model 32 from the storage 20 and outputs the read first derived model 32 to the first derivation unit 43. The RW control unit 42 also reads the second derived model 33 from the storage 20 and outputs the read second derived model 33 to the second derivation unit 44.

[0039] The first derivation unit 43 operates when the instruction receiving unit 40 receives an instruction to analyze the tomographic image 15. The first derivation unit 43 derives indirect finding information 51 by analyzing the tomographic image 15 using the first derivation model 32. The indirect finding information 51 is information about indirect findings that represent at least one of the characteristics of the shape and properties of tissue surrounding a tumor associated with the occurrence of the tumor in the pancreas 16. The first derivation unit 43 outputs the derived indirect finding information 51 to the second derivation unit 44 and the display control unit 45. The word "indirect" in indirect finding is used in contrast to the expression "direct" finding, which refers to a lesion such as a tumor that is directly linked to a disease such as cancer.

[0040] Like the first derivation unit 43, the second derivation unit 44 also operates when the instruction receiving unit 40 receives an instruction to analyze the tomographic image 15. The second derivation unit 44 uses the second derivation model 33 to derive lesion presence possibility information 52 based on the indirect finding information 51. The lesion presence possibility information 52 is information indicating the possibility of the presence of a tumor in the pancreas 16. The second derivation unit 44 outputs the derived lesion presence possibility information 52 to the display control unit 45.

[0041] The display control unit 45 controls the display of various screens on the display 17. The various screens include a first screen 110 (see FIG. 13) for issuing instructions to analyze the tomographic image 15, and a second screen 115 (see FIG. 14) for displaying lesion presence possibility information 52 and the like.

[0042] Indirect findings that show the shape characteristics of the tissue surrounding the tumor (hereinafter referred to as indirect shape findings) include partial atrophy and swelling of the tissue of the pancreas 16, and stenosis and dilation of the pancreatic duct 16D (see Figure 13, etc.). Indirect findings that show the property characteristics of the tissue surrounding the tumor (hereinafter referred to as indirect property findings) include fatty substitution of the tissue of the pancreas 16 (pancreatic parenchyma) and calcification of the tissue of the pancreas 16.

[0043] As an example, as shown in FIG. 4, the first derivation model 32 includes a semantic segmentation model (hereinafter abbreviated as SS model) 60 and a malignancy derivation model 61. As is well known, the SS model 60 is a machine learning model that outputs an output image in which a label representing an extracted object (class) is assigned to each pixel of an input image. In this example, the input image is a tomographic image 15, and the extracted objects are a total of seven classes, including each part of the pancreas 16 exhibiting the above-mentioned indirect findings of atrophy, swelling, stenosis, dilation, fatty substitution, and calcification, as well as the entire pancreas 16. The SS model 60 is constructed using a convolutional neural network (CNN) such as Residual Networks (ResNet) or U-shaped Networks (U-Net). In reality, there are two separate SS models 60: one that labels the parts showing each indirect finding, and one that labels the entire pancreas 16. However, to simplify the explanation, we will refer to them as a single SS model 60.

[0044] 5, the first derivation unit 43 inputs a tomographic image 15 as an input image to the SS model 60. Then, the SS model 60 outputs an output image 65 and an existence probability 66. The first derivation unit 43 also generates position information 67 from the output image 65. The first derivation unit 43 outputs the existence probability 66 and the position information 67 as indirect finding information 51 to the second derivation unit 44, etc.

[0045] The output image 65 is an image in which the above seven classes are labeled for each pixel of the cross-sectional image 15. The entire pancreas 16 is labeled in every cross-sectional image 15, but parts showing indirect findings may not be labeled if they are not depicted in the original cross-sectional image 15, for example.

[0046] The existence probability 66 is the existence probability of each of the six types of indirect findings. The position information 67 is information representing the position of each of the six types of indirect findings. More specifically, the position information 67 is information in which the position coordinates of labeled pixels in the output image 65 are registered for each indirect finding. The existence probability 66 is an example of the "existence probability of an indirect finding" according to the technology of the present disclosure. The position information 67 is an example of the "position information of an indirect finding" according to the technology of the present disclosure.

[0047] As an example, as shown in FIG. 6 , the SS model 60 is composed of a compression unit 70 and an output unit 71. The compression unit 70 has multiple convolution layers that perform convolution processing using filters, and multiple pooling layers that perform pooling processing to reduce the convolution processing data by obtaining local statistics of the convolution processing data. The compression unit 70 converts the tomographic image 15 into a feature map 72. The compression unit 70 outputs the feature map 72 to the output unit 71. Although not shown in the figure, the compression unit 70 also performs skip layer processing and the like to pass the convolution processing data to the output unit 71.

[0048] The output unit 71 includes a decoder unit 73, a probability distribution map generation unit 74, a label assignment unit 75, and a representative value selection unit 76. The decoder unit 73 performs upsampling processing to enlarge the feature map 72 to generate an enlarged feature map. The decoder unit 73 also performs convolution processing during the upsampling processing. The decoder unit 73 also performs merging processing to combine the enlarged feature map with the post-convolution data passed from the compression unit 70 in the skip layer processing. The decoder unit 73 further performs convolution processing after the merging processing. After these various processes, the decoder unit 73 converts the feature map 72 into a final feature map 77.

[0049] The final feature map 77, also called logits, has elements that correspond one-to-one to the pixels of the tomographic image 15. Each element of the final feature map 77 has an element value associated with each class. The decoder unit 73 outputs the final feature map 77 to the probability distribution map generator 74.

[0050] The probability distribution map generation unit 74 uses a well-known activation function such as a softmax function to generate a probability distribution map 78 from the final feature map 77. The probability distribution map generation unit 74 outputs the probability distribution map 78 to the label assignment unit 75 and the representative value selection unit 76.

[0051] As an example, as shown in FIG. 7, the probability distribution map 78, like the final feature map 77, has elements 80 that correspond one-to-one to pixels in the tomographic image 15, and is data in which the probability of each class being present is registered as the element value of each element 80. FIG. 7 illustrates a case in which the probability that the element 80 is a part showing atrophy, i.e., the probability of atrophy being present, is 86% (0.86), and the probabilities of swelling, stenosis, dilation, fatty substitution, and calcification being present are 2%, 1%, 6%, 2%, and 3%, respectively. FIG. 7 also illustrates a case in which the probability of the pancreas 16 being present is 100%. The probability of the presence of indirect findings is 100% when all of them are added together.

[0052] The labeling unit 75 labels each element 80 in the probability distribution map 78 as a class whose existence probability is equal to or greater than a preset threshold. The labeling unit 75 does not label an element 80 whose existence probability of each class is less than the threshold. The threshold is, for example, 80%. Therefore, in the example shown in FIG. 7, the labeling unit 75 labels atrophy, whose existence probability is 86%, and the pancreas 16, whose existence probability is 100%.

[0053] The representative value selection unit 76 selects a representative value of the existence probability for each class of indirect findings. The representative value is, for example, the maximum value or the most frequent value. The representative value selection unit 76 outputs the selected representative value as the existence probability 66.

[0054] As an example, as shown in FIG. 8 , the first derivation unit 43 includes a shape feature derivation unit 85. An output image 65 is input to the shape feature derivation unit 85. Based on the output image 65, the shape feature derivation unit 85 calculates an area ratio 86 of a portion showing indirect shape findings (atrophy, swelling, stenosis, and dilation) relative to the entire pancreas 16. More specifically, the shape feature derivation unit 85 calculates the area ratio 86 by dividing the number of pixels labeled in the portion showing each indirect shape finding in the output image 65 by the number of pixels labeled in the pancreas 16. The shape feature derivation unit 85 outputs the area ratio 86 to the second derivation unit 44, etc., as the indirect finding information 51. The area ratio 86 is an example of a “shape feature of an indirect finding” according to the technology of the present disclosure. The area ratio 86 of a portion showing an indirect characteristic finding may also be calculated.

[0055] As an example, as shown in FIG. 9 , the first derivation unit 43 includes a characteristic feature derivation unit 88. The tomographic image 15 and position information 67 are input to the characteristic feature derivation unit 88. The characteristic feature derivation unit 88 derives a density representative value 89, which is a representative value of pixel values ​​of a portion indicating indirect characteristic findings (fatty substitution and calcification), based on the tomographic image 15 and the position information 67. More specifically, the characteristic feature derivation unit 88 identifies pixels of a portion indicating indirect characteristic findings in the tomographic image 15 from the position information 67. Then, the characteristic feature derivation unit 88 derives, for example, the maximum value, minimum value, mode, or average value of the pixel values ​​of the identified pixels as the density representative value 89. The characteristic feature derivation unit 88 outputs the density representative value 89 to the second derivation unit 44 or the like as indirect finding information 51. The density representative value 89 is an example of a "characteristic feature of an indirect finding" according to the technology of the present disclosure. It is also possible to derive a density representative value 89 of the portion showing the indirect shape finding.

[0056] As an example, as shown in FIG. 10 , the first derivation unit 43 includes a cropped image generation unit 90. The cropped image generation unit 90 receives the tomographic image 15 and position information 67. Based on the position information 67, the cropped image generation unit 90 generates cropped images 91A, 91B, 91C, 91D, 91E, and 91F by cropping box-shaped portions showing each indirect finding from the tomographic image 15. The cropped image 91A is an image obtained by cropping a portion showing atrophy, the cropped image 91B is an image obtained by cropping a portion showing swelling, the cropped image 91C is an image obtained by cropping a portion showing stenosis, and the cropped image 91D is an image obtained by cropping a portion showing dilation. The cropped image 91E is an image obtained by cropping a portion showing fatty substitution, and the cropped image 91F is an image obtained by cropping a portion showing calcification.

[0057] The first derivation unit 43 inputs the cropped images 91A to 91F to the malignancy derivation model 61. Similar to the SS model 60, the malignancy derivation model 61 is configured by a convolutional neural network. The malignancy derivation model 61 derives a tumor malignancy 92 based on indirect findings based on the cropped images 91A to 91F. The malignancy 92 is, for example, on a scale of 1 to 10, with a higher value indicating a higher likelihood that the tumor is malignant. The first derivation unit 43 outputs the malignancy 92 to the second derivation unit 44, etc., as indirect finding information 51. The malignancy 92 is an example of the "malignancy of a lesion based on indirect findings" according to the technology of the present disclosure.

[0058] 11 , the second derivation unit 44 inputs indirect finding information 51, which is composed of an existence probability 66, position information 67, an area ratio 86, a concentration representative value 89, and a malignancy level 92, into the second derivation model 33. The second derivation model 33 is constructed, for example, by a linear discriminant analysis technique or a boosting technique such as XGBoost (Extreme Gradient Boosting). The second derivation model 33 outputs lesion existence possibility information 52 based on the indirect finding information 51.

[0059] The lesion presence possibility information 52 includes a tumor presence probability 100, tumor location information 101, and contribution information 102. The tumor presence probability 100 represents the probability of a tumor being present in the pancreas 16. The tumor location information 101 is information in which the position coordinates of pixels in areas that are suspected to be tumors are registered. The tumor presence probability 100 is an example of a "lesion presence probability" according to the technology of the present disclosure. The tumor location information 101 is an example of "lesion location information" according to the technology of the present disclosure.

[0060] The second derivation model 33 is constructed by a linear discriminant analysis technique or a boosting technique, and therefore is capable of deriving contribution information 102. As shown in FIG. 12 as an example, the contribution information 102 is information in which the contribution to the derivation of tumor presence probability 100 and tumor location information 101 is registered for each piece of indirect finding information 51 (presence probability 66, location information 67, area ratio 86, concentration representative value 89, and malignancy level 92). The greater the contribution of a piece of indirect finding information 51 to the derivation of tumor presence probability 100 and tumor location information 101, the greater the value of the contribution.

[0061] When a tomographic image 15 is transmitted from the PACS server 11 in response to the image distribution request 50, a first screen 110 shown in FIG. 13 as an example is displayed on the display 17 under the control of the display control unit 45. The first screen 110 displays the patient's name, the imaging date and time, whether or not contrast was used, and the tomographic image 15. Below the tomographic image 15, a group of operation buttons 111 for returning the displayed tomographic image 15 is arranged. Also, below the first screen 110, an analysis button 112 is arranged.

[0062] When a doctor wants to check whether or not there is a tumor not depicted in the pancreas 16 shown in the tomographic image 15, the doctor selects the analysis button 112 by placing the cursor 113 on the analysis button 112. When the analysis button 112 is selected, an instruction to analyze the tomographic image 15 is input to the instruction receiving unit 40. This activates the first derivation unit 43 and the second derivation unit 44, and the first derivation unit 43 derives indirect finding information 51, and the second derivation unit 44 derives lesion presence possibility information 52.

[0063] After selecting the analysis button 112, the first screen 110 transitions to a second screen 115 shown in Fig. 14 as an example. A marker 116 indicating an area in the pancreas 16 where a tumor is suspected is displayed on the tomographic image 15 of the second screen 115, based on the tumor location information 101 of the lesion presence possibility information 52. The marker 116 is formed by filling in pixels of a portion that is suspected to be a tumor and is registered in the tumor location information 101, for example, in red.

[0064] The second screen 115 has, at its bottom, an analysis result display area 117 based on the indirect finding information 51 and the lesion presence possibility information 52. The analysis result display area 117 has a display frame 118 for the tumor presence probability 100, a display frame 119 for the presence probability 66 of each indirect finding, the area ratio 86 of the shape indirect finding, and the concentration representative value 89 of the property indirect finding, and a display frame 120 for the malignancy degree 92. By displaying this second screen 115 on the display 17, the display control unit 45 presents the indirect finding information 51 and the lesion presence possibility information 52 to the doctor.

[0065] In the display frames 119 and 120, the display control unit 45 applies red shading 121 to the indirect finding information 51 with the top three contribution degrees registered in the contribution degree information 102. By using this shading 121, the display control unit 45 distinguishes the indirect finding information 51 with the top three contribution degrees to the derivation of the lesion presence possibility information 52 from indirect finding information 51 with contribution degrees lower than third place. The indirect finding information 51 with the top three contribution degrees is an example of "indirect finding information with a relatively high contribution degree" according to the technology of the present disclosure. Furthermore, the indirect finding information 51 with a contribution degree lower than third place is an example of "indirect finding information with a relatively low contribution degree" according to the technology of the present disclosure.

[0066] 14 shows an example in which the contributions of the probability of dilation (66), the area ratio of atrophy (86), and the malignancy level (92) are in the top three. The analysis result display area 117 disappears when the close button 122 is selected. This returns the second screen 115 to the first screen 110.

[0067] Next, the operation of the above configuration will be described with reference to the flowcharts shown in Figures 15 and 16. First, when the operating program 30 is started on the doctor terminal 12, the CPU 22 of the doctor terminal 12 functions as an instruction receiving unit 40, an image acquiring unit 41, a RW control unit 42, a first derivation unit 43, a second derivation unit 44, and a display control unit 45, as shown in Figure 3.

[0068] 15, when a search keyword is input by a doctor via the input device 18 and an image delivery instruction is received by the instruction receiving unit 40 (YES in step ST100), the search keyword is output from the instruction receiving unit 40 to the image acquiring unit 41. Then, an image delivery request 50 including the search keyword is transmitted from the image acquiring unit 41 to the PACS server 11 (step ST110). The PACS server 11 searches for a tomographic image 15 corresponding to the image delivery request 50, and delivers the retrieved tomographic image 15 to the image acquiring unit 41. The tomographic image 15 delivered from the PACS server 11 is acquired by the image acquiring unit 41 (step ST120). The tomographic image 15 is output from the image acquiring unit 41 to the RW control unit 42, and is stored in the storage 20 by the RW control unit 42. Also, as shown in FIG. 13, a first screen 110 including the tomographic image 15 is displayed on the display 17 under the control of the display control unit 45 (step ST130).

[0069] 16, when the doctor selects the analysis button 112 on the first screen 110 and the instruction receiving unit 40 receives an instruction to analyze the tomographic image 15 (YES in step ST200), a message to that effect is output to the first derivation unit 43 and the second derivation unit 44. Then, as shown in FIGS. 5 to 10, the first derivation unit 43 derives indirect finding information 51 from the tomographic image 15 using the first derivation model 32 (step ST210). The indirect finding information 51 is output from the first derivation unit 43 to the second derivation unit 44 and the display control unit 45.

[0070] 11, the second derivation unit 44 derives lesion presence possibility information 52 from the indirect finding information 51 using the second derivation model 33 (step ST220). The lesion presence possibility information 52 is output from the second derivation unit 44 to the display control unit 45.

[0071] As shown in FIG. 14, under the control of the display control unit 45, the second screen 115 including the analysis result display area 117 based on the indirect finding information 51 and the lesion presence possibility information 52 is displayed on the display 17 (step ST230).

[0072] As described above, the CPU 22 of the doctor terminal 12 includes the image acquisition unit 41, the first derivation unit 43, and the second derivation unit 44. The image acquisition unit 41 acquires the tomographic image 15 distributed from the PACS server 11 in response to the image distribution request 50. The first derivation unit 43 analyzes the tomographic image 15 to derive indirect finding information 51 related to indirect findings that represent the shape and properties of the tissue surrounding the tumor associated with tumor development. The second derivation unit 44 derives lesion presence possibility information 52 that indicates the possibility of the presence of a tumor based on the indirect finding information 51. Therefore, by using a method that conforms to the doctor's thinking of using indirect findings as clues to discover tumors that are barely visualized, it is possible to contribute to the discovery of barely visualized tumors.

[0073] 11, the indirect finding information 51 includes a presence probability 66, position information 67, an area ratio 86 of the indirect shape finding, a representative concentration value 89 of the indirect characteristic finding, and a tumor malignancy level based on the indirect finding 92. This allows the characteristics of the indirect finding to be more clearly expressed.

[0074] 11, lesion presence possibility information 52 includes tumor presence probability 100 and tumor position information 101. Therefore, the possibility of tumor presence can be accurately expressed.

[0075] 14, the display control unit 45 presents the doctor with the indirect finding information 51 and the lesion presence possibility information 52. This allows the doctor to diagnose pancreatic cancer while referring to not only the tomographic image 15 but also the indirect finding information 51 and the lesion presence possibility information 52. This reduces the burden on the doctor involved in the diagnosis.

[0076] 14, the display control unit 45 uses shading 121 to distinguish indirect finding information 51 that has a relatively high degree of contribution to the derivation of lesion presence possibility information 52 from indirect finding information 51 that has a relatively low degree of contribution, and presents the information to the doctor. This allows the doctor to see at a glance which indirect finding information 51 has a relatively high degree of contribution. The doctor can verify the validity of the lesion presence possibility information 52.

[0077] The lesion is a tumor, and therefore, lesion presence probability information 52 of tumors directly related to pancreatic cancer can be derived.

[0078] Shape indirect findings include atrophy, swelling, stenosis, and dilation, while property indirect findings include fatty substitution and calcification. These indirect findings frequently occur in the tissue surrounding a tumor as the tumor develops. Therefore, by deriving lesion presence possibility information 52 based on indirect finding information 51 of these indirect findings, the reliability of lesion presence possibility information 52 can be improved.

[0079] The medical image is a non-contrast tomographic image 15 of the abdomen showing the pancreas 16, and is used to diagnose pancreatic cancer. Since tumors related to pancreatic cancer are hardly visualized in the non-contrast tomographic image 15, the effect of the technology of the present disclosure, which can contribute to the discovery of tumors that are barely visualized, can be fully exerted. Furthermore, the non-contrast tomographic image 15 places less of a burden on the patient P when captured than a contrast tomographic image. Therefore, if a way can be opened to detect tumors with a high probability using the non-contrast tomographic image 15, pancreatic cancer can be diagnosed without capturing a contrast tomographic image, which places a relatively large burden on the patient P.

[0080] Although a tumor related to pancreatic cancer has been exemplified as the lesion, the lesion is not limited to this. For example, as shown in Figure 17, the lesion may be a cyst related to pancreatic cancer (neoplastic pancreatic cyst).

[0081] In FIG. 17 , the second derivation model 125 outputs lesion presence possibility information 126 based on the indirect finding information 51. The lesion presence possibility information 126 includes a cyst presence probability 127, cyst location information 128, and contribution information 129. The cyst presence probability 127 represents the probability of a cyst being present in the pancreas 16. The cyst location information 128 is information in which the position coordinates of pixels in areas that are likely to be cysts are registered. The contribution information 129 is information in which the contribution to the derivation of the cyst presence probability 127 and the cyst location information 128 is registered for each piece of indirect finding information 51. The cyst presence probability 127 is an example of a "lesion presence probability" according to the technology of the present disclosure. The cyst location information 128 is an example of a "lesion location information" according to the technology of the present disclosure. The lesion presence possibility information 126 can be derived for cysts directly related to pancreatic cancer. It is also possible to derive both the tumor lesion presence possibility information 52 and the cyst lesion presence possibility information 126.

[0082] [Second embodiment] In the second embodiment shown in FIGS. 18 to 27, a tomographic image 15 in which a tumor is visually depicted is also analyzed.

[0083] 18, the CPU of the doctor terminal of the second embodiment functions as a third derivation unit 135 in addition to the processing units 40 to 45 of the first embodiment (in FIG. 18, units other than the first derivation unit 43 and the second derivation unit 44 are omitted) and also as a third derivation unit 135. In addition to the tomographic image 15, the first derived model 32, and the second derived model 33, a second derived model 136 and a third derived model 137 are stored in the storage of the doctor terminal 12 of the second embodiment.

[0084] As in the first embodiment, the tomographic image 15 and the first derived model 32 are read out from storage by the RW control unit 42 and output from the RW control unit 42 to the first derivation unit 43. The second derived models 33 and 136 are read out from storage by the RW control unit 42 and output from the RW control unit 42 to the second derivation unit 44. Furthermore, the tomographic image 15 and the third derived model 137 are read out from storage by the RW control unit 42 and output from the RW control unit 42 to the third derivation unit 135.

[0085] As in the first embodiment, the first derivation unit 43 derives indirect finding information 51 from the tomographic image 15 using the first derivation model 32. The first derivation unit 43 outputs the indirect finding information 51 to the second derivation unit 44. The third derivation unit 135 derives provisional lesion presence possibility information 138 from the tomographic image 15 using the third derivation model 137. The provisional lesion presence possibility information 138 is information indicating the provisional possibility of the presence of a tumor, which represents at least one of the shape and properties of a visibly rendered tumor. The third derivation unit 135 outputs the provisional lesion presence possibility information 138 to the second derivation unit 44 and the display control unit 45. As in the first embodiment, the second derivation unit 44 derives lesion presence possibility information 52 from the indirect finding information 51 using the second derivation model 33. Furthermore, the second derivation unit 44 uses the second derivation model 136 to derive lesion presence possibility information 139 based on the indirect finding information 51 and provisional lesion presence possibility information 138. The second derivation unit 44 outputs the lesion presence possibility information 139 to the display control unit 45. Note that "provisional" means that it is provisional with respect to the lesion presence possibility information 139 that is ultimately derived.

[0086] The third derivation model 137 includes an SS model 140 and a malignancy derivation model 141. The SS model 140 is constructed by a convolutional neural network, similar to the SS model 60 of the first embodiment. The object to be extracted by the SS model 140 is a portion indicating a tumor.

[0087] As an example, as shown in FIG. 19 , the third derivation unit 135 inputs a tomographic image 15 as an input image to the SS model 140. Then, the SS model 140 outputs an output image 145 and a provisional tumor presence probability 146. The third derivation unit 135 generates provisional tumor location information 147 from the output image 145. The third derivation unit 135 outputs the provisional tumor presence probability 146 and the provisional tumor location information 147 as provisional lesion presence possibility information 138 to the second derivation unit 44, etc. The provisional tumor presence probability 146 is an example of a "provisional lesion presence probability" according to the technology of the present disclosure. The provisional tumor location information 147 is an example of "provisional lesion location information" according to the technology of the present disclosure.

[0088] Output image 145 is an image in which the portion indicating a tumor is labeled as a class. Provisional tumor presence probability 146 is a provisional probability of the presence of a tumor. Provisional tumor position information 147 is information representing the provisional position of a tumor. More specifically, provisional tumor position information 147 is information in which the position coordinates of pixels in the portion indicating a labeled tumor in output image 145 are registered.

[0089] Similar to the SS model 60 of the first embodiment, the SS model 140 generates a probability distribution map indicating the probability of tumor presence, and outputs a tentative tumor presence probability 146 based on the probability distribution map.

[0090] As an example, as shown in FIG. 20 , third derivation unit 135 includes shape feature derivation unit 150. Shape feature derivation unit 150 receives output image 65 output by SS model 60 and output image 145 output by SS model 140. Shape feature derivation unit 150 calculates provisional tumor area ratio 151, which is the area ratio of the tumor-indicating portion to the entire pancreas 16, based on output images 65 and 145. More specifically, shape feature derivation unit 150 calculates provisional tumor area ratio 151 by dividing the number of pixels labeled as the tumor-indicating portion in output image 145 by the number of pixels labeled as pancreas 16 in output image 65. Shape feature derivation unit 150 outputs provisional tumor area ratio 151 to second derivation unit 44, etc., as provisional lesion presence possibility information 138. Provisional tumor area ratio 151 is an example of a "provisional shape feature amount of a lesion" according to the technology of the present disclosure.

[0091] As an example, as shown in FIG. 21 , the third derivation unit 135 includes a characteristic feature derivation unit 155. The characteristic feature derivation unit 155 receives the tomographic image 15 and provisional tumor location information 147. Based on the tomographic image 15 and the provisional tumor location information 147, the characteristic feature derivation unit 155 derives a provisional tumor concentration representative value 156, which is a representative value of pixel values ​​of a portion indicating a tumor. More specifically, the characteristic feature derivation unit 155 identifies pixels of a portion indicating a tumor in the tomographic image 15 from the provisional tumor location information 147. Then, the characteristic feature derivation unit 155 derives, as the provisional tumor concentration representative value 156, for example, the maximum value, minimum value, mode, or average value of the pixel values ​​of the identified pixels. The characteristic feature derivation unit 155 outputs the provisional tumor concentration representative value 156 to the second derivation unit 44, etc., as provisional lesion presence possibility information 138. The provisional tumor concentration representative value 156 is an example of a "provisional characteristic feature of a lesion" according to the technology of the present disclosure.

[0092] 22, the third derivation unit 135 has a cropped image generation unit 160. The cropped image generation unit 160 receives the tomographic image 15 and provisional tumor position information 147. The cropped image generation unit 160 generates a cropped image 161 by cropping a portion representing the tumor from the tomographic image 15 in a box shape, based on the provisional tumor position information 147.

[0093] The third derivation unit 135 inputs the cropped image 161 to the malignancy derivation model 141. Similar to the malignancy derivation model 61, the malignancy derivation model 141 is configured by a convolutional neural network. The malignancy derivation model 141 derives a provisional malignancy 162 of the tumor based on the cropped image 161. Similar to the malignancy 92 in the first embodiment, the provisional malignancy 162 is expressed on a scale of 1 to 10, for example, with a higher value indicating a higher possibility that the tumor is malignant. The third derivation unit 135 outputs the provisional malignancy 162 to the second derivation unit 44, etc., as provisional lesion presence possibility information 138.

[0094] 23 , the second derivation unit 44 inputs indirect finding information 51, which includes a presence probability 66, position information 67, an area ratio 86, a concentration representative value 89, and a malignancy level 92, to the second derivation model 136. The second derivation unit 44 also inputs provisional lesion presence possibility information 138, which includes a provisional tumor presence probability 146, a provisional tumor position information 147, a provisional tumor area ratio 151, a provisional tumor concentration representative value 156, and a provisional malignancy level 162, to the second derivation model 136. The second derivation model 136 is constructed, for example, by linear discriminant analysis or boosting, similar to the second derivation model 33 of the first embodiment. The second derivation model 136 outputs lesion presence possibility information 139 based on the indirect finding information 51 and the provisional lesion presence possibility information 138.

[0095] Similar to the lesion presence possibility information 52 of the first embodiment, the lesion presence possibility information 139 includes a tumor presence probability 170, tumor location information 171, and contribution information 172. As shown in Fig. 24 as an example, the contribution information 172 is information in which the contribution to the derivation of the tumor presence probability 170 and the tumor location information 171 is registered for each indirect finding information 51 (presence probability 66, location information 67, area proportion 86, concentration representative value 89, and malignancy level 92) and for each provisional lesion presence possibility information 138 (provisional tumor presence probability 146, provisional tumor location information 147, provisional tumor area proportion 151, provisional tumor concentration representative value 156, and provisional malignancy level 162).

[0096] 25 as an example, a first screen 180 of the second embodiment is provided with a first analysis button 181 and a second analysis button 182. When the doctor visually determines that a tumor is not depicted in the tomographic image 15, the doctor selects the first analysis button 181. On the other hand, when the doctor visually determines that a tumor is depicted in the tomographic image 15, the doctor selects the second analysis button 182.

[0097] When the first analysis button 181 is selected, a first analysis instruction for the tomographic image 15 is input to the instruction receiving unit 40. As a result, similarly to when the analysis button 112 on the first screen 110 of the first embodiment is selected, the first derivation unit 43 and the second derivation unit 44 are activated, the first derivation unit 43 derives indirect finding information 51, and the second derivation unit 44 derives lesion presence possibility information 52. In this case, the third derivation unit 135 is not activated.

[0098] On the other hand, when the second analysis button 182 is selected, a second analysis instruction for the tomographic image 15 is input to the instruction receiving unit 40. This activates the first derivation unit 43, the second derivation unit 44, and the third derivation unit 135. Then, the first derivation unit 43 derives indirect finding information 51, the third derivation unit 135 derives provisional lesion presence possibility information 138, and the second derivation unit 44 derives lesion presence possibility information 139.

[0099] After the second analysis button 182 is selected, the first screen 180 transitions to a second screen 185 shown in Fig. 26 as an example. A marker 186 indicating a region in the pancreas 16 where a tumor is suspected is displayed on the tomographic image 15 on the second screen 185, based on the tumor position information 171 of the lesion presence possibility information 139. The marker 186 is formed by filling in, for example, red, pixels in a portion that is suspected to be a tumor and is registered in the tumor position information 171.

[0100] The second screen 185 has an analysis result display area 187 at its bottom that is based on indirect finding information 51, provisional lesion presence possibility information 138, and lesion presence possibility information 139. Similar to the analysis result display area 117 of the second screen 115 of the first embodiment, the analysis result display area 187 is provided with a display frame 118 for a tumor presence probability 170, a display frame 119 for the presence probability 66 of each indirect finding, the area proportion 86 of the shape indirect finding, and the concentration representative value 89 of the property indirect finding, and a display frame 120 for the malignancy level 92. The analysis result display area 187 also has display frames 188 for a provisional tumor presence probability 146, a provisional tumor area proportion 151, and a provisional tumor concentration representative value 156, and a display frame 189 for the provisional malignancy level 162. By displaying this second screen 185 on the display 17, the display control unit 45 presents the indirect finding information 51, the provisional lesion presence possibility information 138, and the lesion presence possibility information 139 to the doctor.

[0101] In the display frames 119, 120, 188, and 189, the display control unit 45 applies red shading 190 to the indirect finding information 51 and provisional lesion presence possibility information 138 with the top three degrees of contribution registered in the contribution degree information 172. By using this shading 190, the display control unit 45 distinguishes the indirect finding information 51 and provisional lesion presence possibility information 138 with the top three degrees of contribution to the derivation of the lesion presence possibility information 139 from the indirect finding information 51 and provisional lesion presence possibility information 138 with degrees of contribution lower than third. The indirect finding information 51 and provisional lesion presence possibility information 138 with the top three degrees of contribution are an example of "indirect finding information and provisional lesion presence possibility information with a relatively high degree of contribution" according to the technology of the present disclosure. Furthermore, the indirect finding information 51 and tentative lesion presence possibility information 138 with contribution levels lower than third place are examples of "indirect finding information and tentative lesion presence possibility information with relatively low contribution levels" according to the technology of the present disclosure.

[0102] 26 shows an example in which the contributions of the extension existence probability 66, the provisional tumor area ratio 151, and the provisional malignancy level 162 are in the top three. The analysis result display area 187 disappears when the close button 191 is selected. This returns the second screen 185 to the first screen 180.

[0103] Next, the operation of the above configuration will be described with reference to the flowchart shown in Fig. 27 as an example. Steps ST310 to ST330 when the doctor selects the first analysis button 181 on the first screen 180 and the instruction receiving unit 40 receives a first analysis instruction for the tomographic image 15 (YES in step ST300) are the same as steps ST210 to ST230 shown in Fig. 16 when the analysis button 112 is selected in the first embodiment, and therefore will not be described again.

[0104] When the doctor selects the second analysis button 182 on the first screen 180 and the instruction receiving unit 40 receives an instruction to perform a second analysis of the tomographic image 15 (YES in step ST340), a message to that effect is output to the first derivation unit 43, the second derivation unit 44, and the third derivation unit 135. Then, the first derivation unit 43 derives indirect finding information 51 from the tomographic image 15 using the first derivation model 32 (step ST350). The indirect finding information 51 is output from the first derivation unit 43 to the second derivation unit 44 and the display control unit 45.

[0105] 19 to 22, the third derivation unit 135 uses the third derivation model 137 to derive provisional lesion presence possibility information 138 from the tomographic image 15 (step ST360). The provisional lesion presence possibility information 138 is output from the third derivation unit 135 to the second derivation unit 44.

[0106] 23 , the second derivation unit 44 derives lesion presence possibility information 139 from the indirect finding information 51 and the provisional lesion presence possibility information 138 using the second derivation model 136 (step ST370). The lesion presence possibility information 139 is output from the second derivation unit 44 to the display control unit 45.

[0107] As shown in FIG. 26, under the control of the display control unit 45, a second screen 185 including an analysis result display area 187 based on the indirect finding information 51, the provisional lesion presence possibility information 138, and the lesion presence possibility information 139 is displayed on the display 17 (step ST380).

[0108] As described above, in the second embodiment, the third derivation unit 135 analyzes the tomographic image 15 to derive provisional lesion presence possibility information 138 indicating the provisional possibility of the presence of a tumor. The second derivation unit 44 derives lesion presence possibility information 139 based on the indirect finding information 51 and the provisional lesion presence possibility information 138. Therefore, when a tumor is visibly depicted in the tomographic image 15, the lesion presence possibility information 139 can be derived by taking into account not only the indirect finding information 51 but also the provisional lesion presence possibility information 138. This allows the reliability of the lesion presence possibility information 139 to be improved.

[0109] 23, provisional lesion presence possibility information 138 includes provisional tumor presence probability 146, provisional tumor location information 147, provisional tumor area ratio 151, provisional tumor concentration representative value 156, and provisional tumor malignancy 162. This allows the characteristics of the tumor to be more clearly represented.

[0110] 26, the display control unit 45 presents the doctor with the indirect finding information 51 and the provisional lesion presence possibility information 138. This allows the doctor to diagnose pancreatic cancer while referring to not only the tomographic image 15 but also the indirect finding information 51 and the provisional lesion presence possibility information 138. This reduces the burden on the doctor involved in the diagnosis.

[0111] 26 , the display control unit 45 uses shading 190 to distinguish indirect finding information 51 and provisional lesion presence possibility information 138 that have a relatively high degree of contribution to the derivation of lesion presence possibility information 139 from indirect finding information 51 and provisional lesion presence possibility information 138 that have a relatively low degree of contribution, and presents them to the physician. This allows the physician to see at a glance which indirect finding information 51 and provisional lesion presence possibility information 138 have a relatively high degree of contribution. The physician can verify the validity of the lesion presence possibility information 139.

[0112] Instead of or in addition to the provisional lesion presence possibility information 138 of a tumor, provisional lesion presence possibility information of a cyst may be derived, and the lesion presence possibility information of a cyst may be derived based on the provisional lesion presence possibility information of a cyst.

[0113] The indirect findings may represent either one of the shape and properties of the tissue surrounding the lesion associated with the occurrence of the lesion. The indirect finding information 51 may be at least one of the presence probability 66, location information 67, area ratio 86, concentration representative value 89, and malignancy level 92. The provisional lesion presence possibility information 138 may be at least one of the provisional tumor presence probability 146, provisional tumor location information 147, provisional tumor area ratio 151, provisional tumor concentration representative value 156, and provisional malignancy level 162.

[0114] Lesion presence possibility information 52 (139) may be at least one of tumor presence probability 100 (170) and tumor location information 101 (171). Similarly, lesion presence possibility information 126 may be at least one of cyst presence probability 127 and cyst location information 128.

[0115] The indirect shape finding may be at least one of atrophy, swelling, stenosis, and dilation, and the indirect characteristic finding may be at least one of fatty substitution and calcification.

[0116] The part of the tomographic image 15 that the SS model 60 prioritized when outputting the output image 65 may be displayed separately from the others. Similarly, the part of the tomographic image 15 that the SS model 140 prioritized when outputting the output image 145 may be displayed separately from the others.

[0117] Although the existence probability 66 is derived from the probability distribution map 78, this is not limiting. A machine learning model that outputs the existence probability 66 when the tomographic image 15 is input may also be used.

[0118] The method of deriving lesion presence possibility information 52 from indirect finding information 51 is not limited to the method using second derivation model 33. A rule base in which lesion presence possibility information 52 corresponding to various patterns of indirect finding information 51 is registered may also be used. The same applies to the case where lesion presence possibility information 139 is derive from indirect finding information 51 and provisional lesion presence possibility information 138.

[0119] The shape feature of the indirect findings is not limited to the illustrated area ratio 86. Instead of or in addition to the area ratio 86, the ratio of atrophy or swelling of the head, body, and tail of the pancreas, the diameter of the pancreatic duct 16D where stenosis or dilation is observed, the area of ​​a portion showing fatty substitution, the area of ​​a portion showing calcification, etc. may be used. The characteristic feature of the indirect findings is also not limited to the illustrated concentration representative value 89. Instead of or in addition to the concentration representative value 89, the variance of the distribution of pixel values ​​of pixels in a portion showing fatty substitution, the variance of the distribution of pixel values ​​of pixels in a portion showing calcification, the granularity of a portion showing calcification, etc. may be used. The feature map 72 and / or the final feature map 77 of the SS model 60 may be used as the shape feature of the indirect findings and the characteristic feature of the indirect findings.

[0120] The provisional shape feature of the lesion is not limited to the exemplary provisional tumor area ratio 151. Instead of, or in addition to, the size, circularity, sharpness, etc. of the portion indicating the tumor may be used. The provisional attribute feature of the lesion is also not limited to the exemplary provisional tumor concentration representative value 156. Instead of, or in addition to, the provisional tumor concentration representative value 156, the variance of the pixel value distribution of the pixel in the portion indicating the tumor, the granularity of the portion indicating the tumor, etc. may be used. The feature map and / or the final feature map of the SS model 140 may be used as the provisional shape feature of the lesion and the provisional attribute feature of the lesion.

[0121] Note that the method of displaying indirect finding information 51 with a relatively high contribution to the derivation of the lesion presence possibility information 52 in a manner that distinguishes it from indirect finding information 51 with a relatively low contribution is not limited to the exemplified shading 121. The color of the numerical values ​​of indirect finding information 51 with a relatively high contribution may be different from the color of the numerical values ​​of indirect finding information 51 with a relatively low contribution. For example, the color of the former may be red and the color of the latter may be black. Alternatively, the numerical values ​​of indirect finding information 51 with a relatively high contribution may be displayed in bold. Alternatively, only the indirect finding information 51 with a relatively high contribution may be displayed. The same method can be used to display indirect finding information 51 and tentative lesion presence possibility information 138 with a relatively high contribution to the derivation of the lesion presence possibility information 139 in a manner that distinguishes it from indirect finding information 51 and tentative lesion presence possibility information 138 with a relatively low contribution.

[0122] The indirect finding information 51 , the provisional lesion presence possibility information 138 , and the lesion presence possibility information 52 , 126 , and 139 may be stored in the storage 20 .

[0123] A machine learning model that detects a box-shaped region surrounding a portion showing an indirect finding may be used instead of the SS model 60. Similarly, a machine learning model that detects a box-shaped region surrounding a portion showing a tumor may be used instead of the SS model 140.

[0124] The medical images are not limited to non-contrast tomographic images of the abdomen showing the pancreas. They may also be contrast-enhanced tomographic images of the abdomen showing the pancreas. They may also be tomographic images of the head or chest. The disease that the doctor is attempting to diagnose is not limited to pancreatic cancer, as exemplified above. It may also be pancreatitis. It may also be esophageal cancer, stomach cancer, gastritis, lung cancer, pulmonary edema, liver cancer, hepatitis, colon cancer, etc.

[0125] The medical image is not limited to the tomographic image 15 taken by the CT device 10. For example, it may be a tomographic image taken by an MRI (Magnetic Resonance Imaging) device. Furthermore, the medical image is not limited to a three-dimensional image such as a tomographic image. For example, it may be a two-dimensional image such as a simple radiological image. It may also be a PET (Positron Emission Tomography) image, a SPECT (Single Photon Emission Computed Tomography) image, an endoscopic image, an ultrasound image, or the like.

[0126] The manner in which the lesion presence possibility information 52, etc. is presented to the doctor is not limited to the manner in which the exemplary second screen 115, etc. is displayed on the display 17. The lesion presence possibility information 52, etc. may be printed on a paper medium, or the lesion presence possibility information 52, etc. may be attached to an email and sent to the doctor terminal 12.

[0127] The hardware configuration of the computer constituting the medical image processing device of the present disclosure can be modified in various ways. For example, the medical image processing device can be configured with multiple computers separated as hardware to improve processing power and reliability. For example, the functions of the instruction receiving unit 40, image acquisition unit 41, RW control unit 42, and display control unit 45, and the functions of the first derivation unit 43 and second derivation unit 44 are distributed and performed by two computers. In this case, the medical image processing device is configured with two computers.

[0128] Furthermore, the functions of the medical image processing device may be performed by the PACS server 11 or another server, rather than the doctor terminal 12. In this case, the PACS server 11 or another server distributes various screens, such as the first screen 110, to the doctor terminal 12 in the form of screen data for web distribution created using a markup language such as XML (Extensible Markup Language). The doctor terminal 12 reproduces various screens to be displayed on the web browser based on the screen data, and displays them on the display 17. Note that other data description languages, such as JSON (Javascript (registered trademark) Object Notation), may be used instead of XML.

[0129] The PACS server 11 or another server receives a request to analyze the tomographic image 15 from the doctor terminal 12. The PACS server 11 or another server derives indirect finding information 51 from the tomographic image 15, and derives lesion presence possibility information 52 from the indirect finding information 51. The PACS server 11 or another server generates screen data for the second screen 115 and distributes it to the doctor terminal 12.

[0130] In this way, the hardware configuration of the computer of the medical image processing apparatus can be changed as appropriate depending on the required performance such as processing power, safety, reliability, etc. Furthermore, not only the hardware but also application programs such as the operating program 30 can be duplicated or stored in multiple storages in order to ensure safety and reliability.

[0131] In each of the above embodiments, the following various processors may be used as the hardware configuration of processing units that perform various processes, such as the instruction receiving unit 40, the image acquiring unit 41, the RW control unit 42, the first derivation unit 43, the second derivation unit 44, the display control unit 45, the shape feature derivation units 85 and 150, the property feature derivation units 88 and 155, the cropped image generation units 90 and 160, and the third derivation unit 135. As described above, the various processors include the CPU 22, which is a general-purpose processor that executes software (operation program 30) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD) whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0132] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0133] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by client and server computers. Second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by System on Chip (SoC). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0134] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0135] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is needless to say that it is not limited to the above-described embodiments, and various configurations can be adopted as long as they do not deviate from the gist of the present disclosure.

[0136] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0137] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0138] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0139] 2. Medical System 10 CT device 11 PACS Server 12 Doctor's terminal 13 LAN 15 Tomographic images 16 Pancreas 16D Pancreatic duct 17. Display 18 Input Devices 20. Storage 21 Memory 22 CPU 23 Communications Department 24 Bus Line 30 Operating Program 32 First Derived Model 33, 125, 136 Second derived model 40 Instruction Reception Department 41 Image acquisition unit 42 Read / write control unit (RW control unit) 43 First derivation part 44 Second derivation part 45 Display control unit 50 Image delivery request 51 Indirect finding information 52, 126, 139 Possible lesion information 60, 140 Semantic Segmentation Model (SS Model) 61, 141 Malignancy derivation model 65, 145 output images 66 Probability of Existence 67 Location information 70 Compression section 71 Output section 72 Feature Map 73 Decoder section 74 Probability distribution map generation unit 75 Labeling section 76 Representative value selection section 77 Final feature map 78 Probability Distribution Map 80 elements 85, 150 Shape feature extraction part 86 Area ratio 88, 155 Feature extraction section 89 Representative concentration 90, 160 Cutout image generation section 91A~91F, 161 cropped images 92 malignant grade 100, 170 Probability of tumor presence 101, 171 Tumor location information 102, 129, 172 Contribution Information 110, 180 1st screen 111 Operation buttons 112 Analysis button 113 Cursor 115, 185 2nd screen 116, 186 markers 117, 187 Analysis result display area 118~120, 188, 189 display frame 121, 190 shaded 122, 191 Close button 127 Probability of Cyst Presence 128 Cyst Location Information 135 Third Derivation 137 Third Derived Model 138 Provisional Lesion Presence Information 146 Hypothetical tumor probability 147 Provisional tumor location information 151 Provisional tumor area ratio 156 Tentative representative tumor concentration 162 Provisional malignancy 181 First analysis button 182 Second analysis button P patient ST100, ST110, ST120, ST130, ST200, ST210, ST220, ST230, ST300, ST310, ST320, ST330, ST340, ST350, ST360, ST370, ST380 Step

Claims

1. a processor; a memory connected to or embedded in the processor; The processor: Acquire medical images; an indirect finding that represents at least one of the shape and properties of tissue surrounding the lesion due to the occurrence of the lesion, by analyzing the medical image; the indirect findings representing the shape characteristics include at least one of atrophy, swelling, stenosis, and dilation; The indirect findings characteristic of the condition include at least one of fatty substitution and calcification. Derive indirect finding information regarding indirect findings; deriving lesion presence possibility information indicating the possibility of the presence of the lesion based on the indirect finding information; Medical imaging equipment.

2. 2. The medical image processing device according to claim 1, wherein the indirect finding information is at least one of a probability of existence of the indirect finding, position information of the indirect finding, shape feature values ​​of the indirect finding, property feature values ​​of the indirect finding, and a malignancy level of the lesion based on the indirect finding.

3. 3. The medical image processing apparatus according to claim 1, wherein the lesion existence possibility information is at least one of the existence probability of the lesion and location information of the lesion.

4. The processor: The medical image processing device according to claim 1 , wherein the lesion presence possibility information is presented.

5. The processor: The medical image processing apparatus according to claim 1 , wherein the indirect finding information is presented.

6. The processor: The medical image processing apparatus according to claim 5 , wherein the indirect finding information that contributes relatively highly to the derivation of the lesion presence possibility information is presented separately from the indirect finding information that contributes relatively less.

7. The processor: deriving tentative lesion existence possibility information indicating the tentative existence possibility of the lesion, which represents at least one of the shape and properties of the lesion, by analyzing the medical image; The medical image processing apparatus according to claim 1 , wherein the lesion presence possibility information is derived based on the indirect finding information and the provisional lesion presence possibility information.

8. 8. The medical image processing device according to claim 7, wherein the provisional lesion existence possibility information is at least one of a provisional existence probability of the lesion, provisional position information of the lesion, provisional shape feature of the lesion, provisional property feature of the lesion, and provisional malignancy of the lesion.

9. The processor: The medical image processing apparatus according to claim 7 or 8, wherein the indirect finding information and the provisional lesion presence possibility information are presented.

10. The processor:

10. The medical image processing device according to claim 9, wherein the indirect finding information and the provisional lesion presence possibility information that have a relatively high contribution to the derivation of the lesion presence possibility information are presented separately from the indirect finding information and the provisional lesion presence possibility information that have a relatively low contribution.

11. The medical image processing device according to claim 1 , wherein the lesion includes at least one of a tumor and a cyst.

12. The medical image processing device according to claim 1 , wherein the medical image is a non-contrast tomographic image of the abdomen showing the pancreas, and is used for diagnosing pancreatic cancer.

13. acquiring medical images; an indirect finding that represents at least one of the shape and properties of tissue surrounding the lesion due to the occurrence of the lesion, by analyzing the medical image; the indirect findings representing the shape characteristics include at least one of atrophy, swelling, stenosis, and dilation; The indirect findings characteristic of the condition include at least one of fatty substitution and calcification. Deriving indirect finding information regarding the indirect finding; and deriving lesion presence possibility information indicating the possibility of the presence of the lesion based on the indirect finding information; A method of operating a medical imaging device, comprising:

14. acquiring medical images; an indirect finding that represents at least one of the shape and properties of tissue surrounding the lesion due to the occurrence of the lesion, by analyzing the medical image; the indirect findings representing the shape characteristics include at least one of atrophy, swelling, stenosis, and dilation; The indirect findings characteristic of the condition include at least one of fatty substitution and calcification. Deriving indirect finding information regarding the indirect finding; and deriving lesion presence possibility information indicating the possibility of the presence of the lesion based on the indirect finding information; An operating program for a medical image processing device for causing a computer to execute processing including the steps of:

Images