X-ray imaging diagnostic apparatus, X-ray image processing method, and program

Abstract

Liver disease can be detected from chest X-ray images. [Solution] Server 2 comprises a processor 21 and storage 23 that stores an AI model 3 that has been machine-trained to output the progression level of liver disease when a chest X-ray image is input. The processor 21 inputs a target image, which is a chest X-ray image of the subject, into the AI ​​model 3 to obtain the progression level of the subject's liver disease.

Description

Technical Field

[0001] The present disclosure relates to an X-ray image diagnostic apparatus, an X-ray image processing method, and a program, and more particularly, to an image processing technique for a chest X-ray image.

Background Art

[0002] Conventionally, an X-ray image diagnostic system that images the inside of a subject (typically a human body) by irradiating the subject with X-rays is known (see, for example, Japanese Patent Application Laid-Open No. 2010-166948 (Patent Document 1) and Japanese Patent Application Laid-Open No. 2009-302685 (Patent Document 2)).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] Chest X-ray images are one of the most commonly taken medical images worldwide. There is a demand to detect various diseases of a subject from a chest X-ray image. The inventor has found that a liver disease can be detected from a chest X-ray image.

[0005] The present disclosure has been made to solve the above problems, and one of the objects of the present disclosure is to detect a liver disease from a chest X-ray image.

Means for Solving the Problems

[0006] The X-ray imaging diagnostic apparatus according to the first aspect of this disclosure comprises a processor and storage for a trained model that has been subjected to machine learning to output the progression level of liver disease when a chest X-ray image is input. The processor obtains the progression level of liver disease of a subject by inputting a target image, which is a chest X-ray image of the subject, into the trained model.

[0007] The X-ray imaging diagnostic device relating to the second aspect of this disclosure includes a processor that acquires the progression level of a subject's liver disease by inputting the subject's chest X-ray image into a trained model. The trained model is subjected to machine learning so that when a chest X-ray image is input, it outputs the progression level of the liver disease.

[0008] The X-ray image processing method relating to the third aspect of this disclosure is performed by a computer. The X-ray image processing method includes the step of generating an AI (Artificial Intelligence) model that outputs the progression level of liver disease by machine learning using training data, upon inputting a chest X-ray image.

[0009] The X-ray image processing method relating to the fourth aspect of this disclosure is performed by a computer. The X-ray image processing method includes the step of inputting a chest X-ray image of a subject into a trained model to obtain the progression level of the subject's liver disease from the trained model. [Effects of the Invention]

[0010] According to this disclosure, liver disease can be detected from chest X-ray images. [Brief explanation of the drawing]

[0011] [Figure 1] This diagram schematically shows the overall configuration of the X-ray image processing system according to the embodiment. [Figure 2] A typical hardware configuration for a server, shown in the image. [Figure 3] This is a schematic diagram to explain chest X-ray images. [Figure 4]It is a conceptual diagram for explaining liver disease indicators. [Figure 5] It is a functional block diagram showing the functional configuration of the server. [Figure 6] It is a flowchart showing an example of the processing procedure of the learning process. [Figure 7] It is a flowchart showing an example of the processing procedure of the inference process. [Figure 8] It is a diagram for explaining how to prepare the learning dataset in Example 1. [Figure 9] It is a diagram for explaining the content of the learning dataset in Example 1. [Figure 10] It is a diagram summarizing the performance evaluation results of the AI (Artificial Intelligence) model in Example 1. [Figure 11] It is a diagram showing the performance evaluation results of the AI model in Example 1 by gender. [Figure 12] It is a diagram showing the ROC (Receiver Operating Characteristic) curve of the AI model in Example 1. [Figure 13] It is a diagram showing the saliency map in Example 1. [Figure 14] It is a diagram for explaining how to prepare the learning dataset in Example 2. [Figure 15] It is a diagram for explaining the content of the learning dataset in Example 2. [Figure 16] It is a diagram summarizing the performance evaluation results of the AI model in Example 2. [Figure 17] It is a diagram showing the performance evaluation results of the AI model in Example 2 by gender. [Figure 18] It is a diagram showing the ROC curve of the AI model in Example 2. [Figure 19] It is a diagram showing the saliency map in Example 2.

Modes for Carrying Out the Invention

[0012] <Explanation of Terms> In this disclosure and its embodiments, “chest X-ray image” means an X-ray image covering the chest of a subject. A chest X-ray image may include areas in which the lungs, trachea, diaphragm, heart, arteries, and veins of the subject are captured. In addition to the above, a chest X-ray image may include areas in which at least a portion of the liver of the subject is captured. A chest X-ray image may include areas in which at least a portion of the lateral aspect of the thoracic cavity of the subject is captured.

[0013] In this disclosure and its embodiments, “liver disease” means a pathological condition related to the liver. “Liver disease” includes structural abnormalities (e.g., hepatic cysts, hepatic hemangiomas, etc.) and functional abnormalities of the liver. “Liver impairment” means functional abnormalities of the liver caused by damage to hepatocytes. “Liver impairment” includes acute liver injury and chronic liver injury, and more specifically may include fatty liver (steatosis), acute and chronic hepatitis, cirrhosis, and liver cancer. The causes of liver disease and liver impairment are not particularly limited and may include hepatitis viruses, alcohol, drugs, autoimmune conditions, metabolic conditions, etc.

[0014] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and their descriptions will not be repeated.

[0015] [Embodiment] <Overall System Configuration> Figure 1 is a schematic diagram showing the overall configuration of an X-ray image processing system according to an embodiment. The X-ray image processing system 100 includes, for example, a plurality of X-ray imaging devices 1 and a server 2. Each of the plurality of X-ray imaging devices 1 and the server 2 are connected to each other via a network NW so that they can communicate with one another. Although three X-ray imaging devices 1 are shown in Figure 1, the number of X-ray imaging devices 1 is not particularly limited. There may be many more X-ray imaging devices 1 (for example, tens to tens of thousands).

[0016] Each of the multiple X-ray imaging devices 1 includes an X-ray source (not shown), an X-ray detector 11, and a controller 12. The X-ray detector 11 is, for example, a flat panel detector (FPD) that detects X-rays irradiated from the X-ray source and transmitted through the chest of the subject (patient). The controller 12 controls the X-ray source and the X-ray detector 11 to acquire a chest X-ray image (digital image) and transmits the acquired chest X-ray image to the server 2. The controller 12 is equipped with local storage, a console, and a display.

[0017] Server 2 performs image diagnosis on the chest X-ray images received from X-ray imaging device 1 and transmits the diagnosis results to X-ray imaging device 1. The destination of the diagnosis results may be a terminal (not shown) used by medical professionals (doctors, radiological technologists, etc.) (personal computer, tablet, etc.). Server 2 corresponds to the "X-ray imaging diagnostic device" as described in this disclosure.

[0018] Figure 2 is a block diagram showing a typical hardware configuration of Server 2. Server 2 may be implemented by, for example, a general-purpose computer or by a computer dedicated to X-ray imaging diagnostics. Server 2 includes, as its main hardware elements, a processor 21, memory 22, storage 23, a network interface 24, a display interface 25, and a peripheral device interface 26. The hardware elements of Server 2 are connected to each other by a bus 27.

[0019] The processor 21 is a computing device such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), or GPU (Graphics Processing Unit). The memory 22 includes ROM (Read Only Memory) and RAM (Random Access Memory). The storage 23 is a rewritable non-volatile memory such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The processor 21 performs various processes by reading information (programs, data, etc.) from the memory 22 and storage 23 and loading it into the memory 22.

[0020] Figure 2 shows an example where Server 2 includes one processor 21, but Server 2 may include multiple processors. The same applies to memory 22 and storage 23. In this specification, "processor" is not limited to a processor in the narrow sense that executes processing using a stored-program method, but may also include hardwired circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays). Therefore, the term "processor" can also be interpreted as processing circuitry in which processing is predefined by computer-readable code and / or hardwired circuits.

[0021] The storage area of ​​storage 23 contains the AI ​​(Artificial Intelligence) model 3, the training dataset 41, the profile data 42, the chest X-ray image 43, the liver disease indicators 44, the training program 45, the inference program 46, and the operating system (OS) 47.

[0022] AI Model 3 is a pre-trained model that has undergone training using training data (supervised machine learning). AI Model 3 is configured to perform inference processing that outputs the progression level of liver disease when a chest X-ray image is input. AI Model 3 will be explained in more detail later in Figure 5.

[0023] The training dataset 41 is a set of training data used in the training process of AI model 3. The training dataset 41 includes multiple chest X-ray images 411 and multiple liver test values ​​412 as ground truth data. The chest X-ray images 411 and liver test values ​​412 are associated with each other if they were obtained from the same patient. Preferably, the timing of the chest X-ray images 411 and the timing of the liver test values ​​412 should be as close as possible, and in Examples 1 and 2 described later, this is within 6 months.

[0024] To give a specific example, the liver examination value 412 in Example 1 is a value indicating the amount of ultrasound attenuation by the liver, and is, for example, the CAP (Controlled Attenuation Parameter) (registered trademark) value [unit: dB / m]. The CAP value is obtained by transient elastography such as Fibroscan. The liver examination value 412 in Example 2 is a value indicating the elasticity of the liver, and is, for example, the liver stiffness (E value) [unit: kPa]. The liver stiffness is also obtained by transient elastography, particularly by VCTE (Vibration-Controlled Transient Elastography) (registered trademark).

[0025] Profile data 42 is data indicating the attributes of the patient from whom the chest X-ray image 411 was taken. In the example described later, profile data 42 includes personal information such as the patient's age and sex. Profile data 42 may also include other personal information such as the patient's height, weight, race, medical history, and family history. Furthermore, profile data 42 may also include information indicating the location where the chest X-ray image 411 was taken and the location where the liver test results 412 were taken (such as the name of the medical institution).

[0026] Figure 3 is a schematic diagram illustrating the chest X-ray image 43. In Examples 1 and 2 described later, the chest X-ray image 43 is taken from the rear to the front of a standing subject. As shown in Figure 3, the chest X-ray image 43 includes the area in which the subject's trachea 51, lungs 52, diaphragm 53, heart 54, arteries 55, veins 56, etc. are photographed. In addition, the chest X-ray image 43 may also include the area in which at least a portion of the liver 57 is photographed. The chest X-ray image 43 may also include the area in which at least a portion of the lateral thoracic cavity 58 (the area outside the lungs 52) is photographed. Hereinafter, to distinguish it from the chest X-ray image 411 included in the training dataset 41, the chest X-ray image 43 may be referred to as the "target image".

[0027] Figure 4 is a conceptual diagram illustrating the liver disease index 44. The liver disease index 44 is an index output by the AI ​​model 3 when a chest X-ray image 43 is input to the AI ​​model 3. In other words, the liver disease index 44 is the result of inference processing performed by the AI ​​model 3 and indicates the progression level of the liver disease in the subject. In Example 1, the liver disease is fatty liver, and in Example 2, it is cirrhosis. The liver disease index 44 can be represented, for example, by an integer from 0 to 10 (or a number to one decimal place). In the example shown in Figure 4, the liver disease index 44 increases as the liver disease progresses. However, the method of determining the liver disease index 44 is not limited to this. The liver disease index 44 may decrease as the liver disease progresses, or it may approach a specific value or range as the liver disease progresses. The liver disease index 44 corresponds to the "progression level" in this disclosure. The "progression level" may be provided in a format other than numerical values, and may be a classification, a linguistic expression, etc.

[0028] Referring again to Figure 2, the liver disease index 44 is stored in the storage area of ​​storage 23 in association with the chest X-ray image 43 used in the inference process. However, server 2 may delete the chest X-ray image 43 and the liver disease index 44 from the storage area of ​​storage 23 after, for example, the transmission of the liver disease index 44 from server 2 to the X-ray imaging device 1.

[0029] The learning program 45 is a program for executing a learning process to apply machine learning to the AI ​​model 3. The inference program 46 is a program for executing an inference process using the AI ​​model 3 (the trained model). The OS 47 enables the operation of the learning program 45 and the inference program 46. The learning program 45 and the inference program 46 correspond to the “program” as described in this disclosure.

[0030] The network interface 24 is configured to send and receive information (such as chest X-ray images 43 and liver disease indicators 44) to and from multiple X-ray imaging devices 1 (Figure 1) via the network NW. The display interface 25 is configured to input and output data between the server 2 and the display 91. The peripheral device interface 26 is configured to input and output data between the server 2 and peripheral devices such as a keyboard 92 and a mouse 93.

[0031] Although not shown in the diagram, the AI ​​model 3 (trained model) may be provided from the server 2 to the X-ray imaging device 1 (controller 12), and the AI ​​model 3 may be stored in the local storage area of ​​the X-ray imaging device 1. In other words, the X-ray imaging device 1 may perform inference processing using the AI ​​model 3. In that case, the X-ray imaging device 1 corresponds to the "X-ray diagnostic imaging device" as described in this disclosure.

[0032] <Server Functional Configuration> Figure 5 is a functional block diagram showing the functional configuration of Server 2. Referring to Figures 2 and 5, each function of Server 2 is realized by Server 2 executing the learning program 45 or the inference program 46 on the OS 47. In addition to the AI ​​model 3, Server 2 includes a learning processing unit 6 and an inference processing unit 7.

[0033] The learning processing unit 6 generates training data from the training dataset 41 during the machine learning learning phase and generates an AI model 3 by performing supervised learning using the training data. The learning processing unit 6 includes an input unit 61, a learning unit 62, and a test unit 63.

[0034] The input unit 61 reads the training dataset 41 stored in the storage 23. The training dataset includes a training set, a tuning set (validation set), and a test set. Each data set is associated with profile data 42. The input unit 61 includes a preprocessing unit 611, a data augmentation unit 612, and a profile acquisition unit 613.

[0035] The preprocessing unit 611 performs preprocessing on each chest X-ray image included in the training dataset 41. In this example, the preprocessing unit 611 reduces the length of the chest X-ray image so that the number of pixels on the length of the length of the image becomes a predetermined number (e.g., 320), while maintaining the aspect ratio of the length of the length of the chest X-ray image. Furthermore, the preprocessing unit 611 adds black pixels to both ends of the length of the length so that the number of pixels on the length of preprocessing unit 611 is equal to the number of pixels on the training dataset 41. The preprocessing unit 611 outputs the preprocessed training dataset (multiple chest X-ray images) to the data augmentation unit 612.

[0036] The data augmentation unit 612 performs data augmentation on the preprocessed training dataset. In this example, TrivialAugment Wide is used. However, other known data augmentation methods (such as AutoAugment, Fast AutoAugment, and RandAugment) may also be used. The data augmentation unit 612 outputs the augmented training dataset to the profile acquisition unit 613.

[0037] The profile acquisition unit 613 reads the profile data 42 (personal information such as the patient's age and gender) stored in the storage 23 and associates the data-enhanced training dataset with the profile data 42 (see Figures 9 and 15). The profile acquisition unit 613 outputs the data-enhanced training dataset associated with the profile data 42 to the training unit 62.

[0038] The learning unit 62 generates a trained model by applying machine learning to the AI ​​model 3 using the training dataset from the input unit 61. The learning unit 62 includes a training unit 621 and a tuning unit 622. As described later, the training dataset is divided into a training set, a tuning set, and a test set (see Figures 8, 9, 14, and 15). The training unit 621 performs pre-training of the AI ​​model 3 using the training set. After pre-training, the tuning unit 622 performs learning to infer liver disease indicators (detect liver disease) using the tuning set.

[0039] AI Model 3 is a deep learning model, more specifically a convolutional neural network (CNN) model. AI Model 3 includes a neural network 31 and parameters 32 used by the neural network 31. Parameters 32 include weighting coefficients, hyperparameters, and decision values ​​used to determine the inference result.

[0040] For example, AI Model 3 could use well-known image recognition models such as AlexNet, EfficientNet, or ResNet. For the CNN's loss function, well-known loss functions such as cross-entropy, Dice loss, or Tversky loss could be used. Furthermore, open-source machine learning libraries (deep learning frameworks) such as TensorFlow, Keras, PyTorch, and Chainer may also be used.

[0041] In the examples described later, either the EfficientNet or ResNet image recognition model was used. Cross-entropy was used as the loss function for the CNN. PyTorch was used as the machine learning library. The model that achieved the smallest loss function value on the tuning set was selected as the optimized AI model 3.

[0042] The test unit 63 performs verification tests to evaluate the performance (effectiveness) of the trained AI model 3. More specifically, the test unit 63 analyzes multiple liver disease indicators output from the AI ​​model 3 when a test set is input to the AI ​​model 3, and outputs the analysis results to the display 91 via the display interface 25 (see Figures 10-12 and 16-18 described later). This allows the developer of the AI ​​model 3 to quantitatively evaluate the performance of the AI ​​model 3 (liver disease detection accuracy). Note that the functions of the test unit 63 may be implemented separately on the development terminal used by the developer.

[0043] In the inference phase of machine learning, when a chest X-ray image 43 (target image) of the subject is input to the inference processing unit 7, it uses a trained AI model 3 to estimate the liver disease indicators 44 of the subject. The inference processing unit 7 includes an input unit 71, an inference unit 72, and an output unit 73.

[0044] The input unit 71 receives a chest X-ray image 43 (target image) of the subject. The input unit 71 may read the target image from the storage 23 or receive the target image directly from the X-ray imaging device 1. The input unit 71 includes a preprocessing unit 711. The preprocessing unit 711 performs the same processing on the target image as the preprocessing unit 611 of the learning processing unit 6 (reduction while maintaining the aspect ratio, addition of black pixels). The preprocessing unit 711 outputs the preprocessed target image to the inference unit 72.

[0045] The inference unit 72 inputs the preprocessed target image to the AI ​​model 3 (a trained model) to obtain (or calculate and derive) liver disease indicators 44 from the AI ​​model 3 in the target image. The inference unit 72 outputs the liver disease indicators 44 to the output unit 73.

[0046] The output unit 73 outputs (or transmits) the liver disease indicators 44 from the inference unit 72 to the X-ray imaging device 1 via the network interface 24. The output unit 73 may also store the liver disease indicators 44 from the inference unit 72 in the storage 23.

[0047] <Processing Flow> Figure 6 is a flowchart showing an example of the processing steps for the learning process. The processes shown in this flowchart are executed when predetermined conditions are met (for example, when an operation is received from the developer of AI model 3). Each step is implemented by software processing by server 2 (processor 21), but may also be implemented by hardware processing by electrical circuits located within server 2. Hereafter, steps will be abbreviated as S. The same applies to the flowchart shown later in Figure 7.

[0048] Referring to Figures 2, 5, and 6, in S11, the server 2 (learning processing unit 6) reads the training dataset 41 from the storage 23.

[0049] In S12, Server 2 extracts liver test values ​​(CAP value, E value, etc.) associated with each chest X-ray image from the training dataset 41.

[0050] In S13, Server 2 uses AI Model 3 to perform image recognition on images included in the training dataset 41 and obtains estimated liver disease indicators from AI Model 3.

[0051] In S14, Server 2 performs machine learning on AI model 3 using the liver test values ​​extracted in S12 and the estimated liver disease indicators obtained in S13. As a result of this machine learning, Server 2 updates the parameters 32 of AI model 3 and stores the updated parameters 32 in storage 23.

[0052] In S15, Server 2 determines whether a predetermined number of machine learning cycles have been completed. If the number of training cycles has not reached the specified number (NO in S15), Server 2 returns to S11 and repeats the series of processes. When the number of training cycles reaches the specified number (YES in S15), Server 2 terminates the process.

[0053] Figure 7 is a flowchart showing an example of the inference processing procedure. In the figure, the left side shows the processing performed by the X-ray imaging device 1, and the right side shows the processing performed by the server 2.

[0054] Referring to Figures 2, 5, and 7, in S21, the X-ray imaging device 1 captures a chest X-ray image 43 (target image) of the subject and transmits the captured target image to the server 2. The server 2 (inference processing unit 7) receives the target image from the X-ray imaging device 1.

[0055] In S22, Server 2 inputs the target image into the trained AI model 3 to estimate the liver disease indicators 44 in the target image.

[0056] In S23, Server 2 transmits the liver disease indicators 44 estimated in S22 to X-ray imaging device 1. X-ray imaging device 1 receives the liver disease indicators 44 and displays them on the display (S24). X-ray imaging device 1 may store the liver disease indicators 44 in local storage. After that, X-ray imaging device 1 and Server 2 terminate processing.

[0057] As described above, according to this embodiment, machine learning of the AI ​​model 3 is performed using a training dataset 41 that includes multiple chest X-ray images 411 and multiple liver test values ​​412. Then, by using the trained AI model 3, liver disease can be detected from the subject's chest X-ray image 43 (target image).

[0058] Established diagnostic methods for liver disease include liver biopsy, ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI). Liver biopsy is invasive and carries a risk of complications. Ultrasound (transient elastography) requires a trained operator, and its accuracy may decrease in patients with a high BMI (Body Mass Index). CT raises concerns about radiation exposure for patients undergoing repeated examinations. MRI may have limitations in terms of medical access and examination costs. In contrast, this embodiment allows for the detection of liver disease using standard chest X-ray images. Therefore, it is possible to realize a widely available, inexpensive, and non-invasive diagnostic method that is independent of the operator's skill, the patient's BMI, and does not raise concerns about additional radiation exposure.

[0059] [Example 1] Example 1 describes X-ray imaging diagnostics for detecting fatty liver.

[0060] <Training dataset> Figure 8 illustrates how the training dataset 41 was prepared in Example 1. First, CAP values ​​were collected from patients who underwent CAP testing at two medical institutions, A (MedCity21) and B (Osaka Metropolitan University Hospital). A correlation between CAP values ​​and liver fat mass is known. In this example, the cutoff value for diagnosing a patient with fatty liver was set to CAP value = 275 [dB / m].

[0061] The interquartile range (IQR) of the CAP value is known to represent the quality of the CAP examination. Factors such as patient obesity or ascites, or operator insufficient skill, can lead to a high IQR. Therefore, CAP values ​​with an IQR higher than 40 [dB / m] were excluded from the collected data. This ensures the reliability of the CAP values. Furthermore, CAP values ​​for which no chest X-ray image was available from the same patient were excluded. The remaining pairs of chest X-ray images and CAP values ​​were adopted as the training dataset 41.

[0062] For medical institution A, the training dataset 41 was divided into three subsets. Specifically, the training dataset was randomly divided into a training set, a tuning set, and a test set (internal test set) in a ratio of 8:1:1. This division was performed at the patient level so that all datasets obtained from the same patient were classified into the same subset. As mentioned above, the training set was used to train AI model 3. The tuning set was used to tune AI model 3 (fine-tuning of hyperparameters). AI model 3 with the smallest loss function value was selected as the best-performing model. The internal dataset from medical institution A was used to evaluate the model's performance on other data (new chest X-ray images) from the same medical institution A.

[0063] On the other hand, for medical institution B, all training datasets were used as test sets (external test sets). The external datasets from medical institution B were used to evaluate the performance of AI model 3 on data from medical institution B, which is different from that of medical institution A, which was used to generate AI model 3 (i.e., performance in a real-world environment).

[0064] Figure 9 is a diagram illustrating the contents of the training dataset 41 in Example 1. The training dataset from medical institution A contains 5,499 chest X-ray images from 3,631 patients with an average age of 56 years (standard deviation of 12 years). Of these, 3,662 images were from male patients and 1,837 images were from female patients.

[0065] The training set included 4,443 chest X-ray images from 2,905 patients with a mean age of 56 years (standard deviation 12 years). 2,983 images were from male patients and 1,460 from female patients. The CAP value was 255.92 ± 56.92 (mean ± standard deviation). The IQR for the CAP value was 25.0 ± 13.0 (mean ± standard deviation).

[0066] The tuning set included 527 chest X-ray images from 363 patients with a mean age of 56 years (standard deviation of 11 years). 335 images were from male patients and 192 from female patients. The CAP value was 252.26 ± 59.26. The IQR value for the CAP value was 26.0 ± 14.0.

[0067] The internal test set included 529 chest X-ray images from 363 patients with a mean age of 56 years (standard deviation 11 years). 344 images were from male patients and 185 from female patients. The CAP value was 251.77 ± 56.67. The IQR for the CAP value was 25.0 ± 13.0.

[0068] The external test set from medical institution B contained 1100 chest X-ray images from 783 patients with a mean age of 58 years (standard deviation of 16 years). 604 images were from male patients and 496 from female patients. The CAP value was 227.43 ± 55.59. The IQR for the CAP value was 27.0 ± 13.0.

[0069] <Model Performance Evaluation> Figure 10 summarizes the performance evaluation results of AI Model 3 in Example 1. Figure 11 shows the performance evaluation results of AI Model 3 in Example 1, separated by gender. Referring to Figures 10 and 11, in the internal test set, the Area Under the Curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were 0.83 (95% confidence interval: 0.79~0.86), 77% (74%~81%), 68% (61%~75%), 82% (77%~85%), 65% (59%~72%), and 84% (80%~87%), respectively. On the other hand, with the external test set, the area under the curve, precision, sensitivity, specificity, positive predictive value, and negative predictive value were 0.82 (0.79-0.85), 76% (73-78%), 75% (69-81%), 76% (73-79%), 41% (36-46%), and 93% (91-95%), respectively.

[0070] Figure 12 shows the Receiver Operating Characteristic (ROC) curve for AI Model 3 in Example 1. The vertical axis represents the true positive rate (sensitivity), and the horizontal axis represents the false positive rate (1 - specificity). The same applies to Figure 18, which will be described later. In the figure, the ROC curve using the internal test set is shown at the top, and the ROC curve using the external test set is shown at the bottom.

[0071] As shown in Figures 10 and 11, the accuracy, sensitivity, and specificity of AI Model 3 were consistently good, regardless of whether an internal or external test set was used, or regardless of the patient's gender. Furthermore, Figure 12 shows that AI Model 3 can accurately classify and distinguish between patients with fatty liver and those without. Therefore, it was demonstrated that AI Model 3 has good performance in diagnosing fatty liver from chest X-ray images. This also provides insight that chest X-ray images contain valuable information that can be used for detecting fatty liver using deep learning algorithms.

[0072] <Significance Assessment> Next, we will explain the results of our investigation into which features on chest X-ray images AI Model 3 relies on when detecting fatty liver. Specifically, we created a saliency map visualizing the region focused on by AI Model 3 using the Grad-CAM (Gradient-weighted Class Activation Mapping) method.

[0073] Figure 13 shows the sampling map in Example 1. In Figure 13, the sampling map is superimposed on the chest X-ray images of two patients obtained from medical institution B. The numerical values ​​on the horizontal and vertical axes indicate the pixel positions. The same applies to Figure 19, which will be described later.

[0074] As is clear from Figure 13, the sampling map showed high sampling in the lower half of the chest X-ray image (see arrow AR1). This region corresponds to the patient's liver and diaphragm, among other things. While it is not possible to conclude which features AI Model 3 relies on from the sampling map alone, it strongly suggests that AI Model 3 may be identifying the liver itself. Given that fatty liver impairs blood circulation, it is also possible that AI Model 3 is identifying not only the liver but also the surrounding tissues.

[0075] In addition, the sampling map showed a certain degree of sampling on the lateral side of the patient's thoracic cavity (see arrow AR2). This suggests that AI Model 3 may be recognizing the patient's body type (size). Since obesity is considered a major cause of fatty liver, this result is also considered reasonable.

[0076] [Example 2] Example 2 describes X-ray imaging diagnostics for detecting liver cirrhosis.

[0077] <Training dataset> Figure 14 illustrates how the training dataset 41 was prepared in Example 2. For liver cirrhosis, liver stiffness (E-values) were collected from patients who underwent transient elastography (particularly VCTE®) at two medical institutions, A and B. From the collected E-values, those with an average IQR higher than 30 [kPa] were excluded, and then E-values ​​for which no chest X-ray images were available from the same patient were further excluded. The remaining pairs of chest X-ray images and E-values ​​were adopted as the training dataset 41.

[0078] For each of healthcare institutions A and B, the training dataset was divided into three subsets at the patient level. At healthcare institution A, the dataset obtained during the first period (April 2014 to May 2022) was randomly divided into a training set and a tuning set in a 9:1 ratio. The dataset obtained during the second period (June 2022 to May 2023) was assigned to the test set. The training dataset obtained from healthcare institution B was similarly divided into three subsets. The third period was November 2013 to June 2021, and the fourth period was July 2021 to June 2022.

[0079] In Example 2, a separate AI model 3 was prepared for each medical institution. Specifically, AI model 3 was generated using the training set and tuning set from medical institution A, and this AI model 3 was evaluated using the test set from medical institution A. In addition, AI model 3 was generated using the training set and tuning set from medical institution B, and this AI model 3 was evaluated using the test set from medical institution B.

[0080] Figure 15 is a diagram illustrating the contents of the training dataset 41 in Example 2. The training set from medical institution A contained 5,975 chest X-ray images from 3,760 patients with an average age of 56 years (standard deviation of 12 years). 3,906 images were from male patients and 2,096 from female patients. The E-value was 4.31 ± 3.49 (mean ± standard deviation). The IQR for the E-value was 13.0 ± 10.0 (mean ± standard deviation).

[0081] The tuning set from medical institution A contained 417 chest X-ray images from 693 patients with a mean age of 56 years (standard deviation of 12 years). 439 images were from male patients and 254 from female patients. The E-value was 4.21 ± 3.11. The IQR for the E-value was 13.0 ± 9.0.

[0082] The test set from medical institution B contained 535 chest X-ray images from 536 patients with a mean age of 57 years (standard deviation of 11 years). 439 images were from male patients and 254 from female patients. The E-value was 4.11 ± 1.63. The IQR for the E-value was 14.0 ± 9.0.

[0083] The training set from medical institution B contained 1154 chest X-ray images from 761 patients with a mean age of 56 years (standard deviation of 16 years). 638 images were from male patients and 516 from female patients. The E-value was 10.21 ± 10.75. The IQR for the E-value was 14.0 ± 10.0.

[0084] The tuning set from medical institution B included 115 chest X-ray images from 84 patients with a mean age of 56 years (standard deviation of 16 years). 62 images were from male patients and 53 from female patients. The E-value was 10.93 ± 11.82. The IQR for the E-value was 15.0 ± 9.5.

[0085] The test set from medical institution B contained 101 chest X-ray images from 94 patients with a mean age of 65 years (standard deviation of 12 years). 52 images were from male patients and 49 from female patients. The E-value was 11.85 ± 9.47. The IQR for the E-value was 13.0 ± 10.0.

[0086] <Model Performance Evaluation> Figure 16 is a summary of the performance evaluation results of AI Model 3 in Example 2. Figure 17 is a diagram showing the performance evaluation results of AI Model 3 in Example 2, separated by gender. Figure 18 is a diagram showing the ROC curve of AI Model 3 in Example 2.

[0087] As shown in Figures 16 and 17, for AI model 3 at medical institution A, the AUC, precision, sensitivity, specificity, positive predictive value, and negative predictive value were 0.83 (95% confidence interval: 0.79~0.86), 77% (74%~81%), 68% (61%~75%), 82% (77%~85%), 65% (59%~72%), and 84% (80%~87%), respectively. On the other hand, for AI model 3 at medical institution B, the area under the curve, precision, sensitivity, specificity, positive predictive value, and negative predictive value were 0.82 (0.79~0.85), 76% (73%~78%), 75% (69%~81%), 76% (73%~79%), 41% (36%~46%), and 93% (91%~95%), respectively.

[0088] Thus, regarding liver cirrhosis, the accuracy, sensitivity, and specificity of AI model 3 were good regardless of which medical institution collected the training dataset 41 used for training. Furthermore, AI model 3 was able to accurately classify and identify the presence or absence of liver cirrhosis. Therefore, it was demonstrated that AI model 3 has good performance in diagnosing liver cirrhosis from chest X-ray images.

[0089] <Significance Assessment> Figure 19 shows the severity map in Example 2. As shown in Figure 19, the severity map for liver cirrhosis showed high severity in the region corresponding to the center of the left and right lungs. This is thought to be because the deterioration of blood and fluid circulation resulting from liver cirrhosis manifests as lung disease (pulmonary congestion, pleural effusion, etc.).

[0090] Example 2 describes an example in which liver cirrhosis is the target of detection using AI Model 3. Generally, liver damage progresses in the order of chronic hepatitis, liver cirrhosis, and liver cancer. AI Model 3 is considered capable of detecting not only liver cirrhosis but also all types of liver damage, including chronic hepatitis and liver cancer.

[0091] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than by the description of the embodiments above, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0092] 100 X-ray image processing system, 1 X-ray imaging device, 11 X-ray detector, 12 controller, 2 server, 21 processor, 22 memory, 23 storage, 24 network interface, 25 display interface, 26 peripheral device interface, 27 bus, 3 AI model, 31 neural network, 32 parameters, 41 training dataset, 411 chest X-ray image, 412 liver test values, 42 profile data, 43 chest X-ray image, 44 liver disease indicators, 45 learning program, 46 inference program, 51 trachea, 52 lungs, 53 diaphragm, 54 heart, 55 artery, 56 vein, 57 liver, 58 outside of lungs (outside of thoracic cage), 6 learning processing unit, 61 input unit, 611 preprocessing unit, 612 data expansion unit, 613 profile acquisition unit, 62 learning unit, 621 training unit, 622 tuning unit, 63 Test unit, 7 inference processing unit, 71 input unit, 711 preprocessing unit, 72 inference unit, 73 output unit, 91 display, 92 keyboard, 93 mouse, NW network.

Claims

1. Processor and It includes storage for a pre-trained model that has been subjected to machine learning to output the progression level of liver disease when a chest X-ray image is input, The X-ray imaging diagnostic device includes a processor that inputs a target image, which is a chest X-ray image of the subject, into the trained model to acquire the progression level of the subject's liver disease.

2. The aforementioned target image includes the region in which the lungs, trachea, diaphragm, heart, arteries, and veins of the subject are captured. The X-ray imaging diagnostic apparatus according to claim 1, wherein the processor acquires the progression level of liver damage in the subject.

3. The aforementioned target image further includes a region in which at least a portion of the liver of the subject is captured, The X-ray imaging diagnostic apparatus according to claim 2, wherein the processor acquires the progression level of fatty liver in the subject.

4. The X-ray imaging apparatus according to claim 2, wherein the target image further includes a region in which at least a portion of the outer part of the thoracic cavity of the subject is captured.

5. The X-ray imaging diagnostic apparatus according to claim 2, wherein the processor acquires the progression level of liver cirrhosis in the subject.

6. The X-ray imaging apparatus according to claim 2, wherein the processor acquires the progression level of hepatitis or liver cancer in the subject.

7. The system includes a processor that acquires the progression level of the subject's liver disease by inputting the subject's chest X-ray image into a trained model. The aforementioned trained model has been subjected to machine learning to output the progression level of liver disease when a chest X-ray image is input to an X-ray imaging device.

8. A computer-based X-ray image processing method, An X-ray image processing method comprising the step of generating an AI (Artificial Intelligence) model that outputs the progression level of liver disease when a chest X-ray image is input, using machine learning with training data.

9. The X-ray image processing method according to claim 8, wherein the training data includes a plurality of chest X-ray images and a plurality of liver test values ​​associated with each other as ground truth data.

10. The X-ray image processing method according to claim 9, wherein each of the plurality of liver test values ​​is the ultrasonic attenuation or elasticity value of the liver.

11. A computer-based X-ray image processing method, An X-ray image processing method comprising the step of inputting a chest X-ray image of a subject into a trained model to obtain the progression level of liver disease of the subject from the trained model.

12. A program that, when executed by the computer, causes the computer to perform the method described in any one of claims 8 to 11.

Images