Method and system for quantifying the severity of lung diseases

Abstract

A system, method, and computer program for quantifying the severity of a lung disease. Exemplary aspects include providing an image file of a chest X-ray from a patient to a first artificial intelligence system to perform preprocessing of the image, wherein the lungs are divided into at least four sections, and providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section and calculate a first RALE score, wherein the second artificial intelligence system generates a density segmentation map for each section and calculates a second density score, a second range score, and a second RALE score, analyzing the first RALE score and the second RALE score, and displaying the results to a user.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 364,562, filed on May 12, 2022, entitled "Automated Quantification of Lung Severity Using Chest X - rays", which is hereby incorporated by reference in its entirety.

[0002] This disclosure generally relates to methods and systems for quantifying the severity of lung diseases in medical images, and more particularly, to methods and systems for quantifying the severity of lung diseases in chest X - rays.

Background Art

[0003] Chest X - rays are commonly used in the evaluation of patients suffering from acute respiratory diseases such as COVID - 19, acute respiratory distress syndrome (ARDS), and pneumonia. Billions of chest X - rays are performed annually, but conventional chest X - ray analysis involves qualitative analysis of features of chest X - rays such as focal, patchy, or diffuse densities.

[0004] Conventional chest X - ray analysis requires manual examination of X - rays by a physician or radiologic technologist, and successful detection of abnormalities may depend on the proficiency and / or experience of the radiologic technologist. Thus, conventional analysis may lead to subjective, non - specific, and inconsistent diagnoses.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The methods and systems of this disclosure enable quantification of the severity of lung diseases. The methods and systems of this disclosure may lead to objective and reproducible methods and systems for efficiently and accurately assessing the severity of lung diseases in chest X - rays.

Means for Solving the Problems

[0006] The present disclosure relates to a method for quantifying the severity of a lung disease, which comprises providing an image file of a chest X-ray from a patient to a first artificial intelligence system to perform preprocessing of the image, wherein the preprocessing includes determining vertices and performing segmentation of the image to define lung boundaries, and the lung is divided into at least four sections, and providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on a database including at least two reference image files of the chest X-ray, the at least two reference image files include at least one annotation assigned by a doctor, and the second artificial intelligence system generates a density segmentation map for each section, calculating a first RALE score from the first density score and the first range score of each section, analyzing the density segmentation map of each section to determine a second density score and a second range score for each section and calculating a second RALE score of the image, analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score, and displaying the result to a user, wherein the result includes the overall RALE score, and including.

[0007] The presently disclosed systems and methods may be embodied as a system, method, or computer program product embodied in any tangible expression medium having computer-usable program code embodied in a medium.

[0008] The summary of the above invention, as well as the following drawings and the modes for carrying out the invention, are all exemplary and do not limit the scope of the claims of the present disclosure. Specific details are described in order to better understand the various features, aspects and advantages of the present invention. However, those skilled in the art will understand that these features, aspects and advantages can be implemented without these details. Also, to avoid unnecessarily obscuring other details of the present invention, known structures, methods and / or processes associated with the ways of implementing various features, aspects and / or advantages are not shown in detail and may not be described.

[0009] The present disclosure may be better understood with reference to the accompanying drawings.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0011] The present disclosure generally describes a method and system for quantifying the severity of a lung disease. The methods and systems of the present disclosure may lead to an objective and reproducible method and system for efficiently and accurately assessing the severity of a lung disease in a chest X-ray.

[0012] The present disclosure provides a method 100 (FIG. 1) for quantifying the severity of a lung disease. The method 100 may provide an image file of a chest X-ray from a patient to a first artificial intelligence system 101. As used herein, an "artificial intelligence system" may include at least one computer vision algorithm and / or at least one deep learning network, and the at least one deep learning network may include at least one neural network having a plurality of layers. The use of "first" and "second" does not exclude one or more, two or more, or three or more artificial intelligence systems. The image file of the chest X-ray may include a data file, an image, and / or patient data, and the data and / or image may be formatted in any file format capable of storing a chest X-ray, including but not limited to DICOM, JPEG, TIGG, GIF, and PNG.

[0013] The first artificial intelligence system may preprocess the image 102. The preprocessing may include lung segmentation, scale adjustment, rotation adjustment, angle adjustment, minimization of unnecessary facilities such as IV lines, chemotherapy ports, and EKG wiring, and determination of accurate vertex positions. The segmentation may include defining the lung boundary, the lung may be divided into at least four sections, and anatomical features or image artifacts other than the lung may be excluded. Although the four segmented sections have been described above, more other segmentations than four sections are possible and within the scope of the present disclosure. As used herein, the vertex refers to the center point of the entire lung in a chest X-ray. Once the vertex is determined, method 100 may divide the lung into at least four sections.

[0014] The first artificial intelligence system may be trained using pre-training data, and the pre-training data may include a collection of chest X-rays such as MMIC-CXR. The first artificial intelligence system may be trained based on parameters used to characterize the image files of the present disclosure. The parameters include, but are not limited to, findings or absence of findings, cancer and non-cancer, full breath taken by the patient or non-full breath taken by the patient, and diseased or healthy lungs. As used herein, a finding refers to a visible area that a doctor considers abnormal or not healthy. The training of the first artificial intelligence system may train method 100 to determine weights and teach the appearance of chest X-rays.

[0015] As generally understood in the art, weights are parameters in a deep learning network that may transform input data within the network. The deep learning network may include a series of nodes. Each node may include a set of inputs, weights, and bias values. When an input enters a node, it may be multiplied by the weight value, and the output may be observed or passed to the next node.

[0016] The first artificial intelligence system may include a section module having at least one computer vision algorithm and / or at least one deep learning network to determine vertices and divide the lung into at least four sections (FIG. 3A). The first artificial intelligence system further includes a lung segmentation module having at least one computer vision algorithm and / or at least one deep learning network to segment the lung (FIG. 3A). The section module and the lung segmentation module may operate simultaneously to generate a segmented chest X-ray divided into four sections or quadrants (FIG. 3A).

[0017] Conventional methods and systems for quantifying the severity of lung diseases require chest X-ray images taken in a specific posture, angle, or orientation. Thus, conventional methods may not be able to interpret most chest X-rays with different postures, angles, or orientations. Also, a patient's chest X-ray may show variations compared to a healthy person's chest X-ray. The methods and systems of the present disclosure may consider chest X-rays with different postures, angles, scales, orientations, and patient health levels.

[0018] Conventional methods and systems may analyze the entire chest X-ray or segment the image into two sections. However, these methods and systems cannot accurately evaluate chest X-rays with a low balance of disease presentation in the lung. The methods and systems of the present disclosure may reduce the risk of unbalanced disease presentation by segmenting and dividing the lung into four sections.

[0019] Method 100 may provide each section of the lungs to a second artificial intelligence system 103. The second artificial intelligence system may generate a first density score and a first extent score for each section 104. As used herein, the term "extent" may be read as "consolidation". "Extent" may refer to the degree of alveolar opacity present in a chest X-ray, or may refer to the amount of the most opaque region. As used herein, "density" refers to the density of alveolar opacity present in a chest X-ray. And the second artificial intelligence system may calculate a first RALE score based on the first density score and the first extent score for each section 106.

[0020] As used herein, the term "RALE score" refers to the radiographic assessment of a pulmonary edema score used to evaluate the degree and density of alveolar opacity present in a chest X-ray. Although the RALE score has been described above, other non-invasive measurements for quantifying the severity of lung disease are possible and within the scope of the present disclosure. The RALE score may range from 0 (low severity) to 48 (high severity).

[0021] The second artificial intelligence system may include at least one deep learning network and / or at least one computer vision algorithm to determine the density and extent of each section (Figure 2). The second artificial intelligence system may include at least one individual deep learning network module having at least one deep learning network and / or at least one computer vision algorithm for each section of the lung (Figure 3B) to determine the density and extent of each section. The individual deep learning network module for each section may include a RALE deep learning network. Each RALE deep learning network module may be trained to directly calculate a first density score and a first extent score for the section. At the same time, at least one individual deep learning network module may generate a density segmentation map for each section 105, determine the position of the most dense points in each section, and provide a high-effect visualization of the explainability of RALE deep learning.

[0022] Each section may have a first extent score and a first density score. If the lung is segmented into four sections, the second artificial intelligence system may generate eight components of a first RALE score including four first density scores and four first extent scores.

[0023] In the method of determining the conventional RALE score, less than eight components of the RALE score may be reported, such as only reporting the overall RALE score for a chest X-ray, resulting in inconsistent and inaccurate results. Thus, the conventional method lacks the ability to use the RALE score components for training or understanding how the RALE score is calculated. The methods and systems of the present disclosure may provide more consistent and accurate results by dividing the RALE score into eight components and reporting the components to a database for training medical practitioners and artificial intelligence systems.

[0024] The second artificial intelligence system may be trained based on a database that includes at least two reference image files of chest X-rays. The reference image files include chest X-rays having at least one annotation applied by a physician or radiologic technologist using an annotator, which may include any system or computer program product, and the physician or radiologic technologist may assign a RALE score to the chest X-ray (Figure 4). The database may include at least two reference image files of chest X-rays, for example, at least 2, 100, 1000, 2000, 4000, 6000, 8000, 10000, 50000, and at least 100000 reference image files of chest X-rays. The reference image files may include chest X-rays annotated by a physician or radiologic technologist. The at least one annotation may include a first density score and a first range score for each section or quadrant of the lungs, the presence of atelectasis, the presence of a tube, the image quality, and / or the visibility level. An overall RALE score of the reference image may be calculated using the first density score and the first range score for each section of the reference image. The second artificial intelligence system may be trained using the overall RALE score and / or the first density score and the first range score for each section. The second artificial intelligence system may be trained based on a combination of sections of the lungs having at least one range score and at least one density score. Thus, the second artificial intelligence system may be trained based on at least one density score and at least one range score annotated by a physician or radiologic technologist. The annotated reference image files may include metadata to evaluate the statistics of the dataset and avoid construction errors of the dataset.

[0025] The second artificial intelligence system may be trained using a randomized set that includes at least 90% of the data in the database to randomize the demographics and sources of the image files.

[0026] While generating the first density score and the first range score for each section 104 and calculating the first RALE score 106, the second artificial intelligence system may generate a density segmentation map for each section 105. The density segmentation map may include a visible or computer-visible representation of the density of each section. The density segmentation map may include a three-dimensional or two-dimensional representation of the density of each section of the lung.

[0027] Each individual deep learning network module may detect the point with the highest density in each quadrant and expand the highest density to the segmented region with the highest density. The density segmentation map may further include segmented regions with different densities (Figs. 5A-5D).

[0028] After generating the density segmentation map for each section 105, method 100 may analyze the density segmentation map of each section to determine the second density score and the second range score for each section. Using the second density score and the second range score of each section, the second RALE score of image 107 may be calculated.

[0029] Method 100 may analyze the first RALE score and the second RALE score 108 to determine the difference and the overall RALE score. Determining the difference may involve comparing the first RALE score and the second RALE score to determine the total amount of the difference between the RALE scores. If the difference between the first RALE score and the second RALE score is 3 points, 2 points, 1 point, or 0 points, the first RALE score and the second RALE score may be characterized as matching in method 100.

[0030] If the difference between the first RALE score and the second RALE score is greater than 3, for example, 4, 5, 6, 7, 8, 9, 10, 11, 12 or more, method 100 may recommend that a physician or radiologist according to the present disclosure review and annotate the chest X-ray image file. Further, when a physician or radiologist reviews and annotates a chest X-ray, the annotated chest X-ray image may be added to the database to further train the second artificial intelligence system.

[0031] Method 100 may determine the overall RALE score by calculating the minimum value, maximum value or average value of the first RALE score and the second RALE score. The minimum value, maximum value or average value has been described currently, but the methods and systems of the present disclosure may utilize other calculations that can determine the overall RALE score.

[0032] If the difference between the first RALE score and the second RALE score is 0, 1, 2 or 3, method 100 may input the chest X-ray image file and the overall RALE score into the database, and use either the overall RALE score and / or the components of the first RALE score or the second RALE score to train the second artificial intelligence system to create positive feedback learning. Therefore, the image file may become a reference image file. Thus, method 100 may continuously train the second artificial intelligence system in the same way as the method analyzes the chest X-ray image file. In this way, the methods and systems of the present disclosure may only require clinical involvement when a difference of more than 3 points between the first RALE score and the second RALE score is detected, while still being able to continuously learn. In this way, the methods and systems of the present disclosure may minimize clinical labor while enabling an exponentially large training set to be realized. Further, the methods and systems of the present disclosure may alleviate the dataset bias that limits the conventional use of deep learning in the clinical setting, since the growing and diversifying database can include patient chest X-rays with a wide spectrum of disease severity from multiple health centers in different geographical regions.

[0033] If the image file analyzed by the methods and systems of the present disclosure exhibits a new phenomenon, the first and second artificial intelligence systems may be retrained to take the new phenomenon into account. As used herein, a new event may include a new presentation of a disease or illness.

[0034] Method 100 may display the results to user 109. The results may include an overall RALE score and / or each component of the first and second RALE scores. The results may display a density segmentation map of each section (Figs. 5A - 5D) on a graphical user interface. Each level of density may be represented by a different pattern or different colors such as red, yellow, orange, and blue. The density segmentation map of each section may be displayed as a three - dimensional or two - dimensional representation of the lung. The method may generate a density segmentation map as a two - dimensional or three - dimensional representation of the entire lung by combining each section (Fig. 6). Thus, the methods and systems of the present disclosure may enable explanatory deep learning, and method 100 may display to a physician the visualization of the high impact of the density segmentation map to assist in the understanding of RALE calculation. The density segmentation map may provide visualization of the highest density and provide a less - misunderstood method, resulting in improved clinical reliability.

[0035] Method 100 may result in a correlation with the performance of a trained clinician or a confidence level of the RALE score of at least 0.7, including but not limited to at least 0.8, 0.9, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, and at least 0.99.

[0036] The methods and systems of the present disclosure may be seamlessly incorporated into clinical or hospital workplaces. Conventional methods of determining the RALE score in a clinical or hospital workplace are cumbersome, subjective, inconsistent, and rely on the skills and / or experience of radiologists. The systems and methods of the present disclosure address the problems in the art by providing a quantitative severity assessment of lung diseases without additional effort on the part of physicians or radiologists. The methods and systems of the present disclosure may provide an objective, automated, real-time, accurate, consistent, workplace-independent, and / or device-independent RALE score.

[0037] Conventional methods of determining the RALE score may inaccurately define the apex of the lung, especially when the chest X-ray has a specific lung disease. Incorrect apex positions may result in an incorrect RALE score. Thus, the first artificial intelligence system of the present disclosure automatically and accurately determines the apex position.

[0038] The method may minimize the training error of the artificial intelligence system of the present disclosure by preprocessing and calculating the first RALE score and the second RALE score.

[0039] The methods and systems of the present disclosure may be incorporated into the clinical workflow or into conventional or future visualization software such as CERNER. The methods and systems of the present disclosure may be incorporated into the chest X-ray acquisition system to determine the RALE score in real time when a chest X-ray is obtained.

[0040] Conventional methods and systems may include or merely assist in the diagnosis of a specific lung disease. The methods and systems of the present disclosure may accurately and stably determine the severity of the lung regardless of the lung disease present in the chest X-ray.

[0041] The conventional method for determining the RALE score is time-consuming and inconsistent. In contrast, the methods and systems of the present disclosure provide semi-supervised learning by combining explanatory deep learning density segmentation and RALE score calculation.

[0042] The methods and systems of the present disclosure may provide an assessment of disease progression over time and a measurement of treatment response. The methods and systems of the present disclosure may be used as surrogate markers in clinical trials to enable faster and more efficient trials. As an example, the methods and systems of the present disclosure may provide a quantitative chest X-ray assessment and a measurement of treatment effects directly related to radiation severity, such as a change in metrics from before treatment to after treatment, in a drug and / or treatment trial. Thus, the methods and systems of the present disclosure provide alternative indicators of treatment response, causing an increase in the statistical power of test treatment effects compared to conventional methods, which may result in a reduction in sample size requirements and a more efficient design of clinical trials, as reliable alternative endpoints can significantly save the cost and time of treatment trials.

[0043] The methods and systems of the present disclosure may be incorporated into an integrated delivery network.

[0044] The methods and systems of the present disclosure may be used in a triage setting to provide real-time quantification of lung severity prior to the diagnosis of a specific lung disease.

[0045] FIG. 7 is a block diagram schematically showing the system 200 of the present disclosure for quantifying the severity of lung diseases. The system 200 may include an input system 205. The input system 205 may include, but is not limited to, cloud storage, hospital PACS, computers, smartphones, the web, and any X-ray system, and may include any system that enables a user to input and receive an image file of a chest X-ray. The image file of the chest X-ray may include a data file, an image, and / or patient data, and the data and / or the image may be formatted in any file format capable of storing a chest X-ray, including but not limited to DICOM, JPEG, TIGG, GIF, and PNG.

[0046] The system of the present disclosure may be incorporated into a clinical workflow or may be incorporated into conventional or future visualization software such as CERNER. When a chest X-ray is obtained, the system of the present disclosure may be incorporated into a chest X-ray acquisition system to determine the overall RALE score and / or RALE score components in real time.

[0047] The system 200 may include a database 210. The database 210 may include at least two reference image files of chest X-rays. The reference image files may include chest X-rays having an overall RALE score and / or RALE score components designated by a physician or a radiologic technologist (FIG. 4). The database may include at least two reference image files of chest X-rays, for example, at least 2, 100, 1000, 2000, 4000, 6000, 8000, 10000, 50000, and at least 100000 reference image files of chest X-rays. The reference image files may include chest X-rays annotated by a physician or a radiologic technologist.

[0048] System 200 may include one or more networks and / or communication interfaces 215 that communicate with various computing entities by transmitting, receiving, operating on, processing, displaying, and / or storing data, content, information, and / or similar terms used interchangeably herein.

[0049] System 200 may include a processor 220 that interfaces with a memory 225 (which may be separate from or included as part of the processor 220). Also, the memory 225 may use cloud-type memory. In one aspect, the system may be connected to a base station having memory and processing capabilities. The system may further include an I / O device 230. The processor 220 may interface with a database 210 according to the methods and systems of the present disclosure.

[0050] The memory 225 stores therein a plurality of routings executable by the processor 220. The processor 220 communicating with the memory 225 may be configured to execute a first artificial intelligence system 235. The first artificial intelligence system 235 may include program instructions or computer program code that, when executed by the processor 220, can preprocess the chest X-ray image file provided by the input system 205. The preprocessing may be performed according to the methods of the present disclosure.

[0051] The processor 220 communicating with the memory 225 may be configured to execute a second artificial intelligence system 240. The system 200 may include computer-executable instructions that provide each section of the lungs to the second artificial intelligence system 240 according to the methods of the present disclosure to generate a first density score and a first range score for each section. The second artificial intelligence system 240 may be trained based on the database 210 according to the methods of the present disclosure.

[0052] Memory 225 may further include computer-executable instructions for calculating a first RALE score based on the first density score and the first range score of each section according to the method of the present disclosure. The second artificial intelligence system 240 may be executed by the processor 220 to generate and analyze a density segmentation map of each section according to the method and system of the present disclosure to determine a second density score and a second range score of each section and calculate a second RALE score of the image file, and may include computer-executable instructions.

[0053] The processor 220 communicating with the memory 225 may be configured to execute program instructions to analyze the first RALE score and the second RALE score to determine a difference and an overall RALE score according to the method of the present disclosure. The processor 220 communicating with the memory 225 may be configured to execute program instructions to display the results to the user on the graphical user interface 245 according to the method of the present disclosure. The graphical user interface includes an output system, and the output system may include any system capable of receiving RALE score data, such as cloud storage, a computer, a medical device, a smartphone, and / or a healthcare system. The RALE score data may include the overall RALE score and may include the first RALE score, the second RALE score, and / or at least one component of the first RALE score and the second RALE score.

[0054] The processor 220 may be one or more microprocessors, microcontrollers, application-specific integrated circuits (ASICs), circuits including one or more processing components, a group of distributed processing components, circuits for supporting a microprocessor, or other suitable processing devices interfacing with the memory 225. The processor 220 is also configured to execute computer code stored in the memory 225 to complete and facilitate the operations described herein.

[0055] System 200 includes I / O device 255, and I / O device 255 (including keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other storage media, etc.) may be coupled to the system directly or via an I / O controller. A network adapter may also be coupled to the system to couple the data processing system to other data processing systems, remote printers, or storage devices via an intervening private or public network. Modems, cable modems, and Ethernet cards may be only some of the available types of network adapters.

[0056] One skilled in the art will appreciate that the present disclosure may be implemented as a system, method, or computer program product. Thus, the present invention may take the form of an overall hardware embodiment, an overall software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software aspects and hardware aspects, all generally referred to herein as a "system." Further, the present disclosure may take the form of a computer program product embodied in any tangible expression medium having computer-usable program code embodied in the medium. The computer program of the present disclosure may include at least one non-transitory computer-readable medium including program instructions executable by at least one processor to cause the at least one processor to execute the method of the present disclosure.

[0057] FIG. 8 shows a schematic diagram of a computer network system of the present disclosure including a client computer 300 and a computer program 310 configured to execute the method and system of the present disclosure. Client computer 300 may be any device capable of executing computer program 310 of the present disclosure. The system may interface with at least one network server 320, and at least one network server 320 may interface with database 210 and the system.

[0058] The illustrated system is shown and described herein by specific components and functions, but other aspects of the system may be implemented with fewer or more components or fewer or more functions. Some aspects of the system may include multiple network servers 320, multiple networks, and multiple databases 210. Some aspects may include similar components arranged in a different manner to provide similar functions in one or more aspects.

[0059] The client computer 300 manages the interface between the system user and computer programs 310 and the network server 320. Although this disclosure is described with respect to "computers", any optional device characterized by a data processor and the ability to execute one or more instructions may be described as a computer including, but not limited to, any type of personal computer (PC), server, distributed server, virtual server, cloud computing platform, mobile phone, IP phone, smartphone, mobile device, or personal digital assistant (PDA). Any two or more of such devices that communicate with each other may optionally include a network or computer network.

[0060] The network may communicate with conventional block I / O, for example, via a storage area network (SAN). The network may also communicate with file I / O, for example, using a Transmission Control Protocol / Internet Protocol (TCP / IP) network or similar communication protocol. In one aspect, the storage system includes two or more networks. In other aspects, the client computer 300 is directly connected to the network server 320 via a backplane or system bus. In one aspect, the network server 610 includes a cellular network, other networks of the same type, or combinations thereof.

[0061] The computing system may include a client and a server. The client and the server are generally remote from each other and typically interact via a communication network. The relationship between the client and the server results from computer programs that run on separate computers and have a client-server relationship with each other. In some embodiments, the server sends data, such as an HTML page, to a device for the purpose of causing a user interacting with a user device acting as a client to view the data and receive user input from the user. Data generated at the user device, such as the result of a user interaction, can be received at the server from the device.

[0062] Any combination of one or more computer-usable or computer-readable media may be utilized. The computer-usable or computer-readable media may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. The computer-readable media may also be an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CDROM), an optical storage device, a transmission medium such as those supporting the Internet or an intranet, a magnetic storage device, a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structure in a groove having instructions recorded thereon, and any suitable combination thereof. It should be noted that the computer-usable or computer-readable media may also be paper or another suitable medium upon which a program is printed, as the program can be electronically captured via, for example, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner and then stored in a computer memory. In the context of this specification, the computer-usable or computer-readable media may be any medium that can store, house, communicate, propagate, or transport the program(s) used by or associated with an instruction execution system, apparatus, or device. The computer-usable program code may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, and the like.

[0063] The computer program code for executing the operation of the invention disclosed herein may be written in any combination of one or more programming languages. The programming language may be an object-oriented programming language (such as Java, Smalltalk, C++) or a conventional procedural programming language (such as the "C" programming language), but is not limited thereto. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer via the Internet using an Internet service provider. In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing the state information of a computer-readable program to customize the electronic circuit for implementing the embodiments of the present disclosure.

[0064] The systems and methods of the present disclosure may process data on any commercially available computer. In other embodiments, the computer operating system may include, but is not limited to, Linux, Windows, UNIX, Android, and MAC OS. In one embodiment of the present disclosure, the above-described processing device or any other type of electronic computing platform designed to electronically process the digital data disclosed herein may be used.

[0065] Aspects of the present disclosure are described using flowchart illustrations and / or block diagrams of methods, systems, and program products according to aspects of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable implementation of the steps shown in the flowchart and / or block diagram block or blocks.

[0066] Various embodiments of the present disclosure may be implemented in a data processing system suitable for storing and / or executing program code that includes at least one processor directly or indirectly coupled to a memory element via a system bus. The memory element may include, for example, local memory used during actual execution of program code, bulk storage, and cache memory that temporarily stores at least some program code to reduce the number of times code must be retrieved from bulk storage during execution.

[0067] The computer-readable programs described in this specification may be downloaded from a computer-readable storage medium to an individual arithmetic / processing device, or may be downloaded from an external computer or an external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical transmission fiber, wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each arithmetic / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions to be stored in a computer-readable storage medium in the individual arithmetic / processing device.

[0068] A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, transferred, or transmitted via any suitable means including other ones of memory sharing, message passing, token passing, network transmission.

[0069] Definition All terms used in this specification (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art, unless specifically defined otherwise. Thus, terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or overly formal sense unless explicitly defined in this specification.

[0070] As used herein, the term "lung disease" may include any lung problem that prevents the lungs from functioning properly, and the lung problems include, but are not limited to, COVID-19, acute respiratory distress syndrome (ARDS), pneumonia, and respiratory syncytial virus (RSV).

[0071] As used herein, the term "user" refers to any person, entity, business, individual, institution, healthcare provider, healthcare facility, physician, physician assistant, nurse, nurse practitioner, doctor, and patient who can utilize the methods and systems of the present disclosure.

[0072] As used herein, the term "patient" refers to any animal or human who can receive medical support or diagnosis related to the methods and systems of the present disclosure.

[0073] As used herein, the term "and / or" includes any one or any combination of one or more of the associated listed items. Similarly, as used in the following detailed description, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X adopts A or B" is intended to mean any of the natural inclusive permutations. Thus, "X adopts A or B" is satisfied in any of the above situations when X adopts A, X adopts B, or X adopts both A and B.

[0074] The terms used herein are for illustrative purposes only and are not limited thereto. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. For example, "a" image file may include one or more image files and the like.

[0075] The terms "comprising", "including", "having", "containing" and "characterized by" may be inclusive and accordingly specify the presence of the stated features, elements, compositions, steps, integers, operations and / or components but do not preclude the presence and addition of one or more other features, integers, steps, operations, elements, components and / or their groups. These open-ended terms may be understood as non-limiting terms used to describe and claim the various aspects described herein, but in certain aspects, the terms may instead be understood as more restrictive and limiting terms such as "consisting of" or "consisting essentially of". Accordingly, any given embodiment listing the compositions, materials, components, elements, features, integers, operations and / or process steps described herein also specifically includes embodiments consisting of, or consisting essentially of, such listed compositions, materials, components, elements, features, integers, operations and / or process steps. In the case of "consisting of", alternative embodiments exclude any additional compositions, materials, components, elements, features, integers, operations and / or process steps, whereas in the case of "consisting essentially of", any additional compositions, materials, components, elements, features, integers, operations and / or process steps that substantially affect the basic and novel features may be excluded from such embodiments, but compositions, materials, components, elements, features, integers, operations and / or process steps that do not substantially affect the basic and novel features may be included in the embodiments.

[0076] Method steps, processes and operations described herein need not be construed as necessarily being performed in the particular order described or illustrated, unless specifically specified as an order of performance. It is also understood that additional or alternative steps may be employed, unless otherwise indicated.

[0077] Also, the features described with respect to a particular exemplary embodiment may be combined with other various exemplary embodiments, or combined therewith, in any permutation or combination manner. Different aspects or elements of the exemplary embodiments disclosed herein may be combined in a similar manner. The terms "combination", "combinations of", or "combinations thereof" as used herein refer to all permutations and combinations of the items listed before the term. For example, "A, B, C, or combinations thereof" is intended to include at least one of A, B, C, AB, AC, BC, or ABC, and, if order is important in a particular context, is also intended to include at least one of BA, CA, CB, CBA, BCA, ACB, BAC, or CAB. Continuing with this example, combinations including repetitions of one or more items or terms may be explicitly included, such as BB, AAA, AB, BBC, AAABCCCC, CBBAAA, and CABABB. Those skilled in the art will understand that, unless otherwise explicitly stated from the context, the number of items or terms in any combination is not limited.

[0078] Aspects of the present disclosure may be described herein using flowchart illustrations and / or block diagrams of methods, apparatus (systems) and program products according to aspects of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer readable program instructions. Various illustrative logical blocks, modules, circuits and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software or combinations thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those of ordinary skill in the art will appreciate that the described functionality may be implemented in various ways for each particular application, but such implementation decisions should not be construed as departing from the scope of the present disclosure.

[0079] The flowcharts and block diagrams in the figures illustrate the structure, functionality, and operation of possible realizations of systems, methods, and computer program products according to various aspects of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of one or more executable instructions for implementing the specified logical function(s). In some alternative realizations, the functions noted in the blocks may occur in a different order than shown in the figures. For example, in fact, two blocks shown consecutively may be executed substantially simultaneously or in the reverse order depending on the related functions. Also, note that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations or by a combination of dedicated hardware and computer instructions.

[0080] Also, words such as "next," "following," etc. do not limit the order of steps, and these words may simply be used to guide the reader through the description of the method. The process flowchart may describe the operations as a sequential process, but many of the operations may be performed in parallel or simultaneously. Also, the order of the operations may be changed. The process may correspond to a method, function, procedure, subroutine, subprogram, etc. When the process corresponds to a function, its end may correspond to returning the function to the calling function or main function.

[0081] In this specification, for a better understanding of various embodiments of the systems and methods disclosed herein, a detailed description is provided. However, those skilled in the art will understand that these embodiments may be implemented without these details and / or without any details not described herein. Also, to avoid unnecessarily obscuring the description of other details of the various embodiments, known structures, methods, and / or techniques associated with the ways of implementing the various embodiments may not be shown in detail and may not be described.

[0082] Certain aspects of the present disclosure are provided above in this specification, but the present disclosure may be implemented in many different forms and should not be construed as necessarily limited to only the embodiments disclosed herein. Rather, these embodiments may be provided so that the present disclosure is thorough and complete and conveys the various concepts of the present disclosure to those skilled in the art.

[0083] Furthermore, when the present disclosure describes something as being "based on" something else, such a description refers to a basis that may also be based on one or more other things. In other words, as used herein, unless explicitly indicated otherwise, "based on" comprehensively means "at least partly based on" or "at least partially based on".

[0084] All numerical values described in this specification may be approximate values unless otherwise specified. Thus, the term "substantially" may be presumed if not explicitly stated. It may be understood that the numerical values disclosed in this specification are not strictly limited to the exact numerical values described. Instead, unless otherwise specified, each numerical value described in this specification is intended to mean both the recited value and a functionally equivalent range around that value. At a minimum, and not as an attempt to limit the scope of the claims to the application of the doctrine of equivalents, each numerical value should be construed by applying the ordinary rounding process, taking into account at least the reported number of significant digits. Exemplary degrees of typical error are within 20%, 10%, or 5% of a given value or range of values. Alternatively, the term "about" refers to values within a digit, and in some cases, within 5 times or 2 times a given value. Despite the approximate nature of the numerical values described in this specification, the numerical values described in the specific examples of actual measurements may be reported as accurately as possible. However, any numerical value inherently includes a certain amount of error that necessarily results from the standard deviation found in their individual test measurements.

[0085] All numerical ranges described in this specification include all sub-ranges subsumed therein. For example, the range "1 to 10" or "1 - 10" is intended to include all sub-ranges therebetween, since the disclosed numerical range may be continuous and may include all values between the minimum and maximum values, including the values between the recited minimum value of 1 and the recited maximum value of 10 and all sub-ranges including them. The maximum numerical limitations described in this specification are intended to include all lower numerical limitations. The minimum numerical limitations described in this specification are intended to include all greater numerical limitations.

[0086] Features or functions described with respect to a particular exemplary embodiment may be applicable to various other embodiments and / or may be combined with them. Also, as disclosed herein, different aspects and / or elements of exemplary embodiments may be combined and sub-combined in a similar manner. Further, some exemplary embodiments may be components of a larger system, individually and / or collectively, and other procedures may take precedence over their use and / or may otherwise modify their use. Thus, as disclosed herein, multiple steps may be required before, after, and / or during exemplary embodiments. At least as disclosed herein, any and / or all methods and / or processes may be performed at least in part via at least one entity or actor in any manner.

[0087] All documents cited herein may be incorporated herein by reference to the extent that the incorporated material does not conflict with existing definitions, descriptions, or other documents described herein. If any meaning or definition of a term in this document conflicts with any meaning or definition of the same term in the document incorporated by reference, the meaning or definition assigned to that term in this document shall prevail. The citation of any document shall not be construed as an admission that it is prior art with respect to this application.

[0088] While particular embodiments have been illustrated and described, it will be apparent to those skilled in the art that various other changes and modifications can be made without departing from the spirit and scope of the invention. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation, many equivalents to the specific apparatus and methods described herein, including alternatives, modifications, additions, deletions, variations, and substitutions. Accordingly, the present application, including the appended claims, is intended to cover all such changes and modifications that may fall within the scope of the present application.

[0089] Aspect Aspect 1: A method for quantifying the severity of a lung disease, comprising providing an image file of a chest X-ray from a patient to a first artificial intelligence system to perform preprocessing of the image, wherein the preprocessing includes determining vertices and performing segmentation of the image to define a lung boundary, and the lung is divided into at least four sections, and providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on a database including at least two reference image files of the chest X-ray, the at least two reference image files include at least one annotation assigned by a doctor, and the second artificial intelligence system generates a density segmentation map for each section, calculating a first RALE score from the first density score and the first range score for each section, analyzing the density segmentation map for each section to determine a second density score and a second range score for each section and calculating a second RALE score for the image, analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score, and displaying the result to a user.

[0090] Aspect 2: The method according to Aspect 1, wherein the first artificial intelligence system is trained using pre-training data.

[0091] Aspect 3: The method according to any one of the preceding aspects, wherein the pre-training data includes at least two image files of chest X-rays.

[0092] Aspect 4: The method according to any one of the preceding aspects, wherein the pre-training data includes findings or lack of findings.

[0093] Aspect 5: The method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are identical, and the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

[0094] Aspect 6: The method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are identical, and the image file, the first density score and the first range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

[0095] Aspect 7: The method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are identical, and the image file, the second density score and the second range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

[0096] Aspect 8: The method according to any one of the preceding aspects, wherein the database includes at least 2000 image files of chest X-rays, and the at least one annotation is an overall RALE score assigned by a doctor.

[0097] Aspect 9: The method according to any one of the preceding aspects, wherein the database includes at least 2000 image files of chest X-rays, and the at least one annotation is at least one density score and at least one range score assigned by a doctor.

[0098] Aspect 10: The method according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the density segmentation map of each section on a graphical user interface.

[0099] Aspect 11: The method according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the overall RALE score to the user on a graphical user interface.

[0100] Aspect 12: The method according to any one of the preceding aspects, wherein displaying the result to the user includes displaying at least one first range score and at least one first density score of at least one section to the user on a graphical user interface.

[0101] Aspect 13: The method according to any one of the preceding aspects, wherein the step of displaying the result to the user includes displaying at least one second range score and at least one second density score of at least one section to the user on a graphical user interface.

[0102] Aspect 14: The method according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the density segmentation map of each section as a three-dimensional and / or two-dimensional representation of the lung on a graphical user interface.

[0103] Aspect 15: The method according to any one of the preceding aspects, having an RALE score reliability of at least 0.9.

[0104] Aspect 16: The first RALE score and the second RALE score do not match, the image file is annotated by a doctor to generate an annotated image file, and the annotated image file is added to the database to train the second artificial intelligence system. The method according to any one of the preceding aspects.

[0105] Aspect 17: The method according to any one of the preceding aspects, wherein the first artificial intelligence system includes at least one computer vision algorithm.

[0106] Aspect 18: The artificial intelligence system is the method according to any one of the preceding aspects, including at least one deep learning network.

[0107] Aspect 19: The first artificial intelligence system is the method according to any one of the preceding aspects, including at least one computer vision algorithm and at least one deep learning network.

[0108] Aspect 20: The first artificial intelligence system is the method according to any one of the preceding aspects, including a section module and a lung segmentation module.

[0109] Aspect 21: The second artificial intelligence system is the method according to any one of the preceding aspects, including at least one computer vision algorithm.

[0110] Aspect 22: The second artificial intelligence system is the method according to any one of the preceding aspects, including at least one deep learning network.

[0111] Aspect 23: The second artificial intelligence system is the method according to any one of the preceding aspects, including at least one computer vision algorithm and at least one deep learning network.

[0112] Aspect 24: The second artificial intelligence system is the method according to any one of the preceding aspects, including at least one deep learning network module.

[0113] Aspect 25: The at least one deep learning network module is the method according to any one of the preceding aspects, including at least one computer vision algorithm and / or at least one deep learning network.

[0114] Aspect 26: The method according to any one of the preceding aspects, wherein the second artificial intelligence system includes at least one deep learning network module for each section.

[0115] Aspect 27: The method according to any one of the preceding aspects, wherein the deep learning network includes at least one neural network.

[0116] Aspect 28: The method according to any one of the preceding aspects, wherein the at least one neural network includes a plurality of layers.

[0117] Aspect 29: A system for quantifying the severity of a lung disease, comprising a database having at least two reference images of chest X-rays, at least one processor, at least one communication interface, a user interface, and at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured to store a first artificial intelligence system, a second artificial intelligence system, and computer-executable instructions, and the memory is further configured to execute the instructions by the processor, and the instructions are to provide an image file of the chest X-ray from a patient to the first artificial intelligence system to perform preprocessing of the image, the preprocessing including determining vertices and performing segmentation of the image to define lung boundaries, wherein the lung is divided into at least four sections, and providing each section to the second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on the database having at least two reference images, the at least two reference image files including at least one annotation assigned by a physician, and the second artificial intelligence system generates a density segmentation map for each section, calculating a first RALE score from the first density score and the first range score for each section, analyzing the density segmentation map for each section to determine a second density score and a second range score for each section and calculating a second RALE score for the image file, analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score, and displaying the results to a user on the user interface.

[0118] Aspect 30: The system according to aspect 29, wherein the first artificial intelligence system is trained using pre-training data.

[0119] Aspect 31: The system according to any one of the preceding aspects, wherein the pre-training data includes at least two image files of chest X-rays.

[0120] Aspect 32: The system according to any one of the preceding aspects, wherein the image file and the overall RALE score are added to the at least two reference image files to continuously train the second artificial intelligence system.

[0121] Aspect 33: The system according to any one of the preceding aspects, wherein the pre-training data includes findings or absence of findings.

[0122] Aspect 34: The system according to any one of the preceding aspects, wherein the first RALE score and the second RALE score match, and the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

[0123] Aspect 35: The system according to any one of the preceding aspects, wherein the first RALE score and the second RALE score match, and the image file, the first density score and the first range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

[0124] Aspect 36: The system according to any one of the preceding aspects, wherein the first RALE score and the second RALE score match, and the image file, the second density score and the second range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

[0125] Aspect 37: The system according to any one of the preceding aspects, wherein the database includes at least 2,000 image files of chest X-rays, and the at least one annotation is an overall RALE score assigned by a doctor.

[0126] Aspect 38: The system according to any one of the preceding aspects, wherein the database includes at least 2,000 image files of chest X-rays, and the at least one annotation is at least one density score and at least one range score assigned by a doctor.

[0127] Aspect 39: The system according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the density segmentation map of each section on the user interface.

[0128] Aspect 40: The system according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the overall RALE score to the user on the user interface.

[0129] Aspect 41: The system according to any one of the preceding aspects, wherein displaying the result to the user includes displaying at least one first range score and at least one first density score of at least one section to the user on the user interface.

[0130] Aspect 42: The system according to any one of the preceding aspects, wherein the step of displaying the result to the user includes displaying at least one second range score and at least one second density score of at least one section to the user on the user interface.

[0131] Aspect 43: The system according to any one of the preceding aspects, wherein displaying the result to the user includes displaying the density segmentation map of each section on the user interface as a three-dimensional and / or two-dimensional representation of the lung.

[0132] Aspect 44: The system according to any one of the preceding aspects, having a RALE score reliability of at least 0.9.

[0133] Aspect 45: The first RALE score and the second RALE score do not match, and the image file is annotated by a doctor to generate an annotated image file, and the annotated image file is added to the database to train the second artificial intelligence system. The system according to any one of the preceding aspects.

[0134] Aspect 46: The system according to any one of the preceding aspects, wherein the first artificial intelligence system includes at least one computer vision algorithm.

[0135] Aspect 47: The system according to any one of the preceding aspects, wherein the artificial intelligence system includes at least one deep learning network.

[0136] Aspect 48: The system according to any one of the preceding aspects, wherein the first artificial intelligence system includes at least one computer vision algorithm and at least one deep learning network.

[0137] Aspect 49: The system according to any one of the preceding aspects, wherein the first artificial intelligence system includes a section module and a lung segmentation module.

[0138] Aspect 50: The system according to any one of the preceding aspects, wherein the second artificial intelligence system includes at least one computer vision algorithm.

[0139] Aspect 51: The second artificial intelligence system is the system according to any one of the preceding aspects, including at least one deep learning network.

[0140] Aspect 52: The second artificial intelligence system is the system according to any one of the preceding aspects, including at least one computer vision algorithm and at least one deep learning network.

[0141] Aspect 53: The second artificial intelligence system is the system according to any one of the preceding aspects, including at least one deep learning network module.

[0142] Aspect 54: The at least one deep learning network module is the system according to any one of the preceding aspects, including at least one computer vision algorithm and / or at least one deep learning network.

[0143] Aspect 55: The second artificial intelligence system is the system according to any one of the preceding aspects, including at least one deep learning network module for each section.

[0144] Aspect 56: The deep learning network is the system according to any one of the preceding aspects, including at least one neural network.

[0145] Aspect 57: The at least one neural network is the system according to any one of the preceding aspects, including a plurality of layers.

[0146] Aspect 58: A computer program product for quantifying the severity of a lung disease, comprising at least one non-transitory computer-readable medium including program instructions, wherein when the program instructions are executed by at least one processor, the at least one processor is caused to provide an image file of the chest X-ray from a patient to the first artificial intelligence system to perform preprocessing of the image, the preprocessing including determining vertices and performing segmentation of the image to define a lung boundary, wherein the lung is divided into at least four sections, and providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on a database including at least two reference image files of the chest X-ray, the at least two reference image files including at least one annotation assigned by a physician, and the second artificial intelligence system generating a density segmentation map for each section, calculating a first RALE score from the first density score and the first range score for each section, analyzing the density segmentation map for each section to determine a second density score and a second range score for each section and calculating a second RALE score for the image file, analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score, and displaying the result to a user on a user interface, the result including the overall RALE score, the computer program product.

[0147] Aspect 59: The computer program product according to any one of the preceding aspects.

[0148] Aspect 60: The method, system, and computer program product according to any one of the preceding aspects, wherein the first RALE score includes eight components.

[0149] Aspect 61: The second RALE score, a method, system, and computer program product according to any one of the preceding aspects, comprising eight components.

Example

[0150] Example 1 The system and method of the present disclosure analyzed chest X-ray images of 595 patients with COVID-19 at the time of ICU or hospital admission at the University of Pittsburgh Medical Center facility. After training by senior reviewers, eight physicians at different training levels scored an independent set of chest X-rays, then the score distributions were fed back, and then re-scored independently by the inter-reviewer correlation and intra-class correlation coefficient (ICC) in a binary logistic model, as well as the k-statistic for categorical variables. The inventors trained the method and system of the present disclosure using the mean RALE score from two reviewers who were in substantial agreement after pre-training on the MMIC-CXR dataset using the classes of "no findings" or "clinico-pathological observations".

[0151] The inventors found that while <1% of CXRs showed large RALE discrepancies (≥15-point difference), there was improved inter-rater agreement for the overall RALE score of the disclosed method and system compared to the reviewers' overall RALE score (correlation R = 0.88, p < 0.0001, ICC = 0.91 [0.88 - 0.94], p < 0.0001), with an overall RALE score (correlation R = 0.71, p < 0.0001, ICC = 0.84, 95% confidence interval [0.82 - 0.89], p < 0.0001). Reviewers had moderate agreement (kappa = 0.6) for image quality and slight agreement (kappa = 0.21) for the presence of atelectasis. CXRs with insufficient infiltration had higher central RALE scores (p < 0.01) compared to well-infiltrated images, and the presence of atelectasis was associated with higher mean right lower abdominal density scores. The inventors then trained a deep learning network using RALE score annotations and obtained a Spearman correlation R = 0.87 (p = 1.7 × 10-7) between the predicted RALE score and the RALE score with physician annotations.

[0152] The inventors demonstrated that the disclosed system and method may reliably, accurately, and rapidly predict the RALE score of chest X-rays.

Claims

1. A method for quantifying the severity of a lung disease, comprising: providing an image file of a chest X-ray from a patient to a first artificial intelligence system to perform preprocessing of the image, wherein the preprocessing includes determining vertices and performing segmentation of the image to define a lung boundary, and the lung is divided into at least four sections; providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on a database including at least two reference image files of the chest X-ray, the at least two reference image files include at least one annotation assigned by a doctor, and the second artificial intelligence system generates a density segmentation map for each section; calculating a first RALE score from the first density score and the first range score of each section; analyzing the density segmentation map of each section to determine a second density score and a second range score for each section and calculating a second RALE score for the image; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; displaying the result to a user.

2. The method according to claim 1, wherein the first artificial intelligence system is trained using pre-training data.

3. The method according to claim 2, wherein the pre-training data includes at least two image files of chest X-rays.

4. The method according to claim 2, wherein the pre-training data includes findings or lack of findings.

5. The method according to claim 1, wherein the first RALE score and the second RALE score are identical, the image file and the overall RALE score are added to the database, and are used for continuously training the second artificial intelligence system.

6. The first RALE score and the second RALE score are identical, and the image file, the first density score and the first range score of at least one section are added to the database and used to continuously train the second artificial intelligence system, according to the method of claim 1.

7. The first RALE score and the second RALE score are identical, and the image file, the second density score and the second range score of at least one section are added to the database and used to continuously train the second artificial intelligence system, according to the method of claim 1.

8. The database includes at least 2000 image files of chest X-rays, and the at least one annotation is the overall RALE score assigned by a doctor, according to the method of claim 1.

9. The database includes at least 2000 image files of chest X-rays, and at least one annotation is at least one density score and at least one range score assigned by a doctor, according to the method of claim 1.

10. Displaying the result to the user includes displaying the density segmentation map of each section on a graphical user interface, according to the method of claim 1.

11. Displaying the result to the user includes displaying the overall RALE score to the user on a graphical user interface, according to the method of claim 1.

12. Displaying the result to the user includes displaying at least one first range score and at least one first density score of at least one section to the user on a graphical user interface, according to the method of claim 1.

13. Displaying the result to the user includes displaying at least one second range score and at least one second density score of at least one section to the user on a graphical user interface, according to the method of claim 1.

14. The method according to claim 1, wherein presenting the results to the user includes presenting the density segmentation maps of each section as a three-dimensional and / or two-dimensional representation of the lung on a graphical user interface.

15. The method according to claim 1, having a RALE score reliability of at least 0.

9.

16. The first RALE score and the second RALE score do not match, and the image file is annotated by a doctor to generate an annotated image file, and the annotated image file is added to the database to train the second artificial intelligence system. The method according to claim 1.

17. A system for quantifying the severity of a lung disease, including a database having at least two reference images of chest X-rays, at least one processor, at least one communication interface, a user interface, and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to store a first artificial intelligence system, a second artificial intelligence system, and computer-executable instructions, the memory further being configured to execute the instructions by the processor, the instructions being providing an image file of the chest X-ray from a patient to a first artificial intelligence system to perform preprocessing of the image, the preprocessing including determining vertices and performing segmentation of the image to define a lung boundary, wherein the lung is divided into at least four sections; providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, the second artificial intelligence system being trained based on the database having at least two reference images, the at least two reference image files including at least one annotation assigned by a doctor, the second artificial intelligence system generating a density segmentation map for each section; calculating a first RALE score from the first density score and the first range score of each section; Analyzing the density segmentation map of each section to determine a second density score and a second range score for each section and calculating a second RALE score for the image file; Analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; Displaying the results to the user on the user interface, the system comprising the same.

18. The system according to claim 17, wherein the first artificial intelligence system is trained using pre-training data.

19. The system according to claim 18, wherein the pre-training data includes at least two image files of chest X-rays.

20. The system according to claim 18, wherein the pre-training data includes findings or lack of findings.

21. The system according to claim 17, wherein the first RALE score and the second RALE score match, and the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

22. The system according to claim 17, wherein the first RALE score and the second RALE score match, and the image file and the first density score and the first range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

23. The system according to claim 17, wherein the first RALE score and the second RALE score match, and the image file and the second density score and the second range score of at least one section are added to the database and used to continuously train the second artificial intelligence system.

24. The system according to claim 17, wherein the database includes at least 2000 image files of chest X-rays, and at least one annotation is an overall RALE score assigned by a doctor.

25. The system according to claim 17, wherein the database includes at least 2000 image files of chest X-rays, and at least one annotation is at least one density score and at least one range score assigned by a doctor.

26. The system according to claim 17, wherein displaying the result to the user includes displaying the density segmentation map of each section on the user interface.

27. The system according to claim 17, wherein displaying the result to the user includes displaying the overall RALE score to the user on the user interface.

28. The system according to claim 17, wherein displaying the result to the user includes displaying at least one first range score and at least one first density score of at least one section to the user on the user interface.

29. The system according to claim 17, wherein displaying the result to the user includes displaying at least one second range score and at least one second density score of at least one section to the user on the user interface.

30. The system according to claim 17, wherein displaying the result to the user includes displaying the density segmentation map of each section as a three-dimensional and / or two-dimensional representation of the lung on the user interface.

31. The system according to claim 17, having an RALE score reliability of at least 0.

9.

32. The first RALE score and the second RALE score do not match, and the image file is annotated by a doctor to generate an annotated image file, and the annotated image file is added to the database to train the second artificial intelligence system. The system according to claim 17.

33. A computer program product for quantifying the severity of a lung disease, comprising at least one non-transitory computer-readable medium including program instructions, wherein the program instructions, when executed by at least one processor, cause the at least one processor to provide the image file of the chest X-ray from the patient to the first artificial intelligence system to perform preprocessing of the image, the preprocessing including determining vertices and performing segmentation of the image to define the lung boundary, and the lung is divided into at least four sections. Providing each section to a second artificial intelligence system to generate a first density score and a first range score for each section, wherein the second artificial intelligence system is trained based on a database including at least two reference image files of the chest X-ray, the at least two reference image files include at least one annotation assigned by a doctor, and the second artificial intelligence system generates a density segmentation map for each section; Calculating a first RALE score from the first density score and the first range score of each section; Analyzing the density segmentation map of each section to determine a second density score and a second range score for each section and calculating a second RALE score of the image file; Analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; Causing the computer program product to display the results to the user on a user interface.

Images