Method and system for quantifying severity of lung disease

Abstract

A system, method, and computer program product for quantifying the severity of a lung disease. An example aspect is configured to provide an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first degree score for each segment, to calculate a first RALE score, where the second artificial intelligence system generates a density segmentation map for each segment, to calculate a second density score and a second degree score, to calculate a second RALE score; the first RALE score and the second RALE score are analyzed; and displaying the result to the user.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 364,562, filed on May 12, 2022, entitled AUTOMATED QUANTIFICATION OF LUNG SEVERITY USING CHEST X-RAYS, which is expressly incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to methods and systems for quantifying the severity of lung disease in medical images; and more particularly, to methods and systems for quantifying the severity of lung disease in chest x-rays. Background Art

[0004] Chest X-rays are commonly used to evaluate patients with acute respiratory conditions such as COVID-19, acute respiratory distress syndrome (ARDS), and pneumonia. While billions of chest X-rays are likely performed each year, routine chest X-ray analysis involves qualitative interpretation of features such as focal, plaque, or diffuse densities on the chest X-ray.

[0005] Conventional chest x-ray analysis requires manual review of the x-ray by a physician or radiologist, where successful detection of abnormalities may depend on the radiologist's skill and / or experience. Thus, conventional analysis may result in subjective, nonspecific, and inconsistent diagnoses. Summary of the invention

[0006] The methods and systems of the present disclosure are capable of quantifying the severity of lung disease. The methods and systems of the present disclosure can produce an objective and repeatable method and system to efficiently and accurately assess the severity of lung disease in chest x-rays.

[0007] The present disclosure relates to a method for quantifying the severity of a lung disease, the method comprising: providing an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and performing segmentation of the image to define lung boundaries, and wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on a database comprising at least two reference image files of chest x-rays, wherein the at least two reference image files comprise at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and displaying the result to a user, wherein the result comprises the overall RALE score.

[0008] The systems and methods disclosed herein may be embodied as a system, method, or computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] It should be understood that the foregoing summary of the invention and the following drawings and specific embodiments may be exemplary and may not limit the various aspects of the present disclosure as claimed. Certain details may be set forth to better understand the various features, aspects and advantages of the present invention. However, it will be understood by those skilled in the art that these features, aspects and advantages may be practiced without these details. In other cases, well-known structures, methods and / or processes related to the methods for practicing the various features, aspects and advantages may not be shown or described in detail to avoid unnecessarily obscuring the description of other details of the present invention.

[0010] The present disclosure may be better understood by referring to the accompanying drawings, in which:

[0011] Figure 1 is a flow chart of a method for quantifying the severity of a lung disease according to certain aspects of the present disclosure.

[0012] Figure 2 is a block diagram of a second artificial intelligence system according to certain aspects of the present disclosure.

[0013] Figure 3Ais a diagram of a first artificial intelligence system according to certain aspects of the present disclosure.

[0014] Figure 3B is a diagram of a second artificial intelligence system according to certain aspects of the present disclosure.

[0015] Figure 4 A screenshot of an annotator used by a physician to generate training data for a database is shown in accordance with certain aspects of the present disclosure.

[0016] Figure 5A , 5B , 5C, and 5D are density segmentation maps of each segment of the lung according to certain aspects of the present disclosure.

[0017] Figure 6 is a density segmentation map of the entire lung according to certain aspects of the present disclosure.

[0018] Figure 7 is a block diagram of a system for quantifying severity of lung disease according to certain aspects of the present disclosure.

[0019] Figure 8 is a schematic diagram of a computer network system for quantifying the severity of lung disease according to the present disclosure. DETAILED DESCRIPTION

[0020] The present disclosure generally describes methods and systems for quantifying the severity of lung disease. The methods and systems of the present disclosure can produce an objective and repeatable method and system to efficiently and accurately assess the severity of lung disease in chest x-rays.

[0021] The present disclosure provides a method 100 for quantifying the severity of a lung disease. Figure 1 ). Method 100 may provide an image file of a chest x-ray of a patient to a first artificial intelligence system 101. As used herein, an "artificial intelligence system" may include at least one computer vision algorithm and / or at least one deep learning network, wherein the at least one deep learning network may include at least one neural network having multiple layers. The use of "first" and "second" does not exclude one or more, two or more, or three or more artificial intelligence systems. The image file of a chest x-ray may include a data file, an image, and / or patient data, wherein the data and / or the image may be formatted in any file format capable of storing a chest x-ray (including but not limited to DICOM, JPEG, TIGG, GIF, PNG, etc.).

[0022] The first artificial intelligence system can pre-process the image 102. Pre-processing can include lung segmentation, scale adjustment, rotation adjustment, angle adjustment, minimization of stray equipment (such as IV lines, chemotherapy ports, EKG wiring, etc.), determining accurate vertex locations, etc. Segmentation can include defining lung boundaries, wherein the lungs are divided into at least four segments, and anatomical features or image artifacts other than the lungs can be excluded. Although four segmented segments have been described, other segmentations with more than four segments are possible and within the scope of the present disclosure. As used herein, the vertex refers to the center point of the entire lung in a chest x-ray. Once the vertex is determined, method 100 can divide the lungs into at least four segments.

[0023] The first artificial intelligence system can be trained using pre-training data, where the pre-training data can include a collection of chest x-rays, such as MMIC-CXR. The first artificial intelligence system can be trained based on parameters used to characterize the image files of the present disclosure. Parameters can include, but are not limited to, findings or no findings, cancer or non-cancer, patient taking full breaths or patient taking incomplete breaths, diseased or healthy lungs, etc. As used herein, a finding can refer to a visible area that is considered abnormal or unhealthy by a physician. Training of the first artificial intelligence system can train method 100 to determine weights and teach the appearance of a chest x-ray.

[0024] As is generally understood in the art, weights are parameters within a deep learning network that can transform input data within the network. A deep learning network can contain a series of nodes. Each node can include a set of inputs, weights, and bias values. When an input enters a node, it can be multiplied by the weight value, and an output can be observed or passed to the next node.

[0025] The first artificial intelligence system may include a segment module having at least one computer vision algorithm and / or at least one deep learning network to determine vertices and divide the lung into at least four segments. Figure 3A ). The first artificial intelligence system may further include a lung segmentation module having at least one computer vision algorithm and / or at least one deep learning network to segment the lungs ( Figure 3A The Segment Module and the Lung Segmentation Module can work simultaneously to generate a segmented chest x-ray divided into four segments or quadrants ( Figure 3A ).

[0026] Conventional methods and systems for quantifying the severity of lung disease require chest x-ray images taken at a specific pose, angle, or orientation. Therefore, conventional methods may not be able to interpret a large subset of chest x-rays with different poses, angles, or orientations. In addition, chest x-rays of diseased patients may show differences compared to chest x-rays of healthy patients. The methods and systems of the present disclosure can interpret chest x-rays with different poses, angles, scales, orientations, and patient health levels.

[0027] Conventional methods and systems can analyze the entire chest x-ray, or segment the image into two sections. However, these methods and systems do not accurately evaluate chest x-rays in which the lungs are unbalanced in disease presentation. The methods and systems of the present disclosure can mitigate the risks imposed by unbalanced disease presentation by segmenting and dividing the lungs into four sections.

[0028] The method 100 can provide each segment of the lung to a second artificial intelligence system 103. The second artificial intelligence system can generate a first density score and a first extent score for each segment 104. As used herein, the term "extent" can be used interchangeably with "consolidation". "Extent" can refer to the degree of alveolar opacities present in a chest x-ray or the amount of the highest opacity area. As used herein, "density" can refer to the density of alveolar opacities present in a chest x-ray. The second artificial intelligence system can then calculate a first RALE score based on the first density score and the first extent score for each segment 106.

[0029] As used herein, the term "RALE score" may refer to a radiographic assessment score for pulmonary edema that is used to assess the degree and density of alveolar opacities present in a chest x-ray. Although the RALE score has been described, other non-invasive measurements that quantify the severity of lung disease are possible and within the scope of the present disclosure. The RALE score may range from 0 (low severity) to 48 (high severity).

[0030] The second artificial intelligence system may include at least one deep learning network and / or at least one computer vision algorithm to determine the density and extent of each segment ( Figure 2 The second artificial intelligence system may include at least one separate deep learning network module having at least one deep learning network and / or at least one computer vision algorithm for each segment of the lung. Figure 3B) to determine the density and degree of each segment. The individual deep learning network module for each segment may include a RALE deep learning network. Each RALE deep learning network module may be trained to directly calculate a first density score and a first degree score for the segment. At the same time, the at least one individual deep learning network module may generate a density segmentation map for each segment 105 and determine the location of the highest density point in each segment, thereby providing high-impact visualization for RALE deep learning interpretability.

[0031] Each segment may have a first extent score and a first density score. When the lung is segmented into four segments, the second artificial intelligence system may generate eight components of the first RALE score, including four first density scores and four first extent scores.

[0032] Conventional methods of determining RALE scores may report fewer than eight components of the RALE score, such as reporting only the overall RALE score for a chest x-ray, resulting in inconsistent and inaccurate results. Thus, conventional methods lack the ability to use the RALE score components for training or understanding how the RALE score is calculated. The methods and systems of the present disclosure may provide more consistent and more accurate results by dividing the RALE score into eight components and reporting the components to medical providers and to a database used to train an artificial intelligence system.

[0033] The second artificial intelligence system may be trained based on a database of at least two reference image files including chest x-rays. The reference image files may include a chest x-ray having at least one annotation assigned by a physician or radiologist using an annotator, wherein the annotator may include any system or computer program product in which a physician or radiologist may assign a RALE score ( Figure 4). The database may contain at least two reference image files of chest x-rays, such as at least 2, 100, 1000, 2000, 4000, 6000, 8000, 10000, 50000 and at least 100000 reference image files of chest x-rays. The reference image files may contain chest x-rays annotated by a physician or radiologist. The at least one annotation may include a first density score and a first degree score for each segment or quadrant of the lung, the presence of atelectasis, the presence of intubation, image quality and / or visibility level. The first density score and the first degree score for each segment of the reference image may be used to calculate an overall RALE score for the reference image. The second artificial intelligence system may be trained using the overall RALE score and / or the first density score and the first degree score for each segment. The second artificial intelligence system may be trained based on a combination of segments of the lung having at least one degree score and at least one density score. Thus, the second artificial intelligence system may be trained based on at least one density score and at least one degree score annotated by a physician or radiologist. Annotated reference image files can contain metadata to assess dataset statistics and avoid dataset construction errors.

[0034] The second artificial intelligence system may be trained using a random set comprising at least 90% of the data within the database in order to randomize the demographic characteristics and origin of the image files.

[0035] While generating the first density score and the first extent score for each segment 104 and calculating the first RALE score 106, the second artificial intelligence system can generate a density segmentation map for each segment 105. The density segmentation map can include a visual representation or a computer vision representation of the density in each segment. The density segmentation map can include a 3-dimensional or 2-dimensional representation of the density of each segment of the lung.

[0036] Each individual deep learning network module can detect the highest density point in each quadrant and expand the highest density into the segmented region with the highest density. The density segmentation map can further include segmented regions of different densities ( Figures 5A-5D ).

[0037] After generating the density segmentation map for each segment 105 , method 100 can analyze the density segmentation map for each segment to determine a second density score and a second extent score for each segment. The second density score and second extent score for each segment can be used to calculate a second RALE score for image 107 .

[0038] The method 100 may analyze the first RALE score and the second RALE score 108 to determine a difference and an overall RALE score. Determining the difference may include comparing the first RALE score and the second RALE score to determine the total amount of difference between the RALE scores. If the difference between the first RALE score and the second RALE score is 3 points, 2 points, 1 point, or 0 points, the first RALE score and the second RALE score may be characterized by the method 100 as being consistent.

[0039] If the difference between the first RALE score and the second RALE score is greater than 3, such as 4, 5, 6, 7, 8, 9, 10, 11 or greater than 12, the method 100 can recommend that the chest x-ray image file be reviewed and annotated by a physician or radiologist according to the present disclosure. Once the chest x-ray is reviewed and annotated by a physician or radiologist, the annotated chest x-ray image can be added to the database to further train the second artificial intelligence system.

[0040] Method 100 may determine the overall RALE score by calculating the minimum, maximum, or average of the first RALE score and the second RALE score. Although the minimum, maximum, or average values ​​are described, the methods and systems of the present disclosure may utilize other calculation results capable of determining the overall RALE score.

[0041] When the difference between the first RALE score and the second RALE score is zero, one, two or three, the method 100 can input the image file of the chest x-ray and the overall RALE score into a database, wherein the overall RALE score and / or any of the components of the first RALE score or the second RALE score can be used to train the second artificial intelligence system, thereby forming positive feedback learning. Therefore, the image file can become a reference image file. Therefore, the method 100 can continue to train the second artificial intelligence system while the method analyzes the image file of the chest x-ray. In this way, the method and system of the present disclosure can continue to learn, and the participation of the clinician may only be required when the difference between the first RALE score and the second RALE score exceeds 3 points. Therefore, the method and system of the present disclosure can allow the training set to become exponentially larger while minimizing the labor of the clinician. Further, the method and system of the present disclosure can reduce the dataset bias that currently limits the routine use of deep learning in clinical practice, because the growing and diverse database may contain chest x-rays of patients with a wide range of disease severity from multiple health centers in different geographic areas.

[0042] If the image file analyzed by the method and system of the present disclosure indicates a new phenomenon, the first artificial intelligence system and the second artificial intelligence system can be retrained to explain the new phenomenon. As used herein, a new phenomenon may include any newly existing disease or illness.

[0043] The method 100 may display the results to the user 109. The results may include the overall RALE score and / or individual components of the first RALE score and the second RALE score. The results may also include displaying a density segmentation map ( Figures 5A-5D ). Each density level can be represented by a different pattern or a different color, such as red, yellow, orange, blue, etc. The density segmentation map of each segment can be displayed as a 3D or 2D representation of the lung. The method can generate a density segmentation map as a 2D or 3D representation of the entire lung by combining each segment ( Figure 6 ). Thus, the methods and systems of the present disclosure may enable interpretive deep learning, wherein the method 100 displays a high-impact visualization of a density segmentation map to a physician to assist in understanding the RALE calculation. The density segmentation map may provide a visual effect of the highest density, thereby providing a method that is less prone to misinterpretation, thereby increasing the confidence of the clinician.

[0044] Method 100 may result in a correlation or RALE score confidence in a trained clinician's performance of at least 0.7, including but not limited to at least 0.8, 0.9, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, and at least 0.99.

[0045] The methods and systems of the present disclosure can be seamlessly incorporated into the clinical or hospital workplace. Conventional methods for determining RALE scores in the clinical or hospital workplace are laborious, subjective, inconsistent, non-specific, and rely on the skill and / or experience of the radiologist. The systems and methods of the present disclosure solve the problems existing in the prior art by providing a quantitative severity assessment of lung disease without requiring additional work on the part of the physician or radiologist. The methods and systems of the present disclosure can provide an objective, automatic, real-time, accurate, consistent, workplace-independent, and / or device-independent RALE score.

[0046] Conventional methods of determining RALE scores may inaccurately define the apex of the lung, especially when a chest x-ray may have specific lung disease. An inaccurate apex position may result in an incorrect RALE score. Therefore, the first artificial intelligence system of the present disclosure automatically and accurately determines the apex position.

[0047] The method can minimize the training error of the artificial intelligence system of the present disclosure by preprocessing and calculating the first RALE score and the second RALE score.

[0048] The methods and systems of the present disclosure can be incorporated into clinical workflows and can be integrated into conventional or future visualization software such as CERNER. The methods and systems of the present disclosure can be incorporated into chest x-ray acquisition systems to determine the RALE score in real time once a chest x-ray is obtained.

[0049] Conventional methods and systems may include diagnosis of a specific lung disease, or may only aid in the diagnosis of a specific lung disease. The methods and systems of the present disclosure may accurately and consistently determine lung severity regardless of what lung disease is present in a chest x-ray.

[0050] While conventional methods for determining RALE scores are labor-intensive and inconsistent, the methods and systems of the present disclosure provide semi-supervised learning by combining interpretable deep learning density segmentation with RALE score calculation.

[0051] The method and system of the present disclosure can provide disease progression assessment and treatment response measurement over time. The method and system of the present disclosure can be used as a surrogate marker in clinical trials to achieve faster and more efficient trials. For example, the method and system of the present disclosure can provide quantitative chest x-ray assessment and treatment effect measurement directly for radiographic severity (such as changes in indicators from before treatment to after treatment) in pharmaceutical and / or therapeutic trials. Therefore, compared with conventional methods, the method and system of the present disclosure provides an alternative indicator of treatment response, thereby having stronger statistical power when testing treatment effects, which may lead to reduced sample size requirements and more efficient design of clinical trials, because reliable alternative endpoints can significantly save the cost and time of treatment testing.

[0052] The methods and systems of the present disclosure may be incorporated into an integrated delivery network.

[0053] The methods and systems of the present disclosure may be used in triage situations to provide real-time quantification of lung severity prior to diagnosing a specific lung disease.

[0054] Figure 7 2 is a block diagram schematically illustrating a system 200 for quantifying the severity of a lung disease according to the present disclosure. The system 200 may include an input system 205. The input system 205 may include any system capable of allowing a user to input and receive an image file of a chest x-ray, including but not limited to cloud storage, hospital PACS, computers, smart phones, networks, any x-ray system, etc. The image file of the chest x-ray may include a data file, an image, and / or patient data, wherein the data and / or the image may be formatted in any file format capable of storing a chest x-ray, including but not limited to DICOM, JPEG, TIGG, GIF, PNG, etc.

[0055] The system of the present disclosure can be incorporated into clinical workflows and can be integrated into conventional or future visualization software such as CERNER. The system of the present disclosure can be incorporated into a chest x-ray acquisition system to determine the overall RALE score and / or RALE score components in real time once a chest x-ray is obtained.

[0056] System 200 may include database 210. Database 210 may include at least two reference image files of chest x-rays. The reference image files may include chest x-rays having an overall RALE score and / or RALE score components ( Figure 4 ). The database may include at least two reference image files of chest x-rays, such as at least 2, 100, 1000, 2000, 4000, 6000, 8000, 10000, 50000, and at least 100000 reference image files of chest x-rays. The reference image files may include chest x-rays annotated by a physician or radiologist.

[0057] System 200 may include one or more network and / or communication interfaces 215 for communicating with various computing entities, such as by communicating data, content, information and / or similar terms used interchangeably herein that may be transmitted, received, manipulated, processed, displayed, stored and / or the like.

[0058] The system 200 may include a processor 220 interfaced with a memory 225 (which may be separate from the processor 220 or included as part of the processor). The memory 225 may also employ cloud-based memory. In one aspect, the system may be connected to a base station including memory and processing capabilities. The system may further include an I / O device 230. According to the methods and systems of the present disclosure, the processor 220 may interface with a database 210.

[0059] The memory 225 stores some routines executable by the processor 220. The processor 220 in communication with the memory 225 may be configured to execute a first artificial intelligence system 235. The first artificial intelligence system 235 may include program instructions or computer program codes executable by the processor 220 to pre-process the chest x-ray image file provided by the input system 205. The pre-processing may be performed according to the method of the present disclosure.

[0060] The processor 220 in communication with the memory 225 may be configured to execute the second artificial intelligence system 240. According to the method of the present disclosure, the system 200 may include computer executable instructions to provide each segment of the lung to the second artificial intelligence system 240 to generate a first density score and a first degree score for each segment. According to the method of the present disclosure, the second artificial intelligence system 240 may be trained according to the database 210.

[0061] According to the method of the present disclosure, the memory 225 may further include computer executable instructions to calculate a first RALE score based on the first density score and the first degree score of each segment. According to the method and system of the present disclosure, the second artificial intelligence system 240 may include computer executable instructions executable by the processor 220 to generate and analyze the density segmentation map of each segment to determine the second density score and the second degree score of each segment to calculate the second RALE score of the image file.

[0062] According to the method of the present disclosure, the processor 220 in communication with the memory 225 can be configured to execute program instructions to analyze the first RALE score and the second RALE score to determine the difference and the overall RALE score. According to the method of the present disclosure, the processor 220 in communication with the memory 225 can be configured to execute program instructions to display the results to the user on the graphical user interface 245. The graphical user interface can include an output system, wherein the output system can include any system capable of receiving RALE score data, such as cloud storage, a computer, a medical device, a smart phone, a healthcare data system, and / or the like, wherein the RALE score data can include the overall RALE score, the first RALE score, the second RALE score, and / or at least one component of the first RALE score and the second RALE score.

[0063] The processor 220 may be one or more microprocessors, microcontrollers, application specific integrated circuits (ASICs), a circuit containing one or more processing components, a set of distributed processing components, circuitry for supporting a microprocessor, or other suitable processing device that interfaces with the memory 225. The processor 220 is also configured to execute computer code stored in the memory 225 to perform and facilitate the activities described herein.

[0064] System 200 may include I / O devices 255, which may be coupled to the system directly or through an intervening I / O controller, including but not limited to keyboards, displays, pointing devices, DASD, tapes, CDs, DVDs, thumb drives, and other storage media. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through an intervening private or public network. Modems, cable modems, and Ethernet cards may be just a few of the types of network adapters available.

[0065] As will be appreciated by those skilled in the art, the present disclosure may be embodied as a system, method, or computer program product. Thus, the present invention may take the form of a fully hardware embodiment, a fully software embodiment (including firmware, resident software, microcode, etc.), or an embodiment that combines all software and hardware aspects, which may generally be referred to herein as a "system." In addition, the present disclosure may take the form of a computer program product embodied in any tangible medium of expression having a computer-usable program code embodied in the medium. The computer program product of the present disclosure may include at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to perform the method of the present disclosure.

[0066] Figure 8 A schematic diagram of a computer network system of the present disclosure is depicted, the computer network system comprising a client computer 300 and a computer program 310, the computer program being configured to execute the method and system of the present disclosure. The client computer 300 may be any device capable of executing the computer program 310 of the present disclosure. The system may interface with at least one network server 320, wherein the at least one network server 320 may interface with the database 210 and the system.

[0067] Although the depicted system is shown and described herein as having certain components and functionality, other aspects of the system may be implemented with fewer or more components or with fewer or more functionality. Some aspects of the system may include multiple network servers 320, multiple networks, and multiple databases 210. Some aspects may include similar components arranged in another manner to provide similar functionality in one or more aspects.

[0068] The client computer 300 manages the interface between the system user and the computer program 310 and the network server 320. Although the present disclosure is described with respect to a "computer", it should be noted that any device optionally having a data processor and the ability to execute one or more instructions can be described as a computer, including but not limited to any type of personal computer (PC), server, distributed server, virtual server, cloud computing platform, cellular phone, IP phone, smart phone, mobile device or personal digital assistant (PDA). Any two or more of such devices that communicate with each other can optionally include a network or computer network.

[0069] For example, the network can carry out traditional block I / O communication through a storage area network (SAN). For example, the network can also use a transmission control protocol / internet protocol (TCP / IP) network or similar communication protocols to carry out file I / O communication. On the one hand, the storage system includes two or more networks. On the other hand, the client computer 300 is directly connected to the network server 320 through a backplane or a system bus. On the one hand, the network server 610 includes a cellular network, other similar types of networks or a combination thereof.

[0070] A computing system may include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by means of computer programs running on respective computers and having a client-server relationship with each other. In some aspects, the server transmits data (e.g., an HTML page) to a user device, for example, for the purpose of displaying data to a user interacting with a device used as a client and receiving user input from the user. Data generated at the user device, such as the result of a user interaction, can be received from the device at the server.

[0071] Any combination of one or more computer usable media or computer readable media may be utilized. Computer usable media or computer readable media may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, apparatuses or propagation media. Computer readable media may also be an electrical connection with one or more conductors, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CDROM), an optical storage device, a transmission medium (such as those supporting the Internet or an intranet), a magnetic storage device, a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or a raised structure in a groove on which instructions are recorded), and any suitable combination of the foregoing. Note that the computer usable medium or computer readable medium may be a paper or another suitable medium on which the program is printed, because the program may be electronically captured, for example, by optical scanning of the paper or other medium, and then compiled, interpreted or otherwise processed in a suitable manner as necessary, and then stored in a computer memory. In the context of this document, a computer-usable medium or computer-readable medium may be any medium that can contain, store, communicate, propagate or transport the program for use by or in connection with an instruction execution system, device or apparatus. Computer-usable program code may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc.

[0072] The computer program code for performing the operation of the present invention disclosed by the present invention can be written in any combination of one or more programming languages.Programming language can be but not limited to object-oriented programming language (Java, Smalltalk, C++, etc.) or conventional procedural programming language (" C " programming language, etc.).Program code can be executed completely on user's computer, partly on user's computer, partly on user's computer and partly on remote computer as independent software package, or completely on remote computer or server.In the latter scenario, remote computer can be connected to user's computer through any type of network (including local area network (LAN) or wide area network (WAN)), or can be connected to external computer, and the connection can include utilizing internet service provider to carry out through internet.In certain embodiments, electronic circuit system including for example programmable logic circuit system, field programmable gate array (FPGA) or programmable logic array (PLA) can execute computer-readable program instruction by utilizing the state information of computer-readable program instruction to personalize electronic circuit system, so as to perform various aspects of the present disclosure.

[0073] The system and method of the present disclosure can process data on any commercially available computer. In other aspects, the computer operating system may include but is not limited to Linux, Windows, UNIX, Android, MAC OS, etc. In one aspect of the present disclosure, the aforementioned processing device or any other electronic computing platform can be used, the type of which is designed for electronic processing of digital data as disclosed herein.

[0074] Various aspects of the present disclosure are described with reference to the flowcharts and / or block diagrams of the methods, systems and computer program products according to various aspects of the present disclosure. It will be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and the instructions are executed by the processor of the computer or other programmable data processing device, thereby allowing the implementation of the flowchart and / or block diagram frame or the steps specified in the frame.

[0075] Various embodiments of the present disclosure may be implemented in a data processing system suitable for storing and / or executing program code, the data processing system comprising at least one processor coupled directly or indirectly to a memory element via a system bus. The memory element comprises, for example, a local memory, a bulk memory, and a cache memory used during the actual execution of the program code, which provides temporary storage of at least some program code in order to reduce the number of times the code must be retrieved from the bulk memory during execution.

[0076] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the corresponding computing / processing device.

[0077] A code segment or machine executable instruction may represent a procedure, function, subroutine, program, routine, subroutine, module, software package, class or any combination of instructions, data structures or program statements. A code segment may be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, independent variables, parameters or memory contents. Information, independent variables, parameters, data, etc. may be transmitted, forwarded or transmitted by any suitable means (including memory sharing, message passing, token passing, network transmission, etc.).

[0078] definition

[0079] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. Thus, unless explicitly defined as such herein, terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense.

[0080] As used herein, the term "lung disease" may include any problem in the lungs that prevents the lungs from functioning properly, including but not limited to COVID-19, acute respiratory distress syndrome (ARDS), pneumonia, respiratory syncytial virus (RSV), etc.

[0081] As used herein, the term "user" refers to any person, entity, company, individual, institution, medical provider, medical facility, physician, physician assistant, nurse, nurse practitioner, doctor, patient, etc. who is capable of utilizing the methods and systems of the present disclosure.

[0082] As used herein, the term "patient" refers to any animal or human capable of receiving medical assistance or diagnosis according to the methods and systems of the present disclosure.

[0083] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Likewise, as used in the detailed description below, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the natural inclusive permutations. Thus, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances.

[0084] The terminology used herein is for the purpose of describing specific examples only and is not intended to be limiting. As used herein, the singular forms "a / an" and "the" may be intended to include the plural forms as well, unless the context clearly indicates otherwise. For example, "an" image file may include one or more image files, etc.

[0085] The terms "comprises," "comprising," "including," "having," and "characterized by" may be inclusive, and thus specify the presence of stated features, elements, compositions, steps, integers, operations, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. While these open-ended terms may be understood as non-limiting terms used to describe and claim the various aspects described herein, in certain aspects, the terms may alternatively be understood to substitute for more restrictive and limiting terms, such as "consisting of" or "consisting essentially of." Thus, for any given embodiment reciting a composition, material, component, element, feature, integer, operation, and / or process step, the embodiments described herein are also specifically included as consisting of or consisting essentially of such recited composition, material, component, element, feature, integer, operation, and / or process step. In the case of “consisting of,” alternative embodiments exclude any additional compositions, materials, components, elements, features, integers, operations and / or process steps, while in the case of “consisting essentially of,” any additional compositions, materials, components, elements, features, integers, operations and / or process steps that substantially affect the basic and novel properties may be excluded from such embodiments, but any compositions, materials, components, elements, features, integers, operations and / or process steps that do not substantially affect the basic and novel properties may be included in the embodiments.

[0086] Unless specifically identified as an order of performance, any method steps, processes, and operations described herein should not be construed as necessarily requiring their performance in the particular order discussed or illustrated. It should also be understood that additional or alternative steps may be employed unless otherwise indicated.

[0087] In addition, the features described in certain example embodiments can be combined in various other example embodiments or with the various other example embodiments described in any arrangement or combination. As disclosed herein, different aspects or elements of example embodiments can be combined in a similar manner. As used herein, the terms "combination", "combined" or "combinations thereof" refer to all arrangements and combinations of items listed before the term. For example, "A, B, C or a combination thereof" is intended to include at least one of: A, B, C, AB, AC, BC or ABC, and if the order is important in a particular context, BA, CA, CB, CBA, BCA, ACB, BAC or CAB are also included. Continuing this example, what is clearly included can be a combination containing repetitions of one or more items or terms, such as BB, AAA, AB, BBC, AAABCCCC, CBBAAA, CABABB, etc. It will be understood by the technician that, unless otherwise apparent from the context, the number of items or terms in any combination is generally not limited.

[0088] Various aspects of the present disclosure may be described herein with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to various aspects of the present disclosure.It will be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions.The various illustrative logic blocks, modules, circuits and algorithm steps described in the embodiments disclosed herein can be implemented as electronic hardware, computer software or a combination of the two.In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been generally described above in terms of their functions.Whether such functions are implemented as hardware or software depends on the specific application and design constraints imposed on the overall system.Technicians can implement the functions in different ways for each specific application, but such implementation decisions should not be interpreted as causing departure from the scope of the present disclosure.

[0089] The flow charts and block diagrams in the accompanying drawings show the architecture, function and operation of the possible implementation schemes of the system, method and computer program product according to various aspects of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a module, a segmentation or an instruction portion, which contains one or more executable instructions for implementing the specified logical function. In some alternative embodiments, the function marked in the frame can occur in a sequence different from that marked in the accompanying drawings. For example, in fact, the two frames shown in sequence can be executed substantially simultaneously, or the frames can be executed in reverse order sometimes, depending on the functions involved. It will also be noted that each frame in the block diagram and / or flow chart and the combination of frames in the block diagram and / or flow chart can be implemented by a system based on special hardware that performs a specific function or action or a combination of special hardware and computer instructions.

[0090] Words such as "then," "next," and the like are not intended to limit the order of the steps; these words may be used merely to guide the reader through the description of the method. Although a process flow diagram may describe the operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of the operations may be rearranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, and the like. When a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0091] In the description, certain details are set forth in order to provide a better understanding of the various embodiments of the systems and methods disclosed herein. However, those skilled in the art will appreciate that these embodiments may be practiced without these details and / or in the absence of any details not described herein. In other cases, well-known structures, methods, and / or techniques associated with the methods of practicing the various embodiments may not be shown or described in detail to avoid unnecessarily obscuring the description of other details of the various embodiments.

[0092] Although specific aspects of the present disclosure have been provided above, the present disclosure can be embodied in many different forms and should not be interpreted as necessarily being limited to the embodiments disclosed herein. On the contrary, these embodiments can be provided so that the present disclosure is comprehensive and complete and fully conveys the various concepts of the present disclosure to skilled technicians.

[0093] In addition, when the present disclosure states that something is "based on" other things, such statements mean that it may also be based on one or more other things. In other words, unless otherwise explicitly indicated, as used herein, "based on" inclusively means "at least partially based on" or "at least partially based on".

[0094] Unless otherwise stated, all numerical values ​​stated herein may be approximate. Therefore, the term "about" may be inferred when not explicitly stated. The numerical values ​​disclosed herein may be understood to be not strictly limited to the precise numerical values ​​stated. On the contrary, unless otherwise stated, each numerical value stated herein is intended to mean both the stated value and the functionally equivalent range around the value. At least, and without attempting to limit the application of the principle of equivalents to the scope of the claims, each numerical value should be interpreted at least according to the number of significant figures reported, and by applying conventional rounding methods. Typical exemplary error degrees may be within 20%, 10% or 5% of a given value or value range. Alternatively, the term "about" refers to a value within an order of magnitude, which may be within 5 times or 2 times of a given value. Although approximate values ​​of numerical values ​​are stated herein, the numerical values ​​described in the specific examples of actual measured values ​​may be reported as accurately as possible. However, any numerical value inherently contains certain errors that are inevitably caused by the standard deviation found in its corresponding test measurement results.

[0095] All numerical ranges stated herein include all subranges incorporated therein. For example, a range of "1 to 10" or "1-10" is intended to include all subranges between the stated minimum value 1 and the stated maximum value 10, and includes the end values, because the disclosed numerical ranges can be continuous and include every value between the minimum and maximum values. Any maximum numerical limit stated herein is intended to include all smaller numerical limits. Any minimum numerical limit stated herein is intended to include all larger numerical limits.

[0096] The features or functions described with respect to certain example embodiments may be combined and sub-combined in various other example embodiments, and / or combined and sub-combined with the various other example embodiments. Moreover, as disclosed herein, different aspects and / or elements of the example embodiments may also be combined and sub-combined in a similar manner. Further, some example embodiments (whether individually and / or collectively) may be components of a larger system, in which other procedures may take precedence over and / or otherwise modify their application. Additionally, as disclosed herein, some steps may be required before, after, and / or concurrently with the example embodiments. Note that any and / or all methods and / or processes, at least as disclosed herein, may be performed at least in part in any manner by at least one entity or actor.

[0097] All documents cited herein may be incorporated herein by reference, but only to the extent that the incorporated material does not conflict with existing definitions, statements, or other documents set forth herein. To the extent that any meaning or definition of a term in this document conflicts with any meaning or definition of the same term in a document incorporated by reference, the meaning or definition assigned to the term in this document shall prevail. The citation of any document shall not be construed as an admission that it is prior art with respect to the present application.

[0098] Although specific embodiments have been illustrated and described, it will be apparent to those skilled in the art that various other changes and modifications may be made without departing from the spirit and scope of the invention. Those skilled in the art will recognize or be able to determine, using no more than routine experimentation, many equivalents to the specific apparatus and methods described herein, including substitutions, variations, additions, deletions, modifications, and replacements. Therefore, the present application, including the appended claims, is intended to cover all such changes and modifications that may be within the scope of the present application.

[0099] aspect

[0100] Aspect 1: A method for quantifying the severity of a lung disease, the method comprising: providing an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and performing segmentation of the image to define lung boundaries, and wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on a database comprising at least two reference image files of chest x-rays, wherein the at least two reference image files comprise at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and displaying the results to a user.

[0101] Aspect 2: The method according to claim 1, wherein the first artificial intelligence system is trained using pre-training data.

[0102] Aspect 3: The method according to any one of the preceding aspects, wherein the pre-training data comprises at least two image files of chest x-rays.

[0103] Aspect 4: The method according to any one of the preceding aspects, wherein the pre-training data comprises a discovery or no discovery.

[0104] Aspect 5: A method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

[0105] Aspect 6: A method according to any of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the first density score and the first extent score of at least a segment are added to the database and used to continuously train the second artificial intelligence system.

[0106] Aspect 7: A method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the second density score and the second extent score of the image file and at least one segment are added to the database and used to continuously train the second artificial intelligence system.

[0107] Aspect 8: The method according to any of the preceding aspects, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is an overall RALE score assigned by a physician.

[0108] Aspect 9: The method according to any of the preceding aspects, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is at least one density score and at least one extent score assigned by a physician.

[0109] Aspect 10: The method according to any one of the preceding aspects, wherein displaying the results to a user comprises displaying the density segmentation map of each segment on a graphical user interface.

[0110] Aspect 11: The method according to any of the preceding aspects, wherein displaying the result to a user comprises displaying the overall RALE score to the user on a graphical user interface.

[0111] Aspect 12: The method according to any of the preceding aspects, wherein displaying the results to a user comprises displaying at least one first degree score and at least one first density score for at least one segment to the user on a graphical user interface.

[0112] Aspect 13: The method according to any of the preceding aspects, wherein displaying the results to a user comprises displaying at least one second degree score and at least one second density score for at least one segment to the user on a graphical user interface.

[0113] Aspect 14: The method according to any of the preceding aspects, wherein displaying the results to a user comprises displaying the density segmentation map of each segment as a 3-dimensional and / or 2-dimensional representation of the lung on a graphical user interface.

[0114] Aspect 15: The method according to any of the preceding aspects, wherein the method has a RALE score confidence of at least 0.9.

[0115] Aspect 16: A method according to any one of the preceding aspects, wherein the first RALE score and the second RALE score are inconsistent, and wherein the image file is annotated by a physician to generate an annotated image file, wherein the annotated image file is added to the database to train the second artificial intelligence system.

[0116] Aspect 17: The method according to any one of the preceding aspects, wherein the first artificial intelligence system comprises at least one computer vision algorithm.

[0117] Aspect 18: A method according to any of the preceding aspects, wherein the artificial intelligence system comprises at least one deep learning network.

[0118] Aspect 19: A method according to any one of the preceding aspects, wherein the first artificial intelligence system comprises at least one computer vision algorithm and at least one deep learning network.

[0119] Aspect 20: The method according to any of the preceding aspects, wherein the first artificial intelligence system comprises a segment module and a lung segmentation module.

[0120] Aspect 21: A method according to any of the preceding aspects, wherein the second artificial intelligence system comprises at least one computer vision algorithm.

[0121] Aspect 22: A method according to any one of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network.

[0122] Aspect 23: A method according to any one of the preceding aspects, wherein the second artificial intelligence system comprises at least one computer vision algorithm and at least one deep learning network.

[0123] Aspect 24: The method according to any one of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network module.

[0124] Aspect 25: The method according to any one of the preceding aspects, wherein the at least one deep learning network module comprises at least one computer vision algorithm and / or at least one deep learning network.

[0125] Aspect 26: The method according to any one of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network module for each segment.

[0126] Aspect 27: A method according to any one of the preceding aspects, wherein the deep learning network comprises at least one neural network.

[0127] Aspect 28: A method according to any one of the preceding aspects, wherein the at least one neural network comprises a plurality of layers.

[0128] Aspect 29: A system for quantifying the severity of a lung disease, the system comprising: a database having at least two reference images of a chest x-ray; at least one processor; at least one communication interface; a user interface; and at least one memory, the at least one memory comprising computer program code, the at least one memory and the computer program code being configured to store a first artificial intelligence system, a second artificial intelligence system, and computer executable instructions, the memory being further configured to execute the instructions using the processor, wherein the instructions comprise: providing an image file of a chest x-ray of a patient to the first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and performing segmentation of the image to define lung boundaries, and wherein the lung is divided into at least four segments ; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on a database having at least two reference images, wherein the at least two reference image files contain at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image file; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and displaying the result to a user on the user interface.

[0129] Aspect 30: The system of claim 29, wherein the first artificial intelligence system is trained using pre-training data.

[0130] Aspect 31: The system according to any one of the preceding aspects, wherein the pre-training data comprises at least two image files of chest x-rays.

[0131] Aspect 32: The system according to any of the preceding aspects, wherein the image file and the overall RALE score are added to the at least two reference image files to continuously train the second artificial intelligence system.

[0132] Aspect 33: The system according to any of the preceding aspects, wherein the pre-training data includes findings or no findings.

[0133] Aspect 34: A system according to any of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

[0134] Aspect 35: A system according to any of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the first density score and the first extent score of at least a segment are added to the database and used to continuously train the second artificial intelligence system.

[0135] Aspect 36: A system according to any of the preceding aspects, wherein the first RALE score and the second RALE score are consistent, and wherein the second density score and the second extent score of the image file and at least one segment are added to the database and used to continuously train the second artificial intelligence system.

[0136] Aspect 37: The system according to any of the preceding aspects, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is an overall RALE score assigned by a physician.

[0137] Aspect 38: The system according to any of the preceding aspects, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is at least one density score and at least one extent score assigned by a physician.

[0138] Aspect 39: The system according to any of the preceding aspects, wherein displaying the results to a user comprises displaying the density segmentation map of each segment on the user interface.

[0139] Aspect 40: The system according to any of the preceding aspects, wherein displaying the result to a user comprises displaying the overall RALE score to the user on the user interface.

[0140] Aspect 41: The system according to any of the preceding aspects, wherein displaying the results to a user comprises displaying at least one first extent score and at least one first density score for at least one segment to the user on the user interface.

[0141] Aspect 42: The system according to any of the preceding aspects, wherein displaying the results to a user comprises displaying at least one second degree score and at least one second density score for at least one segment to the user on the user interface.

[0142] Aspect 43: The system according to any of the preceding aspects, wherein displaying the results to a user comprises displaying the density segmentation map of each segment as a 3-dimensional and / or 2-dimensional representation of the lung on the user interface.

[0143] Aspect 44: The system according to any of the preceding aspects, wherein the method has a RALE score confidence of at least 0.9.

[0144] Aspect 45: A system according to any of the preceding aspects, wherein the first RALE score and the second RALE score are inconsistent, and wherein the image file is annotated by a physician to generate an annotated image file, wherein the annotated image file is added to the database to train the second artificial intelligence system.

[0145] Aspect 46: The system according to any of the preceding aspects, wherein the first artificial intelligence system comprises at least one computer vision algorithm.

[0146] Aspect 47: A system according to any of the preceding aspects, wherein the artificial intelligence system comprises at least one deep learning network.

[0147] Aspect 48: A system according to any of the preceding aspects, wherein the first artificial intelligence system comprises at least one computer vision algorithm and at least one deep learning network.

[0148] Aspect 49: The system according to any of the preceding aspects, wherein the first artificial intelligence system comprises a segment module and a lung segmentation module.

[0149] Aspect 50: The system according to any of the preceding aspects, wherein the second artificial intelligence system comprises at least one computer vision algorithm.

[0150] Aspect 51: A system according to any of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network.

[0151] Aspect 52: A system according to any of the preceding aspects, wherein the second artificial intelligence system comprises at least one computer vision algorithm and at least one deep learning network.

[0152] Aspect 53: A system according to any one of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network module.

[0153] Aspect 54: The system according to any of the preceding aspects, wherein the at least one deep learning network module comprises at least one computer vision algorithm and / or at least one deep learning network.

[0154] Aspect 55: The system according to any of the preceding aspects, wherein the second artificial intelligence system comprises at least one deep learning network module for each segment.

[0155] Aspect 56: The system according to any of the preceding aspects, wherein the deep learning network comprises at least one neural network.

[0156] Aspect 57: The system according to any of the preceding aspects, wherein the at least one neural network comprises a plurality of layers.

[0157] Aspect 58: A computer program product for quantifying the severity of a lung disease, the computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising program instructions, the program instructions, when executed by at least one processor, causing the at least one processor to: provide an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and performing segmentation of the image to define lung boundaries, and wherein the lung is divided into at least four segments; provide each segment to a second artificial intelligence system to generate a first density score and a first severity score for each segment, wherein the second artificial intelligence system is based on The second artificial intelligence system is trained on a database comprising at least two reference image files of chest x-rays, wherein the at least two reference image files comprise at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculates a first RALE score based on the first density score and the first extent score for each segment; analyzes the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image file; analyzes the first RALE score and the second RALE score to determine a difference and an overall RALE score; and displays the result to the user on the user interface, wherein the result comprises the overall RALE score.

[0158] Aspect 59: A computer program product according to any one of the preceding aspects.

[0159] Aspect 60: The method, system, and computer program product according to any of the preceding aspects, wherein the first RALE score comprises eight components.

[0160] Aspect 61: The method, system, and computer program product according to any of the preceding aspects, wherein the second RALE score comprises eight components.

[0161] Examples

[0162] Example 1

[0163] The systems and methods of the present disclosure analyzed chest x-ray images of 595 patients with COVID-19 while admitted to the ICU or hospital at the University of Pittsburgh Medical Center facility. After training by a senior reviewer, eight physicians of varying training levels scored an independent set of chest x-rays, followed by feedback on the score distributions, and then independently rescored using inter-reviewer correlation and intraclass correlation coefficients (ICCs) in a two-way random effects model, as well as the kappa statistic for categorical variables. After pre-training on the MMIC-CXR dataset using the "no findings" or "clinical pathology observation" classification, the methods and systems of the present disclosure were trained using the nearly consistent average RALE scores of the two reviewers.

[0164] Interrater agreement for overall RALE scores was found (correlation R=0.71, p<0.0001; ICC=0.84m95% confidence interval [0.82-0.89]mp<0.0001), which improved the overall RALE scores of the disclosed methods and systems with excellent agreement of the reviewers' overall RALE scores (correlation R=0.88, p<0.0001; ICC=0.91[0.88-0.94], p<0.0001), with <1% of CXRs showing large RALE differences (differences of ≥15 points). Reviewers reached moderate agreement on image quality (κ=0.6) and substantial agreement on the presence of atelectasis (κ=0.21). Median RALE scores were higher for CXRs with poor fluoroscopy compared to images with good fluoroscopy (p<0.01), and the presence of atelectasis was associated with higher mean right lower quadrant density scores. Then, a deep learning network was trained with the RALE score annotations, and a Spearman correlation R = 0.87 (p = 1.7 x 10-7) was obtained between the predicted RALE scores and the physician-annotated RALE scores.

[0165] The systems and methods of the present disclosure are demonstrated to reliably, accurately, and rapidly predict the RALE score of chest x-rays.

Claims

1. A method for quantifying the severity of a lung disease, the method comprising: providing an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and segmenting the image to define lung boundaries, and wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on a database comprising at least two reference image files of chest x-rays, wherein the at least two reference image files comprise at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map of each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and Display the results to the user.

2. The method of claim 1 , wherein the first artificial intelligence system is trained using pre-training data.

3. The method of claim 2, wherein the pre-training data comprises at least two image files of chest x-rays. The method according to claim 2 , wherein the pre-training data comprises a discovery or no discovery.

5. The method of claim 1 , wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

6. The method of claim 1 , wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the first density score and the first extent score of at least a segment are added to the database and used to continuously train the second artificial intelligence system.

7. The method of claim 1 , wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the second density score and the second extent score of at least one segment are added to the database and used to continuously train the second artificial intelligence system.

8. The method of claim 1, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is an overall RALE score assigned by a physician.

9. The method of claim 1, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is at least one density score and at least one extent score assigned by a physician.

10. The method of claim 1, wherein displaying the results to a user comprises displaying the density segmentation map for each segment on a graphical user interface.

11. The method of claim 1 , wherein displaying the results to a user comprises displaying the overall RALE score to the user on a graphical user interface.

12. The method of claim 1, wherein displaying the results to a user comprises displaying at least one first extent score and at least one first density score for at least one segment to the user on a graphical user interface.

13. The method of claim 1, wherein displaying the results to a user comprises displaying at least one second extent score and at least one second density score for at least one segment to the user on a graphical user interface.

14. The method of claim 1, wherein displaying the results to a user comprises displaying the density segmentation map of each segment as a 3-dimensional and / or 2-dimensional representation of the lung on a graphical user interface.

15. The method of claim 1, wherein the method has a RALE score confidence of at least 0.

9.

16. The method of claim 1, wherein the first RALE score and the second RALE score are inconsistent, and wherein the image file is annotated by a physician to generate an annotated image file, wherein the annotated image file is added to the database to train the second artificial intelligence system.

17. A system for quantifying the severity of a lung disease, the system comprising: a database having at least two reference images of a chest x-ray; at least one processor; at least one communication interface; a user interface; and at least one memory, the at least one memory including computer program code; the at least one memory and computer program code being configured to store a first artificial intelligence system, a second artificial intelligence system, and computer executable instructions, the memory being further configured to execute the instructions with the processor, wherein the instructions include: providing an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and segmenting the image to define lung boundaries, and wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on the database having at least two reference images, wherein the at least two reference image files contain at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image file; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and The results are displayed to the user on the user interface.

18. The system of claim 17, wherein the first artificial intelligence system is trained using pre-training data.

19. The system of claim 18, wherein the pre-training data comprises at least two image files of chest x-rays.

20. The system of claim 18, wherein the pre-training data comprises either a discovery or a no discovery.

21. The system of claim 17, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the overall RALE score are added to the database and used to continuously train the second artificial intelligence system.

22. The system of claim 17, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the first density score and the first extent score of at least a segment are added to the database and used to continuously train the second artificial intelligence system.

23. The system of claim 17, wherein the first RALE score and the second RALE score are consistent, and wherein the image file and the second density score and the second extent score for at least one segment are added to the database and used to continuously train the second artificial intelligence system.

24. The system of claim 17, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is an overall RALE score assigned by a physician.

25. The system of claim 17, wherein the database contains at least 2000 image files of chest x-rays, and wherein the at least one annotation is at least one density score and at least one extent score assigned by a physician.

26. The system of claim 17, wherein displaying the results to a user comprises displaying the density segmentation map for each segment on the user interface.

27. The system of claim 17, wherein displaying the results to a user comprises displaying the overall RALE score to the user on the user interface.

28. The system of claim 17, wherein displaying the results to a user comprises displaying at least one first extent score and at least one first density score for at least one segment to the user on the user interface.

29. The system of claim 17, wherein displaying the results to a user comprises displaying at least one second extent score and at least one second density score for at least one segment to the user on the user interface.

30. The system of claim 17, wherein displaying the results to a user comprises displaying the density segmentation map of each segment as a 3-dimensional and / or 2-dimensional representation of the lung on the user interface.

31. The system of claim 17, wherein the method has a RALE score confidence of at least 0.

9.

32. The system of claim 17, wherein the first RALE score and the second RALE score are inconsistent, and wherein the image file is annotated by a physician to generate an annotated image file, wherein the annotated image file is added to the database to train the second artificial intelligence system.

33. A computer program product for quantifying severity of a lung disease, the computer program product comprising at least one non-transitory computer readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: providing an image file of a chest x-ray of a patient to a first artificial intelligence system to pre-process the image, wherein the pre-processing comprises determining vertices and segmenting the image to define lung boundaries, and wherein the lung is divided into at least four segments; providing each segment to a second artificial intelligence system to generate a first density score and a first extent score for each segment, wherein the second artificial intelligence system is trained based on a database comprising at least two reference image files of chest x-rays, wherein the at least two reference image files comprise at least one annotation assigned by a physician, and wherein the second artificial intelligence system generates a density segmentation map for each segment; calculating a first RALE score based on the first density score and the first extent score for each segment; analyzing the density segmentation map for each segment to determine a second density score and a second extent score for each segment to calculate a second RALE score for the image file; analyzing the first RALE score and the second RALE score to determine a difference and an overall RALE score; and The results are displayed to the user on the user interface.

Images