Lesion-aware thoracic x-ray synthesis for improved thoracic disease detection

Abstract

Techniques are described for enhancing thoracic X-ray (CXR) images with synthetic lesions by using a lesion aware machine learning framework in conjunction with optimization of a thoracic disease detection model. In an example, a system may include a lesion enhancement component that generates a synthetic lesion image including a synthetic lesion image data object integrated on or within a medical image using a lesion generator trained to generate the synthetic lesion image data object, and editing the synthetic lesion image data object to account for different types of lesions and different anatomical locations of the lesions. The system may also include a training component that uses the synthetic lesion image to train a lesion detector. The training component may further execute an alternating training strategy to integrate training of the lesion generator and the detector for mutual promotion.

Description

Technical Field

[0001] This application relates to a technique for enhancing chest X-ray (CXR) images using synthetic lesions in conjunction with an optimized chest disease detection model with a lesion-aware machine learning framework. Background Art

[0002] Due to its advantages of cost-effectiveness and low-dose radiation, chest X-ray (CXR) is the most common examination method for screening chest diseases. The increasing number of CXR examinations has imposed a heavy workload on radiologists. In addition, due to the complexity of chest anatomy and the subtle changes in the lesion area, CXR interpretation can be challenging even for experienced radiologists.

[0003] With the development of deep learning technology, computer-aided diagnosis (CAD) of chest diseases has made great progress, promising to relieve the burden on radiologists. However, training robust models for disease diagnosis and lesion localization requires a large number of samples with fine-grained labels, which are difficult to obtain due to privacy issues and expensive labeling costs. Summary of the Invention

[0004] The following presents an overview of the invention to provide a basic understanding of one or more embodiments of the invention. This summary of the invention is not intended to identify key or important elements or to delineate any scope of different embodiments or any scope of the claims. Its sole purpose is to present the concepts of the invention in a simplified form as a prelude to the more detailed embodiments presented later. In one or more embodiments, a system, computer-implemented method, apparatus, and / or computer program product are described for facilitating the use of synthetic lesions to enhance chest X-ray (CXR) images using a lesion-aware machine learning (ML) framework associated with an optimized chest disease detection model. In some embodiments, the techniques disclosed in this application can be applied to other types of lesions associated with other types of diseases and body anatomical regions. The techniques disclosed in this application can also be extended to other medical imaging modalities (e.g., magnetic resonance imaging (MRI), computed tomography (CT), and others).

[0005] According to an embodiment, a system is provided that includes a memory and a processor, the memory storing computer-executable components, and the processor executing the computer-executable components stored in the memory. The computer-executable components may include a lesion enhancement component that uses a synthetic lesion generation model to generate synthetic lesion images including synthetic lesion image data objects integrated on or within medical images, where the model is trained to generate synthetic lesion image data objects and clip the synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions. In one or more embodiments, different types of lesions correspond to different types of chest diseases and / or lesions associated with different types of chest diseases (e.g., masses, nodules, pneumonia lesions, tuberculosis lesions, fractures, etc.). In various embodiments, the medical images include CXR images.

[0006] In one or more embodiments, the lesion enhancement component adds the synthetic lesion images to a lesion image training dataset including lesion images, the lesion images including enhanced medical images, and where the computer-executable components further include a training component that employs the lesion image training dataset to train a lesion detection model to detect different types of lesions in the lesion images. In various embodiments, the lesion enhancement component generates the synthetic lesion images in association with the receipt of annotation data for integration on or within medical images, the annotation data indicating a defined disease type, defined anatomical location, and defined size of the corresponding object of the synthetic lesion image data object, and where the training component employs the annotation data correspondingly associated with the synthetic lesion images as ground truth (GT) information associated with training the lesion detection model. The computer-executable components may further include a performance evaluation component that identifies one or more target lesion images of the lesion image training dataset associated with negative performance criteria of the lesion detection model, and where the training component updates the synthetic lesion generation model based on the one or more target lesion images.

[0007] In this regard, the training component may also use one or more machine learning processes to train the synthetic lesion generation model to clip the synthetic lesion data objects to account for different types of lesions and the anatomical locations of the lesions. In some embodiments, the synthetic lesion generation model may also clip the synthetic lesions to account for different sizes and textures of the lesions. In some embodiments, the one or more machine learning processes include an adversarial training process employing a lesion generator network and a discriminator network, where the lesion generator network includes convolutional layers and transformers. In some implementations, the synthetic lesion generation model may include a style variation module that uses noise injection to generate different style variations of the synthetic lesion image data objects.

[0008] In some embodiments, the various elements described in connection with the systems disclosed herein may be implemented in different forms such as computer-implemented methods, computer program products, or other forms. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 An example, non-limiting computing system is presented that facilitates the use of synthetic lesions to enhance CXR images and employs enhanced images associated with optimizing a chest disease detection model, in accordance with one or more embodiments of the subject matter of this application.

[0010] Figure 2 Example CXRs of different types of lesions associated with different types of chest diseases are presented.

[0011] Figure 3 An example process for training a synthetic lesion generation model is presented, in accordance with one or more embodiments of the subject matter of this application.

[0012] Figure 4 An example synthetic lesion generation model is presented, in accordance with one or more embodiments of the subject matter of this application.

[0013] Figure 5 Example components of an example synthetic lesion generation model are illustrated, in accordance with one or more embodiments of the subject matter of this application.

[0014] Figure 6A and Figure 6B A table is presented that illustrates example synthetic lesion images generated by a synthetic lesion generation model, in accordance with one or more embodiments of the subject matter of this application.

[0015] Figure 7 Examples of different types of styled synthetic lesion objects that can be generated by a synthetic lesion generation model via a style variation component are presented, in accordance with one or more embodiments of the subject matter of this application.

[0016] Figure 8A and Figure 8B A flowchart of an example process for training a lesion detector and a lesion generator is presented, in accordance with one or more embodiments of the subject matter of this application.

[0017] Figure 9A and Figure 9B A flowchart of another example process for training a lesion detector and a lesion generator is presented, in accordance with one or more embodiments of the subject matter of this application.

[0018] Figure 10 A high-level illustration of an alternating training framework for training a lesion generator and a lesion detector for mutual improvement is presented, in accordance with one or more embodiments of the subject matter of this application.

[0019] Figure 11 Illustrated is a block diagram of an example, non-limiting computer-implemented method for enhancing CXR images with synthetic lesions in accordance with one or more embodiments of the presently disclosed subject matter.

[0020] Figure 12 Illustrated is a block diagram of another example, non-limiting computer-implemented method for enhancing CXR images with synthetic lesions in accordance with one or more embodiments of the presently disclosed subject matter.

[0021] Figure 13 Illustrated is a block diagram of an example, non-limiting computer-implemented method for enhancing CXR images with synthetic lesions and employing the enhanced images in association with an optimized chest disease detection model in accordance with one or more embodiments of the presently disclosed subject matter.

[0022] Figure 14 Illustrated is a block diagram of an example, non-limiting operating environment that can facilitate one or more embodiments described herein.

[0023] Figure 15 Illustrated is a block diagram of another example, non-limiting operating environment that can facilitate one or more embodiments described herein. DETAILED DESCRIPTION

[0024] The following specific embodiments are merely illustrative and are not intended to limit the application or use of the embodiments and / or the embodiments. In addition, it is not intended to be bound by any express or implied information presented in the aforementioned background technology section, the invention summary section or the specific implementation section.

[0025] The subject disclosure provides systems, computer-implemented methods, apparatus, and / or computer program products that are described for facilitating enhancement of chest X-ray (CXR) images with synthetic lesions using a lesion-aware machine learning (ML) framework associated with an optimized chest disease detection model.

[0026] As described in the background art section, training robust models for disease diagnosis and lesion localization requires a large number of samples with fine-grained labels, which are difficult to obtain due to privacy issues and expensive labeling costs. The technology disclosed in this application addresses this problem by providing a CXR synthesis framework for data augmentation associated with improving the performance of a chest disease detection model. The CXR framework is capable of synthesizing different types of lesions associated with different types of chest diseases at a given location in a normal CXR image, which provides additional training data with bounding box annotations of the lesions. To enable realistic high-quality lesions, adversarial training is performed between a lesion generator and a discriminator, where the generator is prompted to generate lesions indistinguishable from real lesions. To cope with the large variations in size, location, and texture of different lesions, the generator utilizes both convolutional layers and transformers to generate not only fine details locally but also reasonable structures globally. Additionally, the generator is equipped with a style variation module to diversify the synthesized styles via noise injection.

[0027] Furthermore, the technology disclosed in this application explicitly integrates the training of the lesion generator and the detector into the same framework to form a mutually enhancing loop. In this regard, in one or more embodiments, the training of the lesion generator and the detector can be performed in an alternating manner as follows: 1) When optimizing the generator, the trained detector is used to filter simple samples based on their prediction confidence to prompt the synthesis of difficult samples; 2) On the other hand, the trained generator provides the synthesized lesions to the CXR as additional training data for the detection model, which improves the generalization ability of the detector. Thus, these two modes enhance each other's performance in an alternating training manner.

[0028] The effectiveness of the framework proposed in this application has been verified on both public and private data for lesion detection, which covers seven diseases such as pneumonia, nodules, and tuberculosis.

[0029] In some embodiments, the technology disclosed in this application can be applied to other types of lesions associated with other types of diseases and body anatomical regions. The technology disclosed in this application can also be extended to other medical imaging modalities (e.g., magnetic resonance imaging (MRI), computed tomography (CT), etc.).

[0030] The term "medical image" is used to refer to image data depicting one or more anatomical regions of a patient. The medical images or medical image data referred to herein can include any type of medical image associated with various types of medical image acquisition / capture modalities. For example, medical images can include (but are not limited to): radiotherapy (RT) images, X-ray (XR) images, digital radiography (DX) X-ray images, X-ray angiography (XA) images, panoramic X-ray (PX) images, computed tomography (CT) images, mammography (MG) images (including tomosynthesis devices), magnetic resonance imaging (MRI) images, ultrasound (US) images, color flow Doppler (CD) images, positron emission tomography (PET) images, single photon emission computed tomography (SPECT) images, nuclear medicine (NM) images, and similar images.

[0031] Medical images can also include synthetic versions of the native medical images, such as enhanced, modified, or strengthened versions of the native medical images, enhanced versions of the native medical images, and similar versions generated using one or more image processing techniques. In this regard, the term "native" image or "real" image is used herein to refer to the image in its initial captured form and / or received form before being processed by one or more medical image inference models. The term "synthetic" image is used herein to distinguish from the native image or real image, and refers to an image generated or derived from the native image or real image using one or more synthetic image processing techniques (e.g., synthetic lesion object generation). In some embodiments, the term "image data" can include the original measurement data (or simulated measurement data) used to generate the medical image (e.g., the original measurement data captured via a medical image acquisition process).

[0032] The terms "algorithm" and "model" are used interchangeably herein, unless the context warrants a particular distinction between the terms. The terms "artificial intelligence (AI) model" and "machine learning (ML) model" are used interchangeably herein, unless the context warrants a particular distinction between the terms. The AI or ML models referred to herein can include any type of AI or ML model, including (but not limited to): deep learning models, neural network models, deep neural network models (DNN), convolutional neural network models (CNN), generative adversarial neural network models (GAN), etc. The AI or ML models can include supervised learning models, unsupervised learning models, semi-supervised learning models, combinations thereof, and models employing other types of ML learning techniques. The AI or ML models can include a single model or a group containing two or more models (e.g., an enable model or a similar model).

[0033] Reference is now made to the accompanying drawings, in which like reference numerals are used throughout the specification to refer to like elements. In the following description, numerous specific details are set forth for purposes of illustration in order to provide a more thorough understanding of one or more embodiments. It will be apparent, however, that the one or more embodiments may be practiced without these specific details in different circumstances.

[0034] Turning now to the drawings, Figure 1 presented is a non-limiting computing system 100 that facilitates the use of synthetic lesions to enhance CXR images and employs enhanced images that are examples associated with the optimization of a chest disease detection model, in accordance with one or more embodiments of the subject matter of this application.

[0035] Embodiments of the systems and devices described herein may include one or more machine-executable (i.e., computer-executable) components or instructions embodied within one or more machines (e.g., embodied in one or more computer-readable storage media associated with one or more machines). When executed by one or more machines (e.g., processors, computers, computing devices, virtual machines, etc.), such components may cause the one or more machines to perform the described operations. These computer / machine-executable components or instructions (and other components or instructions described herein) may be stored in a memory associated with one or more machines. The memory may also be operatively coupled to at least one processor such that the components may be executed by the at least one processor to perform the described operations. In some embodiments, the memory may include a non-transitory machine-readable medium containing executable components or instructions that, when executed by a processor, facilitate the execution of the operations described for the corresponding executable components. Examples of such memory and processor and other suitable computer or computing-based elements may be referenced Figure 14 found (e.g., processing unit 1404 and system memory 1406, respectively), and may be used in combination to implement one or more systems or components shown and described in conjunction with Figure 1 or other figures disclosed herein.

[0036] In this regard, in one or more embodiments, the computing system 100 may include (or be operatively coupled to) at least one memory 122 for storing computer-executable components and at least one processor (e.g., processing unit 124) for executing the computer-executable components stored in the at least one memory 122. The computer-executable components may include (but are not limited to) a lesion enhancement component 102, a lesion detection component 104, a performance evaluation component 106, and a training component 108. The memory 122 may also include (e.g., store) a model library 114, training data 110, and runtime data 112. Additionally or alternatively, the model library 114, training data 110, and / or runtime data 112 may be associated with one or more additional information storage structures (e.g., a transient memory device, a non-transient memory device, or a similar device) that may be directly or via one or more wired or wireless communication networks coupled to the computing system 100.

[0037] The model library 110 may include one or more models (e.g., ML / AI models and / or other types of models or algorithms) employed by the computing system 100, including both an untrained version of the model and a trained version of the model. These models may include (but are not limited to) a synthetic lesion generation model 116 and a lesion detection model 118. The training data 110 may include training data used by the training component 108 to train and / or retrain or update the synthetic lesion generation model 116 and the lesion detection model 118. The runtime data 114 may include runtime data (or test data) processed by the trained versions of the synthetic lesion generation model 116 and the lesion detection model 118 after at least some training has been completed.

[0038] The computing system 100 may also include one or more input / output devices 126 to facilitate receiving user input associated with training and / or updating the synthetic lesion generation model 116 and / or one or more lesion detection models 118, and / or applying the trained versions of the corresponding models to the corresponding runtime data 114. In this regard, any information received, generated, and / or accessible by the computing system 100 (e.g., training data 112, runtime data 114, synthetic lesion objects, synthetic lesion images including synthetic lesion objects, annotation data, lesion detection model output results, user feedback, etc.) may be presented or submitted to the user via a suitable output device such as a display, a speaker, etc. according to the data format. Refer to Figure 14 Suitable examples of the input / output device 122 are described (e.g., input device 1428 and output device 1436). The computing system 100 may also include a system bus 116 that couples the memory 118, the processing unit 120, and the input / output device 122 to each other.

[0039] In one or more embodiments, the lesion enhancement component 102 may use a synthetic lesion generation model 116 to generate a synthetic lesion image including a synthetic lesion image data object integrated on or within a medical image, wherein the synthetic lesion generation model 116 includes a lesion generator model (e.g., lesion generator 306) that is trained to generate synthetic lesion image data objects and clip the synthetic lesion image data objects to account for different types of lesions, different anatomical locations of the lesions, different sizes of the lesions, and different textures of the lesions. In various embodiments, the medical image and the synthetic lesion image comprise or correspond to a medical image depicting a chest region (i.e., thorax) of a human subject, and the different types of lesions correspond to different types of chest diseases or disorders. In some embodiments, the modality of the medical image and the synthetic lesion image (and the synthetic lesion image data object) is XR (e.g., the medical image corresponds to a CXR). In other embodiments, the modality of the medical image, the synthetic lesion image (and the synthetic lesion image data object) may be CT, MRI, or other medical imaging modalities.

[0040] Reference Figure 2 and in conjunction with Figure 1 , Figure 2 FIG. presents example CXRs having different types of lesions (e.g., mass-like lesions, nodule-like lesions, pneumonia-like lesions, and tuberculosis-like lesions) associated with different types of chest diseases or disorders. The example CXRs are real or native medical images depicting real lesions (as opposed to synthetic lesions generated by the synthetic lesion generation model 116). The corresponding lesions are indicated via rectangular bounding boxes overlaid on the respective CXRs. In this regard, the term "lesion" is used herein to refer to a defined anatomical region of abnormal or altered tissue due to disease or injury. Although four different types of lesions are shown in Figure 1 , the techniques disclosed in this application may be applied to various other types of lesions associated with various additional or alternative chest diseases and disorders (e.g., pneumothorax, pleural effusion, fracture, and other diseases).

[0041] As Figure 1 shown, the different types of lesions associated with different types of chest diseases / disorders are different in location (i.e., anatomical location relative to one or more anatomical structures of the chest region of the body), size (e.g., where the size of the corresponding lesion corresponds to the size of the corresponding bounding box illustrated in Figure 2 ), texture, and appearance. According to one or more embodiments, the synthetic lesion generation model 116 may be configured to generate Figure 2A synthetic version of the lesions shown in [figure] and other types of chest disease / condition lesions: enabling the model to account for the variability in the type (i.e., lesion / disease type), location, size, and texture of different types of lesions associated with different types of chest diseases / conditions. This task requires the synthetic lesion generation model 116 to be powerful in both capturing local details and global plausibility, as generating high-quality lesions for different diseases is challenging due to the large differences.

[0042] For this purpose, in one or more embodiments, the synthetic lesion generation model 116 can be configured to generate a synthetic version of only the lesion itself and / or a portion of the image including the lesion, rather than generating the entire image. The synthetic lesion is further integrated onto or within a normal medical image without a lesion (e.g., a normal CXR) to generate a synthetic lesion image including the synthetic image. For example, the synthetic lesion can correspondingly correspond to an image data object that can be overlaid onto the normal image at a specified location on or within the normal image to generate a synthetic lesion image including the lesion at the specified location. In this way, the computing system 100 can utilize a large number of normal medical images (e.g., normal CXRs), while focusing on lesion synthesis.

[0043] In some implementations of these embodiments, the synthetic lesion generation model 116 can be configured to combine the synthetic lesion image data object with the normal image to generate a synthetic lesion image. In other implementations, the synthetic lesion generation model 116 can be configured to output the synthetic lesion image data object, and the lesion enhancement component 102 can overlay the synthetic lesion image data object at a specified location on the normal image to generate a synthetic lesion image. In either of these cases, the synthetic lesion image data object generated by the synthetic lesion generation model 116 is clipped to account for a specific lesion type among a variety of different lesion types and the specific anatomical location of the lesion. In this regard, once the synthetic lesion generation model 116 is trained (e.g., via the training component 108 discussed in more detail below), the input to the model can include a specified lesion type and a specified anatomical location for integrating the lesion onto or within a normal medical image.

[0044] For example, when applied to CXR, in various embodiments, the input to the synthetic lesion generation model 116 can include selecting a specific type of lesion from among a plurality of predefined types associated with different types of chest diseases / conditions (e.g., pleural effusion, mass, nodule, pneumonia, pneumothorax, tuberculosis, and fracture). The input can also include a specified location on or within a normal CRX for integrating the synthetic lesion. The synthetic lesion image data object generated by the model based on such input will be clipped to reflect the selected lesion type and location. For example, the visual appearance characteristics (e.g., texture, content, geometric configuration / shape, coloring, resolution, pixelation, brightness, etc.) of the synthetic lesion image data object may vary for different types of lesions. Additionally, the visual appearance characteristics of the same lesion type may vary depending on the different anatomical locations selected for integrating the synthetic lesion at the time of input. For example, the visual appearance characteristics (e.g., texture, content, geometric configuration / shape, coloring, resolution, pixelation, brightness, etc.) of the synthetic lesion image data object of the same lesion type may vary depending on the different anatomical locations selected for integrating the synthetic lesion at the time of input. In this regard, in association with training the synthetic image generation model 116, this model can learn not only the differences in the visual characteristics of different types of lesions but also the differences in the visual characteristics of the same type of lesion at different anatomical locations.

[0045] Additionally, the synthetic lesion image data object generated by the synthetic lesion generation model 116 can be clipped to account for different lesion sizes. In this regard, the input to the synthetic lesion generation model 116 can also include a specified lesion size (e.g., which can be specified by the user via a mask and / or bounding box annotation applied to the medical image into which the synthetic lesion will be integrated), and the synthetic lesion generation model 116 can generate a synthetic lesion object having a size corresponding to the specified size. Additionally, the visual appearance characteristics of the synthetic lesion can vary to account for differences in the same type of lesion at the same anatomical location but with different sizes. In this regard, in association with training the synthetic image generation model 116, this model can learn the differences in the visual characteristics of the same type of lesion at the same anatomical location but with different sizes.

[0046] In this regard, it is understood that the synthetic image generation model 116 may include or correspond to one or more generative machine learning models that may be trained to generate synthetic versions of different types of lesions corresponding to different chest diseases, taking into account variations in different types of lesions based on type, location, and size. The lesion enhancement component 102 and / or the synthetic lesion generation model 116 may further combine the synthetic lesions (e.g., synthetic lesion image data objects) with normal images (e.g., normal CRXs without lesions included in the runtime data 112) at specified locations of their respective inputs to generate multiple synthetic lesion images. The multiple synthetic lesion images may provide a wide distribution of different types of lesion images including synthetic lesion image data objects having variability in terms of lesion type, location, and size. Additional details regarding the lesion enhancement component 102 and the synthetic lesion generation model 116 will be described below with reference to Figures 2 to 7 Description.

[0047] In one or more embodiments, the synthetic lesion images generated by the lesion enhancement component 102 using the synthetic image generation model 116 may be used to train (e.g., via the training component 108) the lesion detection model 118 to detect lesions depicted in the synthetic lesion images. In this regard, the synthetic lesion images may be used for image data augmentation to increase the diversity and quantity of training lesion images available for training the lesion detection model 118, which improves the generalization ability of the model. For example, in some embodiments, the lesion detection model 118 may include or correspond to one or more machine learning models that are trained to detect and classify different types of lesions corresponding to different types of lesions represented in the synthetic lesion images. For example, when applied to CXRs, the lesion detection model 118 may include or correspond to one or more deep learning models (or other types of machine learning models) that are configured to detect and classify different types of lesions corresponding to different types of chest diseases / conditions (e.g., pleural effusion, mass, nodule, pneumonia, pneumothorax, tuberculosis, and fracture) in the input CRX images. In some embodiments, the lesion detection model 118 may also be trained to determine the size of the detected lesions and generate a confidence score that represents a measure of the confidence that the lesion detection model has in the accuracy of its inference output.

[0048] In various embodiments, a training dataset (e.g., included in training data 110) can be used to train a lesion detection model 118 (e.g., via a training component 108), the training dataset including lesion images paired with ground truth annotation information indicating lesion type and lesion size. The lesion images can include real lesion images (e.g., real CXRs with real lesions) and / or synthetic lesion images generated via a synthetic lesion generation model 116. In this regard, in one or more embodiments, a lesion augmentation component 102 can employ the synthetic lesion generation model 116 to generate synthetic lesion images including synthetic lesions on normal CXRs (e.g., included in runtime data 112), and add the synthetic lesion images to the training dataset (e.g., included in training data 110) for training the lesion detection model 118 (e.g., by the training component 108). The training dataset for the lesion detection model 118 can also include normal images (e.g., without lesions to train the model to correctly reason when the input image does not depict a lesion).

[0049] The training process can follow conventional supervised and / or semi-supervised machine learning processes, where the lesion detection model 118 is trained to predict whether an input image depicts a lesion and, if so, the type and size of the lesion, by using one or more loss functions that evaluate the loss (e.g., detection loss) based on a comparison of the inference output and the ground truth annotation data associated with (at least some of) the input images. In this regard, because the synthetic lesion generation model 116 is trained to generate enhanced lesion images with input knowledge identifying the type, location, and (in some embodiments) size of the lesion to be generated and applied to normal CRX images, the enhanced lesion images not only increase the quantity and diversity of training lesion images for training the lesion detection model 118, but also contain the necessary ground truth annotation data that has been applied / associated therewith.

[0050] In some embodiments, the lesion detection component 104 may apply a trained version of the lesion detection model 118 to new medical images included in the runtime data 112 to generate corresponding inference output results. For example, the lesion detection component 104 may execute the lesion detection model 118 during the test phase of the training process and / or execute the lesion detection model 118 on real patient images (e.g., real CXRs) in a real clinical workflow. In one or more embodiments applied to the detection of CXRs and different types of chest disease lesions, the inference output results may include information identifying whether one or more lesions are detected in the input CXR, the type of lesion detected (or type of chest disease) among a plurality of defined different types (if a lesion is detected), the size of the detected lesion (e.g., if a lesion is detected), and a confidence score for a measure indicating the confidence associated with the inference output result for a given input image.

[0051] In one or more additional embodiments, the training of the synthetic lesion generation model 116 and the lesion detection model 118 can be integrated into the same framework for mutual improvement. In particular, when optimizing (e.g., retraining / updating) the synthetic lesion generation model 116, the trained version of the lesion detection model 118 can be used to filter "easy" samples based on their prediction confidence to facilitate the synthesis of "difficult" samples. In this regard, an easy sample as used herein refers to an input lesion image in which the trained version of the lesion detection model 118 exhibits good performance (e.g., measured as a function of a high confidence level or another performance evaluation criterion indicative of an acceptable level of model performance accuracy and / or confidence). Similarly, a difficult sample as used herein refers to an input lesion image in which the trained version of the lesion detection model 118 exhibits poor performance (e.g., measured as a function of a low confidence level or another performance evaluation criterion indicative of an unacceptable level of model performance accuracy and / or confidence). For these embodiments, the performance evaluation component 108 can facilitate the evaluation of the performance of the trained version of the lesion detection model when applied by the lesion detection component 104 to one or more lesion images (e.g., real lesion CXRs included in the runtime data 112), the lesion detection component 104 being associated with identifying a subset of the runtime lesion images (e.g., including one or more) for which the performance of the identified lesion detection model 118 is considered inaccurate or insufficient (e.g., based on a low confidence score or another performance evaluation criterion). For example, the performance evaluation component 106 can identify any input lesion images processed by the trained version of the lesion detection model 118 that receive a confidence score below a threshold confidence score. The performance evaluation component 106 can further add the subset of lesion images to a new training data set (e.g., included in the training data 110), and the training component 108 can use the identified subset of lesion images added to the new training data set to further retrain or update the synthetic lesion generation model 116. Further details regarding the alternating training of the synthetic lesion generation model 116 and the lesion detection model 118 are provided below with reference to Figures 8A to 10 Provide additional details regarding the alternating training of the synthetic lesion generation model 116 and the lesion detection model 118.

[0052] Figure 3 An example process 300 for training the synthetic lesion generation model 116 is presented in accordance with one or more embodiments of the subject matter of the present application. Refer to Figures 1 to 3, in one or more embodiments, the synthetic lesion generation model 116 may employ a GAN-based framework that utilizes an adversarial training process between a lesion generator 306' and a discriminator 310. During training, the lesion generator 306' learns to generate synthetic lesion images (e.g., synthetic CXRs) that mimic the distribution of real lesion images in the training set 302 (e.g., real CXRs with real lesions of various types, locations, and sizes), while the discriminator 310 learns to distinguish between real lesion images and synthetic (or "fake") lesion images. Once training has been completed (e.g., after convergence has been reached and / or the loss has reached an acceptable level), the lesion augmentation component 102 may apply the trained version of the lesion generator 306' to a normal CXR (e.g., a real CXR without lesions) to generate synthetic lesions at specified locations and sizes on the normal CXR, thereby generating a synthetic lesion image that includes the synthetic lesions. In this regard, in various embodiments, the synthetic lesion generation model 116 may comprise or correspond to the lesion generator 306' (and vice versa).

[0053] For the remainder of the description and the figures, apostrophes and dashed lines are used to indicate versions of models that are in training, while solid lines with the same reference numeral minus the apostrophe for the same model (e.g., lesion generator 306, rather than lesion generator 306') are used to indicate the trained versions of the models, unless otherwise noted.

[0054] As described above, in one or more embodiments, the training component 108 does not train the synthetic lesion generation model 116 (or more specifically, the lesion generator 306' / 306) to directly generate an entire synthetic CXR image, but rather only generates an image region that includes a lesion (e.g., also referred to as a lesion region). In particular, the lesion generator 306' focuses on synthesizing only the lesion region by taking as input a masked CXR 304, where the masked CXR 304 corresponds to a real CXR from the training set that has a real lesion and a mask 303 with a mask size corresponding to the size of the real lesion formed over the real lesion. In this regard, in association with generating the synthetic lesion image 308, the lesion generator 306' is trained to "fill" the masked region 305 of the masked CXR 304 with the synthetic lesion image data object 307. For example, the synthetic lesion image 308 corresponds to the masked CXR 304 in which the mask 303 has been removed and replaced with the synthetic lesion image data 307. (For illustrative purposes, the masked region 305 is indicated on the synthetic lesion image 308. In practice, the masked region 305 is not marked on the synthetic lesion images generated by the lesion generator 306.) In this regard, the lesion generator 306' can be trained to generate the synthetic lesion image data object 307 for the masked region 305 of the input image and integrate the synthetic lesion image data 307 over the masked region 305 to generate the synthetic lesion image 308. At the same time, the discriminator 310 is trained to distinguish between the synthetic lesion image 308 and the corresponding real version of the input image (i.e., the masked CXR 304 with the mask 303 removed). In this way, the mask 303 defines the location and size of the synthetic lesion image data object 307 generated by the lesion generator 306'. The lesion generator also takes as input the specific type of lesion expected to be covered by the mask 303 that the lesion generator 306' will generate. The training set 302 can contain real lesion images with a variety of different types of lesions (e.g., corresponding to different types of chest diseases) at different locations and with different sizes.

[0055] As Figure 3 shown, in some embodiments, the lesion generator 306' can be trained based on both an adversarial loss (e.g., as a function of the discriminator 310) and a perceptual loss (e.g., using a pre-trained VGG 312). The discriminator 310, the pre-trained VGG 312, and the corresponding parameter settings can be stored and accessed by the training component 108 in the model library 114. The adversarial loss prompts the lesion generator 306' to generate real lesions, while the perceptual loss simplifies the adversarial training by fitting the high-order perceptual information of the image. In one or more embodiments, the lesion generator 306' (L G ) and the discriminator 310 (L D) The loss function can be formulated using the following Equation 1 and Equation 2 respectively, where x and represent the real and generated CXRs respectively.

[0056]

[0057]

[0058] In one or more embodiments, the discriminator 310 can include seven convolutional layers and two fully connected (FC) layers for binary classification, where each convolutional layer has a kernel size of 3×3 and a stride of 2. Additional details regarding the lesion generator 306' will be described below with reference to Figure 4 and Figure 5 described.

[0059] In some embodiments, regularization R 1 can be used to improve the quality and stability of the generated images by regularizing the discriminator 310. For these embodiments, regularization R 1 can be applied to the discriminator 310 and formulated according to Equation 3, where is the gradient of the discriminator.

[0060]

[0061] Perceptual loss can be incorporated into the training of the lesion generator 306' to consider higher-order perceptual information about the image, rather than just pixel-level differences. By incorporating perceptual loss into the training process of the GAN, the training component 108 can prompt the lesion generator 306' to generate images that not only look visually pleasing but also have higher-order semantic meanings. This can result in more realistic and diverse generated images and better retain the details and textures in the initial images. The perceptual loss (L P ) can be formulated according to Equation 4, where represents the conv5_4 layer of the ImageNet pre-trained VGG model 312.

[0062]

[0063] Thus, in one or more embodiments, the loss function L for training the lesion generator 306' can be formulated according to Equation 5, where α and β are balancing coefficients, and these values can be selected / adjusted according to expectations. In an example implementation, we set α = 10 and β = 0.1.

[0064] L = L G + αR 1 + βL P (Equation 5)

[0065] Figure 4 Presents a more detailed illustration of the lesion generator 306 in accordance with one or more embodiments of the subject matter of the present application. As Figure 4 shown, the lesion generator 306 corresponds to a trained version of the lesion generator. As described above, the input to the lesion generator 306 can include a mask 402 applied to a normal CXR, resulting in a masked CXR 404. The applied mask 402 defines the size and location of the synthetic lesion image data object that will be generated and integrated onto the normal CXR. The input to the lesion generator 306 also includes a selected lesion type from among a plurality of defined lesion types corresponding to different types of chest diseases / conditions. The output of the lesion generator 306 includes a synthetic lesion image 416; that is, a CXR having a synthetic lesion (e.g., a synthetic lesion image data object) overlaid on the normal CRX in the area of the normal CXR covered by the mask 402.

[0066] In one or more embodiments, the lesion generator 306 can include an encoder (EC) component 406, a transformer component 408 (e.g., including transformer levels T 1 through T 5 ), a decoder (DC) component 410, a refinement (RF) component 414, and a style variation (SV) component 412. Figure 5 Presents a more detailed view of these respective components of the lesion generator in accordance with one or more embodiments of the subject matter of the present application (where 408-T N can correspond to each of T 1 through T 5 ).

[0067] Referring to Figure 4 and Figure 5 , in various embodiments, in order to generate synthetic lesions with fine details, the lesion generator 306 can utilize convolutional layers in both the encoder component 406 and the decoder component 410 to correspondingly handle local texture processing and reconstruction. For example, the encoder component 406 can employ multiple convolutional layers to downsample and extract local features of the input image of the mask (e.g., the masked CXR 404), and the decoder component 410 can employ multiple convolutional layers to upsample and reconstruct the image. The refinement component 414 is also used to refine the high-frequency details of the synthetic lesion image data. To obtain a structurally reasonable image, a transformer (e.g., the transformer component 408) is introduced as the main body to model the long-range interaction between local features and global features. The transformer component 408 can include five levels of different resolutions (e.g., transformer levels T 1 through T 5)。The style variation component 412 can further impose diversity on the synthesized lesions by injecting a noise vector during the refinement process. In this way, the lesion generator 306 can generate a reasonable synthetic lesion CXR image with real and diverse lesions of different styles. The following refers to Figure 4 and Figure 5 to outline the specific implementation manners of the components of the lesion generator 306 according to one or more embodiments.

[0068] Encoder and decoder components: In one or more embodiments, the input CXR image can be represented as and the mask 402 (binary mask) applied to the image can be represented as M ∈ {0, 1} 1×H×W , where C, H, and W represent the length, height, and width of the channels respectively. Then, the masked CXR 404 can be obtained and defined as CXR I M = I ⊙ M, where ⊙ represents the element-wise product. The masked regions can be defined as zero values and indicate the positions of the synthetic lesions to be generated and applied by the lesion generator 306. The encoder component 406 takes the concatenation of I M and M as input and then downsamples it to 1 / 8 of the initial size with C' channels via three convolutional layers. Each convolutional layer can have a kernel size of 3×3 and a stride of 2. The decoder component 410 upsamples the output of the transformer component 408 to the same resolution as the input with three deconvolutional layers. Each deconvolutional layer can have a kernel size of 4×4 and a stride of 2.

[0069] Transformer component: As Figure 4 shown, in one or more embodiments, the transformer component 408 can include five transformer levels respectively represented as T 1 , T 2 , T 3 , T 4 and T 5 . As shown in Figure 5 (where 408-T N can correspond to each of T 1 to T 5 ), each transformer level can include four transformer blocks and a convolutional layer with a residual connection. In each transformer block (TB), the input first passes through multi-head attention (MHA), and then its output is concatenated with the input to further pass through a fully connected (FC) layer and a multi-layer perceptron (MLP). Formally, for the l-th block in the k-th level, it can be defined according to the following equations 6 and 7, where the input X k,l-1 comes from the previous block and the output is X" k,l .

[0070] X′ k,l= FC([MHA(X k,l-1 ),X k,l-1 ) (Equation 6)

[0071] X" k,l = MLP(X′ k,l ) (Equation 7)

[0072] To generate hierarchical representations with an efficient attention mechanism, MHA can adopt a shifted window design, which is formulated according to Equation 8, where are the query, key, and value matrices; d is the length of these embeddings and M 2 is the number of patches in the window.

[0073]

[0074] Refinement component: As Figure 5 shown, in one or more embodiments, the refinement component 414 can start with four encoder blocks for downsampling. In some embodiments, each encoder block (EB) is a residual block composed of a convolutional layer with a stride of 2. Then, the output of the last encoder block can be upsampled by four decoder blocks (DB), each of which can also be a residual block based on a deconvolutional layer with a stride of 2. In one or more embodiments, there are skip connections between the encoder and decoder blocks with the same resolution. This connection allows for the direct transmission of information from the encoder to the decoder and ensures that important details are preserved throughout the reconstruction process. Additionally, the multi-scale approach of the refinement module also ensures that the finally reconstructed image has accurate and fine-grained details at all resolution levels. In various embodiments, the weights of the convolutional and deconvolutional layers of the refinement component 414 are further renormalized by the style variation component described below.

[0075] Style variation component: In one or more embodiments, the lesion generator 306 can adopt a style variation component 412 to promote the diversity of synthetic lesion image data objects. The style variation component 412 can change the style of the synthetic lesion image data objects by renormalizing the weights of the convolutional and deconvolutional layers of the refinement component 414 with a style vector during the refinement process, where S is controlled by random noise according to Equation 9, where is a random vector from a Gaussian distribution, and SV is a mapping function composed of six FC layers.

[0076] S = SV(z) (Equation 9)

[0077] After mapping, equations 10 and 11 can be used to perform renormalization, where i, j, and k represent the indices of the input channels, output channels, and spatial locations of the convolution, respectively; ∈ is a small constant to avoid division by zero.

[0078] W′ ijk = W ijk · S i (Equation 10)

[0079]

[0080] By modulating the convolution and deconvolution weights of the refinement component 414 with the style vector S, the lesion generator 306 can generate lesions with different styles.

[0081] Figure 6A and Figure 6B presents a table (Table 600) illustrating example synthetic lesion images generated by a synthetic lesion generation model (e.g., synthetic lesion generation model 116 including lesion generator 306) in accordance with one or more embodiments of the subject matter of the present application. In particular, Table 600 illustrates ten different synthetic lesions CXRs generated by a trained version of the lesion generator 306 according to method 300 (e.g., where the full-size CXR corresponds to the synthetic lesion CXR), and where the lesion generator 306 employs the above reference Figure 4 and Figure 5The described architecture. Ten different synthetic lesion CXRs contain two different examples for each of five different types of chest disease / condition lesions; masses, nodules, pneumonia, tuberculosis, and fractures. Each of the ten different synthetic lesion CXRs is generated by the lesion generator 306 using various different normal CXRs (e.g., CXRs without lesions) as input, where a mask is applied to the normal CXR to indicate the lesion area (e.g., defining the lesion location and size) for the integration of the synthetic lesion. The input to the lesion generator 306 also indicates the desired lesion type (e.g., mass, nodule, pneumonia, tuberculosis, and fracture) to be integrated over or within the corresponding lesion area. In Table 600, the lesion areas are marked on the synthetic lesion CXRs with bounding boxes. An enlarged view of the lesion area as the initial normal and the lesion area as containing the synthetic lesion (e.g., corresponding to the synthetic lesion image data object, respectively) is provided on the right side of the corresponding synthetic lesion CXR. As can be seen from Table 600, the techniques disclosed in the present application are capable of replacing a defined area of a normal CXR with synthetic lesion image data of a specific disease. Taking into account the type, location, and size of the lesion, the synthetic lesions not only reflect the characteristics of different diseases but also have semantics and textures consistent with the surrounding areas. For example, it is particularly noteworthy that the synthetic lesions for each same type of disease are significantly different in appearance (e.g., texture, resolution, content, etc.), thus demonstrating how the lesion generator 306 clips the appearance of the synthetic lesions to consider not only the lesion type but also the selected anatomical location and size, while also ensuring that the synthetic lesions have semantics and textures consistent with the surrounding areas of the different initial input CXRs for each same type of disease. The powerful ability of the lesion generator 306 can be attributed to its utilization of both local details and global structure to generate the synthetic lesions.

[0082] Figure 7 Additional examples of synthetic lesion images generated according to the techniques disclosed in the present application are presented, where examples of synthetic lesion objects of different types of styles that can be generated by the lesion generator 306 via the style variation component 412 according to one or more embodiments of the subject matter disclosed in the present application are shown. Similar to Table 600, Figure 7Depicts three different synthetic lesion images generated by the lesion generator 306 (e.g., where the full-size images are synthetic lesion images). The synthetic lesion images correspondingly correspond to CXRs with three different types of synthetic lesions (e.g., nodules, pneumonia, and tuberculosis lesions) generated on or within the regions marked by the bounding boxes overlaid on the corresponding CXRs. An enlarged view of the lesion regions of the synthetic lesions (e.g., synthetic lesion image data objects) including three different example styles (e.g., corresponding to different or random noise injection amounts introduced via the style variation component 412 respectively) is provided on the right side of each synthetic lesion image. By comparing the synthetic lesions of the same lesion type in each of the three different styles, it can be seen that the style variation component 412 provides for generating multiple different synthetic lesion images by using the same input CXR and input parameters (e.g., lesion type, location, and size) by changing the style of the synthetic lesions. For example, the nodules generated in Style 1, Style 2, and Style 3 all have different shapes and positions, and the lesions of pneumonia and tuberculosis generated in Style 1, Style 2, and Style 3 all have different textures.

[0083] As shown in FIGS. 6 and Figure 7 illustrated, the lesion-aware CXR synthesis framework proposed in this application can synthesize realistic and diverse lesions of various chest diseases, which largely alleviates the problem of insufficient data in lesion detection. In this regard, as described above with reference to Figure 1 a trained version of the synthetic lesion generation model 116 (e.g., which can include and / or correspond to a trained version of the lesion generator 306) can be used to generate various synthetic lesion images such as those illustrated in FIGS. 6 and Figure 7 which can be used by the training component 108 to train the lesion detection model 118.

[0084] In addition, in various embodiments, the training component 108 can adopt an alternating training strategy, where the training of the lesion generator 306 is seamlessly integrated into the training of the lesion detection model 118 (also referred to as the lesion detector 803). In this regard, the inventors of the technology of this application believe that the training of the lesion generator should be driven by the performance of the lesion detector, since the latter is the ultimate task we want to improve. On the other hand, the trained lesion generator can improve the detection performance by providing a large number of synthetic lesion CXRs as data augmentation. Therefore, in some embodiments, the training component 108 can integrate the training of the lesion generator 306 and the lesion detector 803 to form a continuous loop such that the corresponding models improve each other's performance in an alternating training manner.

[0085] For example, in one or more embodiments, the training component 108 can use the predictions of the lesion detector as feedback, rather than training the lesion generator independently for all available training data (e.g., real lesion CXRs). More specifically, the training component 108 can first train the lesion detector on an initial training dataset that includes real lesion CXRs. Then, if the predicted confidence score for the lesion region is greater than a threshold, the performance evaluation component 106 can filter out the easy samples of the initial training set. The training component 108 can then use the selected hard samples to train the lesion generator 306' according to process 300. In this way, the lesion generator can be specifically trained to synthesize samples that the lesion detector cannot perform well on, which can be used to update (e.g., retrain / optimize) the lesion detector. Refer to the following Figure 8A and Figure 8B process 800 further describes this process.

[0086] Similarly, after a trained version of the lesion generator 306 has been developed, the lesion augmentation component 102 can use it for data augmentation to improve the performance of the lesion detector. In this regard, when training the lesion detector, in addition to the initial training data, the lesion augmentation component 102 can also introduce a large number of normal CXRs for lesion synthesis. More specifically, the lesion augmentation component 102 can identify the lung regions of the normal CXRs. Then, the lesion augmentation component 102 can generate rectangular masks within the lung regions and select a specific type of lesion to be generated within the mask regions. In some embodiments, the size, position, and lesion type of the mask can be randomly selected. In other embodiments, the size, position, aspect ratio, and lesion type of the mask can be defined based on user input. In other embodiments, to simulate the distribution of different lesions, the lesion type and the position, size, and aspect ratio of the mask are sampled from the GT. Once the input parameters (e.g., lesion type, mask position, mask size, mask aspect ratio, and optional style variations) have been defined for the corresponding normal CXRs, the lesion augmentation component 102 can apply the lesion generator 306 to synthesize lesions on or within the mask regions of the normal CXRs to generate synthetic lesion images. Then, the training component 108 can use the synthetic lesion images to train the lesion detector 803, using the mask regions and the applied input parameters (e.g., defining the lesion type) as ground truth annotations. In this regard, the applied mask can be used as an annotation within the bounding box to define the ground truth size and position of the lesion. In this way, the lesion generator proposed in this application helps improve the generalization ability of the detection model by providing additional augmented and annotated data. Refer to the following Figure 9A and Figure 9B process 900 further describes this process.

[0087] In this regard,Figure 8A and Figure 8B presents a flowchart of an example process 800 for training a lesion detector 803 (e.g., where the lesion detection model 118 may include or correspond to the lesion detector 803, and vice versa) and a lesion generator 306, in accordance with one or more embodiments of the subject matter of the present application.

[0088] Referring Figure 8A , according to process 800, at 802, a training component 108 may use a training set 1 to train a lesion detector 803', where the training set 1 includes ground truth lesion images (e.g., ground truth CXRs with ground truth lesions of various types, locations, and sizes). At 804, a lesion detection component 104 may apply the trained version of the lesion detector 803 to the ground truth lesion images to generate corresponding results (e.g., detection of lesion type, location, size, and confidence score, where the confidence score is a measure indicating the confidence that the lesion detector 803 has of being correct in the lesion type, location, and / or size for a given input image). In one or more embodiments, the ground truth lesion images used at 804 may include some or all of the ground truth lesion images from the training set 1. Additionally or alternatively, the ground truth lesion images used at 804 may include a new set of ground truth CXRs with ground truth lesions excluded from the training set 1 (e.g., a test set, etc.). At 806, a performance evaluation component 106 may identify one or more target lesion images attributable to poor performance results of the lesion detector 803 on the test set. For example, in some embodiments, the performance evaluation component 106 may filter the test set images based on their associated confidence scores and select low-confidence images (e.g., confidence scores below a defined threshold). In other embodiments, other criteria indicating poor model performance may be utilized to identify a subset of test images for training the lesion generator 306 (e.g., images belonging to a defined patient subgroup, images associated with errors attributable to artifacts, or other criteria). At 808, the training component 108 may add the target lesion images (e.g., low-confidence images) to a training set (e.g., training set 2) and train the lesion generator 306 using the target lesion images according to process 300 (e.g., using manually applied masks over the lesions for training purposes).

[0089] From Figure 8BThe dashed line 809 in [Figure 0] continues the process 800. At 810, the lesion enhancement component 102 can employ a trained version of the lesion generator 306 to generate synthetic lesion images. In this regard, as described above, the lesion enhancement component 102 can obtain a set of normal CXRs (e.g., without lesions), apply a mask to the images, and define the type of lesion to be generated on or within the masked region, resulting in a "masked" normal image. According to method 800, in some embodiments, to generate additional training images corresponding to low-confidence images (i.e., training set 2 of method 800), at 810, the lesion enhancement component 102 can control the input parameters used to define the corresponding lesion type, location, and size based on the distribution of the corresponding lesion type, location, and size of the low-confidence image to generate on or within the corresponding normal image. Additionally or alternatively, the lesion enhancement component 102 can randomly define the input parameters while ensuring that a variety of different lesion types, locations, and sizes are applied. In other embodiments, one or more input parameters (e.g., lesion type, mask size, location, and aspect ratio for the corresponding input normal image) can be defined by the user (e.g., based on user input).

[0090] At 812, the training component 108 can add the synthetic lesion images to a new training set (e.g., training set 3) for retraining / updating the lesion detector and use the applied annotation data (e.g., mask and selected / defined lesion type) as paired ground truth (GT). At 814, the training component 108 can then use the synthetic lesion images to retrain / update the lesion detector 803', resulting in an optimized or updated version of the lesion detector 803.

[0091] Figure 9A and Figure 9B [Figure 9] presents a flowchart of another example process 900 for training a lesion detector 803 and a lesion generator 306 in accordance with one or more embodiments of the subject matter of the present application. For the sake of brevity, the repeated description of similar elements employed in the corresponding embodiments is omitted.

[0092] Refer to Figure 9A, according to method 900, at 902, the training component 108 can use training set 1 including a set of real lesion images to train the lesion generator 306' (e.g., according to process 300). At 904, the lesion enhancement component can use the trained version of the lesion generator 306 and a set of normal CXRs, and combine receiving the input of applying a mask to the normal CXR and the information for defining the type of lesion to be generated to generate synthetic lesion images, as described above with reference to 810. At 906, the training component 108 can add the synthetic lesion images to a new training set (e.g., training set 2) for training and / or updating the lesion detector, and use the applied annotation data as paired GT. As Figure 9A shown, in various embodiments, the synthetic lesion images can be used to augment the initial set of real lesion images available for training the detector (e.g., training set 2 can contain real lesion images and synthetic lesion images). At 908, the training component 108 can then use training set 2 to train and / or update the lesion detector.

[0093] Continuing from the dashed line 909 Figure 9B in method 900, at 910, the lesion detection component 104 can apply the trained version of the lesion detector to real lesion images to generate corresponding results (e.g., detection of lesion type, location, size, and confidence score, where the confidence score is a measure indicating the confidence that the lesion detector 803 has in being correct in the lesion type, location, and / or size for a given input image). In one or more embodiments, the real lesion images used at 910 can include some or all of the real lesion images from training set 1. Additionally or alternatively, the real lesion images used at 910 can include a new set of real CXRs with real lesions excluded from training set 1 (e.g., test set, etc.). At 912, the performance evaluation component can identify target lesion images attributed to poor lesion detector performance results (e.g., low confidence images as described with reference to 806). At 914, the training component 108 can use the target lesion images to further retrain or update the lesion generator 306' (e.g., the low confidence images can be applied to a new training set and used to update the lesion generator 306').

[0094] It can be understood that process 800 and / or process 900 can be executed in a continuous manner accordingly to continuously update the lesion detector and the lesion generator based on the performance of the lesion detector.

[0095] In this regard, Figure 10Presents a high-level illustration of an alternating training framework 1000 for training a lesion generator and a lesion detector for mutual improvement in accordance with one or more embodiments of the subject matter of this application. The alternating training framework includes process 1001 and process 1002. Process 1001 uses synthetic lesion images generated by a trained version of the lesion generator 306 to train and / or update the lesion detector 803'. Process 1002 uses low-confidence samples identified based on the performance of a trained version of the lesion detector 803 to train and / or update the lesion generator 306'. As Figure 10 shown, process 1001 and process 1002 can be connected into a continuous loop, where the corresponding processes can be executed in an alternating manner to simultaneously improve the performance of the corresponding models. In some embodiments, process 1001 can be used to initialize process 1000 and then proceed to process 1002 and continue in the direction indicated by the weight transfer arrow (e.g., clockwise). In other embodiments, process 1002 can be used to initialize process 1000 and then proceed to process 1001 and continue in the direction indicated by the weight transfer arrow (e.g., clockwise).

[0096] Figure 11 Illustrates a block diagram of an example, non-limiting computer-implemented method 1000 for enhancing CXR images with synthetic lesions in accordance with one or more embodiments of the subject matter of this application. Process 1100 includes, at 1102, receiving, by a system including a processor (e.g., computing system 100), a request to generate a synthetic lesion image, the synthetic lesion image including a synthetic lesion image data object integrated on or within a medical image (e.g., via lesion enhancement component 102). For example, the request can correspond to a request received from a user, a training component 108 (e.g., associated with guiding the lesion generator 306 to generate a synthetic lesion image at 810 of process 800 and / or 904 of process 900), another system, etc. At 1104, in response to receiving the request, process 1100 includes generating, by the system, a synthetic lesion image using a synthetic lesion generation model that is trained to generate synthetic lesion image data objects and clipping the synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions (e.g., via lesion enhancement component 102).

[0097] Figure 12The block diagram of a non - limiting computer - implemented method 1200 is illustrated, which is for another example of using synthetic lesions to enhance CXR images according to one or more embodiments of the subject matter of the present application. Process 1200 includes, at 1202, training a synthetic lesion generation model by a system including the process (e.g., computing system 100) to generate synthetic lesion images including synthetic lesion image data objects integrated on or within medical images, where the training includes training the synthetic lesion generation model to generate and clip synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions (e.g., via training component 108). At 1204, process 1200 includes using the synthetic lesion generation model to generate synthetic lesion images (e.g., via lesion enhancement component 102).

[0098] Figure 13 The block diagram of a non - limiting computer - implemented method 1300 is illustrated, which is for an example of using synthetic lesions to enhance CXR images and using the enhanced images associated with optimizing a chest disease detection model according to one or more embodiments of the subject matter of the present application. Process 1300 includes, at 1302, training a synthetic lesion generation model by a system including a process (e.g., computing system 100) to generate synthetic lesion images including synthetic lesion image data objects integrated on or within medical images, where the training includes training the synthetic lesion generation model to generate and clip synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions (e.g., via training component 108). At 1304, process 1300 includes using the synthetic lesion generation model to generate synthetic lesion images (e.g., via lesion enhancement component 102). At 1306, process 1300 includes training a lesion detection model by the system to detect different types of lesions using the synthetic lesion images (e.g., via training component 108). At 1308, process 1300 further includes alternating between: updating the synthetic lesion generation model based on the performance of the lesion detection model, resulting in an updated version of the synthetic lesion generation model; and updating the lesion detection model using the updated synthetic lesion images generated by using the updated version of the synthetic lesion generation model (e.g., via training component 108).

[0099] Example operating environment

[0100] One or more embodiments may be a system, method, and / or computer program product at any possible technical detail integration level. The computer program product may include a computer - readable storage medium (or media) having computer - readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0101] A computer-readable storage medium can be a tangible device that is capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium can be, for example but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing devices. A non-exhaustive list of more specific examples of computer-readable storage media includes the following devices: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or raised structures in a groove record thereon instructions, and any suitable combination of the foregoing devices. As used herein, a computer-readable storage medium should not be construed to be a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse transmitted through an optical fiber cable), or an electrical signal transmitted through a wire.

[0102] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or an external storage device via a network, such as, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the corresponding computing / processing device.

[0103] The computer-readable program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit for performing aspects of the present invention.

[0104] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] Such computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, when executed via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. Such computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium storing the instructions comprises an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0106] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other devices implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0107] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or combinations of special-purpose hardware and computer instructions.

[0108] In combination Figure 14 , the systems and processes described below may be implemented in hardware, such as on a single integrated circuit (IC) chip, multiple ICs, an application-specific integrated circuit (ASIC), or similar devices. Additionally, the order in which some or all of the process blocks appear in each process should not be considered restrictive. Rather, it is understood that some of the process blocks may be performed in various orders, not all of which may be explicitly illustrated in the present disclosure.

[0109] Referring Figure 14 , an example environment 1400 for implementing various aspects of the subject matter claimed in the present application includes a computer 1402. The computer 1402 includes a processing unit 1404, a system memory 1406, a codec 1435, and a system bus 1408. The system bus 1408 couples system components, including but not limited to the system memory 1406, to the processing unit 1404. The processing unit 1404 may be any of a variety of available processors. Dual microprocessors and other multi-processor architectures may also be used as the processing unit 1404.

[0110] System bus 1408 can be any of several types of one or more bus structures, including a memory bus or memory controller, a peripheral bus or external bus, or a local bus using various available bus architectures, including but not limited to, Industry Standard Architecture (ISA), Micro Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association Bus (PCMCIA), FireWire (IEEE 1394), and Small Computer System Interface (SCSI).

[0111] In various embodiments, system memory 1406 includes volatile memory 1410 and non-volatile memory 1412 that can employ one or more of the memory architectures disclosed in this application. The Basic Input / Output System (BIOS), which houses basic routines for transferring information between elements within computer 1402 during startup, is stored in non-volatile memory 1412. Additionally, according to the present invention, codec 1435 can include at least one of an encoder or a decoder, where at least one of the encoder or decoder can be composed of hardware, software, or a combination of hardware and software. Although codec 1435 is depicted as a separate component, codec 1435 can be housed in non-volatile memory 1412. By way of illustration and not limitation, non-volatile memory 1412 can include Read-Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash memory, 3D Flash memory, or a resistive memory such as Resistive Random Access Memory (RRAM). In at least some embodiments, non-volatile memory 1412 can employ one or more of the memory devices disclosed in this application. Further, non-volatile memory 1412 can be computer memory (e.g., physically integrated with computer 1402 or its motherboard), or removable memory. Examples of suitable removable memory that can implement the embodiments disclosed in this application can include Secure Digital (SD) cards, CompactFlash (CF) cards, Universal Serial Bus (USB) memory sticks, or similar devices. Volatile memory 1410 includes Random Access Memory (RAM) that serves as an external cache memory, and can also employ one or more of the memory devices disclosed in various embodiments. By way of illustration and not limitation, RAM can take various forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), and Enhanced SDRAM (ESDRAM), among others.

[0112] The computer 1402 may also include removable / non-removable, volatile / non-volatile computer storage media. For example, Figure 14 FIG. illustrates a magnetic disk storage device 1410. The magnetic disk storage device 1410 includes, but is not limited to, devices such as magnetic disk drives, solid state drives (SSDs), flash memory cards, or memory sticks. Additionally, the magnetic disk storage device 1410 may include a separate storage medium or a storage medium combined with other storage media including, but not limited to, optical disk drives such as optical disk ROM devices (CD-ROM), CD recordable drives (CD-R drives), CD rewritable drives (CD-RW drives), or digital versatile disk ROM drives (DVD-ROM). To facilitate connection of the magnetic disk storage device 1410 to the system bus 1408, a removable or non-removable interface such as interface 1416 is typically used. It will be appreciated that the magnetic disk storage device 1410 may store information related to the user. Such information may be stored in a server or in an application running on the user device, or provided to the server or application. In one embodiment, the user may be notified (e.g., via one or more output devices 1436) of the type of information stored to the magnetic disk storage device 1410 or transmitted to the server or application. The user may be provided with the opportunity to opt-in or opt-out of the collection or sharing of such information by the server or application (e.g., via input from one or more input devices 1428).

[0113] It will be appreciated that Figure 14 Software is described that acts as an intermediary between the user and the basic computer resources described in a suitable operating environment 1400. Such software includes an operating system 1410. The operating system 1410 may be stored on the magnetic disk storage 1410 for controlling and allocating the resources of the computer 1402. Application programs 1420 utilize the operating system 1410's management of resources via program modules 1424 and program data 1426 such as boot / shutdown transaction tables and similar programs stored in the system memory 1406 or on the magnetic disk storage device 1410. It will be appreciated that the subject matter claimed herein may be implemented with various operating systems or combinations of operating systems.

[0114] The user inputs commands or information to the computer 1402 through one or more input devices 1428. The input devices 1428 include, but are not limited to, pointing devices such as a mouse, trackball, stylus, touchpad, keyboard, microphone, joystick, gamepad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and similar devices. These and other input devices are connected to the processing unit 1404 via one or more interface ports 1430 through the system bus 1408. One or more interface ports 1430 include, for example, serial ports, parallel ports, game ports, and Universal Serial Bus (USB). Some of the output devices 1436 use the same type of ports as the input devices 1428. Thus, for example, a USB port can be used to provide input to the computer 1402 and output information from the computer 1402 to the output device 1436. The output adapter 1434 is provided to illustrate some output devices 1436 that require a special adapter, such as monitors, speakers, printers, and other output devices 1436. By way of illustration and not limitation, the output adapter 1434 includes video cards and sound cards that provide connection means between the output device 1436 and the system bus 1408. It should be noted that other devices or device systems, such as one or more remote computers 1438, provide both input capabilities and output capabilities.

[0115] The computer 1402 can operate in a networked environment using logical connections to one or more remote computers, such as one or more remote computers 1438. One or more remote computers 1438 can be personal computers, servers, routers, network PCs, workstations, microprocessor-based devices, peer devices, smart phones, tablet computers, or other network nodes, and generally include many of the elements described with respect to the computer 1402. For the sake of brevity, only the memory storage device 1440 is illustrated for one or more remote computers 1438. One or more remote computers 1438 are logically connected to the computer 1402 via a network interface 1442 and then connected via one or more communication connections 1444. The network interface 1442 includes wired or wireless communication networks, such as local area networks (LANs) and wide area networks (WANs) as well as cellular networks. LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ring, and so on. WAN technologies include, but are not limited to, point-to-point links, circuit-switched networks such as Integrated Services Digital Network (ISDN) and its variants, packet-switched networks, and Digital Subscriber Line (DSL).

[0116] One or more communication connections 1444 refer to the hardware / software used to connect network interface 1442 to bus 1408. Although the communication connection 1444 is shown inside computer 1402 for clarity of illustration, it can also be outside computer 1402. For illustrative purposes only, the hardware / software required to connect to network interface 1442 includes internal and external technologies such as modems including conventional telephone-grade modems, cable modems, and DSL modems, ISDN adapters, and wired and wireless Ethernet cards, hubs, and routers.

[0117] Figure 15 FIG. is a schematic block diagram of an example computing environment 1500 with which the subject matter of the present disclosure may interact. System 1500 includes one or more clients 1502. One or more clients 1502 (e.g., corresponding to client system 700 in some embodiments) can be hardware and / or software (e.g., threads, processes, computing devices). System 1500 also includes one or more servers 1504 (e.g., corresponding to vendor system 600 in some embodiments). Thus, system 1500 can correspond to a two-tier client-server model or a multi-tier model (e.g., client, middle-tier server, data server) and other models. One or more servers 1504 can also be hardware and / or software (e.g., threads, processes, computing devices). For example, server 1504 can accommodate threads to perform transformations by adopting the solution of the present application. A possible communication between client 1502 and server 1504 can be in the form of data packets transmitted between two or more computer processes (e.g., process 1001 and process 1002).

[0118] System 1500 includes a communication framework 1506 that can be used to facilitate communication between one or more clients 1502 and one or more servers 1504. One or more clients 1502 are operatively connected to one or more client data repositories 1508 that can be used to store information local to one or more clients 1502. Similarly, one or more servers 1504 are operatively connected to one or more server data repositories 1512 that can be used to store information local to server 1504.

[0119] It should be noted that various aspects or features of the present disclosure can be used in substantially any wireless telecommunication or radio technology, such as Wi-Fi; Bluetooth; Worldwide Interoperability for Microwave Access (WiMAX); Enhanced General Packet Radio Service (Enhanced GPRS); 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE); 3rd Generation Partnership Project 2 (3GPP2) Ultra Mobile Broadband (UMB); 3GPP Universal Mobile Telecommunication System (UMTS); High Speed Packet Access (HSPA); High Speed Downlink Packet Access (HSDPA); High Speed Uplink Packet Access (HSUPA); GSM (Global System for Mobile Communications) EDGE (Enhanced Data Rate for GSM Evolution) Radio Access Network (GERAN); UMTS Terrestrial Radio Access Network (UTRAN); LTE-Advanced (LTE-A); and so on. Additionally, some or all of the aspects described herein can be used in traditional telecommunication technologies such as GSM. Further, mobile and non-mobile networks (such as the Internet, data service networks such as Internet Protocol Television (IPTV), etc.) can utilize the various aspects or features described herein.

[0120] Although the subject matter of the present application has been described above in the general context of computer-executable instructions of a computer program running on one and / or more computers, those skilled in the art will recognize that the present disclosure can also be implemented in conjunction with other program modules. In general, program modules include routines, programs, components, data structures, etc. for performing particular tasks and / or implementing particular abstract data types. In addition, those skilled in the art can understand that the methods of the present invention can be implemented with other computer system configurations, including single-processor or multi-processor computer systems, small computing devices, mainframe computers, and personal computers, handheld computing devices (such as PDAs, telephones), microprocessor-based or programmable consumer or industrial electronic products, and so on. The aspects shown can also be implemented in a distributed computing environment where tasks are performed by remote processing devices linked through a communication network. However, some aspects of the present disclosure, if not all, can be implemented on a stand-alone computer. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0121] As used in this application, the terms "component", "system", "platform", "interface" and like terms can refer to and / or can include computer-related entities or machine-operating related entities having one or more specific functionalities. Entities disclosed herein can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable program, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a server and the server can be components. One or more components can reside within an execution process and / or thread, and a component can be located on one computer and / or distributed between two or more computers.

[0122] In another example, a corresponding component can execute from various computer-readable media on which various data structures are stored. Such a component can communicate, such as via a signal having one or more data packets (e.g., data from one component), via local and / or remote processes, interact with another component in a local system, a distributed system, and / or interact with other systems across a network such as the Internet. As another example, a component can be a device having specific functionality provided by a mechanical component operated by an electrical or electronic circuit, which electrical or electronic circuit is operated by a software or firmware application executed by a processor. In such a case, the processor can be internal or external to the device and can execute at least a portion of the software or firmware application. As yet another example, a component can be a device that provides specific functionality through an electronic component without mechanical parts, where the electronic component can include a processor or other device to execute software or firmware that at least partially imparts the functionality of the electronic component. In one aspect, for example, within a cloud computing system, a component can emulate an electronic component via a virtual machine.

[0123] Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, "X employs A or B" is intended to mean any of the natural inclusive permutations, unless otherwise stated or clear from the context. That is, if X employs A; X employs B; or X employs both A and B, then any of the foregoing instances satisfies "X employs A or B". Further, the articles "a" and "an" as used in this specification and the drawings should generally be construed to mean "one or more", unless otherwise stated or clearly indicated to the contrary from the context as being in the singular form.

[0124] As used herein, the terms "example" and / or "exemplary" are used to mean serving as an example, instance, or illustration. To avoid doubt, the subject matter disclosed herein is not limited by such examples. Additionally, any aspect or design described herein as "example" and / or "exemplary" is not necessarily to be construed as preferred or superior to other aspects or designs, nor does it imply the exclusion of equivalent exemplary structures and techniques known to those of ordinary skill in the art.

[0125] The various aspects or features described herein can be implemented as a method, apparatus, system, or article of manufacture using standard programming or engineering techniques. Additionally, the various aspects or features disclosed in this application can be implemented by program modules for implementing at least one or more of the methods disclosed herein, the program modules being stored in a memory and executed by at least one processor. Other combinations of hardware and software, or hardware and firmware, can enable or implement the various aspects described herein, including one or more of the methods disclosed in this application. The term "article of manufacture" as used herein can encompass a computer program accessible from any computer-readable device, carrier, or storage medium. For example, a computer-readable storage medium can include, but is not limited to, magnetic memory devices (e.g., hard disks, floppy disks, magnetic strips...), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), Blu-ray disks (BD...)), smart cards, and flash memory devices (e.g., cards, sticks, key drives...), or similar devices.

[0126] As used in this specification, the term "processor" can refer to substantially any computing processing unit or device, including but not limited to: a single-core processor; a single processor with software multithreading execution capabilities; a multi-core processor; a multi-core processor with software multithreading execution capabilities; a multi-core processor employing hardware multithreading techniques; a parallel platform; and a parallel platform with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Further, a processor can utilize nanoscale architectures, such as but not limited to molecule- and quantum-dot-based transistors, switches, and gates, in order to optimize space usage or enhance the performance of a user device. A processor can also be implemented as a combination of computing processing units.

[0127] In the present disclosure, terms such as "repository", "storage device", "data repository", "data store", "database", and any other information storage component substantially related to the operation and functionality of a component are used to refer to a "memory component", an entity embodied in a "memory", or a component that includes a memory. It will be understood that the memories and / or memory components described herein can be volatile memories or non-volatile memories, or can include both volatile and non-volatile memories.

[0128] By way of illustration and not limitation, non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memories can include RAMs that can be used, for example, as external caches. By way of illustration and not limitation, RAMs can be obtained in a variety of forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the memory components of the systems or methods disclosed herein are intended to include, but not be limited to, those containing these and any other suitable types of memory.

[0129] It should be realized and understood that the components described for a particular system or method can include the same or similar functionality as the corresponding components (e.g., correspondingly named components or similarly named components) described for other systems or methods disclosed herein.

[0130] The foregoing has included examples of systems and methods that provide the advantages of the present disclosure. Of course, for purposes of describing the present disclosure, it is not possible to describe every possible combination of components or methods, but one of ordinary skill in the art will recognize that many other combinations and permutations of the present disclosure are possible. Additionally, with respect to the terms "comprising", "having", "owning", and similar expressions used in the detailed description, the claims, the appendices, and the drawings, such terms are intended to be inclusive in a manner similar to the term "including" as interpreted when used as a transitional word in a claim.

Claims

1. A system, comprising: a memory that stores computer-executable components; and a processor that executes the computer-executable components stored in the memory, wherein the computer-executable components include: a lesion enhancement component that: uses a synthetic lesion generation model to generate synthetic lesion images, the synthetic lesion images including synthetic lesion image data objects integrated on or within a medical image, the synthetic lesion generation model being trained to generate the synthetic lesion image data objects; and clips the synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions.

2. The system according to claim 1, wherein the different types of lesions correspond to different types of chest diseases.

3. The system according to claim 2, wherein the medical image includes a chest X-ray image.

4. The system according to claim 1, wherein the lesion enhancement component adds the synthetic lesion images to a lesion image training dataset that includes lesion images, the lesion images including enhanced medical images, and wherein the computer-executable components further comprise: a training component that uses the lesion image training dataset to train a lesion detection model to detect the different types of lesions in the lesion images.

5. The system according to claim 4, wherein the lesion enhancement component generates the synthetic lesion images in association with the receipt of annotation data for integration on or within the medical image, the annotation data indicating a defined disease type, a defined anatomical location, and a defined size of the corresponding object of the synthetic lesion image data object, and wherein the training component uses the annotation data correspondingly associated with the synthetic lesion images as ground truth information associated with training the lesion detection model.

6. The system according to claim 4, wherein the computer-executable components further comprise: a performance evaluation component that identifies one or more target lesion images of the lesion image training dataset associated with a negative performance criterion of the lesion detection model, and wherein the training component updates the synthetic lesion generation model based on the one or more target lesion images.

7. The system according to claim 1, wherein the synthetic lesion generation model is further trained to clip the synthetic lesion data objects to account for different lesion sizes and textures.

8. The system according to claim 1, wherein the computer-executable components further comprise: a training component that uses one or more machine learning processes to train the synthetic lesion generation model.

9. The system according to claim 8, wherein the one or more machine learning processes include an adversarial training process that employs a lesion generator network and a discriminator network.

10. The system according to claim 9, wherein the lesion generator network includes convolutional layers and a transformer.

11. The system according to claim 8, wherein the synthetic lesion generation model includes a style variation module that uses noise injection to generate different style variations of the synthetic lesion image data object.

12. A method, comprising: receiving, by a system including a processor, a request for generating a synthetic lesion image, the synthetic lesion image including a synthetic lesion image data object integrated on or within a medical image; and in response to receiving the request, generating, by the system, the synthetic lesion image using a synthetic lesion generation model, the synthetic lesion generation model being trained to generate the synthetic lesion image data object and clip the synthetic lesion image data object to account for different types of lesions and different anatomical locations of the lesions.

13. The method according to claim 12, wherein the different types of lesions correspond to different types of chest diseases, and wherein the medical image includes a chest X-ray image.

14. The method according to claim 12, wherein the request includes annotation data indicating a defined disease type, a defined anatomical location, and a defined size of a corresponding object of the synthetic lesion image data object for integration on or within the medical image, and wherein the generating includes generating the corresponding object and integrating the corresponding object on or within the medical image according to the annotation data.

15. The method according to claim 12, further comprising: adding, by the system, the synthetic lesion image to a lesion image training data set including lesion images, the lesion images including enhanced medical images; and training, by the system, a lesion detection model using the lesion image training data set to detect the different types of lesions in the lesion images.

16. The method according to claim 15, wherein the generating includes generating the synthetic lesion image in association with the receipt of annotation data indicating a defined disease type, a defined anatomical location, and a defined size of a corresponding object of the synthetic lesion image data object for integration on or within the medical image, and wherein the method further comprising: using, by the system, the annotation data correspondingly associated with the synthetic lesion image as ground truth information associated with training the lesion detection model.

17. The method according to claim 15, further comprising: identifying, by the system, one or more target lesion images of the lesion image training data set associated with a negative performance criterion of the lesion detection model; and updating, by the system, the synthetic lesion generation model based on the one or more target lesion images.

18. The method according to claim 12, further comprising: training, by the system, the synthetic lesion generation model using one or more machine learning processes, wherein the one or more machine learning processes include an adversarial training process using a lesion generator network and a discriminator network.

19. A non-transitory machine-readable storage medium comprising executable instructions that, when executed by a processor, facilitate performance of the following operations, the operations comprising: training a synthetic lesion generation model to generate synthetic lesion images, the synthetic lesion images comprising synthetic lesion image data objects integrated on or within a medical image, wherein the training comprises training the synthetic lesion generation model to generate and clip the synthetic lesion image data objects to account for different types of lesions and different anatomical locations of the lesions; and using the synthetic lesion generation model to generate the synthetic lesion images.

20. The non-transitory machine-readable storage medium according to claim 19, the operations further comprising: training a lesion detection model to detect the different types of lesions using the synthetic lesion images; and alternating between: updating the synthetic lesion generation model based on the performance of the lesion detection model, thereby obtaining an updated version of the synthetic lesion generation model; and updating the lesion detection model using updated synthetic lesion images generated by using the updated version of the synthetic lesion generation model.

Images