Concurrent image and corresponding multi-channel auxiliary data generation for generative models

Abstract

The present invention is entitled "Concurrent Image and Corresponding Multi-channel Auxiliary Data Generation for Generative Models". The present invention proposes systems and techniques for providing concurrent image and corresponding multi-channel auxiliary data generation for generative models. In one example, the system generates synthetic multi-channel data associated with a synthetic version of the imaging data. The system also predicts the multi-channel imaging data and the synthetic multi-channel data through a first prediction category set or a second prediction category set. In addition, the system uses the first prediction category set or the second prediction category set of the synthetic multi-channel data to train a generative adversarial network model.

Description

Technical Field

[0001] The present disclosure relates generally to artificial intelligence. Background Art

[0002] Artificial intelligence (AI) can be used for classification and / or analysis of digital images. For example, AI can be used for image recognition. In certain technical applications, AI can be used for generative modeling (such as, for example, generative adversarial networks) to learn data distributions. However, training generative models and / or generating high-quality synthetic images is often difficult to achieve. For example, the accuracy and / or efficiency of pixel-level annotations of synthetic images associated with generative models is often difficult to achieve. In this way, conventional artificial intelligence techniques can be improved. Summary of the invention

[0003] The following content presents a simplified summary of the present specification in order to provide a basic understanding of some aspects of the present specification. This summary is not an exhaustive overview of the present specification. It is neither intended to identify the key or important elements of the present specification, nor to describe any scope of a particular embodiment of the present specification or any scope of the claims. Its sole purpose is to present some concepts of the present specification in a simplified form as a preface to a more detailed description presented later.

[0004] According to one embodiment, the system includes a multi-channel generator component, a discriminator component, and a training component. The multi-channel generator component generates synthetic multi-channel data associated with a synthetic version of the imaging data. The discriminator component predicts the multi-channel imaging data and the synthetic multi-channel data by a first prediction category set or a second prediction category set. The training component trains a generative adversarial network model using the first prediction category set or the second prediction category set of the synthetic multi-channel data.

[0005] According to another embodiment, a method is provided. The method includes generating synthetic multi-channel data associated with a synthetic version of imaging data. The method also includes predicting the multi-channel imaging data and the synthetic multi-channel data by a first prediction category set or a second prediction category set. In addition, the method includes training a generative adversarial network model based on the first prediction category set or the second prediction category set of the synthetic multi-channel data.

[0006] According to yet another embodiment, a computer readable storage device is provided. The computer readable storage device includes instructions that, in response to execution, cause a system including a processor to perform operations including generating synthetic multi-channel data associated with a synthetic version of imaging data. The processor also performs operations including predicting the multi-channel imaging data and the synthetic multi-channel data by a first prediction category set or a second prediction category set. The processor also performs operations including training a generative adversarial network model based on the first prediction category set or the second prediction category set for the synthetic multi-channel data.

[0007] The following detailed description and accompanying drawings set forth certain illustrative aspects of the present specification. However, these aspects are merely indicative of some of the various ways in which the principles of the present specification may be employed. Other advantages and novel features of the present specification will become apparent from the following detailed description of the specification when considered in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Numerous aspects, embodiments, objects and advantages of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings, in which like reference numerals refer to like parts throughout, and in which:

[0009] Figure 1 shows a high-level block diagram of an exemplary generative modeling component according to various aspects and embodiments described herein;

[0010] Figure 2 shows a high-level block diagram of another exemplary generative modeling component in accordance with various aspects and embodiments described herein;

[0011] Figure 3 shows a high-level block diagram of yet another exemplary generative modeling component in accordance with various aspects and embodiments described herein;

[0012] Figure 4 shows a high-level block diagram of yet another exemplary generative modeling component in accordance with various aspects and embodiments described herein;

[0013] Figure 5 A system and an exemplary medical imaging diagnostic process according to various aspects and embodiments described herein are shown, the system including an inference component based on exemplary generative modeling;

[0014] Figure 6 An exemplary system associated with a generative network associated with training according to various aspects and embodiments described herein is shown;

[0015] Figure 7 An exemplary system associated with a multi-channel generator for inference according to various aspects and embodiments described herein is shown;

[0016] Figure 8 An exemplary system associated with post-processing for inference and / or binary mask generation according to various aspects and embodiments described herein is shown;

[0017] Fig. 9 depicts a flow chart of an exemplary method for providing concurrent image and corresponding multi-channel auxiliary data generation for a generative model in accordance with various aspects and embodiments described herein;

[0018] Fig.10 is a schematic block diagram illustrating a suitable operating environment; and

[0019] Fig.11 is a schematic block diagram of a sample computing environment. DETAILED DESCRIPTION

[0020] Various aspects of the present disclosure are now described with reference to the accompanying drawings, wherein the same reference numerals are used to represent the same elements throughout. In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it should be understood that certain aspects of the present disclosure may be practiced without these specific details, or with other methods, components, materials, etc. In other examples, known structures and devices are shown in block diagram form to help describe one or more aspects.

[0021] Systems and techniques for providing concurrent image and corresponding multi-channel auxiliary data generation for a generative model are proposed. For example, novel generative adversarial networks are disclosed herein. During the training phase, the novel generative adversarial network may include a multi-channel generator that generates multi-channel data and a discriminator that receives the multi-channel data to classify real data or synthetic data. Other information (such as fragment information, etc.) can be encoded in another channel so that learning can be performed simultaneously during training. Synthetic multi-channel data can also be generated during the inference phase. In one embodiment, the novel generative adversarial network can use a multi-channel generator associated with a convolutional neural network to generate a synthetic image and two or more segmentations for the synthetic image. The novel generative adversarial network can also use a discriminator associated with a convolutional neural network to classify real images according to multi-channel data related to the synthetic image and two or more segmentations for the synthetic image. In one aspect, a synthetic image for a generative adversarial network can be annotated by simultaneously generating corresponding multi-channel auxiliary data. In another aspect, a loss function generated based on multi-channel data generation can be used to train the generative adversarial network so as to tune the generative adversarial network accordingly. In certain embodiments, a novel generative adversarial network can employ concurrent medical images and corresponding multi-channel auxiliary data to improve annotations associated with disease detection. For example, a novel generative adversarial network can employ concurrent medical images and corresponding multi-channel auxiliary data without the need for radiologists to annotate the synthetic images at the pixel level. In this way, by employing the novel generative adversarial network described herein, the detection and / or positioning of one or more features associated with the image data (e.g., the detection and / or positioning of one or more conditions of a patient associated with the medical imaging data) can be improved. In addition, the accuracy and / or efficiency of the classification and / or analysis of the image data (e.g., medical imaging data) can be improved. In addition, the effectiveness of the machine learning model for the classification and / or analysis of the image data (e.g., medical imaging data) can be improved, the performance of one or more processors executing the machine learning model for the classification and / or analysis of the image data (e.g., medical imaging data) can be improved, and / or the efficiency of one or more processors executing the machine learning model for the classification and / or analysis of the image data (e.g., medical imaging data) can be improved.

[0022] First reference Figure 1, an exemplary system 100 according to one or more embodiments of the present disclosure is shown, which provides concurrent image and corresponding multi-channel auxiliary data generation for a generative model. The system 100 can be employed by various systems, such as, but not limited to, medical device systems, medical imaging systems, medical diagnostic systems, medical systems, medical modeling systems, enterprise imaging solution systems, advanced diagnostic tool systems, simulation systems, image management platform systems, nursing implementation management systems, artificial intelligence systems, machine learning systems, neural network systems, modeling systems, aviation systems, power systems, distributed power systems, energy management systems, thermal management systems, transportation systems, oil and gas systems, mechanical systems, machine systems, equipment systems, cloud-based systems, heating systems, HVAC systems, medical systems, automotive systems, aircraft systems, watercraft systems, water filtration systems, cooling systems, pump systems, engine systems, prediction systems, machine design systems, etc. In one example, the system 100 can be associated with a classification system to facilitate visualization and / or interpretation of medical imaging data. In addition, system 100 and / or components of system 100 can be used to use hardware and / or software to solve problems that are highly technical in nature (e.g., related to processing digital data, related to processing imaging data, related to medical modeling, related to medical imaging, related to artificial intelligence, etc.) that are not abstract and cannot be performed by a human as a series of mental acts.

[0023] The system 100 may include a generative modeling component 102, which may include a multi-channel generator component 104, a discriminator component 106, and a training component 108. In one embodiment, the generative modeling component 102 may be used in a training phase. For example, the generative modeling component 102 may be used for multi-channel generative adversarial network training. Aspects of the systems, devices, or processes explained in the present disclosure may constitute one or more machine-executable components embodied in one or more machines (e.g., embodied in one or more computer-readable media (mediums / media) associated with one or more machines). When executed by one or more machines (e.g., one or more computers, one or more computing devices, one or more virtual machines, etc.), such one or more components may cause one or more machines to perform the operations described. The system 100 (e.g., the generative modeling component 102) may include a memory 112 for storing computer-executable components and instructions. The system 100 (e.g., the generative modeling component 102) may also include a processor 110 to facilitate the operation of the system 100 (e.g., the generative modeling component 102) on the instructions (e.g., computer-executable components and instructions).

[0024] The generative modeling component 102 (e.g., the discriminator component 106 of the generative modeling component 102) can receive imaging data 114. The imaging data 114 can be two-dimensional imaging data and / or three-dimensional imaging data generated by one or more imaging devices. For example, the imaging data 114 can be an image captured via a set of sensors (e.g., a set of sensors associated with an imaging device). In some embodiments, the imaging data 114 can be a series of images captured via a set of sensors (e.g., a set of sensors associated with an imaging device) during a certain time interval. The imaging data 114 can be received directly from one or more imaging devices. Alternatively, the imaging data 114 can be stored in one or more databases that receive and / or store the imaging data 114 associated with one or more imaging devices. In some embodiments, the imaging data 114 can be medical imaging data. For example, the imaging data 114 can be two-dimensional medical imaging data and / or three-dimensional medical imaging data generated by one or more medical imaging devices. In one example, the imaging data 114 can be an electromagnetic radiation image captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). In some embodiments, imaging data 114 can be a series of electromagnetic radiation images captured via a group of sensors (e.g., a group of sensors associated with a medical imaging device) during a certain time interval. In another example, imaging data 114 can be a positron emission tomography (PET) scan image. In yet another example, imaging data 114 can be magnetic resonance imaging (MRI) data. Imaging data 114 can be received directly from one or more medical imaging devices. Alternatively, imaging data 114 can be stored in one or more databases that receive and / or store medical imaging data associated with one or more medical imaging devices. Medical imaging devices can be, for example, x-ray devices, computed tomography (CT) devices, PET scanner devices, MRI devices, another type of medical imaging device, etc.

[0025] In one embodiment, the imaging data 114 may be multi-channel imaging data. For example, the imaging data 114 may be a multi-channel data sample of a real image. In one aspect, the imaging data 114 may include a real version of the imaging data. In addition, the imaging data 114 may include one or more segmentations of the real version of the imaging data. For example, in one embodiment, the imaging data 114 may include a first data channel associated with the real image, a second data channel associated with first segmentation data indicating a segmentation of the real image, and a third data channel associated with second segmentation data indicating a remaining segmentation of the real image. In some embodiments, the imaging data 114 may include mask data. For example, the imaging data 114 may include one or more masks for one or more segmentations of the real version of the imaging data. In one example, the imaging data 114 may include first mask data indicating a first mask for the first segmentation data (e.g., a first ground truth mask) and second mask data indicating a second mask for the second segmentation data (e.g., a second ground truth mask). The mask included in the imaging data 114 may be a filter for masking one or more regions in the real version of the imaging data. For example, the mask included in the imaging data 114 may include one or more weights for one or more regions of interest in the real version of the imaging data. In one example, the mask may include a set of pixels that use binary filtering to define the location of the segmentation. Additionally or alternatively, the generative modeling component 102 (e.g., the multi-channel generator component 104 of the generative modeling component 102) may receive one or more implicit random variables 116. The one or more implicit random variables 116 may, for example, be sampled from a data distribution of random variables. For example, the one or more implicit random variables 116 may be a vector of random variables.

[0026] The multi-channel generator component 104 can generate synthetic multi-channel data based on one or more implicit random variables 116. The synthetic multi-channel data can be a multi-channel data sample. In one aspect, the synthetic multi-channel data can include a synthetic version of the imaging data. In addition, the synthetic multi-channel data can include one or more segmentations of the synthetic version of the imaging data. The synthetic version of the imaging data and the one or more segmentations can be associated with a unique data channel. For example, in one embodiment, the multi-channel generator component 104 can generate a first data channel associated with a synthetic image, a second data channel associated with first segmentation data indicating a segmentation of the synthetic image, and a third data channel associated with second segmentation data indicating a remaining segmentation of the synthetic image. In another embodiment, the multi-channel generator component 104 can generate a first data channel associated with a synthetic image, a second data channel associated with first segmentation data indicating a first segmentation of the synthetic image, a third data channel associated with second segmentation data indicating a remaining portion of the first segmentation of the synthetic image, a fourth data channel associated with third segmentation data indicating a second segmentation of the synthetic image, and a fifth data channel associated with fourth segmentation data indicating a remaining portion of the second segmentation of the synthetic image. However, it should be appreciated that the synthetic multi-channel data generated by the multi-channel generator component 104 may be associated with different numbers of data channels and / or different numbers of segmentations. In one embodiment, the synthetic multi-channel data generated by the multi-channel generator component 104 may include mask data. For example, the multi-channel generator component 104 may generate one or more masks of one or more segmentations of a synthetic version of the imaging data. In one example, the multi-channel generator component 104 may generate first mask data indicating a first mask of the first segmentation data (e.g., a first ground truth mask), the multi-channel generator component 104 may generate second mask data indicating a second mask of the second segmentation data (e.g., a second ground truth mask), etc. The mask included in the synthetic multi-channel data may be a filter for masking one or more regions in the synthetic version of the imaging data. For example, the mask included in the synthetic multi-channel data may include one or more weights for one or more regions of interest in the synthetic version of the imaging data. In one example, the mask may include a set of pixels that use binary filtering to define the position of the segmentation.

[0027] In another embodiment, the multi-channel generator component 104 may employ a deep neural network (such as, for example, a convolutional neural network) to generate synthetic multi-channel data. In some embodiments, the convolutional neural network may be a spring network of a convolutional layer. For example, the convolutional neural network may perform multiple sequential and / or parallel downsampling and / or upsampling of one or more implicit random variables 116 associated with the convolutional layer of the convolutional neural network. In one example, the convolutional neural network may perform a first convolutional layer process associated with sequential downsampling of one or more implicit random variables 116, and a second convolutional layer process associated with sequential upsampling of one or more implicit random variables 116. The spring network of the convolutional layer may include a first convolutional layer process associated with sequential downsampling and a second convolutional layer process associated with sequential upsampling. The spring network of the convolutional layer associated with the convolutional neural network may modify the convolutional layer filter, similar to the function of a spring. For example, the convolutional neural network may analyze the one or more latent random variables 116 based on: first convolutional layer filters including a first size, second convolutional layer filters including a second size different from the first size, and third convolutional layer filters including a first size associated with the first convolutional layer filters.

[0028] The discriminator component 106 may predict the imaging data 114 by the first prediction category set or the second prediction category set. Additionally or alternatively, the discriminator component 106 may predict the synthetic multi-channel data by the first prediction category set or the second prediction category set. In one embodiment, the category set may include multi-channel images (e.g., channel level labels), image labels (e.g., image level labels), and / or clinical labels. The first prediction category set may be associated with a first classification (e.g., a first classification label), and the second prediction category set may be associated with a second classification (e.g., a second classification label). For example, the image level label may correspond to a "true" label of the image (e.g., a "true" classification label) or a "false" label of the image (e.g., a "false" classification label). In another example, the channel level label may correspond to a "true" label of the multi-channel image (e.g., a "true" classification label) or a "false" label of the multi-channel image (e.g., a "false" classification label). In one embodiment, the discriminator component 106 can label the synthetic multi-channel data with a first label (e.g., a "real" label) or a second label (e.g., a "fake" label) based on the following: a first data channel associated with a synthetic image, a second data channel associated with the first segmented data, and a third data channel associated with the second segmented data. In another embodiment, the discriminator component 106 can label the synthetic multi-channel data with a first label (e.g., a "real" label) or a second label (e.g., a "fake" label) based on the following: a first data channel associated with a synthetic image, a second data channel associated with the first segmented data, a third data channel associated with the second segmented data, a fourth data channel associated with the third segmented data, and a fifth data channel associated with the fourth segmented data.

[0029] In some embodiments, the discriminator component 106 can use a deep neural network (such as, for example, a convolutional neural network) to label (e.g., classify) the imaging data 114 and / or the synthetic multi-channel data. In some embodiments, the convolutional neural network can be a spring network of convolutional layers. For example, the convolutional neural network can perform multiple sequential and / or parallel downsampling and / or upsampling on the imaging data 114 and / or the synthetic multi-channel data associated with the convolutional layer of the convolutional neural network. In one example, the convolutional neural network can perform a first convolutional layer process associated with sequential downsampling of the imaging data 114 and / or the synthetic multi-channel data, and a second convolutional layer process associated with sequential upsampling of the imaging data 114 and / or the synthetic multi-channel data. The spring network of the convolutional layer can include a first convolutional layer process associated with sequential downsampling and a second convolutional layer process associated with sequential upsampling. The spring network of the convolutional layer associated with the convolutional neural network can modify the convolutional layer filter, similar to the function of a spring. For example, the convolutional neural network can analyze the imaging data 114 and / or synthesized multi-channel data based on the following: a first convolutional layer filter including a first size, a second convolutional layer filter including a second size different from the first size, and a third convolutional layer filter including a first size associated with the first convolutional layer filter. In some embodiments, the discriminator component 106 can extract information indicating correlation, inference and / or expression from the imaging data 114 and / or multi-channel imaging data based on the convolutional neural network. In one aspect, the discriminator component 106 can use the convolutional neural network to perform learning explicitly or implicitly with respect to the imaging data 114 and / or multi-channel imaging data. The discriminator component 106 can also use an automatic classification system and / or an automatic classification process to facilitate the analysis of the imaging data 114 and / or multi-channel imaging data. For example, the discriminator component 106 can use a probability and / or statistical analysis (e.g., taking into account the analysis utility and cost) to learn the imaging data 114 and / or multi-channel image data and / or generate inferences with respect to it. The discriminator component 106 can use, for example, a support vector machine (SVM) classifier to learn the imaging data 114 and / or multi-channel imaging data and / or generate inferences therefor. In addition or alternatively, the discriminator component 106 can use other classification techniques associated with Bayesian networks, decision trees and / or probabilistic classification models. The classifier used by the discriminator component 106 can be explicitly trained (e.g., via common training data) and implicitly trained (e.g., via receiving external information). For example, with respect to SVM, the SVM can be configured via a learning or training phase within a classifier constructor and a feature selection module. A classifier can be a function that maps an input attribute vector x = (x1, x2, x3, x4, xn) to a confidence that the input belongs to a category - i.e., f(x) = confidence (category).

[0030] The training component 108 may employ the first predicted label or the second predicted label of the imaging data 114 to train the generative artificial intelligence model. Additionally or alternatively, the training component 108 may employ the first predicted label or the second predicted label of the synthetic multi-channel data to train the generative artificial intelligence model. The generative artificial intelligence model may be a model for a generative network. For example, the generative artificial intelligence model may be a model for a generative adversarial network. A generative network (e.g., a generative adversarial network) may be used to generate images and / or corresponding annotations of images. For example, a generative network (e.g., a generative adversarial network) may be used to generate medical images and / or corresponding annotations of medical images. In one embodiment, the training component 108 may generate and / or utilize a loss function to tune the generative artificial intelligence model based on the first predicted label or the second predicted label of the imaging data 114. For example, the training component 108 may generate and / or utilize a loss function to tune the discriminator of the generative adversarial network (e.g., to tune a convolutional neural network for the discriminator of the generative adversarial network) based on the first predicted label or the second predicted label of the imaging data 114. The loss function associated with the imaging data 114 may be, for example, a cross entropy error function. In addition, the generative adversarial network can be modified based on the loss function associated with the imaging data 114 to improve the classification output from the generative adversarial network. In addition or alternatively, the training component 108 can generate and / or utilize a loss function to tune the generative artificial intelligence model based on the first predicted label or the second predicted label of the multi-channel imaging data. For example, the training component 108 can generate and / or utilize a loss function to tune the multi-channel generator of the generative adversarial network based on the first predicted label or the second predicted label of the synthetic multi-channel data (for example, to tune the convolutional neural network of the multi-channel generator for the generative adversarial network). The loss function associated with the synthetic multi-channel data can be, for example, a cross entropy error function. In addition, the generative adversarial network can be modified based on the loss function associated with the synthetic multi-channel data to improve the generation of images according to the data distribution associated with the generative adversarial network. In some embodiments, the loss function generated by the training component 108 may include a set of weights to tune one or more parameters of the generative artificial intelligence model. In one embodiment, the generative modeling component 102 can generate a generative model 118. For example, the generative model 118 can be a generative artificial intelligence model trained by the training component 108. In certain embodiments, the generative model 118 may be associated with a loss function associated with the imaging data 114 and / or a loss function associated with the synthesized multi-channel data.

[0031] It should be appreciated that the technical features of the generative modeling component 102 are highly technical in nature and are not abstract ideas. The processing threads of the generative modeling component 102 that process and / or analyze imaging data 114 and / or one or more implicit random variables 116, etc., cannot be performed by a human (e.g., greater than the capabilities of a single human mind). For example, the amount of imaging data 114 processed by the generative modeling component 102 in a particular time period, the processing speed of the imaging data 114, and / or the type of data of the processed imaging data 114 can be greater, faster, and different, respectively, than the amount, speed, and type of data that can be processed by a single human mind in the same time period. In addition, the imaging data 114 processed by the generative modeling component 102 can be one or more medical images generated by a sensor of a medical imaging device. In addition, the generative modeling component 102 can be fully operational (e.g., fully powered on, fully executed, etc.) for performing one or more other functions while also processing imaging data 114.

[0032] Reference now Figure 2 , shows a non-limiting embodiment of a system 200 according to various aspects and embodiments of the present disclosure. For the sake of brevity, repeated descriptions of similar elements employed in other embodiments described herein are omitted. The system 200 includes a generative modeling component 102. The generative modeling component 102 may include a multi-channel generator component 104, a discriminator component 106, a training component 108, a processor 110, and / or a memory 112. In Figure 2In the illustrated embodiment, the multi-channel generator component 104 may include a synthetic image generator component 202. The synthetic image generator component 202 may generate one or more synthetic images based on one or more implicit random variables 116. In one aspect, the synthetic image generator component 202 may generate a synthetic image for a data channel included in the synthetic multi-channel data. For example, the synthetic image generator component 202 may generate a synthetic image for a first data channel of the synthetic multi-channel data. In addition, at least a second data channel in the synthetic multi-channel data may include a segmentation of a synthetic image generated by the synthetic image generator component 202. In one embodiment, the synthetic image generator component 202 may generate one or more synthetic images based on a data distribution (e.g., a probability distribution) associated with one or more implicit random variables 116. For example, the synthetic image generator component 202 may generate one or more synthetic images based on a vector of data associated with one or more implicit random variables 116. In another embodiment, the synthetic image generator component 202 may generate one or more synthetic images based on a random data distribution associated with one or more implicit random variables 116. For example, in one embodiment, the synthetic image generator component 202 may generate one or more synthetic images based on a random vector of data (e.g., a vector of random data values) associated with one or more hidden random variables 116. In yet another embodiment, the synthetic image generator component 202 may generate one or more synthetic images based on a set of hidden random variables associated with one or more hidden random variables 116.

[0033] Reference now Figure 3 , shows a non-limiting embodiment of a system 300 according to various aspects and embodiments of the present disclosure. For the sake of brevity, repeated descriptions of similar elements employed in other embodiments described herein are omitted. The system 300 includes a generative modeling component 102. The generative modeling component 102 may include a multi-channel generator component 104, a discriminator component 106, a training component 108, a processor 110, and / or a memory 112. In Figure 3In the illustrated embodiment, the multi-channel generator component 104 may include a composite image generator component 202 and / or a segmentation component 302. The segmentation component 302 may generate one or more segmentations associated with one or more composite images generated by the composite image generator component 202. In one aspect, the segmentation component 302 may generate segmentation data for one or more data channels included in the composite multi-channel data. For example, the segmentation component 302 may generate first segmentation data indicating a segmentation of a composite image generated by the composite image generator component 202. The first segmentation data may be included in a data channel (e.g., a second data channel) of the composite multi-channel data. In addition, the segmentation component 302 may generate second segmentation data indicating a remaining segmentation of a composite image generated by the composite image generator component 202. The second segmentation data may be included in a different data channel (e.g., a third data channel) of the composite multi-channel data. The segmentation generated by the segmentation component 302 may correspond to an annotation of a composite image generated by the composite image generator component 202.

[0034] In another embodiment, the segmentation component 302 may additionally or alternatively generate one or more segmentations associated with the imaging data 114. For example, the segmentation component 302 may additionally or alternatively generate one or more segmentations of one or more real images included in the imaging data 114. In one aspect, the segmentation component 302 may generate segmentation data for one or more data channels included in the imaging data 114. For example, the segmentation component 302 may generate first segmentation data indicating a segmentation of a real image included in the imaging data 114 (e.g., a real image included in a first data channel of the imaging data 114). The first segmentation data may be included in a data channel (e.g., a second data channel) of the imaging data 114. Additionally, the segmentation component 302 may generate second segmentation data indicating a remaining segmentation of the real image included in the imaging data 114. The second segmentation data may be included in a different data channel (e.g., a third data channel) of the imaging data 114. The segmentation generated by the segmentation component 302 may additionally or alternatively correspond to an annotation of a real image included in the imaging data 114.

[0035] Reference now Figure 4 , shows a non-limiting embodiment of a system 400 according to various aspects and embodiments of the present disclosure. For the sake of brevity, repeated descriptions of similar elements employed in other embodiments described herein are omitted. The system 400 includes a generative modeling component 102 and a preprocessing component 402. The generative modeling component 102 may include a multi-channel generator component 104, a discriminator component 106, a training component 108, a processor 110, and / or a memory 112. In Figure 4In the illustrated embodiment, the multi-channel generator component 104 may include the synthetic image generator component 202 and / or the segmentation component 302. In some embodiments, the generative modeling component 102 (e.g., the processor 110 of the generative modeling component 102) may generate one or more implicit random variables 116. The pre-processing component 402 may receive the imaging data 114 and / or the mask data 404. Based on the imaging data 114 and / or the mask data 404, the pre-processing component 402 may prepare multi-channel data 406. The multi-channel data 406 may be provided to the generative modeling component 102. The mask data 404 may include one or more ground truth masks for one or more images included in the imaging data 114. In one embodiment, the mask data 404 may be a set of masks from multiple objects. For example, each medical image from the imaging data 114 may be associated with one or more masks. The mask included in the mask data 404 may be a filter for masking one or more regions in an image (e.g., in the imaging data 114). For example, a mask included in mask data 404 may include one or more weights of one or more regions of interest in an image (e.g., in imaging data 114). In one example, a mask included in mask data 404 may include a set of pixels that use binary filtering to define the location of the region of interest. Multi-channel data 406 may be multi-channel imaging data. For example, multi-channel data 406 may be multi-channel data samples of a real image included in imaging data 114. Multi-channel data 406 may additionally or alternatively include one or more segmentations of the real image included in imaging data 114. For example, in one embodiment, multi-channel data 406 may include a first data channel associated with the real image, a second data channel associated with first segmentation data indicating a segmentation of the real image, and a third data channel associated with second segmentation data indicating a remaining segmentation of the real image.

[0036] Reference now Figure 5 , showing a non-limiting embodiment of a system 500 according to various aspects and embodiments of the present disclosure. For the sake of brevity, repeated descriptions of similar elements employed in other embodiments described herein are omitted. The system 500 includes an inference component 102' and a medical imaging diagnostic process 502. The inference component 102' can be an alternative embodiment of the generative modeling component 102. For example, the inference component 102' can be used in the inference stage and / or can be used as an inference component based on generative modeling. The inference component 102' may include a multi-channel generator component 104, a processor 110 and / or a memory 112. In certain embodiments, the inference component 102' may additionally include an identifier component 106. In Figure 5In the illustrated embodiment, the multi-channel generator component 104 may include a synthetic image generator component 202, a segmentation component 302, and / or a post-processing component 503. In some embodiments, the medical imaging diagnosis process 502 may be a component included in the inference component 102' (e.g., a medical imaging diagnosis component). The generative modeling 102 may provide the synthetic multi-channel data 506 and / or other information to the medical imaging diagnosis process 502.

[0037] The post-processing component 503 may generate mask data associated with the segmentation data generated by the segmentation component 302. For example, the post-processing component 503 may generate one or more masks of one or more segmentations generated by the segmentation component 302. The mask generated by the post-processing component 503 may be a binary mask corresponding to an annotation associated with the segmentation. In one embodiment, the post-processing component 503 may generate first mask data indicating a first mask of first segmentation data (e.g., a first ground truth mask), the first segmentation data indicating a segmentation of a synthetic image generated by the synthetic image generator component 202. Additionally, the post-processing component 503 may generate second mask data indicating a second mask of second segmentation data (e.g., a second ground truth mask), the second segmentation data indicating a remaining segmentation of a synthetic image generated by the synthetic image generator component 202. The mask generated by the post-processing component 503 for the synthetic multi-channel data may be a filter for masking one or more regions in the synthetic image generated by the synthetic image generator component 202. For example, a mask included in a composite image generated by composite image generator component 202 may include binary weights of the segmentations in the composite image generated by composite image generator component 202. In one example, a mask included in a composite image generated by composite image generator component 202 may include a set of pixels that use binary filtering to define the locations of the segmentations of the composite image.

[0038] In another embodiment, the post-processing component 503 may additionally or alternatively generate first mask data indicating a first mask of first segmentation data (e.g., a first ground truth mask) indicating a segmentation of a real image included in the imaging data 114. In addition, the post-processing component 503 may generate second mask data indicating a second mask of second segmentation data (e.g., a second ground truth mask) indicating a remaining segmentation of the real image included in the imaging data 114. The mask included in the imaging data 114 may be a filter for masking one or more regions in the real image included in the imaging data 114. For example, the mask included in the imaging data 114 may include binary weights of the segmentation in the real image included in the imaging data 114. In one example, the mask included in the imaging data 114 may include a set of pixels that use binary filtering to define the location of the segmentation associated with the real image. The mask data and / or other information generated by the multi-channel generator component 104 (e.g., the synthetic image generator component 202, the segmentation component 302, and / or the post-processing component 503) may be included in the synthetic multi-channel data 506. For example, the synthesized multi-channel data 506 can be a multi-channel data sample. In one aspect, the synthesized multi-channel data 506 can include a synthesized version of the imaging data (e.g., an image). In addition, the synthesized multi-channel data 506 can include one or more segmentations of the synthesized version of the imaging data. The synthesized version of the imaging data and the one or more segmentations can be associated with unique data channels. For example, in one embodiment, the synthesized multi-channel data 506 can include a first data channel associated with a synthesized image, a second data channel associated with first segmentation data indicating a segmentation of the synthesized image, and a third data channel associated with second segmentation data indicating a remaining segmentation of the synthesized image.

[0039] The medical imaging diagnostic process 502 can use the synthetic multi-channel data 506 and / or other information provided by the generative modeling component 102 to facilitate the classification, positioning, detection and / or segmentation of one or more features associated with the input image. In one embodiment, the medical imaging diagnostic process 502 can perform deep learning to facilitate the classification, positioning, detection and / or segmentation of one or more diseases associated with the input image (e.g., medical image). In another embodiment, the medical imaging diagnostic process 502 can perform deep learning based on a convolutional neural network that receives an input image (e.g., medical image). The diseases classified and / or located by the medical imaging diagnostic process 502 can include, for example, lung disease, heart disease, tissue disease, bone disease, tumor, cancer, tuberculosis, cardiac hypertrophy, lung hypoinflation, lung opacity, hypertension, spinal degenerative disease, calcification, or other types of diseases associated with anatomical regions of the patient's body. In one aspect, the medical imaging diagnostic process 502 can determine a prediction of a disease associated with an input image (e.g., medical image). For example, the medical imaging diagnosis process 502 can determine a probability score of a disease associated with an input image (e.g., a medical image). In one example, the medical imaging diagnosis process 502 can determine a first percentage value representing a negative prognosis likelihood of the disease and a second value representing a positive prognosis likelihood of the disease.

[0040] In some embodiments, the medical imaging diagnostic process 502 can generate a multidimensional visualization associated with the classification and / or location of one or more diseases associated with an input image (e.g., a medical image). For example, the medical imaging diagnostic process 502 can generate a human-interpretable visualization associated with the classification and / or location of one or more diseases associated with an input image (e.g., a medical image). Additionally or alternatively, the medical imaging diagnostic process 502 can generate a human-interpretable visualization of an input image (e.g., a medical image). In one embodiment, the medical imaging diagnostic process 502 can generate deep learning data based on the classification and / or location of a portion of an anatomical region associated with the input image. The deep learning data can include, for example, the classification and / or location of one or more diseases located in the input image. In some embodiments, the deep learning data can include probability data indicating the probability of one or more diseases located in the input image. The probability data can be, for example, a probability array of data values ​​of one or more diseases located in the input image (e.g., a medical image). The multidimensional visualization can be a graphical representation of an input image (e.g., a medical image) that shows the classification and / or location of one or more diseases relative to the patient's body. The medical imaging diagnostic process 502 can also generate a display of a multidimensional visualization of the diagnosis provided by the medical imaging diagnostic process 502. For example, the medical imaging diagnostic process 502 can present a 2D visualization of a portion of an anatomical region on a user interface associated with a display of a user device, and the user device is such as but not limited to a computing device, a computer, a desktop computer, a laptop computer, a monitor device, a smart device, a smart phone, a mobile device, a handheld device, a tablet computer, a wearable computing device, a portable computing device, or another type of user device associated with a display. In one aspect, the multidimensional visualization can include deep learning data. In another aspect, the deep learning data can also be presented on the 3D model as one or more dynamic visual elements. In one embodiment, the medical imaging diagnostic process 502 can change the visual characteristics (e.g., color, size, hue, shadow, etc.) of at least a portion of the deep learning data associated with the multidimensional visualization based on the classification and / or positioning of the portion of the anatomical region. For example, based on the results of deep learning and / or the medical imaging diagnostic process 502, the classification and / or positioning of the portion of the anatomical region can be presented as different visual characteristics (e.g., color, size, hue or shadow, etc.). In another aspect, the medical imaging diagnostic process 502 can allow a user to zoom in or out relative to the deep learning data associated with the multidimensional visualization. For example, the medical imaging diagnostic process 502 can allow a user to zoom in or out relative to the classification and / or location of one or more diseases identified in an anatomical region of a patient's body. In this way, a user can view, analyze, and / or interact with the deep learning data associated with the multidimensional visualization of an input image (e.g., a medical image).

[0041] Reference now Figure 6 , a non-limiting embodiment of a system 600 according to various aspects and embodiments of the present disclosure is shown. For the sake of brevity, repeated descriptions of similar elements used in other embodiments described herein are omitted. System 600 can represent a generative network. For example, system 600 can represent a generative adversarial network. System 600 can include a multi-channel generator 602 and a discriminator 604. Multi-channel generator 602 can be associated with multi-channel generator component 104. Discriminator 604 can be associated with discriminator component 106. Multi-channel generator 602 can be a convolutional neural network that generates multi-channel data samples 612 based on implicit random variables 610. Multi-channel data samples 612 can be, for example, synthetic multi-channel data generated by multi-channel generator component 104. For example, implicit random variables 610 can be included in one or more implicit random variables 116. In one example, implicit random variables 610 can be implicit codes that contribute to the generation of unique synthetic images. Discriminator 604 can be a convolutional neural network that classifies multi-channel data samples 608 from multi-channel data samples 612. For example, the discriminator 604 may generate a classification label 614 corresponding to classifying the multi-channel data sample 608 from the multi-channel data sample 612. The classification label 614 may be a first classification label (e.g., a “real” classification label) or a second classification label (e.g., a “fake” classification label) for the multi-channel data sample 608 and / or the multi-channel data sample 612. The multi-channel data sample 608 may be generated based on the multi-channel real data 606. The multi-channel real data 606 and / or the multi-channel data sample 608 may correspond to the imaging data 114.

[0042] In one embodiment, a loss function 616 may be generated based on the classification label 614. The loss function 616 may be one or more loss functions generated by the training component 108. In one embodiment, the multi-channel real data 606 may be a real image (e.g., a real medical image). In some embodiments, preprocessing 605 may be performed to prepare the multi-channel real data 606. For example, the preprocessing 605 may prepare the multi-channel real data 606 based on the imaging data 601 and / or the mask data 603. The mask data 603 may include one or more corresponding masks for one or more images included in the imaging data 601. In addition, the multi-channel data sample 608 may include multiple data channels. For example, the multi-channel data sample 608 may include a data channel containing a real image (e.g., a real medical image). The multi-channel data sample 608 may also include one or more data channels containing segmentation of the real image. In some embodiments, the multi-channel data sample 608 may also include one or more data channels containing a mask for segmentation of the real image. In another embodiment, the multi-channel data sample 612 may include multiple data channels. For example, the multi-channel data samples 612 may include a data channel containing a synthetic image (e.g., a synthetic medical image). The multi-channel data samples 612 may also include one or more data channels containing a segmentation of the synthetic image. In some embodiments, the multi-channel data samples 612 may also include one or more data channels containing a mask for the segmentation of the synthetic image. In some embodiments, the loss function 616 may tune the multi-channel generator 602 and / or the discriminator 604. In one example, the loss function 616 may be an error gradient for modifying one or more parameters of the multi-channel generator 602 and / or the discriminator 604. For example, the loss function 616 may be generated based on the multi-channel data samples 608 and / or the multi-channel data samples 612 to tune the generation network. In some embodiments, the discriminator 604 may be discarded from the generation network in response to determining that the multi-channel generator 602 is trained. For example, in an embodiment associated with the inference stage, the discriminator 604 may be discarded from the generation network in response to determining that the multi-channel generator 602 is trained.

[0043] Reference now Figure 7, a non-limiting embodiment of a system 700 according to various aspects and embodiments of the present disclosure is shown. For the sake of brevity, repeated descriptions of similar elements employed in other embodiments described herein are omitted. The system 700 may be employed, for example, during an inference phase. The system 700 may include a multi-channel generator 602. In one embodiment, the multi-channel generator 602 may generate a synthetic image 702, a segmentation 704, and a segmentation 706. For example, the multi-channel generator component 104 may generate the synthetic image 702, the segmentation 704, and the segmentation 706. The multi-channel generator 602 may generate the synthetic image 702, for example, based on an implicit random variable 610 and / or other data (e.g., one or more implicit random variables 116). The synthetic image 702 may be, for example, a synthetic medical image (e.g., a synthetic x-ray image, etc.). The segmentation 704 may be a segmentation of the synthetic image 702. For example, the segmentation 704 may correspond to an annotation of the synthetic image 702. The segmentation 706 may be a remaining segmentation of the synthetic image 702. For example, segmentation 706 may be a complementary segmentation compared to segmentation 704. In one embodiment, the composite image 702 may be included in a first data channel of the composite multi-channel data (e.g., multi-channel data sample 612) generated by the multi-channel generator component 104. Segmentation 704 may be included in a second data channel of the composite multi-channel data (e.g., multi-channel data sample 612) generated by the multi-channel generator component 104. In addition, segmentation 706 may be included in a third data channel of the composite multi-channel data (e.g., multi-channel data sample 612) generated by the multi-channel generator component 104. In this way, concurrent images and corresponding multi-channel auxiliary data may be provided for annotation at the pixel level on the composite image without user intervention. It should be appreciated that in some embodiments, the multi-channel generator 602 may generate more than two segmentations (e.g., more than three data channels may be generated).

[0044] Reference now Figure 8, a non-limiting embodiment of a system 800 according to various aspects and embodiments of the present disclosure is shown. For the sake of brevity, repeated descriptions of similar elements used in other embodiments described herein are omitted. System 800 can be used, for example, during the inference phase (e.g., for binary mask generation). System 800 can include post-processing 802. Post-processing 802 can be associated with post-processing component 503. In one embodiment, post-processing 802 can perform processing based on segmentation 704 to generate mask 804. In addition, post-processing 802 can perform processing based on segmentation 706 to generate mask 806. Compared to mask 804, mask 806 can be a relative mask. For example, compared to mask 804, mask 806 can be a complementary mask. Mask 804 and / or mask 806 can be used, for example, for corresponding model training data. Mask 804 can be a binary mask corresponding to an annotation associated with segmentation 704. In one example, mask 804 can be a ground truth mask corresponding to an annotation associated with segmentation 704. The mask 806 may be a binary mask corresponding to the annotation associated with the segmentation 706. In one example, the mask 806 may be a ground truth mask corresponding to the annotation associated with the segmentation 706. In one embodiment, the mask 804 may be included in a second data channel of the synthesized multi-channel data (e.g., the multi-channel data sample 612) generated by the multi-channel generator component 104. Additionally, the mask 806 may be included in a third data channel of the synthesized multi-channel data (e.g., the multi-channel data sample 612) generated by the multi-channel generator component 104.

[0045] Fig. 9 Methods and / or flow charts according to the disclosed subject matter are shown. In order to simplify the description, the method is described and shown as a series of actions. It should be understood and recognized that the present invention is not limited by the actions and / or action sequences shown, for example, the actions can occur in various orders and / or simultaneously, and have other actions not presented and described herein. In addition, all the actions shown may not be required to implement the method according to the disclosed subject matter. In addition, those skilled in the art will understand and recognize that the method can alternatively be represented as a series of interrelated states by state diagrams or events. In addition, it should be further recognized that the methods disclosed below and throughout the specification can be stored on articles of manufacture to facilitate the transmission and transfer of such methods to computers. As used herein, the term "article of manufacture" is intended to encompass a computer program accessible from any computer-readable device or storage medium.

[0046] refer to Fig. 9, shows a non-limiting embodiment of a method 900 according to one aspect of the present invention, the method being used to provide concurrent image and corresponding multi-channel auxiliary data generation for a generative model. At 902, synthetic multi-channel data associated with a synthetic version of imaging data is generated by a system operably coupled to a processor (e.g., by a multi-channel generator component 104). The synthetic multi-channel data can be multi-channel data samples. In one aspect, the synthetic multi-channel data can include a synthetic version of the imaging data. In addition, the synthetic multi-channel data can include one or more segmentations of the synthetic version of the imaging data. The synthetic version of the imaging data and the one or more segmentations can be associated with a unique data channel. For example, in one embodiment, the synthetic multi-channel can include a first data channel associated with a synthetic image, a second data channel associated with first segmentation data indicating a segmentation of the synthetic image, and a third data channel associated with second segmentation data indicating a remaining segmentation of the synthetic image. In another embodiment, the composite multi-channel may include a first data channel associated with a composite image, a second data channel associated with first segmentation data indicating a first segmentation of the composite image, a third data channel associated with second segmentation data indicating a remaining portion of the first segmentation of the composite image, a fourth data channel associated with the third segmentation data indicating a second segmentation of the composite image, and a fifth data channel associated with fourth segmentation data indicating a remaining portion of the second segmentation of the composite image.

[0047] At 904, multi-channel imaging data and / or synthesized multi-channel data are predicted by the system (e.g., by the discriminator component 106) through the first prediction category set or the second prediction category set. In one embodiment, the category set may include a multi-channel image (e.g., a channel level label), an image label (e.g., an image level label), and / or a clinical label. The first prediction category set may be associated with a first classification (e.g., a first classification label), and the second prediction category set may be associated with a second classification (e.g., a second classification label). For example, the image level label may correspond to a "true" label of the image (e.g., a "true" classification label) or a "false" label of the image (e.g., a "false" classification label). In another example, the channel level label may correspond to a "true" label of the multi-channel image (e.g., a "true" classification label) or a "false" label of the multi-channel image (e.g., a "false" classification label). The multi-channel imaging data may be a multi-channel data sample. In one aspect, the multi-channel imaging data may include a true version of the imaging data. In addition, the multi-channel imaging data may include one or more segmentations of the true version of the imaging data. The true version of the imaging data and the one or more segmentations may be associated with a unique data channel. For example, in one embodiment, the multi-channel imaging data may include a first data channel associated with a real image, a second data channel associated with the first segmentation data indicating the segmentation of the real image, and a third data channel associated with the second segmentation data indicating the remaining segmentation of the real image. In another embodiment, the multi-channel imaging data may include a first data channel associated with a real image, a second data channel associated with the first segmentation data indicating the first segmentation of the real image, a third data channel associated with the second segmentation data indicating the remaining portion of the first segmentation of the real image, a fourth data channel associated with the third segmentation data indicating the second segmentation of the real image, and a fifth data channel associated with the fourth segmentation data indicating the remaining portion of the second segmentation of the real image. The real image may be a two-dimensional image or a three-dimensional image generated by one or more medical imaging devices. For example, the real image may be an electromagnetic radiation image captured via a set of sensors (e.g., a set of sensors associated with a medical imaging device). The real image may be received directly from one or more medical imaging devices. Alternatively, the real image may be stored in one or more databases that receive and / or store the real image associated with one or more medical imaging devices. The medical imaging device may be, for example, an x-ray device, a CT device, another type of medical imaging device, etc.

[0048] At 906, a generative adversarial network model is trained by the system (e.g., by the training component 108) based on the first predicted category set or the second predicted category set of the multi-channel imaging data and / or the synthetic multi-channel data. The generative adversarial network model can be, for example, a generative network. The generative adversarial network model can be used to generate images and / or corresponding annotations of images. For example, the generative adversarial network model can be used to generate medical images and / or corresponding annotations of medical images. In one embodiment, a loss function can be generated based on the first predicted label or the second predicted label of the multi-channel imaging data and / or the synthetic multi-channel data. The loss function can be used to tune the generative artificial intelligence model. For example, a multi-channel generator or discriminator of the generative artificial intelligence model can be tuned based on the first predicted label or the second predicted label of the multi-channel imaging data and / or the synthetic multi-channel data. The loss function can be, for example, a cross entropy error function. In addition, the loss function can be used to improve the classification output from the generative adversarial network model. In one embodiment, the loss function can be a loss function associated with the loss between the first data channel, the second data channel, and / or the third data channel. In one embodiment, the loss function may be a loss function associated with a loss between the first data channel, the second data channel, the third data channel, the fourth data channel, and / or the fifth data channel.

[0049] At 908, it is determined whether the generative adversarial network model meets the defined criteria. For example, it can be determined whether the multi-channel generator and / or discriminator for the generative adversarial network model is properly tuned. If not, the method 900 can return to 902. If yes, the method 900 can end. In some embodiments, the method 900 can additionally or alternatively include generating first mask data indicating a first ground truth mask of the first segmented data. In addition, the method 900 can additionally or alternatively include generating second mask data indicating a second ground truth mask of the second segmented data.

[0050] The aforementioned systems and / or devices have been described with respect to the interaction between several components. It should be appreciated that such systems and components may include those components or subcomponents specified herein, some specified components or subcomponents, and / or additional components. Subcomponents may also be implemented as components that are communicatively coupled to other components except those included in the parent component. Further, one or more components and / or subcomponents may be combined into a single component that provides aggregate functionality. Components may also interact with one or more other components that are not specifically described herein for brevity but are known to those skilled in the art.

[0051] To provide context for various aspects of the disclosed subject matter, Fig.10 and Fig.11 The following discussion is intended to provide a brief, general description of a suitable environment in which various aspects of the disclosed subject matter may be implemented.

[0052] refer to Fig.10 , a suitable environment 1000 for implementing various aspects of the present disclosure includes a computer 1012. The computer 1012 includes a processing unit 1014, a system memory 1016, and a system bus 1018. The system bus 1018 couples system components including, but not limited to, the system memory 1016 to the processing unit 1014. The processing unit 1014 can be any of a variety of available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 1014.

[0053] The system bus 1018 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus or external bus, and / or a local bus using any of a variety of available bus architectures including, but not limited to, Industry Standard Architecture (ISA), Micro Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), card bus, Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association (PCMCIA), FireWire (IEEE 1394), and Small Computer System Interface (SCSI).

[0054] The system memory 1016 includes volatile memory 1020 and nonvolatile memory 1022. A basic input / output system (BIOS), which contains the basic routines for transferring information between elements within the computer 1012, such as during startup, is stored in the nonvolatile memory 1022. By way of example and not limitation, the nonvolatile memory 1022 may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). The volatile memory 1020 includes random access memory (RAM), which acts as external cache memory. By way of example and not limitation, RAM can be provided in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM.

[0055] Computer 1012 also includes removable / non-removable, volatile / nonvolatile computer storage media. Fig.10 For example, a disk storage device 1024 is shown. The disk storage device 1024 includes, but is not limited to, devices such as a disk drive, a floppy disk drive, a tape drive, a Jaz drive, a Zip drive, an LS-100 drive, a flash memory card, or a memory stick. The disk storage device 1024 can also include storage media, including but not limited to optical disk drives, such as a compact disk ROM device (CD-ROM), a CD recordable drive (CD-R drive), a CD rewritable drive (CD-RW drive), or a digital versatile disk ROM drive (DVD-ROM), alone or in combination with other storage media. In order to facilitate connecting the disk storage device 1024 to the system bus 1018, a removable / non-removable interface, such as an interface 1026, is typically used.

[0056] Fig.10 Also depicted is software that acts as an intermediary between a user and the basic computer resources described in the appropriate operating environment 1000. For example, such software includes an operating system 1028. The operating system 1028, which may be stored on disk storage 1024, is used to control and allocate the resources of the computer system 1012. System applications 1030 utilize the management of resources by the operating system 1028 through, for example, program modules 1032 and program data 1034 stored on system memory 1016 or disk storage 1024. It should be appreciated that the present disclosure may be implemented with various operating systems or combinations of operating systems.

[0057] The user enters commands or information into the computer 1012 through one or more input devices 1036. The input devices 1036 include, but are not limited to, pointing devices such as a mouse, trackball, stylus, touchpad, keyboard, microphone, joystick, gamepad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, webcam, etc. These and other input devices are connected to the processing unit 1014 through the system bus 1018 via one or more interface ports 1038. The one or more interface ports 1038 include, for example, serial ports, parallel ports, game ports, and universal serial bus (USB). One or more output devices 1040 use some of the same types of ports as the one or more input devices 1036. Thus, for example, a USB port can be used to provide input to the computer 1012 and output information from the computer 1012 to the output device 1040. Output adapters 1042 are provided to illustrate that there are some output devices 1040 such as monitors, speakers, and printers, as well as other output devices 1040 that require special adapters. By way of example and not limitation, output adapters 1042 include video and sound cards that provide a means of connection between output devices 1040 and system bus 1018. It should be noted that other devices and / or systems of devices provide input and output capabilities, such as one or more remote computers 1044.

[0058] The computer 1012 can operate in a networked environment using logical connections to one or more remote computers, such as one or more remote computers 1044. The one or more remote computers 1044 can be personal computers, servers, routers, network PCs, workstations, microprocessor-based devices, peer devices or other public network nodes, etc., and typically include many or all of the elements described relative to the computer 1012. For the purpose of simplicity, only a memory storage device 1046 with one or more remote computers 1044 is shown. The one or more remote computers 1044 are logically connected to the computer 1012 through a network interface 1048 and then physically connected via a communication connection 1050. The network interface 1048 covers wired and / or wireless communication networks, such as local area networks (LANs), wide area networks (WANs), cellular networks, etc. LAN technologies include fiber distributed data interface (FDDI), copper distributed data interface (CDDI), Ethernet, token ring, etc. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks such as integrated services digital networks (ISDN) and variations thereon, packet switching networks, and digital subscriber lines (DSL).

[0059] Communication connection(s) 1050 refers to the hardware / software used to connect the network interface 1048 to the bus 1018. Although the communication connection 1050 is shown within the computer 1012 for clarity of illustration, it may also be external to the computer 1012. For example purposes only, the hardware / software required to connect to the network interface 1048 includes internal and external technologies, such as modems, including conventional telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.

[0060] Fig.11 1 is a schematic block diagram of a sample computing environment 1100 with which the subject matter of the present disclosure can interact. System 1100 includes one or more clients 1110. One or more clients 1110 can be hardware and / or software (e.g., threads, processes, computing devices). System 1100 also includes one or more servers 1130. Therefore, in addition to other models, system 1100 can correspond to a two-tier client-server model or a multi-tier model (e.g., client, middle-tier server, data server). One or more servers 1130 can also be hardware and / or software (e.g., threads, processes, computing devices). For example, server 1130 can accommodate threads to perform conversions by adopting the present disclosure. One possible communication between client 1110 and server 1130 can be in the form of data packets transmitted between two or more computer processes.

[0061] The system 1100 includes a communication framework 1150 that can be used to facilitate communication between one or more clients 1110 and one or more servers 1130. The one or more clients 1110 are operably connected to one or more client data repositories 1120, which can be used to store information local to the one or more clients 1110. Similarly, the one or more servers 1130 are operably connected to one or more server data repositories 1140, which can be used to store information local to the servers 1130.

[0062] It should be noted that various aspects or features of the present disclosure can be utilized in substantially any wireless telecommunication or radio technology, for example, Wi-Fi; Bluetooth; Worldwide Interoperability for Microwave Access (WiMAX); Enhanced General Packet Radio Service (Enhanced GPRS); 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE); 3rd Generation Partnership Project 2 (3GPP2) Ultra Mobile Broadband (UMB); 3GPP Universal Mobile Telecommunications System (UMTS); High Speed ​​Packet Access (HSPA); High Speed ​​Downlink Packet Access (HSDPA); High Speed ​​Uplink Packet Access (HSUPA); GSM (Global System for Mobile Communications) EDGE (Enhanced Data Rates for GSM Evolution) Radio Access Network (GERAN); UMTS Terrestrial Radio Access Network (UTRAN); LTE Advanced (LTE-A); etc. In addition, some or all aspects described herein can be utilized in traditional telecommunication technologies (e.g., GSM). In addition, mobile as well as non-mobile networks (e.g., the Internet, data service networks such as Internet Protocol Television (IPTV), etc.) can utilize aspects or features described herein.

[0063] Although the subject matter has been described above in the general context of computer executable instructions of a computer program running on one and / or multiple computers, it will be appreciated by those skilled in the art that the present disclosure may also or may be implemented in combination with other program modules. Typically, program modules include routines, programs, components, data structures, etc. that perform specific tasks and / or implement specific abstract data types. In addition, it will be appreciated by those skilled in the art that the method of the present invention may be practiced with other computer system configurations, including single-processor or multi-processor computer systems, small computing devices, large computers, and personal computers, handheld computing devices (e.g., PDAs, phones), microprocessor-based or programmable consumer or industrial electronics, etc. The illustrated aspects may also be practiced in a distributed computing environment, in which tasks are performed by a remote processing device linked by a communication network. However, some (if not all) aspects of the present disclosure may be practiced on a stand-alone computer. In a distributed computing environment, program modules may be located in local and remote memory storage devices.

[0064] As used in this application, the terms "component", "system", "platform", "interface", etc. may refer to and / or may include computer-related entities or entities associated with an operating machine having one or more specific functions. The entities disclosed herein may be hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, and / or a computer. By way of example, both an application running on a server and a server may be a component. One or more components may reside within a process and / or a thread of execution, and a component may be located on one computer and / or distributed between two or more computers.

[0065] In another example, the corresponding components can be executed according to various computer-readable media having various data structures stored thereon. The components can communicate via local and / or remote processes, such as according to signals having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems across a network (such as the Internet) via a signal). As another example, the component can be a device with a specific function provided by a mechanical part operated by an electrical or electronic circuit, which is operated by a software or firmware application executed by a processor. In this case, the processor can be inside or outside the device and can execute at least a portion of the software or firmware application. As yet another example, the component can be a device that provides a specific function by an electronic component rather than a mechanical part, wherein the electronic component may include a processor or other devices for executing software or firmware that at least partially gives the electronic component functionality. In one aspect, the component can, for example, simulate the electronic component via a virtual machine in a cloud computing system.

[0066] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise or clear from the context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing instances. Furthermore, the articles "a" and "an" used in the specification and the drawings should generally be construed to mean "one or more" unless specified otherwise or clear from the context to be directed to the singular.

[0067] As used herein, the terms "example" and / or "exemplary" are used to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited to such examples. In addition, any aspect or design described herein as "example" and / or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to exclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.

[0068] Standard programming or engineering techniques can be used to implement various aspects or features described herein as methods, devices, systems, or products. In addition, various aspects or features disclosed in the present disclosure can be implemented by implementing at least one or more program modules in the methods disclosed herein, and the program modules are stored in a memory and executed at least by a processor. Other combinations of hardware and software or hardware and firmware can implement or implement the aspects described herein, including one or more methods disclosed. As used herein, the term "product" can cover computer programs accessible from any computer-readable device, carrier, or storage medium. For example, a computer-readable storage medium can include, but is not limited to, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe ...), an optical disk (e.g., a compact disk (CD), a digital versatile disk (DVD), a blue optical disk (BD) ...), a smart card, and a flash memory device (e.g., a card, a stick, a key drive ...), etc.

[0069] As used in this specification, the term "processor" may refer to substantially any computational processing unit or device, including but not limited to a single-core processor; a single processor with software multithreaded execution capability; a multi-core processor; a multi-core processor with software multithreaded execution capability; a multi-core processor with hardware multithreading technology; a parallel platform; and a parallel platform with distributed shared memory. In addition, a processor may refer to an integrated circuit, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. In addition, the processor may utilize nanoscale architectures (such as, but not limited to, transistors, switches, and gates based on molecules and quantum dots) in order to optimize space usage or enhance the performance of user devices. The processor may also be implemented as a combination of computational processing units.

[0070] In this disclosure, terms such as "storage," "storage device," "data storage," "data storage device," "database," and substantially any other information storage component related to the operation and functionality of the component are used to refer to a "memory component," an entity embodied in a "memory," or a component that includes a memory. It should be appreciated that the memory and / or memory components described herein may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0071] By way of example and not limitation, the non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable ROM (EEPROM), a flash memory, or a non-volatile random access memory (RAM) (e.g., a ferroelectric RAM (FeRAM)). For example, the volatile memory may include a RAM, which may act as an external cache memory. By way of example and not limitation, the RAM may be provided in a variety of forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). In addition, the disclosed memory components of the systems or methods herein are intended to include, but are not limited to, including these and any other suitable types of memory.

[0072] It should be recognized and appreciated that components described with respect to a particular system or method may include the same or similar functionality as corresponding components described with respect to other systems or methods disclosed herein (eg, separately named components or similarly named components).

[0073] What has been described above includes examples of systems and methods that provide the advantages of the present disclosure. Of course, it is not possible to describe every conceivable combination of components or methods for the purpose of describing the present disclosure, but one of ordinary skill in the art will recognize that many additional combinations and permutations of the present disclosure are possible. In addition, to the extent that the terms "including," "having," "having," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be inclusive in a manner similar to the term "comprising," as interpreted when "including" is used as a transitional word in the claims.

Claims

1. A system for image processing, comprising: a memory storing computer executable components; and a processor that executes the computer executable components stored in the memory, wherein the computer executable components include: a multi-channel generator component that generates synthesized multi-channel data associated with a synthesized version of the imaging data; a discriminator component that predicts the multi-channel imaging data and the synthesized multi-channel data by a first prediction class set or a second prediction class set; and A training component, wherein the training component uses the first prediction category set or the second prediction category set of the synthetic multi-channel data to train a generative adversarial network model; wherein the multi-channel generator component generates a first data channel associated with a composite image, a second data channel associated with first segmentation data indicative of a segmentation of the composite image, and a third data channel associated with second segmentation data indicative of a remaining segmentation of the composite image; The computer executable components further include: A post-processing component generates first mask data indicating a first ground truth mask for the first segmented data and second mask data indicating a second ground truth mask for the second segmented data.

2. The system of claim 1, wherein the training component utilizes a loss function to tune the generative adversarial network model based on the first set of prediction categories or the second set of prediction categories of the synthetic multi-channel data.

3. A system according to claim 1, wherein the discriminator component predicts the synthetic multi-channel data through the first prediction category set or the second prediction category set based on the following: the first data channel associated with the synthetic image, the second data channel associated with the first segmented data, and the third data channel associated with the second segmented data.

4. The system of claim 1, wherein the training component tunes the generative adversarial network model based on a loss function associated with losses between a first data channel, the second data channel, and the third data channel.

5. A method for image processing, comprising executing, using a processor operatively coupled to a memory, a computer executable component to perform the following actions: generating synthetic multi-channel data associated with the synthetic version of the imaging data; predicting the multi-channel imaging data and the synthesized multi-channel data by the first prediction category set or the second prediction category set; and training a generative adversarial network model based on the first prediction category set or the second prediction category set of the synthetic multi-channel data, Wherein said generating said synthesized multi-channel data comprises: generating a first data channel associated with a composite image; generating a second data channel associated with first segmentation data indicative of a segmentation of the composite image; and generating a third data channel associated with second segmentation data indicative of a remaining segmentation of the composite image; The method further comprises: First mask data indicating a first ground truth mask for the first segmented data and second mask data indicating a second ground truth mask for the second segmented data are generated. 6 . The method of claim 5 , further comprising utilizing a loss function to tune the generative adversarial network model based on the first set of predicted categories or the second set of predicted categories of the synthetic multi-channel data.

7. A method according to claim 5, wherein the predicting of the multi-channel imaging data includes predicting the synthetic multi-channel data through the first prediction category set or the second prediction category set based on the following: the first data channel associated with the synthetic image, the second data channel associated with the first segmented data, and the third data channel associated with the second segmented data.

8. A computer-readable storage device comprising instructions that, in response to being executed, cause a system comprising a processor to perform operations comprising: generating synthetic multi-channel data associated with the synthetic version of the imaging data; predicting the multi-channel imaging data and the synthesized multi-channel data by the first prediction category set or the second prediction category set; and training a generative adversarial network model based on the first prediction category set or the second prediction category set of the synthetic multi-channel data, Wherein said generating said synthesized multi-channel data comprises: generating a first data channel associated with a composite image; generating a second data channel associated with first segmentation data indicative of a segmentation of the composite image; and generating a third data channel associated with second segmentation data indicative of a remaining segmentation of the composite image; The operations also include: First mask data indicating a first ground truth mask for the first segmented data and second mask data indicating a second ground truth mask for the second segmented data are generated.

Images