Shared potential representation of multi-modal data sets

Abstract

One embodiment of the present invention proposes a technique for determining a potential space between two or more domains. The technique includes executing a first generator neural network to convert a first set of training data associated with a first domain to a first set of training outputs based on a first set of potential values associated with the first domain. The technique further includes executing a second generator neural network to convert the first set of training outputs to a second set of training outputs based on a second set of potential values associated with a second domain. The technique further includes training a first generator neural network based on a first set of losses calculated between the first set of potential values and the second set of potential values.

Description

[0001] Cross-reference to related applications This application claims the benefit of priority to U.S. Patent Application Serial No. 18 / 315,172, filed May 10, 2023, entitled "SHAREDLATENT REPRESENTATIONS OF MULTIMODAL DATASETS". The subject matter of that related application is hereby incorporated herein by reference. Technical Field

[0002] Embodiments of this disclosure generally relate to machine learning, and more specifically, to determining a shared potential representation of both paired multimodal datasets and unpaired multimodal datasets. Background Technology

[0003] Biological systems exhibit a wide range of complex phenomena that influence and / or reflect broader properties such as health, aging, resilience, and / or responses to interventions or treatments. In recent years, various technologies have been developed to collect and analyze biological data in order to understand the underlying patterns or mechanisms of these phenomena and / or properties. For example, imaging techniques such as cryo-electron tomography (CryoET) can be used to generate high-resolution three-dimensional (3D) views of biological samples such as cells, tissues, organisms, or macromolecules. In another example, various “omics” disciplines can be used to study biomolecules that represent the structure, function, and dynamics of biological systems. Among these “omics” disciplines, genomics involves the study of all deoxyribonucleic acid (DNA) within a biological system (e.g., an organism); proteomics can be used to characterize proteins produced or modified by a biological system; transcriptomics involves the study of ribonucleic acid (RNA) molecules transcribed from the genome of a biological system; metabolomics involves the study of small molecule metabolites within a biological system; epigenomics involves the study of epigenetic modifications of the genetic material of a biological system; phenomics involves the study of observable characteristics or traits of a biological system; and metagenomics can be used to study genetic material recovered from environmental samples.

[0004] However, identifying or characterizing patterns or relationships between large biological datasets generated using different technologies can be challenging. More specifically, each type of biological data can provide a noisy and incomplete view of the state or function of a biological system. Furthermore, physiological, functional, imaging, and / or other types of biological data may be difficult to correlate unless these types of data are collected from a single biological sample using a single detection method.

[0005] As mentioned above, what is needed in this field is a more effective technology for analyzing and integrating data across different biological domains. Summary of the Invention

[0006] One embodiment of the present invention provides a technique for determining a latent space between two domains. The technique includes: executing a first generator neural network to transform a first set of training data associated with the first domain into a first set of training outputs based on a first set of latent values ​​associated with the first domain. The technique further includes: executing a second generator neural network to transform the first set of training outputs into a second set of training outputs based on a second set of latent values ​​associated with the second domain. The technique also includes: training the first generator neural network based on a first set of losses calculated between the first set of latent values ​​and the second set of latent values.

[0007] Compared to existing technologies, one technical advantage of the disclosed technology is that the shared latent space learned by the first and second generator neural networks can semantically correlate different types of data in a meaningful way. Therefore, compared to traditional machine learning techniques that cannot generate shared latent spaces across multiple biological (or other types of) domains, the disclosed technology can more completely represent biological systems (or other types of entities) represented using these types of data. Another technical advantage of the disclosed technology is that it can train the first and second generator neural networks to learn the shared latent space even in the absence of paired training data from different domains. Therefore, compared to existing methods that use paired data to learn latent spaces related to multiple domains, the disclosed technology can use larger datasets to generate more complete and semantically meaningful shared latent spaces. These technical advantages provide one or more technical improvements over existing methods. Attached Figure Description

[0008] To gain a more detailed understanding of the features of the various embodiments described above, the inventive concept briefly outlined above can be described in more specific terms with reference to the various embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only illustrate typical embodiments of the inventive concept and should not be considered as any limitation on the scope of the invention; other equally effective embodiments exist.

[0009] Figure 1 A system configured to implement one or more aspects of various embodiments is shown.

[0010] Figure 2 Based on various embodiments Figure 1 A more detailed illustration of the training engine and execution engine is provided.

[0011] Figure 3 Various embodiments are shown. Figure 1 The training engine in the diagram operates when training two generators to learn the potential space shared by two domains.

[0012] Figure 4Examples of performing related operations on data from a first domain and data from a second domain using a shared potential space according to various embodiments are shown.

[0013] Figure 5 This is a flowchart of method steps for training a machine learning model to learn a shared latent space among multiple domains, according to various embodiments.

[0014] Figure 6 This is a flowchart of method steps for analyzing data associated with multiple domains according to various embodiments. Detailed Implementation

[0015] Numerous specific details are set forth in the following description in order to provide a more thorough understanding of the various embodiments. However, those skilled in the art will understand that the inventive concept can be practiced even without one or more of these specific details.

[0016] As mentioned above, data from a specific biological domain may contain noise and cannot fully reflect the overall health, state, or function of a biological system. Furthermore, correlating different types of biological data with each other, and / or with the potential health, state, or function of a biological system, can be very difficult, especially when biological data is not collected from a single biological sample in a single test.

[0017] To better understand biological data and / or other types of data across multiple domains, the disclosed techniques train and execute machine learning models to generate shared latent representations of multimodal datasets. The machine learning model includes multiple encoders and multiple decoders. Each encoder transforms data from the corresponding domain into a fixed-length latent representation in a low-dimensional latent space. Each decoder transforms the latent representation from the low-dimensional latent space into data from the corresponding domain. A given encoder can be paired with a decoder in the same domain to form a generator capable of reconstructing data in that domain. A given encoder in a first domain can also, or alternatively, be paired with a decoder in a second domain to form a generator capable of transforming data from the first domain into data in the second domain.

[0018] For example, a machine learning model may include one or more encoders that transform samples from image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from perturbed cells, data from young systems, data from aged systems, and / or other types of biological data into corresponding latent values. The machine learning model may also include one or more decoders that transform each latent value into one or more samples from the same and / or different domains. In this example, an encoder for image-based data may be paired with a decoder for image-based data to transform image-based data into a latent representation and reconstruct image-based data from the latent representation. An encoder for image-based data may be paired with different decoders for sequencing-based data to perform conversions between image-based and sequencing-based data, compare image-based and sequencing-based data, and / or perform other analyses or predictions related to image-based and / or sequencing-based data.

[0019] To train the machine learning model, the training data is first transformed into a first set of latent values ​​using a first encoder for a first domain. Then, the first set of latent values ​​is transformed into a first set of training outputs using a first decoder for a second domain. The first set of training outputs is transformed into a second set of latent values ​​using a second encoder for the second domain. The second set of latent values ​​is transformed into a second set of training outputs using a second decoder for the first domain. The encoder and decoder are trained using several losses calculated using the first and second sets of latent values, the first and second sets of training outputs, and / or outputs generated by one or more discriminator models based on these latent values ​​and / or training outputs. These losses are used to ensure that at least a portion of the latent space occupied by the latent values ​​is invariant to the domain from which these latent values ​​are derived.

[0020] When it is necessary to learn a shared latent space across more than two domains, a "daisy-chain" approach can be used to train the encoders and decoders for each domain. For example, during the first training phase, the techniques described above can be used to train a first encoder and a first decoder for the first domain, and a second encoder and a second decoder for the second domain, to learn the shared latent space between the two domains. During the second training phase, a third encoder for the third domain can be paired with the first decoder and / or the second decoder, and vice versa. The same training techniques can be repeated using new encoder-decoder pairs, thereby sharing the latent space associated with the third domain with the latent spaces of the first and second domains. Additional training phases can also be added to extend the shared latent space to additional domains.

[0021] System Overview Figure 1 This is a block diagram illustrating a computer system 100 configured to implement one or more aspects of various embodiments. In one embodiment, the computer system 100 includes a desktop computer, laptop computer, smartphone, personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and selectively display images, and is suitable for implementing one or more embodiments. The computer system 100 may also, or alternatively, include machines or processing nodes operating in a data center, cluster, or cloud computing environment that provide scalable computing resources (optionally in the form of a service) over a network.

[0022] As shown in the figure, the computer system 100 includes, but is not limited to, a central processing unit (CPU) 102 and a system memory 104, which are coupled to a parallel processing subsystem 112 via a memory bridge 105 and a communication path 113. The memory bridge 105 is also coupled to an I / O (input / output) bridge 107 via a communication path 106, and the I / O bridge 107 is coupled to a switch 116.

[0023] I / O bridge 107 is configured to receive user input from optional input device 108 (e.g., keyboard or mouse) and forward the input to CPU 102 for processing via communication path 106 and memory bridge 105. In some embodiments, computer system 100 may be a server in a cloud computing environment. In such embodiments, computer system 100 may not have input device 108. Instead, computer system 100 may receive equivalent input by receiving commands transmitted over the network in message form and received via network adapter 118. In one embodiment, switch 116 is configured to provide connectivity between I / O bridge 107 and other components of computer system 100, such as network adapter 118 and various add-on cards 120 and 121.

[0024] In one embodiment, I / O bridge 107 is coupled to system disk 114, which can be configured to store content, applications, and data for use by CPU 102 and parallel processing subsystem 112. In one embodiment, system disk 114 provides non-volatile storage for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROMs (CD-ROMs), DVD-ROMs (Digital Versatile Optical Discs), Blu-ray discs, HD-DVDs (High Definition DVDs), or other magnetic, optical, or solid-state storage devices. In various embodiments, other components, such as universal serial buses or other port connections, compact disc drives, digital versatile optical disc drives, film recording devices, etc., may also be connected to I / O bridge 107.

[0025] In various embodiments, memory bridge 105 may be a northbridge chip, and I / O bridge 107 may be a southbridge chip. Furthermore, communication paths 106 and 113, as well as other communication paths within computer system 100, may be implemented using any technically suitable protocol, including but not limited to AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art.

[0026] In some embodiments, the parallel processing subsystem 112 includes a graphics subsystem that passes pixels to an optional display device 110, which may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, etc. In these embodiments, the parallel processing subsystem 112 includes circuitry optimized for graphics and video processing, such as video output circuitry. Such circuitry may be incorporated into one or more parallel processing units (PPUs) (also referred to herein as parallel processors) included within the parallel processing subsystem 112. In other embodiments, the parallel processing subsystem 112 includes circuitry optimized for general and / or computational processing. Again, such circuitry may be incorporated into one or more PPUs included within the parallel processing subsystem 112, which are configured to perform such general and / or computational operations. In still other embodiments, one or more PPUs included within the parallel processing subsystem 112 may be configured to perform graphics processing, general processing, and computational processing operations. The system memory 104 includes at least one device driver configured to manage the processing operations of one or more parallel processing units (PPUs) within the parallel processing subsystem 112.

[0027] Parallel processing subsystem 112 can be with Figure 1One or more other components can be integrated to form a single system. For example, the parallel processing subsystem 112 can be integrated with the CPU 102 and other interconnect circuitry on a single chip to form a system-on-a-chip (SoC).

[0028] In one embodiment, CPU 102 is the main processor of computer system 100, responsible for controlling and coordinating the operation of other system components. In one embodiment, CPU 102 issues commands to control the operation of the PPU. In some embodiments, communication path 113 is a PCI Express link, in which a dedicated channel is allocated to each PPU, as is known in the art. Other communication paths may also be used. The PPU advantageously implements a highly parallel processing architecture. The PPU can be equipped with any number of local parallel processing memories (PP memories).

[0029] It should be understood that the system illustrated herein is merely illustrative and can be varied and modified according to actual circumstances. First, the system's functionality can be distributed across multiple nodes in a distributed and / or cloud computing system. Second, the connection topology, including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112, can be modified as needed. For example, in some embodiments, the system memory 104 can be directly connected to the CPU 102 instead of via memory bridge 105, while other devices communicate with the system memory 104 through memory bridge 105 and CPU 102. In another example, the parallel processing subsystem 112 can be connected to I / O bridge 107 or directly to the CPU 102 instead of via memory bridge 105. In a third example, I / O bridge 107 and memory bridge 105 can be integrated into a single chip instead of existing as one or more separate devices. Third, Figure 1 One or more of the components shown may be missing. For example, switch 116, network adapter 118, and add-on cards 120, 121 can be omitted and will be directly connected to I / O bridge 107.

[0030] In one or more embodiments, the computer system 100 is configured to execute a training engine 122 and an execution engine 124 residing in system memory 104. The training engine 122 and the execution engine 124 may be stored in system disk 114 and / or other storage media and loaded into system memory 104 when executed.

[0031] More specifically, training engine 122 and execution engine 124 include the functionality to train and execute machine learning models to learn shared latent representations of data across multiple domains. For example, training engine 122 and execution engine 124 can be used to generate machine learning models that can translate between data samples of image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from perturbed cells, data from young systems, data from old systems, and / or other types of biological data and latent values ​​in a shared low-dimensional latent space.

[0032] This machine learning model comprises multiple generator neural networks, which are trained together with multiple discriminator neural networks using a CycleGAN paradigm. During the training of the machine learning model, the discriminator neural networks are used to classify the latent values ​​and / or generative outputs from the generator neural networks. The classification outputs from the discriminator neural networks are also used to update the parameters of the generator neural networks, ensuring that the latent values ​​generated by the generator neural networks for different domains reside in the same shared latent space. The operation of the training engine 122 and the execution engine 124 will be described in detail below.

[0033] Shared latent representations of multimodal datasets Figure 2 Based on various embodiments Figure 1 A more detailed illustration of the training engine 122 and execution engine 124 is provided. As described above, the training engine 122 and execution engine 124 run to train and execute a machine learning model, thereby learning a latent manifold 240 shared by data samples 232 from multiple domains 220(1)-220(N) (each of which is referred to individually as domain 220 in this document).

[0034] In some embodiments, domain 220 represents different types of biological data samples that can be collected or generated from a biological system. For example, domain 220 may represent (but is not limited to) image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from disturbed cells, data from biological systems of different ages, data collected through medical scanning techniques, and / or other types of biological data.

[0035] In various embodiments, data samples can be generated through any number of data generation steps, such as imaging, sequencing, omics measurements, and generative machine learning. Examples of imaging techniques include optical microscopy or cryo-electron microscopy. Examples of omics measurements include transcriptomics (e.g., RNAseq), genomics, proteomics, metabolomics, and epigenomics (e.g., ATACseq, DNA methylation). Furthermore, data samples can be perturbed in a variety of ways. Examples of perturbations include: (i) environmental stimuli, such as temperature changes, osmotic shocks, pressure changes, stress, starvation; (ii) perturbations using chemical, pharmaceutical, or biological agents; and (iii) gene function manipulation, such as gene knockout (e.g., CRISPR knockout), transcript knockdown, or CRISPR modification.

[0036] Field 220 may also or alternatively represent other types of data. For example, field 220 may include text, images, audio, video, point clouds, grids, sensor data, and / or other types of data related to users, objects, environments, computer systems, machines, virtual worlds, and / or other types of entities.

[0037] like Figure 2 As shown, the machine learning model includes multiple generators 202(1)-202(N). Generator 202(1) includes encoder 204(1) and decoder 206(1), and generator 202(N) includes individual encoders 204(N) and decoders 206(N). Each encoder 204(1)-204(N) transforms a set of inputs into a corresponding set of latent values ​​210(1)-210(N), and each decoder 206(1)-206(N) transforms a given set of latent values ​​210(1)-210(N) into a corresponding set of decoder outputs 212(1)-212(N). In this paper, each of the generators 202(1)-202(N) is individually referred to as generator 202, each of the encoders 204(1)-204(N) is individually referred to as encoder 204, and each of the decoders 206(1)-206(N) is individually referred to as decoder 206. Each set of potential values ​​210(1)-210(N) is referred to as potential value 210 in this paper, and each set of decoder outputs 212(1)-212(N) is referred to as decoder output 212 in this paper.

[0038] In some embodiments, encoder 204 and decoder 206 comprise residual neural networks having a rectified linear unit (ReLU) activation function, a dropout layer, and a normalization layer. The last layer of one or more encoders 204 and / or one or more decoders 206 may contain a tanh activation function. Encoder 204 and decoder 206 may also or alternatively comprise other types of neural network architectures. For example, encoder 204 and / or decoder 206 may comprise a diffusion model, a domain-invariant variational autoencoder (DIVA), a model-based autoencoder for generating discrete omics samples with known statistical properties, and / or other types of neural networks capable of transforming data samples in one or more domains to latent values ​​in a latent space.

[0039] The training engine 122 uses multiple sets of training data 214(1)-214(N) (each set of data is separately referred to as training data 214) to train the generator 202. Figure 2 As shown, each set of training data 214 is associated with a different domain 220. For example, a set of training data 214 for a given domain 220 may include (but is not limited to): image-based data generated using a specific imaging or scanning technique; sequencing-based data generated using a specific technique and / or for a specific type of organism; specific types of omics data; specific types of physiological data; data collected from specific types of homeostatic cells; data collected from cells that have been perturbed in a specific manner; data from biological systems of a specific age; and / or other data distributions generated in a specific manner and / or collected from specific sources (e.g., an organism or a type of organism).

[0040] More specifically, training engine 122 trains each generator 202 using training data 214 from one or more domains 220. During the training of a given generator 202, training engine 122 inputs the training data 214 associated with a domain 220 into an encoder 204 within that generator 202 and uses the encoder 204 to transform the input training data 214 into a set of latent values ​​210. Training engine 122 also uses a decoder 206 within the generator 202 to transform the set of latent values ​​210 into a corresponding set of decoder outputs 212. Training engine 122 computes one or more losses 208 using the training data 214, latent values ​​210, and / or decoder outputs 212, and uses training techniques (e.g., gradient descent and backpropagation) to update the parameters of the encoder 204 and decoder 206 within the generator 202 in a manner that reduces the losses 208.

[0041] Figure 3 Various embodiments are illustrated. Figure 1The training engine 122 in the diagram operates when training two generators 202(1) and 202(2) to learn the latent space shared by the two domains 220. (See diagram for details.) Figure 3 As shown, one domain 220 is represented by A, and the other domain 220 is represented by B. A generator 202(1) is represented by G. A2B It indicates that it includes: encoder 204 (1), using e A The representation, and decoder 206(1), use d B In generator 202(1), e A Encoder 204(1) converts the data in domain A into data from domain Z. A The latent representation in the latent space. d B Decoder 206(1) will Z A The latent representation in the latent space is transformed into data in domain B. Therefore, generator 202(1) can be used to transform data in domain A into data in domain B.

[0042] Another generator 202(2) uses G B2A It indicates that it includes: encoder 204 (2), using e B This indicates that, and decoder 206(2), is represented by d. A In generator 202(2), e B Encoder 204(2) converts the data in domain B into data from domain Z. B The latent representation in the latent space. d A Decoder 206(1) will Z B The latent representation in the latent space is transformed into data in domain A. Therefore, generator 202(2) can be used to transform data in domain B into data in domain A.

[0043] In one or more embodiments, training engine 122 trains generators 202(1) and 202(2) in a forward phase 312 and a backward phase 314. In the forward phase 312, training engine 122 inputs training data 214(1) from domain A into encoder 204(1) of generator 202(1) and uses encoder 204(1) to convert training data 214(1) into Z. A The first set of latent values ​​210(1) in the latent space. The training engine 122 inputs the latent values ​​210(1) into the decoder 206(1) of the generator 202(1), and uses the decoder 206(1) to convert the first set of latent values ​​210(1) into the first set of decoder outputs 212(1), denoted as B*. Then, the training engine 122 inputs the decoder outputs 212(1) into the encoder 204(2) of the generator 202(2), and uses the encoder 204(2) to convert the decoder outputs 212(1) into Z.B The second set of latent values ​​210 (2) in the latent space. The training engine 122 inputs the latent value 210 (2) into the decoder 206 (2) of the generator 202 (2) and uses the decoder 206 (2) to convert the latent value 210 (2) into the second set of decoder output 212 (2), denoted as Â.

[0044] The training engine 122 also uses several losses (e.g., Figure 2 The generator 202(1)-202(2) is trained using losses 208 in the training data 214(1), latent values ​​210(1)-210(2), and decoder outputs 212(1)-212(2). These losses include forward loop consistency loss 302 and latent discriminator loss 304.

[0045] In some embodiments, the forward recurrent consistency loss 302 represents a measure of the difference between training data 214(1) in domain A and decoder output 212(2), which corresponds to a reconstruction of training data 214(1) in the same domain. For example, the forward recurrent consistency loss 302 may include mean squared error (MSE), mean absolute error (MAE), and / or other types of reconstruction loss between training data 214(1) and decoder output 212(2). In another example, decoder output 212(2) may include parameters of the distribution from which the generated data can be sampled, and the forward recurrent consistency loss 302 may be computed based on the likelihood that the training data comes from the distribution specified by decoder output 212(2).

[0046] The latent discriminator loss 304 is computed using the output generated by a first discriminator neural network (not shown) from two sets of latent values ​​210(1) and 210(2). More specifically, the first discriminator neural network includes a classifier that attempts to classify a given latent value as belonging to Z. A Potential space or Z B Latent space. Therefore, the latent discriminator loss 304 may include binary cross-entropy loss and / or other types of classification loss, which are calculated using correct and incorrect predictions of the class generated by the first discriminator neural network based on two sets of latent values ​​210(1)-210(2).

[0047] After the forward phase 312 is completed, the training engine 122 executes the backward phase 314. In the backward phase 314, the training engine 122 inputs the training data 214(2) from domain B into the encoder 204(2) of the generator 202(2), and uses the encoder 204(2) to convert the training data 214(2) into Z. BThe third set of latent values ​​210 (3) in the latent space. The training engine 122 inputs the latent value 210 (3) into the decoder 206 (2) of the generator 202 (2), and uses the decoder 206 (2) to convert the third set of latent values ​​210 (3) into the third set of decoder output 212 (3), denoted as A*. Then, the training engine 122 inputs the decoder output 212 (3) into the encoder 204 (1) of the generator 202 (1), and uses the encoder 204 (1) to convert the decoder output 212 (3) into Z. A The fourth set of latent values ​​210 (4) in the latent space. The training engine 122 inputs the latent value 210 (4) into the decoder 206 (1) of the generator 202 (1) and uses the decoder 206 (1) to convert the latent value 210 (4) into the fourth set of decoder output 212 (1), denoted as B̂.

[0048] The training engine 122 also uses a backward recurrent consistency loss 310 to train generators 202(1)-202(2), which is calculated based on training data 214(2) in domain B and decoder output 212(4), where decoder output 212(4) corresponds to a reconstruction of training data 214(2) in the same domain. Similar to the forward recurrent consistency loss 302, the backward recurrent consistency loss 310 may include MSE, MAE, likelihood-based loss and / or other types of loss, all of which are calculated using training data 214(2) and decoder output 212(4).

[0049] like Figure 3 As shown, the training engine 122 further uses a feature discriminator loss 306 and a guidance loss 308 to train generators 202(1) and 202(2), which are calculated using the decoder output 212(1) generated in the forward phase 312 and the training data 214(2) input to generator 202(2) at the beginning of the backward phase 314. The feature discriminator loss 306 is calculated using features generated from the decoder output 212(1) and the training data 214(2). For example, the training engine 122 can use an embedding model to convert the decoder output 212(1) and the training data 214(2) into a corresponding feature set. The training engine 122 can also use a second discriminator neural network (not shown) to classify a given feature set as real data or fake data belonging to domain B. Then, the training engine 122 can calculate the feature discriminator loss 306 as a binary cross-entropy loss, least squares loss and / or other types of classification loss, which are calculated based on the correct and incorrect predictions generated by the second discriminator neural network based on the decoder output 212 (1) and training data 214 (2).

[0050] In some embodiments, training engine 122 uses a guidance loss 308 to mitigate pattern collapse in generator 202 (1), where the decoder output 212 (1) produced by generator 202 (1) fails to reflect the complete distribution of data in domain B. For example, training engine 122 can use clustering techniques to group training data 214 (2) in domain B into multiple clusters. Training engine 122 can also train a classifier to predict the cluster to which a given data sample from domain B belongs. Training engine 122 can use this classifier to generate cluster predictions for a batch of decoder outputs 212 (1) and compute a loss that is inversely proportional to the cluster “diversity” predicted by the classifier for that batch of decoder outputs 212 (1). Thus, training engine 122 can use the loss computed based on the classifier output to ensure that generator 202 (1) can generate all patterns from the data distribution in domain B.

[0051] In another example, training engine 122 can use the implicit maximum likelihood estimation (IMLE) technique to minimize the distance between real samples in training data 214(2) and pseudo-decoder outputs 212(1) generated by generator 202(1). The IMLE technique may include selecting a batch of random decoder outputs 212(1) from generator 202(1) and updating the parameters of generator 202(1) such that the selected decoder outputs 212(1) samples move toward the closest real data point in training data 214(2).

[0052] In one or more embodiments, the training engine 122 trains the encoder 204, decoder 206, and / or generator 202 using a total loss calculated as a weighted combination of forward recurrent consistency loss 302, latent discriminator loss 304, feature discriminator loss 306, guidance loss 308, and backward recurrent consistency loss 310. The weights used in the weighted combination may reflect the relative contributions of the forward recurrent consistency loss 302, latent discriminator loss 304, feature discriminator loss 306, guidance loss 308, and backward recurrent consistency loss 310 to the total loss. For example, the training engine 122 may use a weight of 1.0 for the forward recurrent consistency loss 302 and backward recurrent consistency loss 310, a weight of 0.01 for the feature discriminator loss 306, a weight of 0.005 for the latent discriminator loss 304, and a weight of 0.01 for the guidance loss 308 to calculate the total loss. These weights can be adjusted to reflect different types of domains 220, the dimension of the latent space, the neural network architecture of the encoder 204 and decoder 206, and / or other factors. After calculating the total loss, the training engine 122 can backpropagate the total loss across the individual layers of the generator 202(1) and / or 202(2) and update the parameters (e.g., neural network weights) of one or more components of the generator 202(1) and / or 202(2) based on the negative gradient of the backpropagated loss using stochastic gradient descent.

[0053] Training engine 122 may also, or alternatively, use different losses and / or combinations of losses to train different components of generators 202(1) and / or 202(2). For example, training engine 122 may use latent discriminator loss 304 to train encoders 204(1)-204(2) and discriminator neural networks associated with latent values ​​210(1)-210(2). Training engine 122 may also, or alternatively, use forward recurrent consistency loss 302 and backward recurrent consistency loss to train generators 202(1) and / or 202(2) in an end-to-end manner. Training engine 122 may also, or alternatively, use feature discriminator loss 306 and guidance loss 308 to train generator 202(1) and discriminator neural networks associated with decoder output 212(1) and training data 214(2).

[0054] The encoder 204, decoder 206, and / or generator 202 are trained in this way using a forward recurrent consistency loss 302, a latent discriminator loss 304, a feature discriminator loss 306, a guidance loss 308, and a backward recurrent consistency loss 310, so that the encoder 204, decoder 206, and / or generator 202 can learn a shared latent manifold in which latent values ​​210(1)-210(4) associated with different domains 220 lie within the shared latent manifold. This shared latent manifold allows for the identification, comparison, and / or analysis of states associated with biological systems and / or other entities represented by data points in domain 220, as will be described in further detail below.

[0055] Training engine 122 also includes the ability to perform additional training on generators 202(1)-202(2) using variations of forward phase 312 and backward phase 314. For example, training engine 122 can use... Figure 3 The generators 202(1) and 202(2) shown are configured to perform a training phase comprising a forward phase 312 and a backward phase 314. The training engine 122 may also, or alternatively, perform different or additional training phases comprising modified forward and backward phases. The modified forward phase may include: using generator 202(2) to convert training data 214(2) from domain B into decoder output 212(3) in domain A; and then using generator 202(1) to convert decoder output 212(3) into additional decoder output 212(4) in domain B. The modified backward phase may include: using generator 202(1) to convert training data 214(1) from domain A into decoder output 212(1) in domain B; and then using generator 202(2) to convert decoder output 212(1) into additional decoder output 212(2) in domain A. In the modified forward and backward phases, the training engine 122 may use different discriminator neural networks to compute the feature discriminator loss 306, which is designed to distinguish between the training data 214(1) and the decoder output 212(3). In the modified forward and backward phases, the training engine 122 may also use the training data 214(1) and the decoder output 212(3) to compute the guidance loss 308.

[0056] In another example, training engine 122 may execute a given forward phase 312 and / or backward phase 314 multiple times. In a given forward phase 312 and / or backward phase 314, training engine 122 may train an encoder 204 and a decoder 206 associated with one domain 220, while keeping another encoder 204 and decoder 206 associated with another domain 220 fixed. Training engine 122 may also, or alternatively, train a generator 202, while keeping another generator 202 fixed. After completing this forward phase 312 and / or backward phase 314 (e.g., after a certain number of training iterations or epochs), training engine 122 may execute a subsequent forward phase 312 and / or backward phase 314, where components that remained fixed in the previous epoch (e.g., an encoder 204, a decoder 206, and / or a generator 202) will be trained, while components already trained in the previous epoch will remain fixed.

[0057] In the third example, training engine 122 may execute a forward phase 312 and a backward phase 314, in which one or more encoders 204, decoders 206, and / or generators 202 are trained, while one or more discriminators used to generate the latent discriminator loss 304 and / or feature discriminator loss 306 associated with these encoders 204, decoders 206, and / or generators 202 remain fixed. Training engine 122 may then execute a subsequent forward phase 312 and a backward phase 314, in which the encoders 204, decoders 206, and / or generators 202 trained in the previous phase remain fixed, while the discriminators used to generate the latent discriminator loss 304 and / or feature discriminator loss 306 associated with these encoders 204, decoders 206, and / or generators 202 are trained. The training engine 122 can continue to alternate between training the individual components of the generator 202 and training the discriminators associated with these components, in order to maintain a balance between the performance of the encoder 204, decoder 206 and / or generator 202 and the performance of the corresponding discriminators.

[0058] Although the operation of the training engine 122 has been described above for learning the shared latent space between two domains 220, it should be recognized that the training engine 122 can be used to extend the shared latent space to cover more than two domains 220. For example, the training engine 122 can train the encoder 204 and decoder 206 of more than two domains 220 in a "daisy-chain" manner, which involves multiple training phases. In the first training phase, the training engine 122 can use the methods described above... Figure 3The technique described herein trains a first encoder 204 and a first decoder 206 for a first domain 220, and a second encoder 204 and a second decoder 206 for a second domain 220, thereby enabling the two encoders 204 and the two decoders 206 to learn a shared latent space between the two domains 220. In a second training phase, the training engine 122 may pair a third encoder 204 for a third domain 220 with the first and / or second decoder 206, and pair a third decoder 206 for a third domain 220 with the first and / or second encoder 204. The training engine 122 may also repeat the same training technique using new encoder-decoder pairs, thereby sharing the latent space associated with the third domain 220 with the latent spaces of the first and second domains 220. The training engine 122 may also repeat this process to extend the shared latent space to additional domains 220 beyond the third domain 220.

[0059] Back Figure 2 The discussion continues, after training the generator 202 using training data 214 from multiple domains 220 and loss 208, the execution engine 124 uses a shared manifold 240 of latent values ​​210 to perform additional analysis 238 and / or prediction 236 related to data samples 232 in some or all of the domains 220. (See also...) Figure 2 As shown, execution engine 124 can use one or more encoders 204 to transform data samples 232 from the corresponding domain 220 into latent representations 234 located in manifold 240. Execution engine 124 can also, or alternatively, use one or more decoders 206 to transform the latent representations 234 in manifold 240 into data samples 232 in the corresponding domain 220. Execution engine 124 can also perform analysis 238, comparing the latent representations 234 and / or data samples 232. Execution engine 124 can further use data samples 232, latent representations 234, and / or analysis 238 to generate predictions 236 associated with the latent representations 234 and / or data samples 232.

[0060] Figure 4 The illustration shows examples of using a shared potential space to perform operations related to data 402 from a first domain and data 404 from a second domain, according to various embodiments. Figure 4 As shown, data 402 includes image-based data that can be generated using immunostaining, image processing, and multi-parameter analysis techniques. Data 404 includes omics-based deoxyribonucleic acid (DNA) methylation data that can be generated using various sequencing technologies.

[0061] Data 402 and 404 can be compared and / or transformed into each other using components 406 and 408 included in one or more generators 202. For example, component 406 may include an encoder specific to the domain associated with data 402, while component 408 may include a decoder specific to the domain associated with data 404. Component 406 can be used to transform images contained in data 402, segmented objects in the images, image-based features, and / or image-based analyses into a shared latent space associated with the two domains (e.g., Figure 2 The latent representation 234(1) is in manifold 240. The value of latent representation 234(1) can be used as input to the corresponding latent representation 234(2) in component 408. Then, component 408 can be used to convert latent representation 234(2) into DNA methylation values ​​corresponding to data 404.

[0062] In another example, component 406 may include an encoder specific to the domain associated with data 402, while component 408 may include an encoder specific to the domain associated with data 404. In this example, components 406 and 408 may be used to transform data 402 and 404 into corresponding latent representations 234(1) and 234(2) in a shared latent space, respectively. Cosine similarity, Euclidean distance, dot product, and / or other vector similarity or distance metrics may be computed between latent representations 234(1) and 234(2) to determine the degree of similarity or dissimilarity and / or state differences between the biological systems represented by data 402 and 404. Additional points in the latent space representing interpolations between latent representations 234(1) and 234(2) may also be determined, or alternatively. These additional points may then be transformed into corresponding data points in those domains using decoders of the respective domains associated with the shared latent space. These data points will represent biological system states between the state associated with data 402 and the state associated with data 404. This process can be repeated for other potential representation pairs and / or sets 234 to explore the shared latent space, determine the biological states represented by the potential representations 234 within the shared latent space, determine the similarities and / or relationships between biological states and / or corresponding potential representations 234, and / or perform other types of operations or analyses related to the potential representations 234, the corresponding datasets, and / or the biological systems represented by the data.

[0063] Although Figure 4The example illustration depicts the use of a shared latent space to perform analysis and / or prediction associated with image-based data 402 and DNA methylation data 404, but it should be understood that the functionality of components 406 and 408 can also be used to perform various types of analysis 238 and / or prediction 236 associated with other domains. For example, components 406 and 408 may include encoders, decoders, and / or other elements of one or more generators 202 associated with image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from perturbed cells, data from young systems, data from old systems, and / or other types of biological data. These generators 202 may also, or alternatively, be associated with text, images, audio, video, point clouds, meshes, sensor data, and / or other data generated or collected by users, objects, environments, computer systems, machines, virtual worlds, and / or other entities. These generators 202 can be used to transform between data points in the corresponding domain and potential representations 234 in a shared potential space, determine relationships or similarities between data points, determine or predict the states or attributes represented by data points and / or potential representations 234, predict the response of biological systems and / or entities to aging and / or disturbances, predict diseases and / or other risks of biological systems using data associated with multiple domains of the biological system, and / or perform other operations using data points and / or potential representations.

[0064] Figure 5 This is a flowchart of method steps for training a machine learning model to learn a shared latent space among multiple domains, according to various embodiments. Although the method steps are combined... Figures 1 to 3 The system shown is described, but those skilled in the art will understand that any system configured to perform some or all of the method steps in any order falls within the scope of this disclosure.

[0065] As shown in the figure, in step 502, training engine 122 collects multiple sets of training data associated with multiple domains. For example, training engine 122 may collect image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from perturbed cells, data from young systems, data from aged systems, and / or other types of biological data. Training engine 122 may also, or alternatively, collect text, images, audio, video, point clouds, meshes, sensor data, and / or other data generated or collected by users, objects, environments, computer systems, machines, virtual worlds, and / or other types of entities. Training engine 122 may obtain each set of data from repositories, streaming data sources, synthetic data generators, and / or other data sources.

[0066] In step 504, training engine 122 executes a first generator neural network to transform a first set of training data in the first domain into a first set of training outputs in the second domain based on a first set of latent values ​​associated with the first domain. For example, training engine 122 may use a first encoder in the first generator neural network to transform the input training data into a first set of latent values ​​in the latent space associated with the first domain. Training engine 122 may also use a first decoder in the first generator neural network to transform the first set of latent values ​​into the first set of training outputs.

[0067] In step 506, training engine 122 executes a second generator neural network to transform the first set of training outputs into a second set of training outputs in the first domain based on a second set of latent values ​​associated with the second domain. Continuing the example above, training engine 122 can use a second encoder in the second generator neural network to transform the first set of training outputs into a second set of latent values ​​in the latent space associated with the second domain. Training engine 122 can also use a second decoder in the second generator neural network to transform the second set of latent values ​​into a second set of training outputs.

[0068] In step 508, training engine 122 uses some or all of the latent values ​​and / or training outputs generated in previous steps to compute a first set of losses. For example, training engine 122 may compute a forward recurrent consistency loss as a measure of the difference between the first set of training data in the first domain and the second set of training outputs (corresponding to the reconstruction of the first set of training data). Training engine 122 may also, or alternatively, use the outputs generated by the first discriminator neural network based on the two sets of latent values ​​to compute a latent discriminator loss.

[0069] In step 510, training engine 122 trains the first generator neural network and / or the second generator neural network based on the first set of losses. For example, training engine 122 can use gradient descent and backpropagation to update the weights of the first and / or second generator neural networks in such a way as by reducing the forward recurrent consistency loss, the latent discriminator loss, and / or the weighted combination of the forward recurrent consistency loss and the latent discriminator loss.

[0070] In step 512, training engine 122 executes a second generator neural network to transform a second set of training data in the second domain into a third set of training outputs in the first domain based on a third set of latent values ​​associated with the second domain. For example, training engine 122 can use a second encoder in the second generator neural network to transform the second set of training data into a third set of latent values ​​in the latent space associated with the second domain. Training engine 122 can also use a second decoder in the second generator neural network to transform the third set of latent values ​​into a third set of training outputs.

[0071] In step 514, training engine 122 executes a first generator neural network to transform the third set of training outputs into a fourth set of training outputs in the first domain based on a fourth set of latent values ​​associated with the second domain. Continuing the example above, training engine 122 can use a first encoder in the first generator neural network to transform the third set of training outputs into a fourth set of latent values ​​in the latent space associated with the first domain. The training engine can also use a first decoder in the first generator neural network to transform the fourth set of latent values ​​into a fourth set of training outputs.

[0072] In step 516, training engine 122 computes a second set of losses using some or all of the latent values ​​and / or training outputs generated in previous steps. For example, training engine 122 may compute a backward cycle consistency loss as a measure of the difference between the second set of training data in the second domain and the fourth set of training outputs (corresponding to the reconstruction of the second set of training data).

[0073] In another example, training engine 122 can use an embedding model to transform a given training output set and the corresponding training dataset in the same domain into a corresponding feature set. Training engine 122 can also use a second discriminator neural network to classify a given feature set as real or fake data in the domain. Then, training engine 122 can compute the feature discriminator loss 306 as an adversarial loss and / or classification loss based on the correct and incorrect predictions generated by the second discriminator neural network from the training output and training data.

[0074] In the third example, training engine 122 can use clustering techniques to group training data in a given domain into multiple clusters. Training engine 122 can train a classifier to predict the cluster to which a given data sample from that domain belongs. Training engine 122 can use the classifier to generate cluster predictions for a batch of decoder outputs 212(1) and calculate a loss that is inversely proportional to the “diversity” of the clusters predicted by the classifier for that batch of decoder outputs 212(1).

[0075] In the fourth example, training engine 122 can use the implicit maximum likelihood estimation (IMLE) technique to minimize the distance between real samples in the training dataset in a given domain and spurious training outputs in the same domain generated by the generator neural network. The IMLE technique may include selecting a batch of random training outputs from the generator neural network and calculating the loss based on the distance between the selected training outputs and the nearest real data points in the training dataset.

[0076] In step 518, training engine 122 trains the first and / or second generator neural network based on the second set of losses. For example, training engine 122 may calculate the total loss as a weighted combination of forward recurrent consistency loss, latent discriminator loss, feature discriminator loss, guidance loss, and / or backward recurrent consistency loss. Training engine 122 may also update the parameters of the first and / or second generator neural network in a manner that reduces the total loss.

[0077] In step 520, training engine 122 trains one or more discriminator neural networks based on a third set of losses associated with the latent values ​​and / or training outputs generated by the first and / or second generator neural networks. For example, training engine 122 can use updated generator neural networks to generate additional latent values ​​and training outputs based on training data in the first and second domains. Training engine 122 can use the first discriminator neural network to classify latent values ​​as associated with the first or second domain. Training engine 122 can use the classification output from the first discriminator neural network to compute a new latent discriminator loss and train the first discriminator neural network based on the new latent discriminator loss. Training engine 122 can also, or alternatively, use a second discriminator neural network to classify data associated with the corresponding domain and / or features generated from the data as real or fake. Training engine 122 can use the classification output to compute a discriminator loss and train the second discriminator neural network based on that discriminator loss.

[0078] In step 522, training engine 122 determines whether to continue training the generator and discriminator neural networks. For example, training engine 122 may decide to continue training the generator and discriminator neural networks until one or more conditions are met. These conditions include (but are not limited to) the convergence of the parameters of the generator and / or discriminator neural networks; the loss decreasing below a threshold; or reaching a certain number of training steps, iterations, batches, and / or epochs. Once these conditions are met, training engine 122 terminates the process of training the generator and discriminator neural networks.

[0079] If further training of the generator and discriminator neural networks is required, training engine 122 will repeat steps 504-520 to further train the generator and discriminator neural networks. For example, training engine 122 can repeat steps 504-520 multiple times for iterations to train different components of the generator and / or discriminator neural networks. In a given iteration, training engine 122 can train one generator neural network while keeping another generator neural network fixed. Training engine 122 can also, or alternatively, train the encoder and decoder associated with one domain while keeping the encoder and decoder associated with another domain fixed. In the next iteration, training engine 122 can train the generator neural network (or encoder and decoder) that remained fixed in the previous iteration, while keeping the generator neural network (or encoder and decoder) trained in the previous iteration fixed.

[0080] In another example, training engine 122 may execute a first training phase, which trains the first and second generator neural networks and their corresponding discriminator neural networks for a certain number of iterations in steps 504-520. Then, training engine 122 may swap the first and second generator neural networks and execute a second training phase, which trains the swapped generator neural networks and discriminator neural networks for an additional number of iterations in steps 504-520.

[0081] Training engine 122 may also repeat steps 504-520 to train additional generator neural networks and discriminator neural networks associated with the additional domains. For example, after the first and second generator neural networks have been trained, training engine 122 may create a new “first” generator neural network by pairing a third encoder for the third domain with the decoders in the first and second generator neural networks. Training engine 122 may also create a new “second” generator neural network by pairing a third decoder for the third domain with the encoders in the first and / or second generator neural networks. Training engine 122 may also perform steps 504-522 once or more using the new first and second generator neural networks (and the corresponding discriminator neural networks), thereby sharing the latent space associated with the third domain with the latent spaces of the first and second domains. Training engine 122 may continue to add new domains to the shared latent space in this manner until the shared latent space contains all domains associated with the training data collected in step 502, and / or domains associated with training data collected independently of step 502.

[0082] Figure 6 This is a flowchart of method steps for analyzing data associated with multiple domains, according to various embodiments. Although the method steps are combined... Figures 1 to 3 The system shown is described, but those skilled in the art will understand that any system configured to perform some or all of the method steps in any order falls within the scope of this disclosure.

[0083] As shown in the figure, in step 602, the execution engine 124 receives a first generator neural network and a second generator neural network, which are trained to learn a shared latent space between the two domains. For example, the first and second generator neural networks have been trained according to... Figure 5 After training according to the steps discussed herein, execution engine 124 can receive the first generator neural network and the second generator neural network from training engine 122, a repository, and / or other sources.

[0084] In step 604, execution engine 124 executes one or more components of the first generator neural network to perform a transformation between a first set of data and a first set of latent values ​​associated with the first domain. For example, execution engine 124 may use an encoder in the first generator neural network to transform data points in the first domain into corresponding latent values. Execution engine 124 may also, or alternatively, use a decoder in the first generator neural network to transform latent values ​​in the latent space into corresponding data points in the first domain.

[0085] In step 606, execution engine 124 executes one or more components of the second generator neural network to transform between a second set of data and a second set of latent values ​​associated with the second domain. For example, execution engine 124 may use an encoder in the second generator neural network to transform data points in the second domain into corresponding latent values. Execution engine 124 may also, or alternatively, use a decoder in the second generator neural network to transform latent values ​​in the latent space into corresponding data points in the second domain.

[0086] In step 608, execution engine 124 generates one or more prediction results based on the first set of potential values ​​and the second set of potential values. For example, execution engine 124 can use the first set of potential values ​​and the second set of potential values ​​to determine similarity and / or other relationships between corresponding data points, predict states and / or state changes associated with data points, transform data points in one domain into corresponding data points in another domain, and / or perform other operations or analyses involving one or two domains.

[0087] In summary, the disclosed technique trains and executes a machine learning model to generate shared latent representations of multimodal datasets. The machine learning model includes multiple encoders and multiple decoders. Each encoder transforms data from a corresponding domain into a fixed-length latent representation in a low-dimensional latent space. Each decoder transforms the latent representation from the low-dimensional latent space into data in the corresponding domain. A given encoder can be paired with a decoder for the same domain to form a generator capable of reconstructing data in that domain. An encoder for a given first domain can also, or alternatively, be paired with a decoder for a second domain to form a generator capable of transforming data in the first domain into data in the second domain.

[0088] For example, a machine learning model may include one or more encoders that transform samples from image-based data, sequencing-based data, omics data, physiological data, data collected from homeostatic cells, data collected from perturbed cells, data from young systems, data from aged systems, and / or other types of biological data into corresponding latent values. The machine learning model may also include one or more decoders that transform each latent value into one or more samples from the same and / or different domains. In this example, an encoder for image-based data may be paired with a decoder for image-based data to transform the image-based data into a latent representation and reconstruct the image-based data from the latent representation. An encoder for image-based data may be paired with different decoders for sequencing-based data to perform conversions between image-based and sequencing-based data, compare image-based and sequencing-based data, and / or perform other analyses or predictions related to the image-based and / or sequencing-based data.

[0089] To train a machine learning model, the training data is transformed into a first set of latent values ​​using a first encoder for a first domain. The first set of latent values ​​is transformed into a first set of training outputs using a first decoder for a second domain. The first set of training outputs is transformed into a second set of latent values ​​using a second encoder for the second domain. The second set of latent values ​​is transformed into a second set of training outputs using a second decoder for the first domain. The encoder and decoder are trained using several losses, which are computed using the first and second sets of latent values, the first and second sets of training outputs, and / or outputs generated by one or more discriminator models based on these latent values ​​and / or training outputs. These losses can be used to ensure that at least a portion of the latent space occupied by the latent values ​​is invariant with respect to the domain from which the latent values ​​originate.

[0090] When it is necessary to learn a shared latent space across more than two domains, a "daisy-chain" approach can be used to train the encoders and decoders for each domain. For example, in the first training phase, the techniques described above can be used to train a first encoder and a first decoder for the first domain, and a second encoder and a second decoder for the second domain, to learn the shared latent space between the two domains. In the second training phase, a third encoder for the third domain can be paired with the first decoder and / or the second decoder, and vice versa. The same training techniques can be repeated using new encoder-decoder pairs, thereby sharing the latent space associated with the third domain with the latent spaces of the first and second domains. Additional training phases can also be added to extend the shared latent space to additional domains.

[0091] Compared to existing technologies, one technical advantage of the disclosed technique is that the shared latent space learned by the machine learning model can semantically associate different types of data. Therefore, compared to traditional machine learning techniques that cannot generate shared latent spaces across multiple biological domains (or other types of domains), the disclosed technique can more completely represent biological systems (or other types of entities) represented using these types of data. Another technical advantage of the disclosed technique is that it can train machine learning models to learn shared latent spaces even in the absence of paired training data from different domains. Therefore, compared to existing techniques that use paired data to learn latent spaces associated with multiple domains, the disclosed technique can use larger datasets to generate more complete and semantically meaningful shared latent spaces. These technical advantages provide one or more technical improvements over existing methods.

[0092] 1. In some embodiments, a computer-implemented method for determining a latent space between two domains includes: executing a first generator neural network to convert a first set of training data associated with the first domain into a first set of training outputs based on a first set of latent values ​​associated with the first domain; executing a second generator neural network to convert the first set of training outputs into a second set of training outputs based on a second set of latent values ​​associated with a second domain; and training the first generator neural network based on a first set of losses calculated between the first set of latent values ​​and the second set of latent values.

[0093] In various embodiments, training data can be generated through any number of data generation steps, such as imaging, sequencing, omics measurements, and generative machine learning. Examples of imaging techniques include optical microscopy or cryo-electron microscopy. Examples of omics measurements include transcriptomics (e.g., RNAseq), genomics, proteomics, metabolomics, and epigenomics (e.g., ATACseq, DNA methylation). Furthermore, training data can be perturbed in a variety of ways. Examples of perturbation include: (i) environmental stimuli, such as temperature changes, osmotic shocks, pressure changes, stress, starvation; (ii) perturbation using chemicals, drugs, or biological agents; and (iii) gene function manipulation, such as gene knockout (e.g., CRISPR knockout), transcript knockdown, or CRISPR modification.

[0094] 2. The computer-implemented method according to Clause 1 further includes: executing a discriminator neural network to convert the first set of potential values ​​into a first set of predictions; executing the discriminator neural network to convert the second set of potential values ​​into a second set of predictions; and calculating the first set of losses based on the first set of predictions and the second set of predictions.

[0095] 3. The computer-implemented method according to any of the provisions of 1-2, wherein the first set of losses includes binary cross-entropy loss.

[0096] 4. The computer-implemented method according to any one of the provisions of 1-3 further includes: training the second generator neural network based on a second set of losses calculated between a third set of potential values ​​generated by the first generator neural network and a fourth set of potential values ​​generated by the second generator neural network.

[0097] 5. The computer-implemented method according to any one of the provisions of 1-4, further comprising: training the first generator neural network based on a second set of losses calculated between the second set of training outputs and the first set of training data.

[0098] 6. The computer-implemented method according to any one of the provisions of 1-5, further comprising: executing the second generator neural network to convert the second set of training data into a third set of training outputs; executing the first generator neural network to convert the third set of training outputs into a fourth set of training outputs; and training at least one of the first generator neural network or the second generator neural network based on a second set of losses calculated between the fourth set of training outputs and the second set of training data.

[0099] 7. The computer-implemented method according to any one of the provisions of 1-6 further comprises: training the second generator neural network based on a third loss calculated between the third set of training outputs and the first set of training data.

[0100] 8. The computer-implemented method according to any one of the provisions of 1-7, further comprising: training the first generator neural network based on a second set of losses calculated between the first set of training outputs and a third set of training data associated with the second domain.

[0101] 9. The computer-implemented method according to any one of the provisions of 1-8 further comprises: training at least one of the first generator neural network or the second generator neural network based on a second set of losses calculated using a set of classifiers associated with the first set of training outputs.

[0102] 10. A computer-implemented method according to any one of the provisions of 1-9, wherein at least one of the first domain or the second domain includes at least one of image-based data, sequencing-based data, omics data, physiological data, homeostatic cell data, disturbed cell data, young cell data, or aged cell data.

[0103] 11. In some embodiments, one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: executing a first generator neural network to convert a first set of training data associated with a first domain into a first set of training outputs based on a first set of latent values ​​associated with a first domain; executing a second generator neural network to convert the first set of training outputs into a second set of training outputs based on a second set of latent values ​​associated with a second domain; and training the first generator neural network based on a first set of losses calculated between the first set of latent values ​​and the second set of latent values.

[0104] 12. One or more non-transitory computer-readable media as described in Clause 11, wherein the instructions further cause the one or more processors to perform the steps of: executing a discriminator neural network to generate a first set of predictions associated with the first set of potential values; executing the discriminator neural network to generate a second set of predictions associated with the second set of potential values; and calculating the first set of losses based on the first set of predictions and the second set of predictions.

[0105] 13. One or more non-transitory computer-readable media according to any of the provisions of 11-12, wherein the instructions further cause the one or more processors to perform the steps of: executing a discriminator neural network to generate a first set of predictions associated with the first set of training outputs; executing the discriminator neural network to generate a second set of predictions associated with a second set of training data in the second domain; and training the first generator neural network based on a second set of losses calculated using the first set of predictions and the second set of predictions.

[0106] 14. One or more non-transitory computer-readable media according to any one of the provisions of 11-13, wherein the instructions further cause the one or more processors to perform the steps of: executing the second generator neural network to convert a second set of training data associated with the second domain into a third set of training outputs; executing the first generator neural network to convert the third set of training outputs into a fourth set of training outputs; and training at least one of the first generator neural network or the second generator neural network based on a second set of losses calculated between the fourth set of training outputs and the second set of training data.

[0107] 15. One or more non-transitory computer-readable media according to any of the provisions of 11-14, wherein the instructions further cause the one or more processors to perform the steps of: executing a discriminator neural network to generate a first set of predictions associated with the third set of training outputs; executing the discriminator neural network to generate a second set of predictions associated with the first set of training data; and training a second generator neural network based on a third set of losses calculated using the first set of predictions and the second set of predictions.

[0108] 16. One or more non-transitory computer-readable media according to any of the provisions of 11-15, wherein the instructions further cause the one or more processors to perform the steps of: executing a third generator neural network to convert the first set of training data into a third set of training outputs based on a third set of latent values ​​associated with the first domain; executing a fourth generator neural network to convert the third set of training outputs into a fourth set of training outputs based on a fourth set of latent values ​​associated with the third domain; and training at least one of the third generator neural network or the fourth generator neural network based on a second set of losses calculated between the third set of latent values ​​and the fourth set of latent values.

[0109] 17. One or more non-transitory computer-readable media according to any of the provisions of 11-16, wherein the first generator neural network includes a first encoder associated with the first domain and a first decoder associated with the second domain.

[0110] 18. One or more non-transitory computer-readable media according to any of the provisions of 11-17, wherein the second generator neural network includes a second encoder associated with the second domain and a second decoder associated with the first domain.

[0111] 19. One or more non-transitory computer-readable media according to any of the provisions of 11-18, wherein at least one of the first generator neural network or the second generator neural network comprises a residual block having a modified linear unit (ReLU) activation function.

[0112] 20. In some embodiments, a system includes: one or more memories storing instructions, and one or more processors coupled to the one or more memories, wherein the processors, when executing the instructions, are configured to perform the following steps: executing one or more components of a first generator neural network to transform between a first set of data and a first set of latent values ​​associated with a first domain; executing one or more components of a second generator neural network to transform between a second set of data and a second set of latent values ​​associated with a second domain, wherein the first generator neural network and the second generator neural network are trained using one or more losses computed between a third set of latent values ​​associated with the first domain and a fourth set of latent values ​​associated with the second domain; and generating one or more predictions based on the first set of latent values ​​and the second set of latent values.

[0113] Any element of any claim and / or any combination of any element described in this application, in whatever manner, is within the intended scope of this invention and its protection.

[0114] The descriptions of various embodiments are presented for illustrative purposes and are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations can be readily made by those skilled in the art without departing from the scope and spirit of the described embodiments.

[0115] Various aspects of this embodiment can be embodied as a system, method, or computer program product. Therefore, various aspects of this disclosure can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which are generally collectively referred to herein as a “module,” a “system,” or a “computer.” Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described in this disclosure can be implemented as a circuit or set of circuits. Moreover, various aspects of this disclosure can take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0116] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, such as, but not limited to, any one of the foregoing, or a suitable combination of any of the foregoing. More specific examples (not an exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or a suitable combination of any of the foregoing. In this document, a computer-readable storage medium may be any tangible medium capable of containing or storing a program for use by or in connection with an instruction execution system, apparatus, or device.

[0117] Various aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams illustrating methods, apparatus (systems), and computer program products of various embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine. When these instructions are executed by the processor of the computer or other programmable data processing apparatus, the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams are implemented. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0118] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, code segment, or portion of code, containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the order in which the functions marked in the blocks are executed may differ from the order shown in the figures. For example, two blocks shown consecutively in the figures may actually execute substantially simultaneously, or, depending on the functions involved, these blocks may sometimes execute in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, as well as combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware system or a combination of dedicated hardware and computer instructions that performs the specified function or action.

[0119] While the above description relates to embodiments of this disclosure, other embodiments of this disclosure may be designed without departing from the basic scope of this disclosure, the scope of which is defined by the following claims.

Claims

1. A computer-implemented method for determining a potential space between two or more domains, the method comprising: Execute a first generator neural network to transform a first set of training data associated with the first domain into a first set of training outputs based on a first set of latent values ​​associated with the first domain. The second generator neural network is executed to transform the first set of training outputs into the second set of training outputs based on the second set of potential values ​​associated with the second domain. as well as The first generator neural network is trained based on a first set of losses calculated between the first set of potential values ​​and the second set of potential values.

2. The computer-implemented method according to claim 1 further includes: Execute the discriminator neural network to transform the first set of potential values ​​into the first set of predictions; The discriminator neural network is executed to convert the second set of potential values ​​into a second set of predictions; as well as The first set of losses is calculated based on the first set of predictions and the second set of predictions.

3. The computer-implemented method according to claim 2, wherein, The first set of losses includes binary cross-entropy loss.

4. The computer-implemented method according to claim 1 further includes: The second generator neural network is trained based on a second set of losses calculated between a third set of potential values ​​generated by the first generator neural network and a fourth set of potential values ​​generated by the second generator neural network.

5. The computer-implemented method according to claim 1, further comprising: The first generator neural network is trained based on a second set of losses calculated between the second set of training outputs and the first set of training data.

6. The computer-implemented method according to claim 1, further comprising: The second generator neural network is executed to transform the second set of training data into the third set of training output; The first generator neural network is executed to convert the third set of training outputs into a fourth set of training outputs; as well as Based on the second set of losses calculated between the fourth set of training outputs and the second set of training data, at least one of the first generator neural network or the second generator neural network is trained.

7. The computer-implemented method according to claim 6, further comprising: The second generator neural network is trained based on the third set of losses calculated between the third set of training outputs and the first set of training data.

8. The computer-implemented method according to claim 1, further comprising: The first generator neural network is trained based on a second set of losses calculated between the first set of training outputs and a third set of training data associated with the second domain.

9. The computer-implemented method according to claim 1, further comprising: At least one of the first generator neural network or the second generator neural network is trained based on a second set of losses calculated using a set of classifiers associated with the first set of training outputs.

10. The computer-implemented method according to claim 1, wherein, At least one of the first domain or the second domain includes at least one of image-based data, sequencing-based data, omics data, physiological data, homeostatic cell data, disturbed cell data, young cell data, or aged cell data.

11. One or more non-transitory computer-readable media storing instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: Execute a first generator neural network to transform a first set of training data associated with the first domain into a first set of training outputs based on a first set of latent values ​​associated with the first domain. The second generator neural network is executed to transform the first set of training outputs into the second set of training outputs based on the second set of potential values ​​associated with the second domain. as well as The first generator neural network is trained based on a first set of losses calculated between the first set of potential values ​​and the second set of potential values.

12. One or more non-transitory computer-readable media according to claim 11, wherein, The instructions also cause the one or more processors to perform the following steps: Execute the discriminator neural network to generate a first set of predictions associated with the first set of potential values; The discriminator neural network is executed to generate a second set of predictions associated with the second set of potential values; as well as The first set of losses is calculated based on the first set of predictions and the second set of predictions.

13. One or more non-transitory computer-readable media according to claim 11, wherein, The instructions also cause the one or more processors to perform the following steps: Execute the discriminator neural network to generate a first set of predictions associated with the first set of training outputs; The discriminator neural network is executed to generate a second set of predictions associated with a second set of training data in the second domain; as well as The first generator neural network is trained based on a second set of losses calculated using the first set of predictions and the second set of predictions.

14. One or more non-transitory computer-readable media according to claim 11, wherein, The instructions also cause the one or more processors to perform the following steps: The second generator neural network is executed to transform the second set of training data associated with the second domain into a third set of training outputs; The first generator neural network is executed to convert the third set of training outputs into a fourth set of training outputs; as well as Based on the second set of losses calculated between the fourth set of training outputs and the second set of training data, at least one of the first generator neural network or the second generator neural network is trained.

15. One or more non-transitory computer-readable media according to claim 14, wherein, The instructions also cause the one or more processors to perform the following steps: The discriminator neural network is executed to generate a first set of predictions associated with the third set of training outputs; The discriminator neural network is executed to generate a second set of predictions associated with the first set of training data; as well as The second generator neural network is trained based on a third set of losses calculated using the first set of predictions and the second set of predictions.

16. One or more non-transitory computer-readable media according to claim 11, wherein, The instructions also cause the one or more processors to perform the following steps: A third generator neural network is executed to transform the first set of training data into a third set of training outputs based on a third set of latent values ​​associated with the first domain; The fourth generator neural network is executed to transform the third set of training outputs into the fourth set of training outputs based on the fourth set of latent values ​​associated with the third domain; as well as At least one of the third generator neural network or the fourth generator neural network is trained based on a second set of losses calculated between the third set of potential values ​​and the fourth set of potential values.

17. One or more non-transitory computer-readable media according to claim 11, wherein, The first generator neural network includes a first encoder associated with the first domain and a first decoder associated with the second domain.

18. One or more non-transitory computer-readable media according to claim 17, wherein, The second generator neural network includes a second encoder associated with the second domain and a second decoder associated with the first domain.

19. One or more non-transitory computer-readable media according to claim 11, wherein, At least one of the first generator neural network or the second generator neural network includes a residual block with a modified linear unit ReLU activation function.

20. A system comprising: One or more memories, their storage instructions, and One or more processors coupled to the one or more memories, and the processors, when executing the instructions, are configured to perform the following steps: Execute one or more components of the first generator neural network to transform between a first set of data and a first set of latent values ​​associated with a first domain; Execute one or more components of a second generator neural network to transform between a second set of data and a second set of latent values ​​associated with a second domain, wherein the first generator neural network and the second generator neural network are trained using one or more losses computed between a third set of latent values ​​associated with the first domain and a fourth set of latent values ​​associated with the second domain; and Generate one or more predictions based on the first set of potential values ​​and the second set of potential values.

Images