Reciprocating generative model

Abstract

Reciprocal generative models are provided. For each generative model in a set of K generative models that classify sensor data into K classes, in- distribution samples are sampled from training data as being classified as belonging to a class of the generative model, and out-of-distribution samples are sampled from the training data as being classified as not belonging to a class of the generative model. Out-of-distribution samples are also generated from each remaining reciprocal generative model in the set of reciprocal generative models other than the generative model to provide additional samples classified as not belonging to a class of the generative model. Parameters of the generative model are updated to minimize a loss function to maximize a likelihood of the samples belonging to the class and to maximize the loss function with respect to both the sampled out-of-distribution samples and the generated out-of-distribution samples to minimize a likelihood of the samples not belonging to the class.

Description

Technical Field

[0001] This disclosure relates to reciprocating generative models. Background Technology

[0002] Generative models learn joint probability distributions The discrimination model learns the conditional probability distribution. Identify distribution It can be used to directly present examples Classification However, generative models can be used to fit each example individually to a probability distribution for each class. When classifying new points, generative models can be used to find out which distribution the point is most likely to come from.

[0003] There is substantial work focused on applications of deep generative models for classification tasks. Two representative examples include “Semi-supervised Learning with Deep Generative Models” (Kingma, Rezende, Mohamed, Welling, 2014 arXiv 1406.298) and “Adversarial Autoencoders” (Makhzani, Shlens, Jaitly, Goodfellow, Frei, 2015 arXiv 1511.05644). Summary of the Invention

[0004] Based on one or more illustrative examples, a method for training an ensemble of reciprocal generative models for K classes is described. The ensemble of K generative models classifying sensor data into the K classes is initialized such that, during training, each of the K generative models is configured to classify an example into or out of a corresponding class among the K classes. For each of the K generative models, in-distribution samples are sampled from the training data of the generative model if they are classified as belonging to the class of the generative model, out-of-distribution samples are sampled from the training data of the generative model if they are classified as not belonging to the class of the generative model, and out-of-distribution samples are generated from each residual reciprocal generative model in the ensemble of reciprocal generative models other than the generative model to provide additional samples classified as not belonging to the class of the generative model. The parameters of the generative model are updated based on in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples. This minimizes the loss function with respect to in-distribution samples, thereby maximizing the likelihood that a sample belongs to a class of the generative model. Furthermore, it maximizes the loss function with respect to both sampled and generated out-of-distribution samples, thereby minimizing the likelihood that a sample does not belong to a class of the generative model. A set of K trained generative models is then used to classify sensor data with respect to these K classes.

[0005] According to one or more illustrative examples, a system is provided for training an ensemble of reciprocating generative models for K classes. The system includes a processor of a computing device, the processor being programmed to initialize an ensemble of K generative models for classifying sensor data into the K classes, such that, when trained, each of the K generative models is configured to classify an example into or from a corresponding class among the K classes. The processor is further programmed to: for each of the K generative models, sample in-distribution samples from the training data of generative models classified as belonging to a class of the generative model; sample out-of-distribution samples from the training data of generative models classified as not belonging to a class of the generative model; generate out-of-distribution samples from each of the remaining inverse generative models in the reciprocal generative model set (excluding the generative model) to provide additional samples classified as not belonging to a class of the generative model; and update the parameters of the generative model based on the in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples to minimize the loss function with respect to the in-distribution samples, thereby maximizing the likelihood that the samples belong to a class of the generative model, and to maximize the loss function with respect to both the sampled out-of-distribution samples and the generated out-of-distribution samples, thereby minimizing the likelihood that the samples do not belong to a class of the generative model. The processor is also programmed to classify the sensor data with respect to the K classes using the set of K trained generative models.

[0006] According to one or more illustrative examples, a non-transitory computer-readable medium includes instructions for training a set of reciprocating generative models for K classes, the instructions causing the processor, when executed by a processor, to perform operations. The operations include initializing a set of K generative models for classifying sensor data into the K classes, such that, when trained, each of the K generative models is configured to classify an example into or from a corresponding class among the K classes. The operation further includes: for each of the K generative models, sampling in-distribution samples from the training data for generative models classified as belonging to a class of the generative model, sampling out-of-distribution samples from the training data for generative models classified as not belonging to a class of the generative model, generating out-of-distribution samples from each of the remaining inverse generative models in the reciprocal generative model set (excluding the generative model) to provide additional samples classified as not belonging to a class of the generative model, and updating the parameters of the generative model based on the in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples to minimize the loss function with respect to the in-distribution samples, thereby maximizing the likelihood that the samples belong to a class of the generative model, and maximizing the loss function with respect to both the sampled out-of-distribution samples and the generated out-of-distribution samples, thereby minimizing the likelihood that the samples do not belong to a class of the generative model. The operation further includes classifying the sensor data with respect to the K classes using the set of K trained generative models. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of an exemplary embodiment of a system for training and using a reciprocating generative model; Figure 2 This is an example diagram of the training process for a reciprocal generative model; and Figure 3 This is an example diagram illustrating an exemplary embodiment of the process for training and using a reciprocating generative model. Detailed Implementation

[0008] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various forms and alternative forms. The figures are not necessarily to scale; some features may be enlarged or minimized to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those skilled in the art to adopt the embodiments in various ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any one figure may be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of illustrated features provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of this disclosure may be desired for a particular application or implementation.

[0009] As discussed in detail in this paper, a method for training generative and deep generative models for their application in classification with a reciprocal principle is described, wherein the loss function is maximized with respect to out-of-distribution samples (from the training set and / or generated) and minimized with respect to in-distribution samples.

[0010] Generative models—that is, models based on samples from sensors and related tags joint distribution The model used for modeling—long used to solve classification problems of sensor data. Given a multi-class problem (where...) Can be taken One of the values ​​(each indicating a different class) can be trained. There are 10 different generative models, among which Each of the generative models returns ,sample Originating from the first There are several classes (where their distribution is determined by...). (Modeling) the likelihood of the associated distribution. The model can also output the likelihood of its distribution. The generated sample.

[0011] One of the major obstacles associated with the use of generative models, and particularly with deep generative models (models utilizing deep neural network architectures), in their use for sensor data classification, is their inability to accurately identify out-of-distribution samples. This means that by training generative models via maximizing the likelihood of in-distribution samples, the system also risks increasing the likelihood of out-of-distribution samples. This high likelihood risk of out-of-distribution samples hinders the use of generative models as classifiers.

[0012] The disclosed method develops a training method for... A collection of generative models (one generative model per class) and a training dataset. Classify sensor data into A method for mutually inverse generative models of different classes. For each class The training process simultaneously trains the corresponding generative model through the following steps: 1) Crossing the first... Class In-distribution samples Minimize the parameters of the generative model above the expected value of the distribution. Loss items ; and 2) Maximizing the parameters of the generative model above the expected value of the distribution of out-of-distribution samples. Loss items The out-of-distribution sample consists of samples that do not belong to the first... Class training set The sample taken and the remaining The samples generated by the generative model are approximated by the two. (Belongs to class) The samples, as in-distribution samples, contribute positively to the training of the first sample. One generative model is generated, and it contributes (negatively) as an out-of-distribution sample to the training of the remaining generative models.

[0013] Such technology offers practical applications by being applicable to sensor signals received from a wide variety of sensors, such as video, radar, LiDAR, ultrasound, and / or motion sensors, and can be used to compute control signals for controlling physical systems, such as computer-controlled machines like robots, vehicles, household appliances, power tools, manufacturing machines, personal assistants, or access control systems. The described technology does this by classifying sensor data and explicitly training on in-distribution samples to find surrogates for out-of-distribution samples.

[0014] Figure 1 This is a schematic diagram of an exemplary embodiment of a system 100 for training and using a reciprocating generative model 114. System 100 includes a processor 102 operatively connected to a memory 104, a display device 106, a human-machine interface (HMI) control 108, and a network device 110. As described in more detail below, during operation, system 100 explicitly trains the generative model 114 to minimize a loss function with respect to an in-distribution set of samples of training data 116, and to maximize a loss function with respect to an out-of-distribution set of samples, including the use of training data 116 and generated data.

[0015] In system 100, processor 102 may include one or more integrated circuits that implement the functionality of a central processing unit (CPU) and / or a graphics processing unit (GPU). In some examples, processor 102 is a system-on-a-chip (SoC) that integrates the functionality of both a CPU and a GPU. The SoC may optionally include other components, such as memory 104 and network device 110, into a single integrated device. In other examples, the CPU and GPU are connected to each other via peripheral connectivity devices, such as PCI express or another suitable peripheral data connection. In one example, the CPU is a commercially available central processing device that implements an instruction set, such as one of the x86, ARM, Power, or MIPS instruction set families. Additionally, alternative embodiments of processor 102 may include a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or any other suitable digital logic device.

[0016] During operation, processor 102 executes stored program instructions retrieved from memory 104. The stored program instructions include software that controls the operation of processor 102 to perform the operations described herein. Memory 104 may include both non-volatile memory and volatile memory devices. Non-volatile memory includes solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when system 100 is disabled or loses power. Volatile memory includes static and dynamic random access memory (RAM), which stores program instructions and data during operation of system 100.

[0017] The GPU may include hardware and software for displaying at least two-dimensional (2D) and optionally three-dimensional (3D) graphics to display device 106. Display device 106 may include an electronic display screen, projector, printer, or any other suitable device for reproducing the graphics display. In some examples, processor 102 uses the hardware functionality in the GPU to execute software programs to accelerate the performance of machine learning or other computational operations described herein.

[0018] HMI control 108 may include any of a variety of devices that enable system 100 to receive control input from workers or other users. Examples of suitable input devices for receiving human-machine interface input may include keyboards, mice, trackballs, touchscreens, voice input devices, graphic tablets, etc.

[0019] Network device 110 may include any of a variety of devices that enable system 100 to send and / or receive data from external devices. Examples of suitable network devices 110 include network adapters or peripheral interconnect devices that receive data from another computer or external data storage device, which can be useful for receiving large datasets in an efficient manner.

[0020] Machine learning application 112 can use various algorithms to perform aspects of the operations described herein. In examples, as discussed above, machine learning application 112 may include instructions stored in memory 104 and executable by processor 102. Computer-executable instructions can be compiled or interpreted from computer programs created using a variety of programming languages ​​and / or techniques, which are not limited and include, individually or in combination, Java, C, C++, C#, Visual Basic, JavaScript, Python, Perl, PL / SQL, etc. Generally, processor 102 receives instructions, for example, from memory 104, computer-readable media, etc., and executes these instructions to perform one or more processes, including one or more processes described herein. Such instructions and other data can be stored and transmitted using a variety of computer-readable media.

[0021] In machine learning systems, model-based reasoning refers to inference methods that operate based on a machine learning model 114 that is analyzing a worldview. Generally, the machine learning model 114 is trained to learn a function that provides a precise correlation between input and output values. At runtime, the machine learning engine uses the knowledge encoded in the machine learning model 114 against observed data to derive conclusions such as diagnoses or predictions. The example machine learning library is the TensorFlow engine, available from Alphabet Inc. in Mountain View, California, although other machine learning systems can be used as an adjunct or alternative.

[0022] Different types of machine learning models are in use. Generative models learn joint probability distributions. The discrimination model learns the conditional probability distribution. Identify distribution It can be used to directly present examples Classification However, the generative model 114 can be used to fit examples individually to each class with probability distributions. When classifying new points, the generative model can be used to determine which distribution the point is most likely to come from.

[0023] As described in this article, model 114 is a generative model 114. Regarding generative model 114, let... Let X be a machine learning system that classifies sensor data x into K classes. Composed of K different generative models This enables each individual to generate a model. Corresponding to the first Generative models that model the distribution of each class. In deep generative model 114, each generative model... Through the parameter vector of the deep neural network A parameterized deep neural network is used for modeling. From the generative model 114, we can obtain the model from the first... Distributions associated with each class Generate the likelihood of sample x, and model 114 can be derived from the likelihood of the first sample x. Samples are generated from distributions associated with each class. .

[0024] In this system, 100 maximizes the cross-class of its own. In the process of estimating the expected value of the likelihood on the training data of 116 samples, each model is... The parameters. Conversely, those models 114 can also be trained to minimize the parameters that do not belong to the class. The negative log-likelihood over the 116 training data samples is as follows: The expected value of the negative log-likelihood can be interpreted as a loss function across the sample distribution. .

[0025] Therefore, this disclosure describes a method for creating a reciprocal generative model 114 and a reciprocal deep generative model 114, and how to train them from labeled training data 116. These models 114 are trained together as an ensemble using the training data 116 to simultaneously maximize the likelihood of samples derived from a target distribution and minimize the likelihood of samples derived from outside the target distribution. Each model 114 in the ensemble of models 114 is responsible for a single class. .

[0026] The classification system built on the reciprocal generative model 114 achieves high performance while being robust to out-of-distribution samples or even distribution shifts (where there are shifts in the distribution characteristics associated with the data and classes). This approach addresses the significant gaps associated with the use of discriminative models (lack of robustness to out-of-distribution samples or distribution shifts) while maintaining the high performance levels typically associated with discriminative models.

[0027] Figure 2 This is an example diagram of the training process 200 for the reciprocal generative model 114. As shown, thick solid arrows indicate training examples where the loss is minimized (in-distribution training examples); thin solid arrows indicate training examples where the loss is maximized (out-of-distribution training examples); and dashed arrows indicate generated examples where the loss is minimized (synthetic out-of-distribution training examples).

[0028] Given by (for Parameterized A set of generators, by ensuring that the loss function crosses from the th generator... The sample size of each class (within the distribution) is minimized, and the sample size across classes not originating from the first class is minimized. Maximizing samples from the nth class (outside the distribution) allows for the construction of a reciprocal generative model 114. This is achieved by optimizing the following two distinct tasks to obtain the model for the nth class. Parameters of the individual reciprocal generative model 114 of each class :1) Regarding its belonging to its first Minimize the loss (maximize the likelihood) for samples of each class, thereby accurately modeling the samples from the target distribution; and 2) regarding samples not belonging to the class. Maximizing the loss (minimizing likelihood) for each class of samples results in inaccurate modeling of samples not derived from the target distribution. By improving (2), model 114 is minimizing its... Out of distributionThe performance on the proxy of the samples, which is obtained by combining training samples from the residual class with generated samples from the residual inverse generative model 114.

[0029] against Training the reciprocal generative model 114 of each class can be performed as follows. The input used may include the training set. (For example, training data 116) A set of 114 (randomly) initialized generative models Initialize the binary stopping condition to false. Balance factor between in-distribution samples and out-of-distribution samples And the balance factor between real samples and synthetic samples. , loss function .

[0030] Given these inputs, the following procedure can be performed: WHILE It is fake FOR In 1- For samples within the distribution Sampling is performed so that the label of each sample is... 2-For out-of-distribution samples Sampling is performed so that the label of each sample is not... 3- Generate out-of-distribution samples from the remaining inverse generative model 4-Update Parameters of a reciprocal generative model Update stop conditions Therefore, a generative model 114 can be used, which is explicitly trained to minimize the loss function with respect to the in-distribution sample set and maximize the loss function with respect to the out-of-distribution sample set, which is either from the training set or generated—if the software used to train the generative model 114 is available.

[0031] Figure 3 This is an example diagram of an exemplary embodiment of the process 300 for training and using the reciprocating generative model 114. In the example, process 300 can be executed using system 100 after training process 200.

[0032] At operation 302, the set of reciprocating generative models 114 is initialized. The set of K generative models can be configured to classify sensor data into K classes, such that once trained, each generative model 114 in the set of K generative models 114 is configured to classify an example into or out of one of the K classes.

[0033] At operation 304, in-distribution samples are sampled for each reciprocating generative model 114. In the example, in-distribution samples are pulled from the training data 116 as samples classified as belonging to the class of the corresponding generative model 114.

[0034] At operation 306, out-of-distribution samples are sampled for each reciprocating generative model 114. In the example, out-of-distribution samples are pulled from the training data 116 as samples classified as not belonging to the class of the corresponding generative model 114.

[0035] At operation 308, out-of-distribution samples are generated from each of the remaining inverse generative models 114 in the set of reciprocal generative models 114, excluding generative model 114. These additional samples are generated as surrogates to provide supplementary samples that are classified as not belonging to the class of generative model 114. By using these additional generated samples, the ability of generative model 114 to accurately identify out-of-distribution samples is improved.

[0036] At operation 310, the parameters of the generative model 114 are updated. The update is performed based on the in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples to minimize the loss function with respect to the in-distribution samples, thereby maximizing the likelihood that the samples belong to the class of the generative model 114, and to maximize the loss function with respect to both the sampled out-of-distribution samples and the generated out-of-distribution samples, thereby minimizing the likelihood that the samples do not belong to the class of the generative model 114.

[0037] At operation 312, the set of K trained generative models 114 is used to classify the sensor data into K classes. In the example, the sensor data can be received from a wide variety of sensors, such as video, radar, LiDAR, ultrasonic, and / or motion sensors. By using the generative models 114, the sensor data can be accurately classified into K classes. Based on the classification of the sensor data, system 100 can calculate one or more control signals for controlling a physical system, such as a computer-controlled machine, like a robot, vehicle, household appliance, power tool, manufacturing machine, personal assistant, or access control system. After operation 312, process 300 ends.

[0038] The processes, methods, or algorithms disclosed herein may be deliverable to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, processes, methods, or algorithms may be stored in various forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media such as ROM devices and information changeably stored on writable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. Processes, methods, or algorithms may also be implemented in a software executable object. Alternatively, processes, methods, or algorithms may be embodied, wholly or partially, using suitable hardware components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or a combination of hardware, software, and firmware components.

[0039] While exemplary embodiments have been described above, they are not intended to describe all possible forms covered by the claims. The language used in this specification is descriptive rather than limiting, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other embodiments of the invention that may not be explicitly described or illustrated. While various embodiments may have been described as providing advantages over or preferred over other embodiments or prior art implementations in one or more desired features, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, suitability, weight, manufacturability, ease of assembly, etc. Accordingly, any embodiment described as less desirable in one or more features than other embodiments or prior art is not outside the scope of this disclosure and may be desirable for a particular application.

Claims

1. A method for training a set of reciprocating generative models for K classes to control a physical system, the method comprising: Initialize a set of K generative models that classify video sensor data into the K classes, such that when trained, each generative model in the set of K generative models is configured to classify an example into or out of one of the K classes. For each of the K generative models Sample in-distribution samples from the training data, such as those classified as belonging to the class of the generative model. Sample out-of-distribution samples from the training data, such as those classified as classes not belonging to the generative model. From each of the remaining inverse generative models in the reciprocal generative model set (excluding the generative model), off-distribution samples are generated to provide additional samples classified as classes not belonging to the generative model. The parameters of the generative model are updated based on in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples. The loss function is minimized with respect to in-distribution samples, thereby maximizing the likelihood that a sample belongs to the class of the generative model. The loss function is maximized with respect to both sampled out-of-distribution samples and generated out-of-distribution samples, thereby minimizing the likelihood that a sample does not belong to the class of the generative model. The set of K generative models, as trained, is used to classify video sensor data with respect to the K classes; and In response to signals generated based on classification of the K classes using video sensor data, control one or more of a robot, vehicle, household appliance, power tool, manufacturing machine, personal assistant, or access control system.

2. The method of claim 1, further comprising, in updating the parameters of each generative model, utilizing (i) a balance factor between in-distribution samples and out-of-distribution samples and (ii) a balance factor between real samples and synthetic samples.

3. The method of claim 1, wherein the parameters of the generated model are updated according to the following variance formula: in Indicate the parameters of the corresponding generative model. It is the balance factor between in-distribution samples and out-of-distribution samples. , It is a balance factor between real out-of-distribution samples and generated out-of-distribution samples from the training data. L It is a loss function. It is an in-distribution sample. It is an out-of-distribution sample, and These are out-of-distribution samples generated.

4. The method of claim 1, wherein the set of K generative models is initialized to random values ​​before learning.

5. A system for training a set of reciprocating generative models for K classes to control a physical system, comprising: The processor of the computing device is programmed to: Initialize a set of K generative models that classify video sensor data into the K classes, such that when trained, each generative model in the set of K generative models is configured to classify an example into or out of one of the K classes. For each of the K generative models Sample in-distribution samples from the training data, such as those classified as belonging to the class of the generative model. Sample out-of-distribution samples from the training data, such as those classified as classes not belonging to the generative model. From each of the remaining inverse generative models in the reciprocal generative model set (excluding the generative model), off-distribution samples are generated to provide additional samples classified as classes not belonging to the generative model. The parameters of the generative model are updated based on in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples. The loss function is minimized with respect to in-distribution samples, thereby maximizing the likelihood that a sample belongs to the class of the generative model. The loss function is maximized with respect to both sampled out-of-distribution samples and generated out-of-distribution samples, thereby minimizing the likelihood that a sample does not belong to the class of the generative model. The video sensor data is classified about the K classes using a set of trained generative models; and In response to signals generated based on classification of the K classes using video sensor data, control one or more of a robot, vehicle, household appliance, power tool, manufacturing machine, personal assistant, or access control system.

6. The system of claim 5, wherein the processor is further programmed to update the parameters of each generative model using (i) a balance factor between in-distribution samples and out-of-distribution samples and (ii) a balance factor between real samples and synthetic samples.

7. The system of claim 5, wherein the processor is further programmed to update the parameters of the generative model according to the following variance formula: in Indicate the parameters of the corresponding generative model. It is the balance factor between in-distribution samples and out-of-distribution samples. , It is a balance factor between real out-of-distribution samples and generated out-of-distribution samples from the training data. L It is a loss function. It is an in-distribution sample. It is an out-of-distribution sample, and These are out-of-distribution samples generated.

8. The system of claim 5, wherein the processor is further programmed to initialize the set of the K generative models to random values ​​prior to learning.

9. A non-transitory computer-readable medium comprising instructions for training a set of reciprocal generative models for K classes, the instructions, when executed by a processor, causing the processor to perform operations, the operations including: Initialize a set of K generative models that classify video sensor data into the K classes, such that when trained, each generative model in the set of K generative models is configured to classify an example into or out of one of the K classes. For each of the K generative models Sample in-distribution samples from the training data, such as those classified as belonging to the class of the generative model. Sample out-of-distribution samples from the training data, such as those classified as classes not belonging to the generative model. From each of the remaining inverse generative models in the reciprocal generative model set (excluding the generative model), off-distribution samples are generated to provide additional samples classified as classes not belonging to the generative model. The parameters of the generative model are updated based on in-distribution samples, out-of-distribution samples, and generated out-of-distribution samples. The loss function is minimized with respect to in-distribution samples, thereby maximizing the likelihood that a sample belongs to the class of the generative model. The loss function is maximized with respect to both sampled out-of-distribution samples and generated out-of-distribution samples, thereby minimizing the likelihood that a sample does not belong to the class of the generative model. The video sensor data is classified about the K classes using a set of trained generative models; and In response to signals generated based on classification of the K classes using video sensor data, control one or more of a robot, vehicle, household appliance, power tool, manufacturing machine, personal assistant, or access control system.

10. The medium of claim 9, wherein the instructions further cause the processor to perform operations including updating the parameters of each generative model using (i) a balance factor between in-distribution samples and out-of-distribution samples and (ii) a balance factor between real samples and synthetic samples.

11. The medium of claim 9, wherein the instructions further cause the processor to perform an operation comprising updating parameters of the generative model according to the following variance formula: in Indicate the parameters of the corresponding generative model. It is the balance factor between in-distribution samples and out-of-distribution samples. , It is a balance factor between real out-of-distribution samples and generated out-of-distribution samples from the training data. L It is a loss function. It is an in-distribution sample. It is an out-of-distribution sample, and These are out-of-distribution samples generated.

12. The medium of claim 9, wherein the instructions further cause the processor to perform an operation comprising initializing the set of the K generative models to random values ​​prior to learning.

Images