Generation and discriminant training as variable resolution gaming

Abstract

Computer-implemented systems and methods train generators and discriminators through machine learning, wherein the generators and discriminators are trained using simulated multiplayer games in adversarial relationships. Model parameters for the generator and discriminator may be updated non-simultaneously. In addition, the simulated multiplayer gaming may include a two-person zerosum gaming.

Description

[0001] Priority claim

[0002] This application claims priority to U.S. Provisional Application Serial No. 63 / 362,267, filed on March 31, 2022, which is incorporated herein by reference in its entirety. Background Art

[0003] In recent years, generative adversarial networks (GANs) have produced remarkable results, such as generating realistic images of fake faces, i.e. images that look like human faces but are not the faces of any real people. The training process for GANs trains both the generator and the discriminator in an adversarial relationship. The discriminator is trained to distinguish between real images and artificially generated images, and the generator is trained to fool the discriminator.

[0004] However, there are difficulties in training GANs and related generators. Some of these difficulties have existed since the earliest GAN systems and have been the focus of considerable research. Research has reduced the severity or frequency of these difficulties but has not eliminated them. Examples of difficulties include: (1) the need to carefully balance the quality of the partially trained discriminator and the partially trained generator throughout the training process; (2) instabilities during training; (3) mode collapse or partial mode collapse, i.e., the tendency of the generator to generate realistic images for some subset of data classes while ignoring other subsets; (4) the trained discriminator is unusable for other applications and is typically discarded; and (5) the generator receives little or no training on real data and only or mostly training on generated data, and therefore learns slowly on areas of real data for which there is no generated data. Summary of the invention

[0005] In one general aspect, the present invention relates to a computer-implemented machine learning system and method for training a generator and a discriminator together but in an adversarial relationship. The generator and the discriminator are trained according to a multi-player simulated game, in which each player, e.g., a generator player and a discriminator player, can adopt pure strategies and mixed strategies. The game can be a zero-sum game without causing instability during the training process. In fact, the zero-sum game formulation is preferred because it is a more natural representation of the competitive relationship between the discriminator and the generator, and because the stability and convergence of the training process are even better for the zero-sum formulation than for the non-zero-sum formulation. Finite zero-sum games have von Neumann solutions, not just Nash equilibria. Unlike Nash equilibria, the value of the von Neumann solution is unique. If another player uses the von Neumann solution, no player can improve their results by deviating from the von Neumann solution. This property further ensures stability during the training process.

[0006] In other embodiments, gaming may include additional players and alliances between players.

[0007] The benefits of training a generator and / or a discriminator according to embodiments of the present invention will be apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Various embodiments of the present invention are described in conjunction with the following drawings.

[0009] Figure 1 is a flow chart of an illustrative embodiment of aspects of the present invention for training a set of discriminators and a set of adversarial data generators.

[0010] Figure 2 are illustrative embodiments of aspects of the invention.

[0011] Figure 3 is a diagram of a computer system that can be used to implement various embodiments of the invention.

[0012] Figure 4 is a diagram of an illustrative example of a feedforward neural network.

[0013] Figure 5 is a flow chart of an illustrative embodiment of aspects of the invention.

[0014] Figure 6 A three-player cooperative game is depicted according to various embodiments of the invention.

[0015] Figure 7 is a system diagram of an illustrative embodiment of aspects of the invention.

[0016] Figure 8 is a flow chart of a development process according to various embodiments of the present invention that facilitates progressively training a generator and a discriminator for a range of domains of increasing complexity.

[0017] Fig. 9 and Fig.10 is a diagram of a machine learning system for training a generator and a discriminator together but in an adversarial relationship based on a multi-player simulated game according to various embodiments of the present invention. DETAILED DESCRIPTION

[0018] Fig. 9 and Fig.10 900 and a generator 901 in an adversarial relationship. The generator 901 may include an adversarial generator (also in Figure 6 601), and the discriminator 900 may include a discriminator decision module 902 (also sometimes referred to herein as a "discriminator" and in the Figure 6The discriminator decision module 902 is trained to determine or discriminate whether the input data item to the discriminator decision module 902 is generated by the generator 901 or whether it comes from another data sample source 922, which can be embodied as, for example, Figure 2 and Fig.10 The dedicated data generator and selector 222 in the discriminator can be a component of the discriminator or at least have a cooperative relationship with the discriminator. Specifically, the discriminator decision module 902 can be trained by machine learning to estimate the probability that the input sample to the discriminator decision module 902 comes from the adversarial generator 901 and from the data source 922. The generator 901 is trained by machine learning to maximize the probability that the discriminator decision module 902 is wrong about the source of the input item to the discriminator decision module 902 (in Fig. 9 Specifically, according to various embodiments of the present invention, a multi-player game is used to train the discriminator decision module 902 and the generator 901, where the players can adopt pure strategies and mixed strategies, as described below. In various embodiments, the game may include two players, such as Fig. 9 As shown: a generator player controlling the generator 901, and a discriminator player controlling aspects of both the discriminator decision module 902 and the data source 922. The player is implemented by a computer system, such as Figure 3 In other embodiments, there may be additional players (also implemented by the computer system 300 in the simulated game), such as Figure 6 As shown, this will be further described below.

[0019] Figure 1 is a flow chart of an illustrative embodiment of aspects of the present invention for training a set of "local" discriminators 971 for the discriminator decision module 902 and a set of adversarial data generators ("local generators" 977) for the generator 901. Each local discriminator 971 also has an associated local region and is trained to discriminate whether a presented data item (i.e., a data item presented to the local discriminator 971) is from a corresponding local generator 977. Thus, there may be a local discriminator 971 for each local generator 977, although in other embodiments, the number of local discriminators 971 and local generators 977 may not be equal. As described below, the discriminator 900 may include or at least cooperate with a plurality of local region detectors 973, each of which is trained to discriminate whether a data item presented to the local region detector is accepted or rejected as a member of a specified set associated with the corresponding local region detector 977. Thus, the number of local region detectors 973 may be equal to the number of local region generators 977, although in other embodiments, the number may not be equal. Figure 1The steps in the flowchart may be performed on a computer system (such as Figure 3 The system is implemented in software on a computer system 300 shown in FIG.

[0020] Figure 1 A key aspect of the illustrated illustrative embodiment is that the computer system 300 separates the modeling and training of policies for the discriminator and data generator set (boxes 111-114 and 121-124) from the tactical modeling and training of the discriminator and data generator set (boxes 131-134 and 141-144). As a preview, in boxes 111-114, the system 10 trains a hybrid strategy for the discriminator 900, which involves playing simulated games 114. In boxes 121-124, the system 10 trains a hybrid strategy for the generator 901, which involves playing simulated games 124. In boxes 131-134, the learning parameters of the machine learning component for the discriminator 900 are updated. In boxes 141-144, the learning parameters of the machine learning component for the generator 901 are updated. The updates to the policies and learning parameters can be based on the accumulated statistics 151 from the games. Preferably, processes 111-114, 121-124, 131-134 and 141-144 are not performed simultaneously. Instead, they can be performed one at a time and in no particular order, although it is preferred that the frequency of gaming (i.e., steps 111-114 and 121-124) is performed more frequently than updating the learning parameters (i.e., steps 131-134 and 141-144).

[0021] In the sense of the classical game theory of John von Neumann and Oskar Morgenstern, strategies can include pure strategies and mixed strategies. In this case, a pure strategy is a game strategy in which the player adopts the strategy or tactic that provides the best payoff. In a mixed strategy, the player chooses a strategy or tactic based on a probability distribution. Tactics can include local models that help simplify and stabilize the training of the discriminator and the set of data generators.

[0022] During training, in the illustrative embodiment, the computer system 300 may also iteratively update the discriminator model 971 (blocks 131-134, see also Fig.10 971) and iterative updates of the generator model (boxes 141-144; see also Fig.10 977), for example, the model of the local generator used for the generator 901, i.e., the iterative update of the discriminator model is not performed simultaneously with the iterative update of the generator model 977, which eliminates potential circular updates and increases convergence.

[0023] This illustrative embodiment illustrates several general principles that the computer system 300 can implement and combine in various ways in different aspects and different embodiments of the invention. In addition, the computer system 300 can use different embodiments and different design choices for different applications. For example, in some applications, once the training of the generator 901 and the discriminator 902 is completed, only the local generator 977 of the generator 901 is important for operational use, and the discriminator model 971 of the discriminator 902 is discarded or not used, as typically occurs in adversarial generator training systems. However, in some embodiments of the invention, the discriminator model 971 is equally important or more important than the generator model 977, and therefore, the discriminator model 971 is retained for operational use once training is completed.

[0024] There are options in the arrangement of the components of the system and in the order in which operations are performed during training. These options can be actively controlled by the computer system 300 and / or by human developers working as a human team plus the AI ​​training process control system 221 (see Figure 2 The training process control system 221 enables custom control and adjustment of the training process to proactively fine-tune the training during the training process rather than following pre-specified rules. The optional involvement of a human team enables human guidance during the training process.

[0025] Several broad principles are shown in the illustrative embodiment, which can be applied to other embodiments. First, the training process consists of a classic strategy game, in which the discriminator player 900 (also represented by Figure 1 111-114 in FIG. 111-114 ) and generator player 901 (also represented by Figure 1 121-124 in FIG. 12A and FIG. 12B ) actively selects a strategy. This feature is in contrast to a system for training an adversarial generator, in which the discriminator system is a passive pattern recognition system that does not have the ability to select a strategy.

[0026] Second, the game can be a zero-sum game without causing instability in the training process. In fact, the zero-sum game formulation is preferred not only because it is a more natural representation of the competitive relationship between the discriminator and the generator, but also because the stability and convergence of the training process are even better for the zero-sum formulation than for the non-zero-sum formulation.

[0027] Third, we enable the discriminator player to actively choose strategies to actively detect mode collapse or partial mode collapse during the training of the generator system, and then enable the generator system to learn to correct the mode collapse.

[0028] Fourth, the game part of the training process can be a finite zero-sum game, allowing the training of mixed strategies for two players in a simple way.

[0029] Fifth, fine-tuning of parameters in the pattern recognition system used by players ( Figure 1 Boxes 131-134 and 141-144) in FIG. 1 are a training process separate from the training of the hybrid strategy.

[0030] Sixth, fine-tuning of parameters in the pattern recognition system can be performed independently in each of the multiple local virtual "regions", thereby simplifying the training process.

[0031] Seventh, finite zero-sum games have von Neumann solutions, not just Nash equilibria. Unlike Nash equilibria, the value of a von Neumann solution is unique. If another player uses a von Neumann solution, no player can improve their results by deviating from the von Neumann solution. This property further ensures stability during training.

[0032] Eighth, on the other hand, if one player uses a non-optimal solution (e.g., partial mode collapse of the generator), then the second player can exploit the non-optimal solution to achieve better results than the von Neumann solution. This mechanism is one of the ways to detect and correct mode collapse and other defects in partially trained generator systems.

[0033] Ninth, in order to support the discriminator player to actively select strategies, the computer system 300 can implement a dedicated data generator and selector system 222 (see Figure 2 and Fig.10 ) as a data source 922 of other data items for the discriminator 902. This subsystem enables the computer system 300 to supply an unlimited amount of training data examples for training the local discriminator system 971 of the discriminator 902.

[0034] Tenth, the updating of the mixed strategy of the discriminator player (block 112) is separate from the updating of the mixed strategy of the generator player (block 122). Figure 1 In the illustrated embodiment, the mixed strategy is trained by simulating repeated games. Preferably, the updating of the mixed strategy of the discriminator player is performed alternately with the updating of the generator player, which is an iterative process that is more stable than simultaneous updating.

[0035] Eleventh, the parameter update of the discriminator pattern recognition model is separate from the parameter update of the generator pattern recognition model, and is preferably performed alternately, which is more stable than simultaneous updates, such as simultaneous gradient descent.

[0036] Figure 1The illustrative embodiment shown in FIG. 1 includes a loop from block 171 back to block 101. In block 101, the computer system 300 selects or updates a data space. In the loop back to block 101, the computer system 300 may refine a previously selected or updated data space or select a new data space. For example, in image generation, the computer system 300 may initially select a data space for a low-resolution image with a small number of pixels, and may increase the resolution and number of pixels in each successive loop through block 101.

[0037] The data space refinement and loop stopping criteria in block 171 may be predetermined criteria set by the system designer or controlled by the collaborative training process control system (see Figure 2 The collaborative training process control system 221 may include a human team and one or more AI systems working in collaboration. In this context, the term "collaboration" refers to the collaboration between the human team and the AI ​​system. Figure 2 In block 229 of , the word “cooperation” refers to the cooperation between the generator and the discriminator of discriminator 900 , as opposed to adversarial generator 901 . The AI ​​system of training process control system 221 may be implemented on computer system 300 or a separate computer system similar to computer system 300 .

[0038] In block 102, the computer system 300 obtains or updates a set of one or more feature spaces and a mapping from the data space selected in block 101 to each feature space. In some embodiments, the data space selected by the computer system 300 in block 101 may be used as the feature space, and the mapping may be an identity function. Preferably, in block 103, the process control system (see Figure 2 221), the computer system 300 selects to facilitate the local area detector 973 (see Fig.10 ) is the feature space for training.

[0039] Computer system 300 may train the feature space using any of several methods known to those skilled in the art of neural network training and / or may train the feature space using methods particularly relevant to various embodiments of the present invention.

[0040] For example, computer system 300 can train an autoencoder with a bottleneck layer and use the bottleneck layer activations as features in the feature space, which is well known to those skilled in the art of training neural network autoencoders. Computer system 300 can then use the encoder of the autoencoder as a feature mapping system. Specific to some embodiments of the present invention, the training of the autoencoder can include backpropagation from the target to the bottleneck layer in addition to the target of reproducing the input, which is a defining feature of the autoencoder. For example, Figure 8 As shown, computer system 300 can back-propagate to the bottleneck layer from the training of classifier 801 , from the supervised training of cluster 802 , and / or from the unsupervised training of cluster 803 .

[0041] As another example, the computer system 300 may select the activations of nodes in a fully connected layer in a convolutional neural network classifier as the feature space, where the convolutional layer is followed by one or more fully connected layers. In this example, the computer system 300 may train a modified version of the convolutional classifier using additional backpropagation from the training of steps 801, 802, and / or 803.

[0042] As another example, in some embodiments, the set of variables in the feature space may include one or more human-understandable features specified by the system designer and / or the human team in the collaborative human plus AI training process control system 221. In these embodiments, the computer system 300 may pre-train one or more human-understandable features using supervised training, and may also train the full set of human-understandable features using the training methods discussed above.

[0043] In block 103, reference is made to Fig.10 , the computer system 300 trains multiple local region detectors 973 of the discriminator 900. The local region detectors 973 are separate from the local discriminator models 971 of the discriminator decision module 902, and each local region detector 973 can be associated with one or more corresponding discriminator models 971 of the discriminator decision module 902. The number of local region detectors 973 can be the same as or different from the number of local discriminators 971 (e.g., Fig.10The local region detector 973 is a machine learning system trained to discriminate whether a presented data item is accepted or rejected as a member of a specified set. The concept accepted by the local detector 973 is a generalization of the concept of a local region in a vector space. The detector 973 trained by the computer system 300 in box 103 is called a "local region detector", and the set of data items accepted by the local region detector 973 is called a "local region". A "local region" in this sense is not necessarily similar to a "region" in the sense of a small, connected region in a vector space.

[0044] The local region detector 973 can be any type of machine learning system, including decision trees, random forests, mixtures of parameter probability distributions (such as Gaussian distributions), support vector machines, or neural networks. The local region detector 973 can be trained by supervised learning, unsupervised learning (such as unsupervised clustering), by partially supervised training using separate detectors for separate categories, or by semi-supervised learning (where some or all of the training data is labeled by the classification system itself).

[0045] Preferably, the feature space is selected by the computer system 300 to facilitate the training of the local region detector 973 and other goals. For example, in the case of using a convolutional neural network for image recognition, the space of activation vectors of the nodes of one of the fully connected layers after the convolutional layer can be used as a feature space to facilitate clustering. As another example, the bottleneck layer of an autoencoder neural network can be used as a feature space. Such an autoencoder neural network can be trained using unsupervised learning.

[0046] In some embodiments, computer system 300 may refine local region detector 973 by constructing a hierarchy in which a data set accepted by a previous local region detector may be subdivided into acceptance regions of two or more local region detectors that are applied to data examples accepted by the previous region detector. For example, Figure 8 The embodiment of one aspect of the present invention shown in makes extensive use of this process of subdividing local areas. Figure 8 .

[0047] The computer system 300 uses the local area detector 973 to define the pure strategy of the discriminator player.

[0048] In box 104, the computer system 300 initializes the discriminator and generator pattern recognition models, i.e., the discriminator model 971 of the discriminator 902 and the model of the local generator 977 of the generator 901. The number of local generators 977 may be equal to the number of local area detectors 973 and / or local discriminators 971, or the number of local generators 977 may be different from the number of local area detectors 973 and local discriminators 971. In box 104, the computer system 300 also initializes or updates the mixed strategies of the discriminator player and the generator player. For example, the computer system 300 may initialize all pure strategies to be equally possible. In some embodiments, the computer system 300 may initialize the mixed strategy of the player based on the previous mixed strategy of the player.

[0049] In block 105, the computer system 300 selects whether to train the discriminator player's mixed strategy (sub-block 111), train the generator player's mixed strategy (sub-block 121), train the parameters of the discriminator player's pattern recognition model (sub-block 131), and / or train the parameters of the generator player's pattern recognition model (sub-block 141). When using the selection process of block 105, in some embodiments, it is preferred that the computer system 300 alternates between the training processes rather than updating the two competing systems simultaneously.

[0050] In block 111, the computer system 300 selects a mixed strategy to update the discriminator player. In the illustrative embodiment, a pure strategy includes the computer system 300 selecting one of the local area detectors 973. The selection of the local area detector may affect the generation of a data item generated by the data source 922, which may be input to the discriminator decision module 902 depending on the switch 941. A mixed strategy includes the computer system 300 randomly selecting a pure strategy according to a specified probability distribution in the set of pure strategies.

[0051] Collectively, in blocks 112-114, the computer system 300 finds a new mixed strategy for the discriminator player that performs better relative to the current actual or estimated mixed strategy of the generator player, unless the current strategy of the generator player is an optimal von Neumann mixed strategy.

[0052] In one example embodiment, in block 112, the computer system 300 improves the discriminator player's current mixed strategy by finding a pure strategy that performs better than the discriminator player's current mixed strategy relative to the generator player's current mixed strategy. For example, the computer system 300 may calculate the payoff of each pure strategy relative to the generator player's current mixed strategy and select the strategy with the best score. Unless the generator player is using the optimal von Neumann mixed strategy relative to the current set of local region detectors 973, the computer system 300 may improve the performance of the discriminator player's current mixed strategy by increasing the probability of the selected pure strategy. This improvement may be achieved even if the discriminator player is already using the optimal von Neumann mixed strategy.

[0053] That is, if the Generator player's current mixed strategy is not optimal, then the Discriminator player can take advantage of this situation. In repeated games, this strategy of the Discriminator player only works until the Generator player fixes the Generator's mixed strategy. The Discriminator player then needs to find another flaw until both players are using the optimal von Neumann strategy, which is the desired long-term outcome. Temporarily deviating from the optimal strategy helps speed up learning for the player who has not yet found the optimal strategy.

[0054] The process by which the computer system 300 causes a player who is already close to an optimal strategy to temporarily use a non-optimal strategy eliminates the need to carefully balance the capabilities of the discriminator system and the generator system at each training stage during the training process.

[0055] In block 113, the computer system 300 selects data items from the data source 922 based on the pure policy selected by the computer system 300 in block 112. That is, for example, the computer system 300 selects data items accepted by the local area detector 973 specified by the pure policy selected by the computer system 300 in block 112. In the illustrative embodiment, the computer system 300 selects data items from the local area detector 973. Figure 2 Such data items are obtained by a dedicated data generator and selector 222 in .

[0056] Computer system 300 may select data items from a training data set or from data created by computer system 300 through data augmentation. Optionally, in some embodiments, computer system 300 may select data items generated by computer system 300 using supplemental collaboration generator 229 (see Figure 2 ). Computer system 300 can use the trimmed probability distribution in collaboration generator 229 and can also perform additional validation on the data (see Figure 2The complementary collaborative generator 229 differs from the adversarial generator 901 in that it is designed to collaborate with, rather than compete with, the discriminator decision module 902 and is able to use real data as the basis for its generation process.

[0057] In block 114 , computer system 300 then conducts a simulated game in which computer system 300 selects from data source 922 the data examples selected by computer system 300 for the discriminator player in block 113 or the data examples obtained by computer system 300 from the opponent generator player.

[0058] According to various embodiments, the simulated play of the game proceeds as follows. First, the computer system 300, on behalf of the discriminator player, but without explicit knowledge of the discriminator player, selects a local area detector 973 based on the current probability of the discriminator player's mixed strategy. For example, if the first local area detector has a 60% probability under the current discriminator player's mixed strategy, the probability of detecting the first local area detector is 60%.

[0059] Second, the computer system 300 acting as the discriminator player receives data from a data source 922, such as from a dedicated data generator and selector system 222 (see Figure 2 ) obtains data examples, i.e., data examples selected by computer system 300 in block 113. As described herein, data items from dedicated data generator and selector system 222 may be affected by selected local area detector 973.

[0060] Third, the computer system 300 obtains a data example from the adversarial generator system 901. The computer system 300 first randomly selects a generator model from the generator model set 977 of the generator 901 based on the mixed strategy of the generator player. The computer system 300 then generates a data example using the selected generator model 977 of the generator 901. The data example is generated by the computer system 300 for the adversarial generator system without regard to the local region detector 973 of the discriminator 900 or the selection of the local region detector 973 in the "first" step described above.

[0061] Fourth, in an example embodiment, the computer system 300 randomly selects whether to present a data item from the data source 922 (e.g., the dedicated data generator and selector system 222) in the above-mentioned "second" step, or to present an accepted data item obtained from the adversarial generator system 901 in the above-mentioned "third" step. The discriminator player (e.g., the discriminator decision module 902) does not know and will not learn which data source the computer system 300 has selected during the game. In some embodiments, the computer system 300 acting as a discriminator player can first select the local area detector with the highest detection score, and then determine the discrimination decision using the relevant local discriminator. In some embodiments, for example, if there is overlap between local detector areas, the computer system 300 can determine the discrimination decision as a collective decision of multiple local discriminators.

[0062] Fifth, the computer system 300 then obtains from the discriminator system 900 whether the discriminator system 900 classifies the presented data item as real or not generated by the adversarial generator 901 (e.g., from a data source) or generated by the adversarial generator 901. The classification may include a calculation of the likelihood that the presented data item is from the adversarial generator 901 (and / or a calculation of the likelihood that the presented data item is from another data source (e.g., data source 922) accordingly).

[0063] Sixth, the computer system 300 then Fig. 9 At block 951 of , it is determined whether the discriminator decision module 902 has made a correct discrimination. In some embodiments, the output of the discriminator decision module 902 is preferably a continuous value, piecewise differentiable function of each input variable. Similarly, for training purposes, the output of the generator 901 is a continuous value, although when the generator is deployed in an application (e.g., when generating digital images), the output value may be quantized. In order to determine whether the discriminator system 900 is correct for the presented data example, the computer system 300 compares the output of the discriminator decision module 902 with a specified threshold to determine whether the output indicates that the discriminator system has characterized the presented data example as obtained from a dedicated data generator and selector system 222 (or other data source 922) or from an adversarial generator system 901. For example, all output values ​​greater than or equal to the threshold can be interpreted as indicating data examples from the dedicated data generator and selector system 222, and all output values ​​less than the threshold can be interpreted as indicating data examples from the adversarial generator 901. The computer system 300 then determines whether the indicated judgment is correct or incorrect because the computer system 300 (but not the discriminator decision module 902) knows where the data item was selected in step 4 above.

[0064] Seventh, the computer system 300 then allocates a profit to each player. In an example embodiment, if the discriminator classification is correct, the computer system 300 allocates a score of +1 to the discriminator player, and if the discriminator classification is incorrect, a score of -1 is allocated. If no acceptable data examples are received from the adversarial generator player before the stopping criteria are reached in step (3) above, the computer system 300 also allocates a score of +1 to the discriminator player. In a preferred zero-sum embodiment, the computer system 300 allocates the negative of the profit allocated to the discriminator to the generator player. In some non-zero-sum embodiments, the computer system 300 allocates the negative of the profit allocated to the discriminator to the generator player only when the data item presented to the discriminator system is from the adversarial generator. Otherwise, in this example non-zero-sum embodiment, the computer system 300 allocates 0 profit to the generator player.

[0065] Eighth, in some embodiments, for any presented generated data item within a specified anti-plagiarism distance of any one of a specified set of specified training data items (such as copyrighted artwork or well-known earlier artwork), the computer system 300 may assign a payoff of +1 to the discriminator player and a payoff of -1 to the generator player. The computer system 300 may assign these payoffs regardless of whether the discriminator classification is correct or not. In some non-zero-sum embodiments, the computer system 300 may modify only the payoff of the generator player.

[0066] In block 121, the computer system 300 selects a mixed strategy to train against the generator player. Collectively, in blocks 122-124, the computer system 300 finds a new mixed strategy for the generator player that performs better relative to the discriminator player's current actual or estimated mixed strategy, except in the case where the discriminator player's current strategy is an optimal von Neumann mixed strategy.

[0067] In one example embodiment, in block 122, the computer system 300 selects a pure strategy for the adversary generator player. That is, the computer system 300 may select a local generator 977 of the generator 901. In some embodiments, the computer system 300 trains one or more local generators 977 to generate data primarily in a particular local region, i.e., data accepted by a particular local region detector 973 of the discriminator 902. However, in some embodiments, the computer system 300 may train the local generator 977 without considering the acceptance region of the local region detector 973.

[0068] In box 123 , the computer system 300 generates a data example using the local generator 977 selected in box 122 .

[0069] In block 124, the computer system 300 conducts a round of simulated gaming. The computer system 300 selects the data example generated by the adversarial generator 901 in block 123 or the data example from the data source 922, such as from the dedicated data generator and selector system 222 (see Figure 2 ) The computer system 300 then presents the selected data example to the discriminator model 971 of the discriminator 902. The discriminator model 971 of the discriminator 902 is unaware of whether the computer system 300 selected the adversarial data example generated in block 123 or the supplementary reality data item from the data source 922.

[0070] In block 124, the computer system 300 calculates the payoffs allocated to each player in the same manner as in block 114. The payoffs may be calculated, for example, as described above.

[0071] It should be noted that preferably, boxes 114 and boxes 124 are not combined into a single box because the simulated game in box 114 uses the selected pure discriminator strategy against the current mixed strategy of the adversary generator player, while the simulated game in box 124 uses the selected pure strategy of the adversary generator player against the current mixed strategy of the discriminator player.

[0072] In block 151, the computer system 300 accumulates statistics from the simulated games in blocks 114 and 124, such as an empirical estimate of each player's mixed strategy, as represented by the relative fraction of the number of times each pure strategy was chosen in the simulated play of the game. The empirical estimate may be a weighted average of previous games, where earlier games are not taken into account.

[0073] In block 131, the computer system 300 selects to update the parameter training of one of the local discriminator models 971 (e.g., pattern recognition systems) of the discriminator 902. In some embodiments, more than one local discriminator model / system 971 may be updated at a time, or block 131 may be selected in multiple consecutive loops from block 174 to block 105.

[0074] In block 132, computer system 300 selects local discriminator 971 and optionally local detector 973 of discriminator 902. In some embodiments, each local discriminator 971 is associated with a specific local detector 973, in which case the selection of local detector 973 is implicit in the selection of local discriminator 971.

[0075] In block 133, computer system 300 receives data from data source 922 (e.g., Figure 2The data examples are obtained by the computer system 300 from the dedicated data generator and selector 222 of the discriminator 900 and the local generator 977 of the adversarial generator 901. In some embodiments, the data examples are limited to the data examples accepted by the local region detector 973 of the discriminator 900 selected by the computer system 300 in block 132.

[0076] These obtained data examples are used to train the local discriminator 971 selected in box 132, and the selections are not represented as actions in the simulated game.

[0077] The number of data examples of each type to be obtained by the computer system 300 may be controlled by the system design and / or the training process control system 221. For example, the training process control system 221 may specify an amount of data examples estimated to be sufficient to accurately estimate the learning parameters in the selected local discriminator pattern recognition system 971.

[0078] In box 134, the computer system 300 uses the data examples obtained in box 133 to update the training of the local discriminator pattern recognition system 971 selected in box 132. These data examples may or may not have classification labels, but in contrast to the data presented to the discriminator system during simulated game play in box 114 or box 124, in box 134, each data example is labeled as obtained from a dedicated data generator and selector 222 or obtained from an adversarial generator system. In some embodiments, the output activations of the alternative outputs of the discriminator system can be constrained or normalized to sum to 1.0. The target values ​​in training the selected local discriminator pattern recognition system 971 can be 1.0 (correct response, whether realistic or adversarial), and 0.0 (incorrect response). In some embodiments, the computer system 300 can use less extreme target values ​​(such as 0.9 and 0.1) to train the system to respond more smoothly to small changes in the data.

[0079] As shown by the connection from box 134 back to itself, this training update can be an iterative process, such as gradient descent for a neural network model or an EM (Expectation Maximization) algorithm for a probability distribution mixture model.

[0080] In box 141, the computer system 300 selects to update the training of parameters of one of the local adversarial generator systems 977 of the generator 901.

[0081] In box 142, computer system 300 selects local adversarial generator 977 of generator 901.

[0082] In box 143 , the computer system 300 uses the selected adversarial generator to generate data examples to train the parameters of the local adversarial generator 977 selected in box 142 .

[0083] In block 144, the computer system 300 updates the parameters in the local adversarial generator 977. As with block 134, the parameter update at block 144 may be an iterative process, such as gradient descent of a neural network model.

[0084] If the local adversarial generator 977 is a neural network, the computer system 300 can train the neural network by mini-batch based gradient descent. For each data item, an estimate of the gradient is calculated by back propagation, which is a calculation method well known to technicians in the field of neural network training.

[0085] For such training, the computer system 300 needs to calculate the derivative of the output of the discriminator system with respect to each input variable of the discriminator (that is, each output variable of the generator network).

[0086] If the discriminator system is a neural network, the computer system 300 can calculate the derivative of the output score with respect to each input variable through back-propagation.

[0087] In some embodiments, for other forms of discriminators, the computer system 300 may calculate the derivative of the output score of the discriminator 902 with respect to each input variable of the discriminator. For example, the computer system 300 may perform such calculations on a discriminator model that includes a mixture of one or more parameter probability distributions.

[0088] If the local adversarial generator 977 is a model that can be trained by examples, for example, if the local adversarial generator model 977 is a mixture of parameter probability distributions, the computer system 300 can use a different algorithm to train the generator model 977. For example, in such an embodiment, the computer system 300 can use the EM algorithm, which is well known to those skilled in the art of statistical estimation. In some embodiments, the computer system 300 can use gradient descent to train the generator model 977 of a mixture of parameter probability distributions.

[0089] Block 161 is a no-op that collects control flows from blocks 151, 134, and 144. From block 161, the computer system 300 proceeds to block 174. In block 174, the computer system 300 determines whether to continue training using the current local region detector 973, feature space, and data space based on criteria specified by, for example, the training process control system 221. If the computer system 300 determines to continue the current training, the computer system 300 returns to block 105, otherwise the computer system 300 proceeds to block 173.

[0090] In block 173, the computer system 300 determines whether to refine the local region detector 973. For example, if the computer system 300 determines that replacing the current local region detector 973 with multiple detectors 973 may better meet the specified cost performance criteria, the computer system 300 may return to block 103. In some embodiments, the computer system 300 may determine to proceed to block 172 based on a specified limit on the number of training passes specified by the training process control system 221 or other stopping criteria.

[0091] In block 172 , the computer system 300 determines whether to return to block 102 to update the feature space or the mapping from the data space to the feature space, or proceed to block 171 based on the criteria specified by the training process control system 221 .

[0092] In block 171 , the computer system 300 determines whether to return to block 101 to update the data space or terminate the training process based on the stopping criteria specified by the training process control system 221 .

[0093] Figure 2 is an illustrative embodiment of aspects of the present invention, wherein computer system 300 implements dedicated subsystems to provide Figure 1 The discriminator system and local discriminators in blocks 113, 114, 124, 131-134 obtain or create additional data.

[0094] Figure 2 Box 221 represents a collaborative human and AI training process control system. The training process and control system include a human team and one or more AI systems implemented on a computer system (such as computer system 300). An example of such a collaborative human and AI system is described in PCT application WO2021 / 194516A1 of D5AI, entitled “Data-dependent Node-to-node Knowledge Sharing by Regularization in Deep Learning”, which is incorporated herein by reference in its entirety.

[0095] The AI ​​system is trained to tune hyperparameters, set criteria for stopping conditions, and perform validation and other testing of decisions such as when and how much to refine the data space, select among candidate feature spaces, control initialization of the discriminator model, and other tasks mentioned herein in discussing various aspects of the invention. The AI ​​system can be pre-trained on these control tasks, starting with trial and error, or using reinforcement learning on other tasks.

[0096] Typically, in the collaborative training process control system 221, the AI ​​system makes routine decisions, especially those that need to be made multiple times or are based on collecting a large amount of information. For example, when testing the possible plagiarism of images, audio, text, or other outputs of the adversarial generator, hundreds of thousands of tests or more may need to be performed. In this example, the testing will be completed by the AI ​​system, except that the human team can choose to actively intervene, or the AI ​​system can actively request human judgment, where there is a specified limit on the amount of human help requested.

[0097] The human team can guide the AI ​​system, for example, to set a guidance scheme for AI control hyperparameters. In some embodiments, the human team can actively guide the adaptive training of the AI ​​system to improve the performance of the AI ​​system on the training process control task.

[0098] Figure 2 Box 222 in represents a dedicated data generator and selector system, which includes a set of subsystems for enhancing and supplementing the available data for training the discriminator and generator systems.

[0099] In block 225, computer system 300 obtains the "real" data. Figure 2 In the context of box 225 of , "real" data includes data designated for training the discriminator 902 and the adversarial generator system 901 and other data from the same source that is separate from the selected training data. Typically, the computer system 300 can use unsupervised training, so the training data does not need to be labeled. However, in some optional aspects, the present invention can use supervised training, where some of the real data of box 225 can be labeled. In this case, some of the data separate from the training data can include labeled data.

[0100] Under the guidance of a human team or a computer system 300 implementing an AI system in a process control system, the computer system 300 provides additional real data during the training process at block 225. In addition to providing the data required to play the game, the computer system 300 can also provide additional data for various purposes.

[0101] For example, at block 244 , computer system 300 may set aside data to test the performance of the discriminator system on data that has not been used to train the discriminator system.

[0102] Furthermore, during the training process, the computer system 300 may gradually add some temporarily held-out data to the training set to improve the generalization ability of the discriminator system.

[0103] Computer system 300 may also obtain new data to add to real data repository 225, for testing at box 244 (as indicated by the dashed arrow from box 225 to box 244), and / or to add to training data.

[0104] In some embodiments, some data sets outside the current training may have been previously used to train a set of discriminators and adversarial generators. In such embodiments, the computer system 300 may store both real data and adversarial data in the repository 243. During the training of the current discriminator and generator system, the computer system 300 may retrieve these previous data. In such embodiments, as part of the testing in block 244, the computer system 300 may compare the performance of the current discriminator on such data with the performance of the discriminator that has been trained on the reserved data.

[0105] During the training process, the computer system 300 may retrieve some real data from the repository 225 and apply a data augmentation process at block 228. In some embodiments, the computer system 300 may also apply data augmentation to previous real data and / or previous adversarial data in the repository 243. In certain aspects of some embodiments, in block 227, the computer system 300 may limit the data to be selected to data accepted by the local region detector 973 specified by the computer system 300.

[0106] Any technique suitable for data enhancement may be used at box 228. For example, at box 228, the computer system 300 may perform data enhancement by selecting training data items and making changes to the selected data items. In addition, the system designer and / or the training process control system 221 may specify a class of small changes, such as translation, rotation, color filtering, or other perturbations, which are designed to change the data items without changing their classification. In an illustrative embodiment, the classification of small changes is limited to not changing the classification of the data items from data to be classified as real by the discriminator to data that is desired to be classified as artificial or forged. The computer system 300 may limit the magnitude of the type of small changes specified by the system designer and / or the training process control system 221 through hyperparameters adjusted by previous experience in similar tasks. Preferably, when selecting small changes, the computer system 300 may use a random process so that there is no limit to the number of different new data items that can be created and selected.

[0107] In some embodiments, the computer system 300 can use simulated adversarial attacks to enhance data items. Although sharing adversarial properties, the adversarial attack process is different from the adversarial generator. An adversarial attack is an attack on a class classifier. The attack makes small changes to a selected data item. The purpose of an adversarial attack is to make changes to the data item so that the correct category of the changed data item is still the same category as the original selected data item, but causes the class classifier to mistake the new item for a different category. Simulated adversarial attacks are well known to those skilled in the art of training defenses against adversarial attacks.

[0108] To compute the adversarial attack, the computer system 300 may use a previously trained classifier system that has been trained on a training set that overlaps with the training sets of the discriminator and generator systems. As an aspect of the present invention, the purpose of the simulated adversarial attack used by the computer system 300 is not to cause the classifier system to mistake new data items for different categories, but simply to find new directions for making small changes to selected data items that are beyond the directions selected for the categories of small changes caused by defined transformations (such as translations, rotations, color changes, etc.). Since the purpose of the simulated adversarial attack is not to produce a specified category change, the computer system 300 may select random weights for all categories of the classifier as targets for the direction of the small adversarial attack, optionally including the original classification. Thus, the direction of the simulated adversarial attack is determined by a random vector in N-dimensional space, where N is the number of categories in the pre-trained classifier. As is well known to those skilled in the art of adversarial attacks and defenses, the magnitude of the change can be determined by applying L to the change vector. p The norm is constrained by a hyperparameter ε, where common values ​​of p include 0, 1, 2, and infinity. Well-known methods for computing adversarial attacks include the Fast Gradient Sign Method (FGSM), the Projected Gradient Descent (PGD) method, etc.

[0109] In some embodiments, the computer system 300 may generate additional data in the supplementary collaborative generator system 229. Unlike training with adversarial data, the computer system 300 uses real data in training the collaborative generator system 229. The generator in the system 229 may be a modified version of any of the following: (1) a generative adversarial network (GAN), (2) a generator 901 trained according to an embodiment of the present invention, or (3) some other form of generator that can be trained for other purposes in addition to being trained for the purpose of deceiving the discriminator. However, in the training process of the modified version of the generator, the computer system 300 may include real data so labeled. It should be noted that such a modified generator is not suitable for use as an adversarial generator. Instead, the computer system 300 may use it as a collaborative generator. In certain aspects of some embodiments, in box 227, the computer system 300 may limit the data to be selected to data accepted by the local area detector 973 specified by the computer system 300.

[0110] In some example embodiments, in the supplementary collaborative generator 229, the computer system 300 may generate data by random variation around real data items or around feature space interpolation points between two or more real data items. The random variation in the collaborative data generator may use a trimmed probability distribution, such as a Gaussian distribution, to remove data samples if the data sample is greater than a specified number of standard deviations from the mean, where the restrictions are controlled by, for example, hyperparameters specified by the training process control system 221. The restrictions may be fractions of standard deviation <1.0. For multivariate Gaussians, restrictions may be imposed on each variable and / or on the vector norm of the vector of random variables, respectively. Preferably, the training process control system 221 is pre-trained to set trimming restrictions to ensure that data generation is true to the standards set by the system designer and / or the human team in the collaborative training process control system 221. In some embodiments, the human team may examine selected examples of the data generated by the supplementary collaborative generator and may adjust the trimming restrictions.

[0111] In box 226, the computer system 300 can perform additional validation tests on the data generated by the collaborative generator 229. For example, the computer system 300 can test the data items using a collection of one or more previously trained discriminator systems. The previously trained discriminator system may have been trained with data that is different from the data currently used to train the discriminator and adversarial generator system. Preferably, the previously trained discriminator system has been trained using the techniques described herein to train the discriminator system to generalize new data that was not included in the training data of the previously trained discriminator system. In some embodiments, human judgment can be used as a validation technique. Examples of human validation can be initiated by a human or an AI system in a collaborative training process control system. Preferably, humans will control the relative frequency of human validation.

[0112] In some embodiments, the computer system 300 may compute a consensus decision between multiple previously trained discriminator systems. For example, the computer system 300 may accept a data item as real data only if the data item is accepted by a specified majority of previously trained discriminator systems.

[0113] After training as described herein, the generator (or local generator) or the discriminator (or local discriminator or local detector) can be deployed in an operational (e.g., non-training) setting, although the generator and / or discriminator can be further trained after deployment. Thus, in some embodiments, the computer system 300 retains the discriminator system not only for the reality verification system of block 226, but also for other applications. In some applications, the discriminator system is used, but the adversarial generator system is not used. In some embodiments, the computer system 300 can train the discriminator system on additional tasks, such as with Figure 7 Related Discussion In some embodiments, computer system 300 may collaborate to generate the additional data in block 229 and / or enhance the additional data in block 228 as additional data for one or more additional targets.

[0114] In some of these applications, the computer system 300 can train the system to reject data other than that created by the adversarial generator. For example, the computer system 300 can train a classifier system to detect and classify not only a set of target categories, but also categories other than the target category, and classify other data as not belonging to any category. The computer system 300 can use such a classifier system as a discriminator, for example, by including the training data for the discriminator in the target category and rejecting all other data. In another example embodiment, the computer system 300 can distinguish data in the target category and other categories from all other data (including non-real data).

[0115] In applications for such real versus non-real data discriminators, the desired discrimination may be between all real data and all non-real data, not just non-real data created by an adversarial generator trained to attempt to fool the discriminator.

[0116] In some embodiments, the computer system 300 can obtain data from a category classifier that has been trained using additional classification options to represent data that is not in any specified category. In such embodiments, in block 241, the computer system 300 obtains data that such a classifier has been classified as not in a specified set of categories.

[0117] In some embodiments, the computer system 300 may obtain data from a classifier that has been trained using additional categories that are not in the specified target category set. In such embodiments, in block 242, the computer system 300 obtains data that such a classifier has classified as a category that is not in the specified category set.

[0118] In block 223, the computer system 300 selects the data obtained in blocks 225, 226, 228, 241, and / or 242 and generates a corresponding data item based on the data item in the block 223. Figure 1 The data is provided to the discriminator system 902 and / or the individual local area detectors 973 and local discriminators 971 in accordance with the requests in boxes 113, 114, 124, 133 and 143.

[0119] When implementing aspects of the collaborative human plus AI training process control system 221 , the computer system 300 may control the amount of data that the computer system 300 obtains from each source in blocks 114 and 124 .

[0120] In some embodiments, computer system 300 may implement a step-by-step refinement process, where at the final stage, one or more local regions may be small regions for which there are few data examples in the original training data set. In this case, computer system 300 may obtain additional data that is not included in the original training data set from real data repository 225. In some embodiments, computer system 300 may use collaboration generator 229 to obtain the additional data.

[0121] In some embodiments, the computer system 300 can continuously obtain additional data in order to train the discriminator and adversarial generator system to better generalize to new data. In some embodiments, the computer system 300 can delete some data from future rounds of the training process. The computer system 300 can add this deleted data to the repository 243. In some embodiments, the computer system 300 can implement a systematic process of continuously changing data controlled by the training process control system 221.

[0122] In some embodiments, the computer system 300 may implement a mixed strategy for a discriminator player by requesting an amount of real data from the local detector 973 that is proportional to the relative probability of the local detector 973 in the player's mixed strategy.

[0123] Figure 3 300 is a diagram of a computer system 300 that can be used to implement the above-described embodiments, such as the above-described processes described in conjunction with the various figures. The computer system 300 shown includes a plurality of processor units 302A-B, and in the illustrated embodiment, each processor unit includes a plurality (N) of processor cores 304A-N. Each processor unit 302A-B may include onboard memory (ROM or RAM) (not shown) and off-board memory 306A. The on-board memory may include primary, volatile and / or non-volatile memory (e.g., memory directly accessible to the processor cores 304A-N). The off-board memory 306A-B may include auxiliary, non-volatile memory (e.g., memory that the processor cores 304A-N cannot directly access), such as ROM, HDD, SSD, flash memory, etc. The processor cores 304A-N may be CPU cores, GPU cores, and / or AI accelerator cores. The GPU cores run in parallel (e.g., a general purpose GPU (GPGPU) pipeline), and therefore can generally process data more efficiently than a group of CPU cores, but all cores of the GPU execute the same code at the same time. AI accelerators are a class of microprocessors designed to accelerate artificial neural networks. They are also often used as coprocessors in devices with a host CPU 310. AI accelerators typically have tens of thousands of matrix multiplier units that operate at lower precision than a CPU core, e.g., 8-bit precision in an AI accelerator versus 64-bit precision in a CPU core.

[0124] In various embodiments, different processor cores 304 may implement different steps of various processes and programs. For example, in one embodiment, the core of the first processor unit 302A may implement the training process of the local discriminator 971, and the core of the second processor unit 302B may implement the training process of the adversarial generator 977. In addition, different core groups in the first and / or second processor units 302A, 302B may be responsible for independent training of different local discriminators 971 or different local adversarial generators 977. As another example, another multi-processor unit 302C may implement the AI ​​system in the training process control system 221. In addition, different core groups in the processor unit 302C may be responsible for different AI systems. As a further example, another multi-processor unit 302D may implement the supplementary collaboration generator 229. In addition, different cores in another multi-processor unit 302E may respectively implement data enhancement 228 and data selection 223.

[0125] One or more host processors 310 may coordinate and control processor units 302A-B. The processes depicted in the various figures may be embodied as a set of instructions stored in a memory (e.g., an integrated memory of processing units 302A, 302B or an off-board memory 306A coupled to processing units 302A, 302B or other processing units), which is coupled to one or more processors (e.g., at least one of the processor core groups 304A-N of processing units 302A, 302B or another processor communicatively coupled to processing units 302A, 302B), such that when executed by one or more processors, the instructions cause the processors to perform the above processes by, for example, controlling machine learning systems 221 and 222 or 701, 702, and 703 stored in processing units 302A, 302B, and 302C.

[0126] In other embodiments, computer system 300 can be implemented with one processor unit. In embodiments where multiple processor units are present, the processor units can be co-located or distributed. For example, the processor units can be interconnected using suitable wired and / or wireless data communication links over a data network (e.g., LAN, WAN, the Internet, etc.). Suitable data links, such as a data bus (preferably a high-speed data bus) or a network link (e.g., Ethernet) can be used to share data between the various processing units.

[0127] The software of the various computer systems described herein and other computer functions described herein can be implemented in the form of computer software using any suitable computer programming language such as .NET, C, C++, Python and using conventional functions or object-oriented techniques. The programming language of the computer software and other computer-implemented instructions can be translated into machine language by a compiler or assembler before execution and / or can be directly translated by an interpreter at runtime. Examples of assembly languages ​​include ARM, MIPS, and x86; examples of high-level languages ​​include Ada, BASIC, C, C++, C#, COBOL, CUDA, Fortran, Java, Lisp, Pascal, ObjectPascal, Haskell, ML; examples of scripting languages ​​include Bourne scripts, JavaScript, Python, Ruby, Lua, PHP, and Perl.

[0128] Figure 4 is a diagram of an illustrative example of a feedforward neural network. The local generator 977 of the generator 901 and the local region detector 973 and the local discriminator 971 of the discriminator 902 and / or the collaborative data generator 229 may be implemented, for example, with a feedforward neural network. Figure 4The example network in has an input layer, an output layer, and three hidden or inner layers. The example network is a fully connected layered network in which each non-input node receives connections from a node in the previous layer. More generally, in a feedforward network, any non-input node can receive connections from an appropriate subset of nodes in the previous layer, and can also receive connections from any node in any lower layer. In fact, any directed acyclic graph can define the connections in a feedforward neural network.

[0129] More generally, a neural network can have connections from a node in a higher layer to itself or to nodes in a lower layer, creating a recurrent neural network. However, without loss of generality, any recurrent neural network with a finite limit T on the number of connection steps from a node to itself or to a lower node can be unfolded into a large feed-forward neural network with T copies of the base network. The process of unfolding a recurrent neural network is well known to those skilled in the art of training neural networks.

[0130] Figure 5 is a flow diagram of an illustrative embodiment of aspects of the invention in which a computer system 300 may implement one or more forms of statistical smoothing. "Statistical smoothing" is a general term that refers to any of a number of techniques used in various machine learning and statistical estimation systems. Typically, statistical smoothing is used to normalize estimates of learning parameters so that the statistical or machine learning model is able to better generalize to new data not included in the training of the system.

[0131] When training a discriminator and a generator in an adversarial relationship, additional smoothing is also required. This additional requirement not only appears in the embodiments of the present invention, but also appears in the training of any system in which the discriminator and the generator are in an adversarial relationship, in which the generator attempts to deceive the discriminator.

[0132] For example, in the zero-sum two-player game formulation of the present invention, if unconstrained and trained on a finite amount of training data, the von Neumann optimal solution for the discriminator is to memorize the training data and then accept all and only data items that completely match the training data items.

[0133] However, this game-theoretic optimal solution is not the desired outcome of the development process for these adversarial discriminator-generator systems. However, the desired outcome is to have the generator generate data items that are so similar to the training data items that the discriminator will be fooled. Computer system 300 can use statistical smoothing and other regularization methods so that the discriminator and adversarial generator can be trained toward the goal of perfect performance without having to fully achieve that goal.

[0134] Figure 5 The flowchart of is an illustrative example of one way to organize some statistical smoothing or regularization methods that can be used during training, such as Figure 1 As shown. Due to the separation of the training of the hybrid strategy from the training of the model parameters, the separation of the updating of the discriminator model parameters from the updating of the adversarial generator parameters, the optional use of collaborative generators, the ability to mix neural networks with other kinds of machine learning systems, and the collaboration of human teams and AI systems in the training process control system 221, the computer system 300 can choose from a variety of statistical smoothing and regularization methods. Figure 5 The examples shown in are merely a sample of the many ways in which computer system 300 may be used.

[0135] In block 501, the computer system 300 sets a hyperparameter value that controls the degree of smoothness or regularization, as specified by the system design of the training process control system 221. Smoothness may affect the aesthetics of the generated examples and the performance of the discriminator, collaborative generator, and / or adversarial generator on new data. The AI ​​system in the training process control system 221 may measure performance on reserved data. Preferably, human involvement will be involved in judging the aesthetics (511).

[0136] In block 502, computer system 300 selects one or more available statistical smoothing and / or regularization methods. Computer system 300 may select any of the methods shown in blocks 503-510 or any well-known method that may be suitable for the type of machine learning system used in embodiments of the present invention.

[0137] For statistical smoothing purposes, it may be sufficient to use one or more of the various smoothing and regularization methods available. For example, if computer system 300 uses anti-plagiarism regularization (506) when training the adversarial generator, it is generally not necessary to use anti-plagiarism in the training of the collaborative generator because, in typical embodiments, computer system 300 allows the collaborative generator to copy training data examples. Computer system 300 can apply these statistical smoothing and / or regularization methods in various orders and during various stages of the training process.

[0138] In block 503, in a discriminator or generator implemented as a neural network, the computer system 300 may limit or regularize the weights of the connections in the neural network. For example, the computer system 300 may impose a maximum value on the magnitude (absolute value) of any weight. In this and other limits used in blocks 503, 504, and 505, the computer system 300 may impose an absolute limit or regularization penalty on weights that exceed a specified value. In some embodiments, the computer system 300 may only apply a regularization penalty. The regularization penalty may, for example, be equal to a hyperparameter multiplied by the L1 or L2 norm of the connection weight.

[0139] In block 504, in a discriminator or generator implemented as a neural network, the computer system 300 may limit the magnitude of activations of certain nodes in the neural network. For example, the computer system 300 may use an activation function such as a sigmoid function or a hyperbolic tangent. During training, the computer system 300 may apply a regularization penalty to nodes with unbounded activation functions. The computer system 300 may specify a node-specific value above which a regularization penalty is applied. The computer system 300 may apply and adjust node-specific values ​​controlled by the training process control system 221 that are customized for the situation at the point in training where regularization is applied.

[0140] In block 505, in a discriminator or generator using a parametric probability model, such as a Gaussian mixture model (GMM), the computer system 300 may impose a limit or regularization on the minimum value of a parameter, such as a standard deviation, as a measure of spread. The computer system 300 may apply this limit or regularization uniformly to all probability distributions in the mixture, or may selectively apply this limit or regularization using node-specific hyperparameters.

[0141] In block 506, the computer system 300 may apply anti-plagiarism regularization to the adversarial generator. As the name implies, in some applications, anti-plagiarism regularization may be required without smoothing. However, even in cases where plagiarism avoidance is not required, the computer system 300 may apply anti-plagiarism regularization as a method of statistical smoothing. In fact, the purpose of anti-plagiarism regularization is to prevent the adversarial generator from exactly copying the training data examples, which would otherwise be a von Neumann optimal solution.

[0142] Computer system 300 may determine an anti-plagiarism penalty based on the distance between the generated example and the closest data item in the training data. As a plagiarism prevention mechanism, computer system 300 may only need to search for the closest training data item in a particular list, such as copyrighted works of art or works of old masters. However, for statistical smoothing, if the training data is dense enough in space, computer system 300 may check all training data items or a representative sample of training data items. The method of measuring distance may depend on the type of data. For example, for images, the distance may be L p Norm, in pixel value vector space or feature space, 0≤p≤∞.

[0143] In block 507, the computer system 300 may train overlapping local region detectors 973. That is, two or more local region detectors 973 may all accept a data item. For statistical smoothing purposes, for a generated data item, the computer system 300 may randomly select which local region detector 973 is associated with the item. In some embodiments, when the same data item is presented later, the computer system 300 may independently and randomly select an associated local region detector 973.

[0144] In block 508, the computer system 300 may train the discriminator, the local detector, and / or the collaborative generator on a plurality of tasks, such as Figure 7 As shown in the example in . Training a machine learning system on multiple tasks has an indirect regularization effect. During training, the computer system 300 automatically adjusts the learned parameters to achieve the best compromise that satisfies the entire set of objectives, which will usually not be perfect on any one objective. In the training of the discriminator in simulated games, it is usually impossible to memorize the training data of the discriminator relative to the adversarial generator because there are tradeoffs in the solutions to the multiple tasks. It should be noted that Figure 7 The example tasks in are those that occur naturally in a variety of applications and are not specifically designed for statistical smoothing purposes and to avoid von Neumann solutions for memorizing the training data.

[0145] In block 509, the computer system 300 may use target values ​​that are less extreme than the limits of the output value range when training the discriminator. For example, if the value range of the output variable in the discriminator is [0, 1], the computer system 300 may use target values ​​such as 0.1 and 0.9.

[0146] In block 510, computer system 300 may add additional data to the training set. In contrast to many machine learning tasks, in various embodiments of the present invention, there is no limit to the amount of training data that may be obtained. Computer system 300 may always use Figure 2 The computer system 300 may also obtain more data from the repositories 225 and 243. In a preferred embodiment, the computer system 300 obtains such additional data specified by the training process control system 221 to improve the generalization performance to new data. The statistical smoothing effect of the additional data is an additional benefit.

[0147] In box 511, the computer system 300 tests the performance of the discriminator system on the data retained from the training and / or the newly generated data. In some embodiments, the computer system 300 can obtain a judgment on the aesthetics or other subjective qualities of the output of the collaborative generator or the adversarial generator. Preferably, the subjective quality is judged by humans, although the AI ​​system in the training process control system 221 can provide assistance. In addition, the human subjective judgment required by the computer system 300 in box 511 does not require humans to have expertise in machine learning. The person providing the subjective judgment may be a domain expert, or may generally be an end user in the application that the system is training. Therefore, the process in box 511 is an example of AI guided by end users.

[0148] If the computer system 300 determines in block 511 that the smoothing control hyperparameters need to be adjusted, the computer system 300 proceeds to block 512. Otherwise, the computer system 300 completes Figure 5 The process shown in .

[0149] Figure 6 Schematic diagram of the present invention as an embodiment of a three-player cooperative game. In the von Neumann-Morgenstern theory of n-player cooperative games, the game analysis involves alliances between groups of players, which is in contrast to the Nash equilibrium analysis of non-cooperative games.

[0150] In the illustrative example, the third player 603 includes a collaboration generator 607, a game strategy AI 604, a data and model repository 605, and a feature mapping system set 606. These components are roughly equivalent to Figure 2 The dedicated data generator and similar subsystem in the selector 222. However, in Figure 6 In the illustrated embodiment, the situation is slightly different because the third system 603 can also form an alliance 612 with the confrontation generator player 601 instead of forming an alliance 613 with the discriminator decision module 602.

[0151] exist Figure 6 In one embodiment shown, the game between any one player and the opposing two-player alliance is equivalent to a two-player game, preferably a zero-sum game.

[0152] In one example embodiment, the game between the adversary generator player 601 and the alliance 613 of the discriminator player 602 and the data enhancer and selector player 603 is equivalent to Figure 1 and Figure 2The two-player zero-sum game shown in the embodiment of FIG. 6 is different in that player 603 can switch alliances and the distribution of the game proceeds among the three players changes. In a preferred embodiment, the computer system 300 controls the switching of alliances, such as controlled by the training process control system 221. In a preferred embodiment, the alliance 611 of the confrontation generator player 601 and the discriminator player 602 is not used.

[0153] In some embodiments, the computer system 300 randomly switches between the alliance 613 of the discriminator player 602 and the data enhancer and selector player 603 and the alliance 612 of the confrontation generator player 601 and the data enhancer and selector player 603. The probability and / or frequency of switching alliances can be controlled by the training process control system 221.

[0154] The game between the discriminator player 602 and the coalition 612 of the counter generator player 601 and the data enhancer and selector player 603 enables the computer system 300 to arrange the payoff so that the payoff goes directly from the discriminator player 602 to the player 603, while Figure 1 In the zero-sum version of the two-player game, the payoff goes to the adversarial generator player.

[0155] During the game, the discriminator player 602 receives data items, but Figure 1 Similar to the two-player game shown, the discriminator player does not know that the computer system 300 is from player 603 (equivalent to Figure 2 's dedicated data generator and selector 222) or obtains data items from player 601 (against the generator player).

[0156] In an example embodiment, if the computer system 300 obtains a data item from the data augmentation and selector player 603 during the game play, the discriminator player 602 pays or receives a payment to the player 603. For example, the payoff to the player 603 may be +1 or -1, depending on whether the discriminator correctly identifies the data item obtained from the player 603. That is, if the discriminator 902 incorrectly identifies the data item, the payoff to the player 603 may be +1 (conversely, the payoff to the discriminator is -1); and if the discriminator correctly identifies the data item, the payoff to the player 603 may be -1 (conversely, the payoff to the discriminator is +1).

[0157] On the other hand, if computer system 300 obtains a data item from adversary generator player 601 during game play, discriminator player 602 pays or receives payment to or from adversary generator player 602. The payoff to discriminator player 602 may be +1 or -1, depending on whether the discriminator correctly identifies that the data item was obtained from adversary generator player 601.

[0158] In one example embodiment, during the game, the computer system 300 randomly selects whether the data item to be presented to the discriminator player 602 is obtained from the opponent generator player 601 or from the data augmentation and selector player 603, where the alliance of the opponent generator 601 and the player 603 has no control over the data source selection. This embodiment results in a training process similar to Figure 1 and Figure 2 The training process for the two-player zero-sum game shown in has similar results.

[0159] In another example embodiment, the computer system 300 may give some control over the selection of the source of the data items to be presented to the discriminator player 602 to a coalition of the adversarial generator player 601 and the data augmentation and selector player 603, where the amount of control over the selection is specified by the training process control system 221. This flexibility enables the training process control system 221 to adapt to application goals outside of gaming. For example, based on the application, the computer system 300 or a human team in the training process control system may specify that avoiding mode collapse is more important than proportionally modeling the probability distribution of data between local detector regions 973, or vice versa. The computer system 300 can adjust the ratio of data obtained from the data augmentation and selector player 603 to the data obtained from the adversarial generator player 601 to achieve the desired balance. In some embodiments, the computer system 300 can change the ratio as specified by the training process control system 221, because the ratio may need to be different as the detector region becomes smaller.

[0160] like Figure 5 As discussed in the discussion of block 508 of , in some embodiments of the invention, the discriminator and the collaborative generator may be trained with multiple objectives. Figure 7 is an illustrative embodiment of a system including a generator 701, a classifier or discriminator 703, and a discriminator 702 trained with multiple objectives, with a list of example tasks in which such a system may be used during training.

[0161] exist Figure 7 In some embodiments, the discriminator 702 may be Figure 1 and Figure 2 The collaborative generator 701 may be similar to one of the local discriminators 971 in the illustrative embodiment of the present invention shown in FIG. Figure 2 Collaboration Generator 229 in. Figure 7 The system shown in also includes one or more additional discriminators or classifiers 703, which are trained to perform tasks such as those listed in box 720. Although only one additional discriminator or classifier is shown in the figure, any number of additional discriminators / classifiers 720 can be used, as indicated by the ellipsis "...".

[0162] exist Figure 7 In the illustrated embodiment, there are one or more connections from a node in the discriminator 702 to an additional discriminator or classifier 703. Connections from the discriminator 702 may be established by the computer system 300 from any internal node or input node of the discriminator 702. Connections from input nodes of the discriminator 702 are equivalent to connections from output nodes of the collaboration generator 701, as shown.

[0163] The computer system 300 can establish a connection from a node in the discriminator 702 to any node of the discriminator or classifier 703, including input nodes and output nodes.

[0164] An example task is to allow the end user to select human preferences from a generated list of examples. The end user's human preferences can be, for example, colors, objects, tones, shapes, etc. in images; sounds, melodies, instruments, etc. in audio; words, sentence structures, etc. in text. The embodiments that allow the end user to select human preferences are different from Figure 5 In this embodiment, the computer system 300 presents a collection of two or more images or other generated items to a human. The human may be an end user or a system developer. If presented with a pair of items, the human may specify a preference for one or the other or indicate no preference.

[0165] In some embodiments, if more than two items are presented at a time, the human will select the most preferred item. In some embodiments, the human can divide the presented items into two groups, a preferred group and a less preferred group.

[0166] In these embodiments, subsystem 703 distinguishes between preferred and non-preferred items. If a human participant indicates a preference, the computer system 300 back-propagates the derivative of the preference as a training target for the discriminator 703. The computer system 300 also back-propagates the derivative of the target from the nodes in the discriminator 703 that receive connections from the discriminator 702 back to each connected source node, and then returns to the sub-network of the source node. Finally, the computer system 300 back-propagates the derivative to the collaborative generator 701 through the input node of the discriminator 702. Through the addition rule of derivatives, the computer system 300 adds these derivatives back-propagated from the discriminator 703 to the derivatives back-propagated from the target output by the computer system 300 in the training of the discriminator 703, whose main task is to distinguish the data items obtained from the collaborative generator from the data items obtained from the adversarial generator.

[0167] There is no direct backpropagation to the adversarial generator, except for the items it generates. However, Figure 1The competition process of the illustrated embodiment will cause the adversarial generator to learn to imitate the data items generated by the collaborative generator 701 to satisfy human preferences back-propagated through the discriminator 703.

[0168] In some embodiments, the collaboration generator 701 may generate data that is accepted by more than one local detector 973. If the computer system 300 back-propagates a human preference that favors one of the local detectors 973, the effect may be to increase the number of items generated in that local detector's region, which may effectively change the discriminator player's mixed strategy.

[0169] In some applications, it is useful to distinguish between data items in a manifold that represent real data and data outside the manifold. That is, the discriminator should be trained to reject all non-real data, including data far from the manifold, not just data generated by an adversarial generator, which once well trained will only generate data close to the manifold.

[0170] In examples 2, 3, and 4 in the list in box 720, the discriminator 703 is trained to distinguish real data from a specified source from data from various other sources (including non-real data).

[0171] In Example 2, the discriminator 703 distinguishes real data from any non-real data. Figure 1 In the data space of block 101 or Figure 1 Non-real data is obtained by randomly sampling from the feature space of box 102. In both cases, the real data is usually on or near the lower dimensional manifold, so most of the data randomly selected from the higher dimensional space will be non-real data. The back propagation and training process of this example is the same as that of the human preference discriminator discussed above.

[0172] In Example 3, system 703 is a classifier that is trained to classify data into a specified set of categories, where the number of categories is greater than two. However, in the illustrative embodiment of Example 3, the categories of classifier 703 are divided into a target set of categories to be distinguished from other categories. This process may be useful, for example, if there is a feature that exists in one of the category sets but not in another category set. The computer system 300 will then train the discriminator 14 to recognize the presence or absence of the feature.

[0173] In Example 4, system 703 is a classifier trained to classify data into two or more categories. In a preferred embodiment, in Example 4, classifier 703 also has an output indicating that the data item does not match the model of any category. Computer system 300 can use such a classifier when training a data system in which only part of the data is labeled as a category. Unlabeled data may contain data items that do not belong to any category. In this case, computer system 300 can train classifier 703 with one or more additional output nodes, which represent data that learns to represent clusters in data that do not belong to any conventional category. Computer system 300 can back-propagate the features that the data item is one of the known categories or not one of the known categories to discriminator 702. Computer system 300 will then train discriminator 702, both in the task of distinguishing data obtained from data enhancer and selector 222 from data obtained from adversarial generators, and in the task of distinguishing data in classifier 703 of known categories rather than classifier 703 of known categories. As mentioned before, training the discriminator 702 on multiple tasks leads to statistical smoothing and regularization, which also helps the discriminator 702 to generalize better to new data.

[0174] In Example 5, the computer system 300 trains the discriminator 703 to distinguish data generated by the first generator from data generated by the second generator. The effects of back propagation and training the discriminator with dual tasks are similar to the previous examples. It should be noted that in the task of distinguishing between the two generators, the computer system 300 has no restrictions on the amount of data that the computer system 300 can obtain from each generator, so if the computer system 300 uses a neural network or other general approximator for designing the discriminator 703, the computer system 300 should be able to train the discriminator 703 to any accuracy. On the other hand, as in other examples, the dual-task generation in training the discriminator 702 should improve the statistical smoothing and regularization of generalization.

[0175] In Example 6, computer system 300 trains discriminator 703 to detect adversarial attacks. An adversarial attack is a modified data item that is modified in a way that does not change the correct classification label, but causes the attacked classifier to incorrectly identify the category of the data item. In many adversarial attacks, the modified data item is almost indistinguishable from the original data item to a human observer. Defending against adversarial attacks is an important problem in computer security. Defending against adversarial attacks involves detecting that an adversarial attack has occurred and then correcting the error. In this example, backpropagation and training of the discriminator 702 on the dual task has the same beneficial effects as described above.

[0176] In Example 7, the computer system 300 trains the discriminator 703 to detect plagiarism. Figure 1 and Figure 2 The embodiment shown in is directly useful in the training process of the embodiment shown in . In Example 7, the computer system 300 creates training data for the discriminator 703 by selecting data examples from a set of training data for training the discriminator 703 to distinguish those examples from new data items, where the computer system 300 creates each new item by modifying the selected data examples by an amount small enough that the new item plagiarizes the original item. The computer system 300 can then use the discriminator 703 to detect data generated by the adversarial generator that should be subject to anti-plagiarism penalties. Similarly, backpropagation to the discriminator 702 has the beneficial effect of training the discriminator 702 on a dual task.

[0177] In Example 8, the computer system 300 trains the discriminator 703 to distinguish between undisturbed data and data that has been disturbed by adding noise or making other disturbances to the original data items. In Example 8, the computer system 300 can then use the discriminator 703 as a component in a system to remove noise from a signal, image, or other data item. In Example 8, the computer system 300 will calculate the back propagation and dual training of the discriminator 702 as described above. In Example 9, as an additional purpose, the computer system 300 can train a discriminator or detector as a subsystem of a network that performs a general classification task, where training data exists in the data space of the discriminator or detector. In some embodiments, the computer system 300 can save one or more discriminators and / or one or more detectors for later use in additional tasks, such as Examples 1 to 9. In some embodiments, the computer system 300 can perform additional training for such additional tasks.

[0178] In some embodiments, computer system 300 may use simple discriminator models that require little computing resources to train during some stages of the learning process. Computer system 300 may subsequently train more complex discriminator models.

[0179] For example, Figure 8 An example embodiment of aspects of the present invention is shown in which a computer system 300 can use a simple discriminator ( Figure 8 809) to quickly divide the feature space into local areas. Figure 1 102) or data space ( Figure 1 101) changes, the computer system 300 can use this process to quickly define the local area (see Figure 1 103).

[0180] Figure 8An embodiment of aspects of the invention is shown, where the computer system 300 can be used in a development process including multiple stages, where a data space can be replaced by a new data space, for example in a series of refinements of image resolution. In this embodiment, at certain stages, the computer system 300 can train a simple discriminator (809) and optionally skip training of more refined generators and discriminators.

[0181] Figure 8 is a flow chart of an illustrative embodiment of a development process for creating data spaces and successive refinements of local regions within each data space. Figure 8 If the process starts from scratch, the computer system 300 can use a uniform discriminator as the discriminator for each region and use a uniform probability distribution to randomly generate data for each region. In box 801, the computer system 300 trains a classifier for the current data space. In box 802, the computer system 300 trains clusters within each category. In box 803, the computer system 300 can train additional clusters on unlabeled data, or use supervised or unsupervised training to train additional clusters on labeled data that has been set aside in training in box 802. In box 804, the computer system 300 trains or supplements the basic training of the feature encoder by backpropagation during training in boxes 801, 802, and 803. In box 805, the computer system 300 can select a region, such as in a previous loop from box 103 to box 173. Figure 1 Alternatively, the computer system 300 may initialize the set of regions with a single region covering the current data space or feature space. In block 806, the computer system 300 obtains an adversarial generator for the selected local region and optionally a collaborative generator for the selected region. In block 807, the computer system 300 obtains the adversarial generator for the selected local region and optionally the collaborative generator for the selected region. Figure 2 The dedicated data generator and selector system 222 of the computer system 300 obtains a specified amount of collaborative data in the selected area. In block 808, the computer system 300 obtains a specified amount of adversarial data from the adversarial generator system.

[0182] In box 809, the computer system 300 trains a simple discriminator for the region. Examples of simple discriminators may include (without loss of generality): (1) a uniform distribution or zero discriminator, (2) linear regression, (3) a discriminator with a linear hyperplane decision boundary, (4) a single-node artificial neural network, (5) a support vector machine with a limit on the number of variables, (6) a neural network with a limit on the number of training cycles (not necessarily trained to convergence), or any other machine learning system that is easy to represent and / or train. The task of the simple discriminator is only to perform a preliminary separation of the data obtained from the dedicated data generator and selector 222 from the data generated by the adversarial generator. In box 816, through optional further discriminator training of the computer system 300, the computer system 300 will use this simple discriminator to divide the current local region into multiple smaller regions.

[0183] In block 810 , the computer system 300 determines whether to perform further training to refine the generator and discriminator trained in block 809 based on criteria specified by the training process control system 221 .

[0184] For example, in some embodiments, in an early stage of data space refinement, the computer system 300 may determine not to refine the generator and discriminator for any region because new discriminators and generators will be trained in subsequent refinements of the data space.

[0185] As another example, the computer system 300 may determine not to refine the discriminators and generators for regions that are smaller than the criteria set by the training process control system 221. For example, the computer system 300 may determine not to refine any region contained in the anti-plagiarism region. More generally, in some embodiments, if the entire region is contained within a hypersphere with a radius less than a specified anti-plagiarism distance, the computer system 300 may select a uniform distribution in block 809 and determine not to refine the discriminators and generators for the region.

[0186] In block 811, the computer system 300 may divide the selected region into a plurality of regions based on the classification categories of one or more local discriminators 971. In some embodiments, the computer system 300 may construct a decision tree in which branch points correspond to the discriminators used to successively divide each region. The computer system 300 may use the leaves of the decision tree as local region detectors 973. In some embodiments, the computer system 300 may train a separate detector to detect data in the leaves of the decision tree.

[0187] In block 812, the computer system 300 determines whether to continue partitioning the regions. If so, the computer system 300 returns to block 806. Otherwise, the computer system 300 completes this round. However, the computer system 300 may repeat the process in subsequent rounds of data space or feature space refinement. Figure 8In some embodiments, the computer system 300 may perform a process of Figure 8 process.

[0188] In block 813, the computer system 300 trains the discriminator and the generator, for example, using Figure 1 In block 814, the computer system 300 obtains adversarial data. In block 815, the computer system 300 obtains collaborative data. In some embodiments, in block 816, the computer system 300 may train a region divider discriminator based on the adversarial and collaborative data obtained by the computer system 300 in blocks 814 and 815. In some embodiments, the computer system 300 may use the discriminator trained by the computer system 300 in block 813 as a region divider.

[0189] The generator and discriminator described herein (e.g., local generator 977 and / or local discriminator 971) can be used to generate data and discriminate data (as appropriate) after being trained as described herein. The data can be images, text, medical-related diagnostic data, etc.

[0190] After the training described herein, the generator (or one or more local generators) or discriminator (or one or more local discriminators and / or detectors) or collaborative generator can be deployed in an operating environment, although the generator, discriminator and / or collaborative generator can continue to be trained after deployment. For example, a generator (or one or more local generators or collaborative generators) can be deployed to generate data to train another machine learning system, such as a machine learning classifier. The generated data can be images (e.g., synthetic images) with examples of medical conditions (positive and negative), which are used to train a medical imaging system to detect medical conditions in images through machine learning. For example, once the generator is trained, it can be deployed to generate MRI scan images, tomographic images, such as CT (computed tomography), OCT (optical coherence tomography) or PET (positron emission tomography), X-ray images and / or ultrasound scans to train corresponding classifiers of medical conditions that can be detected in scans / images through machine learning. The generator can also be used to generate images or videos of objects, which can be used to train a computer vision system to detect objects in images or videos. For example, the computer vision system can be part of a robot or an autonomous vehicle. The generator can also be deployed to generate synthetic cyber threats that can be used to train cybersecurity systems to

[0191] The discriminator, any local discriminator, and / or any local detector may be deployed after training as described herein. In various embodiments, after the collaborative training as described herein, the discriminator and / or detector may also be trained before deployment using additional training data in the data space of the discriminator or detector. The discriminator and / or detector may also be trained before deployment to perform the above combined Figure 7 The additional tasks described.

[0192] Therefore, in one general aspect, the present invention relates to a computer system and method for training a generator and a discriminator in an adversarial manner. In one embodiment, the method includes: training the generator and the discriminator together in a multi-player simulated game by machine learning through a computer system, wherein the simulated game includes multiple rounds, wherein in each round, the discriminator determines whether the selected data item presented to the discriminator is from the generator or from a data source different from the generator. The training includes: (1) training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; (2) training the generator to generate data that the discriminator incorrectly determines is not from the generator; (3) updating the iterative update of the model parameters for the generator and the discriminator, wherein the model parameters for the generator and for the discriminator are updated non-simultaneously.

[0193] In another embodiment, the method includes: training the generator and the discriminator together in a multi-player simulated game by machine learning through a computer system, wherein the simulated game includes multiple rounds, wherein in each round, the discriminator determines whether the selected data item presented to the discriminator is from the generator or from a data source different from the generator. The training includes: (i) training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; (ii) training the generator to generate data that the discriminator incorrectly determines is not from the generator; and iteratively updating model parameters for the generator and the discriminator. Training may include two rounds. In the first round of the simulated game, the training includes: (a) updating the current mixed strategy for the discriminator by the computer system, thereby generating an updated mixed strategy for the discriminator; (b) obtaining a first data item from the data source based on the updated mixed strategy for the discriminator; (c) generating a second data item by the generator using the current mixed strategy for the generator; (d) inputting a first selected data item to the discriminator by the computer system, wherein the first selected data item is the first data item or the second data item, wherein the computer system makes a first selection of the first data item or the second data item, and wherein the discriminator is unaware of the first selection of the computer system; (e) determining by the discriminator whether the first selected data item is generated by the generator; (f) determining by the computer system whether the discriminator correctly determines whether the first selected data item is generated by the discriminator; and (g) allocating a first payoff to the generator and the discriminator by the computer system based on whether the discriminator correctly determines whether the first selected data item is generated by the discriminator. In the second round of the simulated game, the training may include: (h) updating the current mixed strategy for the discriminator by the computer system, thereby generating an updated mixed strategy for the generator; (i) obtaining a third data item from the data source based on the updated mixed strategy for the discriminator; (j) generating a fourth data item by the generator using the updated mixed strategy for the generator; (k) inputting a second selected data item to the discriminator by the computer system, wherein the second selected data item is the third data item or the fourth data item, wherein the computer system makes a second selection of the third data item or the fourth data item, and wherein the discriminator is unaware of the second selection of the computer system; (l) determining by the discriminator whether the second selected data item is generated by the generator; (m) determining by the computer system whether the discriminator correctly determines whether the second selected data item is generated by the discriminator; and (n) allocating a second payoff to the generator and the discriminator by the computer system based on whether the discriminator correctly determines whether the second selected data item is generated by the discriminator.

[0194] In various embodiments, the simulated game is a two-player zero-sum game, such as a two-player finite zero-sum game.

[0195] In various embodiments: in the first round of the simulated game, updating the current mixed strategy used for the discriminator includes updating the current mixed strategy used for the discriminator based on the payoffs of each round of the simulated game before the first round; and in the second round of the simulated game, updating the current mixed strategy used for the generator includes updating the current mixed strategy used for the generator based on the payoffs of each round of the simulated game before the second round.

[0196] In various embodiments, updating the current mixed strategy for the discriminator includes finding a pure strategy for the discriminator that performs better than the current mixed strategy for the discriminator relative to the current mixed strategy for the generator; and updating the current mixed strategy for the generator includes finding a pure strategy for the generator that performs better than the current mixed strategy for the generator relative to the current mixed strategy for the discriminator.

[0197] In various embodiments, the training further includes, after the second round: iteratively updating, by the computer system, the model parameters for the discriminator; and iteratively updating, by the computer system, the model parameters for the generator, such that the updating of the model parameters for the generator and the updating of the model parameters for the discriminator are not updated simultaneously.

[0198] In various embodiments, the discriminator includes a plurality of local region detectors. Furthermore, each of the plurality of local region detectors is trained by machine learning to discriminate whether a data item presented to the local region detector is accepted or rejected as a member of a specified set associated with the local region detector. Furthermore, in a first round of the simulated game, updating the current mixed strategy for the discriminator includes selecting one of the plurality of local region detectors such that the first data item from the data source is based on the selected one of the plurality of local region detectors.

[0199] In various embodiments, the designated set for the first local area detector partially overlaps with the designated set for the second local area detector.

[0200] In various embodiments, the discriminator includes a plurality of local discriminators.

[0201] In various embodiments, the generator includes a plurality of local generators. In addition, each of the plurality of local generators is trained by machine learning to generate data items in a local data region associated with the local generator. In addition, in a second round of the simulated game, updating the current hybrid strategy for the discriminator includes selecting one of the plurality of local region generators so that the fourth data item is generated by the selected local generator.

[0202] In various embodiments, the discriminator includes a plurality of local discriminators, wherein each of the plurality of local discriminators is trained to determine whether a data item presented to the local discriminator is from the generator.

[0203] In various implementations, each of the plurality of local generators comprises a neural network; each of the plurality of local region detectors comprises a neural network; and each of the plurality of local discriminators comprises a neural network.

[0204] In various embodiments, before the first round of the simulated game, the method further includes: using the computer system to train the multiple local generators through machine learning; using the computer system to train the multiple local detectors through machine learning; and using the computer system to train the multiple local discriminators through machine learning.

[0205] In various implementations, the data source includes a collaborative generator trained to collaborate with the discriminator.

[0206] In various embodiments, the training of the collaborative generator is controlled by a collaborative human plus AI training process control system.

[0207] In various embodiments, the discriminator includes a neural network having a first node; the first node is connected to a second node in an additional machine learning system; the additional machine learning system is trained to perform a task separate from the tasks of the discriminator and the generator; and training the discriminator includes training the discriminator with an additional objective from the additional machine learning system, such that training the discriminator with the additional objective includes back-propagating a derivative from the second node of the additional machine learning system to the first node of the discriminator.

[0208] In various embodiments, the collaborative generator includes a neural network having a third node; the third node is connected to a fourth node in the additional machine learning system; and the method also includes training the collaborative generator using an additional objective from the additional machine learning system, such that training the collaborative generator using the additional objective includes backpropagating derivatives from the fourth node of the additional machine learning system to the second node of the collaborative generator.

[0209] In various embodiments, the generator includes a generator selected from the group consisting of: a digital image generator, a digital audio generator, and a text generator.

[0210] In various implementations, the method further includes deploying the generator to generate data after training.

[0211] In various embodiments, the method further includes additionally training the discriminator to perform a second task. The second task may include a determination of whether a data item input to the discriminator will attract people with known preferences; a determination of whether a data item input to the discriminator is real or arbitrary; a determination of whether the data item input should be classified into a specified set of two or more classification categories; a determination of whether the data item input should be classified into one of a plurality of classification categories; a determination of whether the data item input to the discriminator is from the first generator or from the second generator; a determination of whether the data item input to the discriminator is attacked by an adversarial attack; a determination of whether the data item input to the discriminator is a plagiarized work; a determination of whether the data item input to the discriminator is perturbed; and / or a classification task.

[0212] The examples presented herein are intended to illustrate potential and specific embodiments of the present invention. It is understood that these examples are primarily intended to illustrate the present invention for those skilled in the art. One or more specific aspects of the examples are not necessarily intended to limit the scope of the present invention. In addition, it should be understood that the drawings and descriptions of the present invention have been simplified to illustrate elements relevant to a clear understanding of the present invention, while other elements have been eliminated for clarity. Although various embodiments are described herein, it is apparent that those skilled in the art may make various modifications, changes, and adjustments to these embodiments, thereby obtaining at least some advantages. Therefore, the disclosed embodiments are intended to include all such modifications, changes, and adjustments, without departing from the scope of this embodiment herein.

Claims

1. A method of training a generator and a discriminator in an adversarial manner, the method comprising training the generator and the discriminator together in a multi-player simulated game by machine learning via a computer system, wherein the simulated game comprises a plurality of rounds, wherein in each round, the discriminator determines whether a selected data item presented to the discriminator is from the generator or from a data source different from the generator, and wherein the training comprises: training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; training the generator to generate data that the discriminator incorrectly determines is not from the generator; as well as Iterative updates of model parameters for the generator and the discriminator are performed, wherein the model parameters for the generator and the discriminator are updated non-simultaneously.

2. A method of training a generator and a discriminator in an adversarial manner, the method comprising training the generator and the discriminator together in a multi-player simulated game by machine learning via a computer system, wherein the simulated game comprises a plurality of rounds, wherein in each round, the discriminator determines whether a selected data item presented to the discriminator is from the generator or from a data source different from the generator, and wherein the training comprises: training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; training the generator to generate data that the discriminator incorrectly determines is not from the generator; as well as Iteratively updating model parameters for the generator and the discriminator, The training includes: In the first round of the simulated game: updating, by the computer system, a current hybrid strategy for the discriminator, thereby generating an updated hybrid strategy for the discriminator; obtaining a first data item from the data source based on the updated hybrid strategy for the discriminator; generating, by the generator, a second data item using a current mixing strategy for the generator; inputting a first selected data item to the discriminator via a computer system, wherein the first selected data item is the first data item or the second data item, wherein the computer system makes a first selection of the first data item or the second data item, and wherein the discriminator is unaware of the first selection of the computer system; determining, by the discriminator, whether the first selected data item is generated by the generator; determining, by the computer system, whether the discriminator correctly determined whether the first selected data item was generated by the discriminator; and allocating, by the computer system, a first benefit to the generator and the discriminator based on whether the discriminator correctly determines whether the first selected data item is generated by the discriminator; and In the second round of the simulated game: updating, by the computer system, the current hybrid strategy for the discriminator, thereby generating an updated hybrid strategy for the generator; obtaining a third data item from the data source based on the updated hybrid strategy for the discriminator; generating a fourth data item by the generator using the updated hybrid strategy for the generator; inputting a second selected data item to the discriminator via a computer system, wherein the second selected data item is the third data item or the fourth data item, wherein the computer system makes a second selection of the third data item or the fourth data item, and wherein the discriminator is unaware of the second selection of the computer system; determining, by the discriminator, whether the second selected data item is generated by the generator; determining, by the computer system, whether the discriminator correctly determined whether the second selected data item was generated by the discriminator; and A second benefit is allocated to the generator and the discriminator by the computer system based on whether the discriminator correctly determines whether the second selected data item is generated by the discriminator.

3. The method of claim 2, wherein: The simulated game is a two-player zero-sum game.

4. The method of claim 3, wherein: The two-person zero-sum game includes a two-person limited zero-sum game.

5. The method of claim 2, wherein: The model parameters for the generator and the discriminator are updated non-simultaneously.

6. The method according to any one of claims 2 to 5, wherein: In a first round of the simulated game, updating the current mixed strategy for the discriminator includes updating the current mixed strategy for the discriminator based on the payoffs of each round of the simulated game before the first round; as well as In the second round of the simulated game, updating the current mixed strategy for the generator includes updating the current mixed strategy for the generator based on the payoffs of each round of the simulated game before the second round.

7. The method of claim 6, wherein: Updating the current mixed strategy for the discriminator includes finding a pure strategy for the discriminator that performs better with respect to the current mixed strategy of the generator than the current mixed strategy of the discriminator; as well as Updating the current mixed strategy for the generator includes finding a pure strategy for the generator that performs better with respect to the current mixed strategy of the discriminator than the current mixed strategy for the generator.

8. The method according to any one of claims 2 to 5, wherein: The training further comprises, after the second round: iteratively updating, by the computer system, model parameters for the discriminator; and The model parameters for the generator are iteratively updated by the computer system such that the model parameters for the generator and the model parameters for the discriminator are updated non-simultaneously.

9. The method according to any one of claims 2 to 5, wherein: The discriminator includes a plurality of local area detectors; training each of the plurality of local region detectors by machine learning to discriminate whether a data item presented to the local region detector is accepted or rejected as a member of a specified set associated with the local region detector; and In a first round of the simulated game, updating the current hybrid strategy for the discriminator includes selecting one of the plurality of local region detectors such that the first data item from the data source is based on the selected one of the plurality of local region detectors.

10. The method of claim 9, wherein: The specified set for the first local area detector partially overlaps with the specified set for the second local area detector.

11. The method of claim 9, wherein: The discriminator includes a plurality of local discriminators.

12. The method of claim 11, wherein: The generator includes a plurality of local generators; training each of the plurality of local generators by machine learning to generate data items in a local data region associated with the local generator; as well as In a second round of the simulated game, updating the current mixed strategy for the discriminator includes selecting one of the plurality of local region generators so that a fourth data item is generated by the selected local generator.

13. The method of claim 12, wherein: Each of the plurality of local discriminators is trained to determine whether a data item presented to the local discriminator is from the generator.

14. The method of claim 13, wherein: Each of the plurality of local generators comprises a neural network; Each of the plurality of local region detectors comprises a neural network; and Each of the plurality of local discriminators includes a neural network.

15. The method according to claim 14, further comprising before the first round of the simulated game: Using the computer system to train the plurality of local generators through machine learning; Using the computer system to train the plurality of local area detectors by machine learning; as well as The plurality of local discriminators are trained by machine learning using the computer system.

16. The method according to any one of claims 2 to 5, wherein: The data source includes a collaborative generator trained to collaborate with the discriminator.

17. The method of claim 16, wherein: The training of the collaborative generator is controlled by a collaborative human plus AI training process control system.

18. The method of claim 16, wherein: The discriminator includes a neural network having a first node; The first node is connected to a second node in an additional machine learning system; The additional machine learning system is trained to perform a task separate from the tasks of the discriminator and the generator; as well as Training the discriminator includes training the discriminator with an additional objective from the additional machine learning system, wherein training the discriminator with the additional objective includes back-propagating a derivative from a second node of the additional machine learning system to a first node of the discriminator.

19. The method of claim 18, wherein: The collaboration generator includes a neural network having a third node; The third node is connected to a fourth node in the additional machine learning system; and The method also includes training the collaborative generator using additional objectives from the additional machine learning system, wherein training the collaborative generator using the additional objectives includes back-propagating derivatives from a fourth node of the additional machine learning system to a second node of the collaborative generator.

20. The method according to any one of claims 1 to 5, wherein: The generator comprises a generator selected from the group consisting of: a digital image generator, a digital audio generator, and a text generator.

21. The method of any one of claims 1 to 5, further comprising deploying the generator to generate data after the training.

22. The method of any one of claims 1 to 5, further comprising additionally training the discriminator to perform a second task.

23. The method of claim 22, wherein: The second task involves determining whether a data item input to the discriminator will appeal to a person with known preferences.

24. The method of claim 22, wherein: The second task includes determining whether a data item input to the discriminator is real or arbitrary.

25. The method of claim 22, wherein: The second task includes a determination of whether a data item input should be classified into a specified set of two or more classification categories.

26. The method of claim 22, wherein: The second task includes a determination of whether a data item input should be classified into one of a plurality of classification categories.

27. The method of claim 22, wherein: The second task includes determining whether a data item input to the discriminator comes from the first generator or from the second generator.

28. The method of claim 22, wherein: The second task includes determining whether a data item input to the discriminator is an adversarial attack.

29. The method of claim 22, wherein: The second task includes determining whether a data item input to the discriminator is plagiarized work.

30. The method of claim 22, wherein: The second task includes determining whether a data item input to the discriminator is corrupted.

31. The method of claim 22, wherein: The second task includes a classification task.

32. A computer system for training a generator and a discriminator in an adversarial manner, the computer system comprising: one or more processor cores; as well as a memory in communication with the one or more processor cores, wherein the memory stores computer instructions that, when executed by the one or more processor cores, cause the one or more processor cores to train the generator and the discriminator together in a multi-player simulated game through machine learning, wherein the simulated game includes a plurality of rounds, wherein in each round the discriminator determines whether a selected data item presented to the discriminator is from the generator or from a data source different from the generator, and wherein the memory stores instructions that cause the one or more processor cores to train the generator and the discriminator together by: training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; training the generator to generate data that the discriminator incorrectly determines is not from the generator; as well as Iterative updates of model parameters for the generator and the discriminator are performed, wherein the model parameters for the generator and the discriminator are updated non-simultaneously.

33. A computer system for training a generator and a discriminator in an adversarial manner, the computer system comprising: one or more processor cores; as well as a memory in communication with the one or more processor cores, wherein the memory stores computer instructions that, when executed by the one or more processor cores, cause the one or more processor cores to train the generator and the discriminator together in a multi-player simulated game through machine learning, wherein the simulated game includes a plurality of rounds, wherein in each round the discriminator determines whether a selected data item presented to the discriminator is from the generator or from a data source different from the generator, and wherein the memory stores instructions that cause the one or more processor cores to train the generator and the discriminator together by: training the discriminator to perform a first task, wherein the first task is whether the selected data item presented to the discriminator is from the generator or from the data source; training the generator to generate data that the discriminator incorrectly determines is not from the generator; as well as Iteratively updating model parameters for the generator and the discriminator, The training includes: In the first round of the simulated game: updating a current hybrid strategy for the discriminator, thereby producing an updated hybrid strategy for the discriminator; obtaining a first data item from the data source based on the updated hybrid strategy for the discriminator; generating a second data item by the generator using a current mixing strategy for the generator; inputting a first selected data item to the discriminator, wherein the first selected data item is the first data item or the second data item, wherein the computer system makes a first selection between the first data item or the second data item, and wherein the first selected data item is unknown to the discriminator; determining, by the discriminator, whether the first selected data item is generated by the generator; determining whether the discriminator correctly determined whether the first selected data item was generated by the discriminator; and assigning a first benefit to the generator and the discriminator based on whether the discriminator correctly determines whether the first selected data item is generated by the discriminator; and In the second round of the simulated game: updating the current mixing strategy for the discriminator to produce an updated mixing strategy for the generator; obtaining a third data item from the data source based on the updated hybrid strategy for the discriminator; generating a fourth data item by the generator using the updated hybrid strategy for the generator; inputting a second selected data item to the discriminator, wherein the second selected data item is the third data item or the fourth data item, wherein the computer system makes a second selection of the third data item or the fourth data item, and wherein the discriminator is unaware of the second selected data item; determining, by the discriminator, whether the second selected data item is generated by the generator; determining whether the discriminator correctly determined whether the second selected data item was generated by the discriminator; and A second benefit is allocated to the generator and the discriminator based on whether the discriminator correctly determines whether the second selected data item is generated by the discriminator.

34. The computer system of claim 33, wherein: The simulated game is a two-player zero-sum game.

35. The computer system of claim 34, wherein: The two-person zero-sum game includes a two-person limited zero-sum game.

36. The computer system of claim 33, wherein: The model parameters for the generator and the discriminator are updated non-simultaneously.

37. A computer system as claimed in any one of claims 33 to 36, wherein: The memory stores instructions that cause the one or more processor cores to perform the following operations: In a first round of the simulated game, updating the current mixed strategy for the discriminator by updating the current mixed strategy for the discriminator based on the payoffs of the rounds of the simulated game before the first round; as well as In a second round of the simulated game, the current mixed strategy for the generator is updated by updating the current mixed strategy for the generator based on the payoffs of the rounds of the simulated game before the second round.

38. The computer system of claim 37, wherein: The memory stores instructions that cause the one or more processor cores to perform the following operations: updating the current mixed strategy for the discriminator by finding a pure strategy for the discriminator that performs better with respect to the current mixed strategy of the generator than the current mixed strategy of the discriminator; as well as The current mixed strategy for the generator is updated by finding a pure strategy for the generator that performs better with respect to the current mixed strategy of the discriminator than the current mixed strategy of the generator.

39. A computer system as claimed in any one of claims 33 to 36, wherein: The memory stores instructions that cause the one or more processor cores to perform the following operations after the second round: iteratively updating model parameters for the discriminator; and The model parameters for the generator are iteratively updated so that the model parameters for the generator and the model parameters for the discriminator are updated non-simultaneously.

40. A computer system as claimed in any one of claims 33 to 36, wherein: The discriminator includes a plurality of local area detectors; training each of the plurality of local region detectors by machine learning to discriminate whether a data item presented to the local region detector is accepted or rejected as a member of a specified set associated with the local region detector; and The memory stores instructions that cause the one or more processor cores to update the current hybrid strategy for the discriminator by selecting one of the multiple local area detectors in a first round of the simulated game, so that the first data item from the data source is based on the selected one of the multiple local area detectors.

41. The computer system of claim 40, wherein: The specified set for the first local area detector partially overlaps with the specified set for the second local area detector.

42. The computer system of claim 40, wherein: The discriminator includes a plurality of local discriminators.

43. The computer system of claim 42, wherein: The generator includes a plurality of local generators; training each of the plurality of local generators by machine learning to generate data items in a local data region associated with the local generator; as well as The memory stores instructions that cause the one or more processor cores to update the current mixed strategy for the discriminator by selecting one of the multiple local generators in a second round of the simulated game, so that the fourth data item is generated by the selected local generator.

44. The computer system of claim 43, wherein: The discriminator includes a plurality of local discriminators, wherein each of the plurality of local discriminators is trained to determine whether a data item presented to the local discriminator is from the generator.

45. The computer system of claim 44, wherein: Each of the plurality of local generators comprises a neural network; Each of the plurality of local region detectors comprises a neural network; and Each of the plurality of local discriminators includes a neural network.

46. ​​The computer system of claim 45, further comprising the memory storing instructions that cause the one or more processor cores to perform the following operations prior to the first round of the simulated game: Training the multiple local generators by machine learning; Training the plurality of local detectors by machine learning; and The multiple local discriminators are trained by machine learning.

47. A computer system as claimed in any one of claims 33 to 36, wherein: The data source includes a collaborative generator trained to collaborate with the discriminator.

48. The computer system of claim 47, further comprising a collaborative human plus AI training process control system configured to control the training of the collaborative generator.

49. The computer system of claim 47, wherein: The discriminator includes a neural network having a first node; The first node is connected to a second node in an additional machine learning system; The additional machine learning system is trained to perform a task separate from the tasks of the discriminator and the generator; as well as The memory stores instructions that cause the one or more processor cores to train the discriminator by training the discriminator with an additional objective from the additional machine learning system, such that training the discriminator with the additional objective includes back-propagating a derivative from a second node of the additional machine learning system to a first node of the discriminator.

50. The computer system of claim 49, wherein: The collaboration generator includes a neural network having a third node; The third node is connected to a fourth node in the additional machine learning system; and The memory stores instructions that cause the one or more processor cores to train the collaborative generator using additional objectives from the additional machine learning system, such that training the collaborative generator using the additional objectives includes back-propagating derivatives from a fourth node of the additional machine learning system to a second node of the collaborative generator.

51. A computer system as claimed in any one of claims 32 to 36, wherein: The generator comprises a generator selected from the group consisting of: a digital image generator, a digital audio generator, and a text generator.

52. A computer system as claimed in any one of claims 32 to 36, wherein: The memory stores instructions that cause the one or more processor cores to deploy the generator to generate data after training.

53. A computer system as claimed in any one of claims 32 to 36, wherein: The memory stores instructions that cause the one or more processor cores to additionally train the discriminator to perform a second task.

54. The computer system of claim 53, wherein: The second task involves determining whether a data item input to the discriminator will appeal to a person with known preferences.

55. The computer system of claim 53, wherein: The second task includes determining whether a data item input to the discriminator is real or arbitrary.

56. The computer system of claim 53, wherein: The second task includes a determination of whether a data item input should be classified into a specified set of two or more classification categories.

57. The computer system of claim 53, wherein: The second task includes a determination of whether a data item input should be classified into one of a plurality of classification categories.

58. The computer system of claim 53, wherein: The second task includes determining whether a data item input to the discriminator comes from the first generator or from the second generator.

59. The computer system of claim 53, wherein: The second task includes determining whether a data item input to the discriminator is an adversarial attack.

60. The computer system of claim 53, wherein: The second task includes determining whether a data item input to the discriminator is plagiarized work.

61. The computer system of claim 53, wherein: The second task includes determining whether a data item input to the discriminator is corrupted.

62. The computer system of claim 53, wherein: The second task includes a classification task.

Images