Generator network for generating images with a predetermined count of objects
By training the generator and discriminator network adversarially, the problem of time-consuming manual annotation in image classifier training is solved, generating realistic images that match object counts, improving training efficiency and accuracy, and realizing automated spatial layout generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2021-09-29
- Publication Date
- 2026-06-05
AI Technical Summary
Training an image classifier requires a large number of manually annotated real images, especially rare traffic scenarios, making the training process time-consuming and expensive. Furthermore, existing generative adversarial networks struggle to generate realistic images that match a specific object count.
By configuring the generator network to be trained adversarially with the generator and discriminator networks, images are generated using noise samples and object count mappings. Accurate matching of object counts is achieved through weight sharing and optimization of the common loss function. The generator network is modified to contain dense blocks to accommodate multi-object generation.
Generator networks can generate image sets with rich semantic meaning, reduce the need for manual labeling, improve the training efficiency and accuracy of image classifiers, and automatically infer reasonable spatial layouts.
Smart Images
Figure CN114332552B_ABST
Abstract
Description
[0001] The present invention relates to adversarial training of generator networks for producing synthetic images, which can be used in particular to train image classifiers or to obtain spatial design for machines. Background Technology
[0002] Image classifiers require training images, for which the "true" classification scores the classifier should assign are known. Obtaining a large set of training images with sufficient variability is time-consuming and expensive. For example, if an image classifier is to classify traffic conditions captured using one or more sensors carried by a vehicle, a lengthy test drive is required to obtain a sufficient number of training images. The "true" classification scores needed for training often need to be obtained by manually annotating the training images, which is also time-consuming and expensive. Furthermore, some traffic conditions, such as blizzards, occur only occasionally during the capture of training images.
[0003] To alleviate the scarcity of training images, generative adversarial networks (GANs) can be trained to generate synthetic images that resemble realistic images, which can then be used as training images for image classifiers. Conditional GANs (cGANs) can be used to generate synthetic images that belong to a specific pattern in the distribution of realistic images. For example, a conditional GAN can generate synthetic images that belong to a specific class in a classification.
[0004] DE 10 2018 204 494 B3 discloses a method for generating synthetic radar signals as training material for a classifier. Summary of the Invention
[0005] The inventors have developed a method for training a generator network that is configured to generate images with multiple objects.
[0006] In this method, a set of training images is provided. Furthermore, for each training image, at least one actual count of the objects contained in the corresponding training image is provided. Specifically, this actual count of objects can be a multi-class count vector, which includes separate counts for multiple object classes.
[0007] Furthermore, a generator network is provided, configured to map a combination of noise samples and at least one target count of an object to a generated image. Such a generator network can be generated, for example, from a generator network configured to map noise samples to a generated image. For instance, the input layer of such an existing generator network can be modified to accommodate the input of the target count. Like the actual counts, the target counts can be multi-class count vectors.
[0008] A discriminator network is provided. This discriminator network is configured to map an image to a combination of: a determination that the image is either a training image or a generated image produced by a generator network, and at least one predicted count of an object in the image. Similarly, the predicted count of the object can be a multi-class count vector. The discriminator network is only needed during training. After training, the completed generator network can be used independently.
[0009] For training, noise samples and object counts are randomly drawn. Using a generator network, these noise samples and object counts are mapped to the generated image.
[0010] The generated images are pooled together with the training images. From this pooling, images are randomly sampled and fed into the discriminator network. For each sampled image, the discriminator network outputs a determination of whether the corresponding image is a training image or a generated image, as well as at least one predicted count of the objects in that image. This predicted count can also be a multi-class count vector.
[0011] The discriminator parameters characterizing the discriminator network behavior were optimized with the objective of improving the accuracy of the discriminator network in distinguishing between generated and training images. Simultaneously, the generator parameters characterizing the generator network behavior were optimized with the objective of degrading said accuracy.
[0012] This part of the training is adversarial: through continuous competition, the generator network is trained to generate increasingly better "forgeries" to "fool" the discriminator network, and the discriminator network is trained to become increasingly adept at detecting these "forgeries".
[0013] Building upon this adversarial training, both the generator and discriminator parameters are trained with the objective of improving the matching between the predicted counts of objects on one hand and the actual or target counts of objects on the other. Specifically, if the image input to the discriminator network is a training image, the actual counts of objects known from that training image are compared with the predicted counts of those objects. If the image input to the discriminator network is a generated image, the target counts of the objects used to generate that generated image are compared with the predicted counts of those objects.
[0014] This part of the training is not strictly adversarial. Instead, the generator and discriminator networks work together to improve matching: based on training images with known actual counts of their objects, the discriminator network learns to accurately predict that object count. This ability of the discriminator network is then used as a metric to determine whether the generator network has produced an image containing the object count requested by the target count. Just as in the adversarial part of the training, the discriminator network is still rating what the generator network is doing. However, the generator network's intention is not to "fool" the discriminator network, but rather to diligently do something better based on the discriminator network's rating.
[0015] The possibility of generating images with expected counts of objects allows for the production of images with richer and more realistic scenes. Many realistic scenes contain more than one object of at least one type. When training image classifiers to evaluate realistic scenes, for example, to enable driving vehicles in a way that is at least partially automated, it is important that the classifier outputs sufficient results if multiple objects are present in the scene. For example, most traffic scenes involve multiple moving objects (such as other vehicles or pedestrians). Furthermore, traffic signs frequently appear in clusters in a single location, and they must all be properly processed to determine the appropriate behavior of vehicles. For example, in a single location, there might be a posted speed limit of 30 km / h, along with an indication that this is valid between 9 PM and 5 AM. A higher speed limit of 50 km / h might be in effect between 5 AM and 9 PM. In addition, there might be a no-entry sign combined with an indication that this only applies to vehicles weighing 3.5 tons or more.
[0016] A generator network trained according to the method described herein can produce realistic images of any scene for which the correct semantic meaning, i.e., the "ground truth," is known, since the image is initially generated based on this. Therefore, a large set of training images for an image classifier can be generated without having to manually annotate or "label" them with the correct semantic meaning.
[0017] In a manner similar to that of a generator network, the discriminator network used during training can be generated from an existing GAN discriminator network by modifying its output layer, which previously only output a determination of whether the input image is a "real" training image or a generated "fake" image, so that it now also outputs at least one predicted count of the object.
[0018] However, in a particularly advantageous embodiment, the discriminator network includes:
[0019] ● A first network, comprising multiple convolutional layers and classification layers, is configured to map the results of these convolutional layers to a determination of whether an image input to a discriminator network is a training image or a generated image; and
[0020] ● A second network comprising multiple convolutional and regression layers is configured to map the results of these convolutional layers to at least one predicted count of an object in an image.
[0021] In this way, each of the first and second networks can specialize in its respective task with maximum flexibility. If the first and second networks are completely independent of each other, there is no need to weigh the trade-off between generating images from the best distinctions of the training images on one hand and accurately predicting object counts on the other.
[0022] In another particularly advantageous embodiment, during the training of the discriminator network,
[0023] ● The update parameters characterizing the behavior of at least one convolutional layer in the first network are also updated in the corresponding convolutional layer in the second network, and / or
[0024] ● The update parameters characterizing the behavior of at least one convolutional layer in the second network are also updated in the corresponding convolutional layer in the first network.
[0025] For example, such "weight sharing" can be achieved by using parameters in the same memory space to set the behavior of convolutional layers in the first network and corresponding convolutional layers in the second network. This "weight sharing" has a dual effect. First, it reduces memory consumption. Because neural networks like generator and discriminator networks are frequently implemented on GPUs, and GPU memory is much more expensive than general-purpose RAM, available GPU memory can be a limiting factor for the size of networks that can be implemented. Second, "weight sharing" has been found to have a surprising regularization effect on the discriminator network as a whole.
[0026] In a particularly advantageous embodiment, the optimization of the generator and discriminator parameters is performed with the objective of optimizing the common loss function value, which includes:
[0027] ● The first contribution of the discriminator network to recognizing that the lower the training image, the better;
[0028] ● The discriminator network's second contribution is that it recognizes that the lower the generated image quality, the better; and
[0029] ● A third contribution: If the image input to the discriminator network is a training image, the lower the predicted count of the object matches the actual count of the object, the better; or if the image input to the discriminator network is a generated image, the lower the predicted count of the object matches the target count of the object, the better.
[0030] As discussed earlier, the third contribution "knows" whether the image input to the discriminator network is a training image or a generated image. Therefore, the discriminator network's ability to accurately predict object counts can be trained on training images and used on generated images. The contributions can be weighted relative to each other to manipulate the importance of the counting target.
[0031] For example, a third contribution could be the norm of the difference between the actual or target count of one object and the predicted count of another object. This norm could be, for example, the L1 norm or the L2 norm.
[0032] Advantageously, the generator parameters can be optimized with the objective of maximizing the value of the common loss function, and the discriminator parameters can be optimized with the objective of minimizing the value of the common loss function. Maximizing the loss function value will then primarily affect the second contribution: if the generator network becomes better at producing "forgeries"—which tends to make the discriminator network perform worse—the second contribution increases.
[0033] In theory, a generator network could attempt to maximize the loss function by only outputting generated images that consistently contain incorrect object counts. However, if the generator network's training branches towards this "solution," it's highly likely that the generated images will simultaneously become less realistic. Consequently, the second contribution to the loss function will become smaller, as the discriminator network will more easily distinguish generated images from training images. Furthermore, if the discriminator network receives "bad examples" of all generated images, its training to accurately predict object counts will be compromised: it will then no longer accurately predict the actual number of objects in the training images.
[0034] Therefore, the net effect of the third contribution in the common loss function is that the discriminator network is trained on the training images to accurately predict the actual count of objects in the training images, while the generator network is trained to produce the desired number of object instances.
[0035] In another particularly advantageous embodiment, the provision of training images may specifically include:
[0036] ● Transform the first training image into a new training image by means of rotation, scaling and / or translation;
[0037] ● Associate the new training image with one or more actual counts of objects associated with the first training image; and
[0038] ● Expand the training image set with new training images.
[0039] In this way, if there is only a small amount of training material available for a specific number of object instances, the variability of that training material can be increased without the need for further manual labeling efforts.
[0040] As discussed earlier, a generator network can be modified to produce the target number of object instances simply by changing the input layer to accommodate that target number. However, depending on the complexity of the generator network required to produce realistic images of complex objects, the additional input for the target number may not pass through that generator network. For example, if the generator network has an encoder-decoder architecture with a "bottleneck" of very low dimensionality in the latent space, the multi-class count vectors may be insufficiently represented in the latent space.
[0041] Therefore, the present invention also provides a method for manufacturing a generator network configured to generate images having multiple objects.
[0042] The method begins by providing a generator network configured to generate images of at least one instance of an object having at least one type. Specifically, the generator network includes at least one residual layer, a convolutional layer, a combination of convolutional layers and sampling layers, and / or fully connected layers.
[0043] The generator network is then modified by replacing the remaining layers, combinations of convolutional layers and sampling layers, and / or fully connected layers with dense blocks comprising multiple convolutional layer sequences. In this paper, the result of at least one of these convolutional layers is fed into multiple subsequent convolutional layers in the sequence. These "skipped connections" in the dense block preserve counting information, such as multi-class counting vectors. Simultaneously, such a dense block provides at least the versatility and flexibility of the replaced layers, ensuring that the generator network's performance in generating realistic images in the desired domain should be at least as good as it was before the modification.
[0044] As discussed previously, images with multiple object instances generated by generator networks are very useful and versatile training material for image classifiers. Therefore, this invention also provides a method for training an image classifier.
[0045] This method begins by providing a generator network trained using the training method described above. In this paper, "providing" specifically means that the generator network does not need to be trained by the same entity performing the image classifier training. Instead, a second entity can obtain a fully trained generator from the first entity and then continue training the image classifier.
[0046] A generator network is used to generate classifier training images. These classifier training images contain a predetermined number of object instances. The objects have a predetermined type.
[0047] For each classifier training image, labels are obtained using a predetermined type of object and a predetermined number of instances, indicating which one or more classes the image classifier will map to for that classifier training image. The specific way the labels are determined based on object type and number of instances depends on the use case at hand. For example, in a simple use case, the presence of at least one instance of a particular type of object can trigger the inclusion of that type of object (such as "car", "traffic sign", or "pedestrian") in the label. However, the presence of multiple instances of a particular object may also generate its own label. For example, the presence of a large number of pedestrians might cause the classifier training image to be labeled as "gathering", "group", or "crowd". The presence of multiple trees might cause the classifier training image to be labeled as "path" or "forest".
[0048] Training images are provided to an image classifier and mapped to classes. Specifically, in addition to the training images generated by the generator network as described above, the training images can also include other images whose labels come from any other source (such as manually labeled images, like images actually captured using a physical camera). For example, the total set of training images can be obtained starting with multiple manually labeled images actually captured with a camera. This initial set of training images can then be increasingly expanded using training images generated by the generator network.
[0049] The differences between the classes output by the image classifier and the labels associated with the training images are rated using a predetermined classifier cost function. The classifier parameters, characterizing the behavior of the image classifier, are optimized with the objective of improving the rating through the classifier cost function.
[0050] As discussed earlier, the use of generator networks allows for a much higher degree of versatility and variability in the training materials used to train image classifiers. Therefore, image classifiers are more likely to output appropriate results for their applications in a wider range of situations.
[0051] It has also been found, surprisingly, that training the generator network as described above enables it to infer a reasonable spatial layout including the target count of target object instances, even if the training is not based on any spatial information. This could be developed to automatically obtain a reasonable spatial layout for machines.
[0052] Therefore, the present invention also relates to a method for obtaining the spatial layout of a machine, the method comprising a predetermined number of instances of one or more predetermined components.
[0053] This method begins by training a generator network according to the generator network training method described above. In this paper, the training images include images showing various counts of instances of a predetermined component. At least one actual count associated with each training image is the count of instances of the predetermined component in the corresponding training image.
[0054] Noise samples are extracted. These noise samples, along with a predetermined number of instances of one or more components, are fed into the generator network. The generator network then outputs a generated image with the desired spatial layout.
[0055] Previously, designing spatial layouts was a manual engineering task. This method not only automates that manual task, producing the same end result without requiring manual effort, but also generates entirely new spatial layouts that are most likely not produced by manual engineering.
[0056] In particular, even if the available training images only show a specific count of instances of the desired object, the generator network can interpolate and extrapolate this to other unseen counts. That is, to generate a spatial design with a specific desired count of object instances, training material showing that instance count is not required. Therefore, the time saved by training the generator network cannot be eliminated because manual engineering is no longer necessary.
[0057] Furthermore, once the generator network is trained, more spatial designs can be easily obtained by simply extracting new noise samples and feeding them into the generator.
[0058] In a particularly advantageous embodiment, the machine includes an electric motor, and at least one of the predetermined components is a magnet. For example, the power of the electric motor may depend on the number of electromagnetic coils used to drive it. Based on previous versions of electric motors with a lower number of magnets, a spatial layout with a higher number of magnets and higher power can be extrapolated.
[0059] The methods described above can be implemented, in whole or in part, by a computer. Therefore, they can be embodied in a computer program that can be loaded onto one or more computers. Accordingly, the present invention also provides a computer program having machine-readable instructions that, when executed by one or more computers, cause the one or more computers to perform one or more methods as described above. In this regard, embedded systems and control units, for example, used in vehicles or other machines, capable of executing program code, will also be considered computers.
[0060] The present invention also provides a non-transitory computer-readable storage medium and / or downloadable product having a computer program. The downloadable product is, for example, a digital product that can be offered for sale in an online store and downloaded immediately, and is deliverable via a computer network; that is, a digital product downloaded by a user of a computer network.
[0061] In addition, one or more computers may be equipped with computer programs, non-transitory computer-readable storage media, and / or downloadable products. Attached Figure Description
[0062] In the following text, figures are used to illustrate the invention, without any intention of limiting the scope of the invention. The figures show:
[0063] Figure 1 An exemplary embodiment of the method 100 for training a generator network;
[0064] Figure 2 shows an example image generated by the generator network based on training images with handwritten digits from the MNIST dataset;
[0065] Figure 3 An exemplary embodiment of the method 200 for manufacturing generator network 1;
[0066] Figure 4 An exemplary embodiment of the method 300 for training image classifier 3;
[0067] Figure 5 An exemplary embodiment of a method 400 for obtaining machine space layout. Detailed Implementation
[0068] Figure 1 This is a schematic flowchart of a method 100 for training generator network 1.
[0069] In step 110, a set of training images 11 is provided. For each training image 11, at least one actual count 15a-15d of the objects 14a-14d contained in the corresponding training image 11 is provided.
[0070] Specifically, according to box 111, the first training image 11 can be transformed into a new training image 11' by means of rotation, scaling, and / or translation. According to box 112, the new training image 11' can be associated with one or more actual counts 15a-15d of objects 14a-14d already associated with the first training image 11. According to box 113, the set of training images 11 can be expanded with the new training image 11'.
[0071] In step 115, a generator network 1 is provided. This generator network 1 is configured to map a combination of noise samples 17 and at least one target count 16a-16d of objects 14a-14d to the generated image 13. In step 130, noise samples 17 and target counts 16a-16d of objects 14a-14d are extracted. In step 140, the generator network 1 maps these noise samples 17 and target counts 16a-16d to the generated image 13.
[0072] The generated image 13 associated with the target counts 16a-16d of objects 14a-14d, together with the training image 11 associated with the actual counts 15a-15d of objects 14a-14d, is pooled in pool P.
[0073] In step 120, a discriminator network 2 is provided. The discriminator network 2 is configured to map image 21 to a combination of the following: determining 23 whether the image is training image 11 or generated image 13 produced by generator network 1, and at least one predicted count 18a-18d of objects 14a-14d in image 21.
[0074] According to box 121, the discriminator network 2 may include a first network 2a that outputs decision 23 and a second network 2b that outputs prediction counts 18a-18d for objects 14a-14d. According to box 121a, the update parameters (weights) of the first network 2a may also be updated at corresponding positions in the second network 2b. According to box 121b, the update weights of the second network 2b may also be updated at corresponding positions in the first network 2a.
[0075] In step 150, images 21 are randomly drawn from pool P. In step 160, these randomly drawn images 21 are fed to discriminator network 2 and mapped to decision 23 and prediction counts 18a-18d for objects 14a-14d.
[0076] In step 170, the discriminator parameters 22, which characterize the behavior of the discriminator network 2, are optimized with the objective of improving the accuracy of the discriminator network 2 in distinguishing between the generated image 13 and the training image 11. In step 180, the generator parameters 12, which characterize the behavior of the generator network 1, are optimized with the objective of degrading the accuracy.
[0077] Furthermore, in step 190, both generator parameter 12 and discriminator parameter 22 are trained with the following objective: to improve the matching between the predicted counts 18a-18d of one object 14a-14d and the actual counts 15a-15d or target counts 16a-16d of the other object 14a-14d.
[0078] The final fully trained state obtained by generator parameter 12 is represented by reference symbol 12. * The final fully trained state obtained from discriminator parameter 22 is labeled with reference symbol 22. * mark.
[0079] According to box 195, the common loss function can be used to optimize 170, 180, and 190.
[0080] Figure 2 shows an example of a generated image 13 produced by a generator network 1, which has been trained on training images 11 containing handwritten digits from the well-known MNIST dataset. In the example shown in Figure 2, instances of four objects 14a (digit 6), 14b (digit 9), 14c (digit 1), and 14d (digit 2) are illustrated. Image 13 is generated by the generator network from noisy samples 17 and multi-class count vectors, for each object 14a-14d, the multi-class count vectors giving the target counts 16a-16d of the instances that will appear in the generated image 13.
[0081] Figure 2a and Figure 2b Two generated images 13 are shown, each including an instance of object 14a (number 6) and an instance of object 14b (number 9). Because Figure 2b From and generated by it Figure 2a The noise sample 17 was generated differently from the noise sample 17, so the numbers look different and are located at the same level as... Figure 2a In different positions.
[0082] Figure 2c and Figure 2d Two generated images 13 are shown, each including one instance of object 14c (digit 1) and two instances of object 14d (digit 2). Again, the use of new noise samples 17 causes new changes to be generated digits, and also causes digits to move to new positions.
[0083] Figure 3 This is a schematic flowchart of an exemplary embodiment of a method 200 for manufacturing a generator network 1. In step 210, a generator network 1 is provided, which has been configured to generate realistic images, but has not yet been configured to adapt to a request for a specific count 16a-16d of instances of objects 14a-14d. In step 220, the generator network 1 is modified by replacing previously existing layers with dense blocks having multiple convolutional layers and "skip connections" between non-adjacent convolutional layers. In step 230, the modified generator network 1' is trained using the method 100 described above.
[0084] Figure 4 This is a schematic flowchart of an exemplary embodiment of a method 300 for training an image classifier 3.
[0085] In step 310, a generator network 1 trained using the method 100 described above is provided. In step 320, this generator network 1 is used to generate classifier training images 31, which contain a predetermined number 16a-16d instances of objects 14a-14d. Similar to... Figure 1Above the expected number of instances 16a-16d, noisy samples 17 are used to generate each image 13, and each image 13 is then used as a classifier training image 31.
[0086] Objects 14a-14d have predetermined types. In step 330, these predetermined types and corresponding numbers 16a-16d of the instances are used to generate labels 31a for each classifier training image 31. Label 31a indicates one or more classes 33 to which the image classifier 3 will map the classifier training image 31.
[0087] In step 340, the classifier training image 31 is provided to the image classifier 3 and mapped to class 33 by the image classifier 3. In step 350, these classes are compared with labels 31a, and the differences between class 33 and corresponding labels 31a are rated using the classifier cost function 34. In step 360, the classifier parameters 32, which characterize the behavior of the image classifier 3, are optimized with the objective of improving the rating 350a through the classifier cost function 34. The final fully trained state of the classifier parameters 32 is represented by reference symbol 32. * mark.
[0088] Figure 5 This is a schematic flowchart of an exemplary embodiment of a method 400 for obtaining the spatial layout of a machine. The machine will include a predetermined number 16a-16d of instances of one or more predetermined components.
[0089] In step 410, the generator network 1 is trained using the training method 100 described above. The training images 11 used include images showing various counts of instances of a predetermined component. At least one actual count 15a-15d associated with each training image 11 is a count of instances of the predetermined component in the corresponding training image 11.
[0090] In step 420, noise sample 17 is extracted. In step 430, noise sample 17 and a predetermined number 16a-16d of instances of predetermined components are supplied to generator network 1. Thus, a generated image 13 with the desired spatial layout is obtained.
Claims
1. A method (100) for training a generator network (1), the generator network (1) being configured to generate an image (13) having multiple objects (14a-14d), the method comprising: ● Provide a set of (110) training images (11), and for each training image (11), provide at least one actual count (15a-15d) of the objects (14a-14d) contained in the corresponding training image (11). ● Provide (115) a generator network (1), which is configured to map a combination of noise samples (17) and at least one target count (16a-16d) of objects (14a-14d) to a generated image (13). ● Provide a discriminator network (2) (120), which is configured to map an image (21) to a combination of the following: a determination (23) that the image is a training image (11) or a generated image (13) generated by the generator network (1), and at least one predicted count (18a-18d) of objects (14a-14d) in the image (21). ● Extract (130) noise samples (17) and target counts of objects (14a-14d) (16a-16d); ● The generator network (1) maps (140) the target counts (16a-16d) of the noise samples (17) and the objects (14a-14d) to the generated image (13). ● Randomly draw (150) images (21) from the pool (P) that includes the generated image (13) and the training image (11); ● Randomly selected images (21) are fed (160) to the discriminator network (2) and mapped to a combination of the following: a determination (23) that the corresponding image (21) is a training image (11) or a generated image (13), and at least one predicted count (18a-18d) of the object (14a-14d) in the image (21). ● Optimize the discriminator parameters (22) that characterize the behavior of the discriminator network (2) with the following objective (170): improve the accuracy of the discriminator network (2) in distinguishing between generated images (13) and training images (11); ● Generator parameters (12) characterize the behavior of the generator network (1) using the objective optimization (180) that degrades the accuracy; and ● Further optimize (190) both generator parameters (12) and discriminator parameters (22) with the following objectives: improve the matching between the predicted count (18a-18d) of one object (14a-14d) and the actual (15a-15d) or target (16a-16d) count of the other object (14a-14d).
2. The method (100) according to claim 1, wherein, Provides a discriminator network (2), which includes: ● A first network (2a) comprising multiple convolutional layers and a classification layer is configured to map the results of these convolutional layers to a determination (23) of whether an image input to the discriminator network (2) is a training image (11) or a generated image (13); and ● A second network (2b) comprising multiple convolutional and regression layers is configured to map the results of these convolutional layers to at least one predicted count (18a-18d) of objects (14a-14d) in the image (21).
3. The method (100) according to claim 2, wherein, During the training of the discriminator network (2), ● The update parameters characterizing the behavior of at least one convolutional layer in the first network (2a) are also updated in the corresponding convolutional layer in the second network (2b) (121a), and / or ● The update parameters characterizing the behavior of at least one convolutional layer in the second network (2b) are also updated in the corresponding convolutional layer in the first network (2a) (121b).
4. The method (100) according to any one of claims 1 to 3, wherein, The optimization of generator parameters (12) and discriminator parameters (22) is performed with the objective of optimizing the common loss function value (195), which includes: ● The discriminator network (2) makes the first contribution to recognizing the training image (11) as low as possible; ● The discriminator network (2) identifies the second contribution of the generated image (13) as low as possible; and ● If the image (21) input to the discriminator network (2) is a training image (13), then the lower the match between the predicted count (18a-18d) of the object (14a-14d) and the actual count (15a-15d) of the object (14a-14d) becomes, the better; or if the image (21) input to the discriminator network (2) is a generated image (13), then the lower the match between the predicted count (18a-18d) of the object (14a-14d) and the target count (16a-16d) of the object (14a-14d) becomes, the better.
5. The method (100) according to claim 4, wherein, The third contribution is the norm of the difference between the actual (15a-15d) or target (16a-16d) counts of one object (14a-14d) and the predicted counts (18a-18d) of the other object (14a-14d).
6. The method (100) according to any one of claims 4 to 5, wherein, The generator parameters are optimized by maximizing the value of the common loss function (12), and the discriminator parameters are optimized by minimizing the value of the common loss function (22).
7. The method (100) according to any one of claims 1 to 6, wherein, The actual counts (15a-15d) of objects (14a-14d), the target counts (16a-16d) of objects (14a-14d), and the predicted counts (18a-18d) of objects (14a-14d) are multi-class count vectors, which include separate counts for multiple object classes (14a-14d).
8. The method (100) according to any one of claims 1 to 7, wherein, The provision of training images (110) specifically includes: ● Transform the first training image (11) (111) into a new training image (11') by means of rotation, scaling and / or translation; ● Associate the new training image (11') with one or more actual counts (15a-15d) of the objects (14a-14d) associated with the first training image (11); and ● Expand the training image (11) set with new training images (11').
9. A method (200) for manufacturing a generator network (1), the generator network (1) being configured to generate an image (13) having a plurality of objects (14a-14d), the method comprising: ● Provide (210) a generator network (1), the generator network (1) being configured to generate an image (13) of at least one instance of an object (14a-14d) of at least one type, wherein the generator network (1) includes at least one combination and / or fully connected layer of a residual layer, a convolutional layer, a convolutional layer and a sampling layer; ● Modify (220) generator network (1) by replacing the remaining layers, convolutional layers, combinations of convolutional layers and sampling layers and / or fully connected layers with dense blocks comprising multiple convolutional layer sequences, wherein the result of at least one of these convolutional layers is fed into multiple subsequent convolutional layers in the sequence; and ● Train (230) the modified generator network (1') using the method (100) according to any one of claims 1 to 8.
10. A method (300) for training an image classifier (3), comprising: ● Provide (310) a generator network (1) trained using the method (100) according to any one of claims 1 to 8; ● Use a generator network (1) to generate (320) a predetermined number (16a-16d) of classifier training images (31) containing instances of objects (14a-14d) with a predetermined type; ● Using the predetermined type of objects (14a-14d) and the predetermined number of instances (16a-16d), obtain (330) labels (31a) for each classifier training image (31), the labels (31a) indicating the one or more classes (33) to which the image classifier (3) should map the classifier training image (31). ● Provide (340) classifier training images (31) to the image classifier (3) so that they are mapped to classes (33); ● Rating (350) the difference between the class (33) thus obtained and the label (31a) associated with the training image (31) using a predetermined classifier cost function (34); and ● Classifier parameters (32) characterize the behavior of the image classifier (3) by improving the rating (350a) through the classifier cost function (34).
11. A method (400) for obtaining the spatial layout of a machine, the method comprising a predetermined number (16a-16d) of instances of one or more predetermined components, the method comprising: ● The generator network (1) is trained (410) using the method (100) of any one of claims 1 to 8, wherein the training images (11) include images showing various counts of instances of a predetermined component, and wherein at least one actual count (15a-15d) associated with each training image (11) is a count of instances of the predetermined component in the corresponding training image (11); ● Extract (420) noise samples (17); and ● A predetermined number (16a-16d) of noise samples (17) and instances of predetermined components are supplied (430) to the generator network (1) to obtain a generated image (13) with the desired spatial layout.
12. The method (400) according to claim 11, wherein, The machine includes an electric motor, and at least one of the predetermined components is a magnet.
13. A computer program product comprising machine-readable instructions that, when executed by one or more computers, cause the one or more computers to perform one or more methods (100, 200, 300, 400) of any one of claims 1 to 12.
14. A temporary computer-readable storage medium comprising machine-readable instructions that, when executed by one or more computers, cause the one or more computers to perform one or more methods (100, 200, 300, 400) of any one of claims 1 to 12.
15. One or more computers having a computer program product of claim 13 and / or a non-transitory computer-readable storage medium of claim 14.