Transferring knowledge from teacher neural network to student neural network
By providing students' neural network with style enhancement training images and combining the predetermined loss function optimization parameters, the calculation cost and speed problems of training students' neural networks on resource-constrained hardware platforms are solved, and behavior consistency with teachers' neural networks is achieved, and resource utilization efficiency is improved.
Patent Information
- Application Number
- CN202510170068.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-02-15
- Filing Date
- 2025-02-17
- Publication Date
- 2025-08-15
AI Technical Summary
In the prior art, neural networks with larger architectures are expensive and slow to use when used on resource-constrained hardware platforms, making it difficult to effectively train student neural networks with smaller architectures to mimic their behavior.
By providing a style-enhanced version of the training image, the output and intermediate work product consistency evaluation of the student neural network and the teacher neural network under different styles are optimized using predetermined loss functions, the parameters of the student neural network are extended, and the training data set is reduced.
The student neural network is effectively trained to maintain consistency with the teacher neural network under different style conditions, improve performance on resource-constrained hardware platforms, and reduce training data requirements and computing costs.
Smart Images
Figure CN120494039A_ABST
Abstract
Description
Technical Field
[0001] The present invention involves training a student neural network with a smaller architecture based on the knowledge of a given trained teacher network with a larger architecture. Background Art
[0002] During neural network training, the learned knowledge is stored in the parameters that characterize the neural network's behavior. Therefore, the capacity for storing knowledge is proportional to the number of trainable parameters. So-called base models contain very many parameters and have been trained using large amounts of data. Examples of such base models include the CLIP model (Contrastive Language-Image Pretraining), which learns correspondences between natural language and images, and the SAM model (Arbitrary Segmentation Model), which can cut out any object from any image.
[0003] Due to the large number of parameters, using a large base model in real-world applications can be computationally expensive and slow. In particular, the model will have to fit into bulk storage when at rest and into RAM when in use. For applications running on hardware platforms that do not allow the use of the full base model due to lack of resources, a student model with a smaller architecture is trained to mimic the behavior of the large base model, which is then called the teacher model. If a sufficiently large set of training examples is used, and the student model gives essentially the same output as the teacher model for each training example, the trained student model can be used as a fast, resource-efficient, but still sufficiently accurate approximation to the teacher model. Summary of the Invention
[0004] The present invention provides a method for training a student neural network to adopt the behavior of a given teacher neural network. The given teacher neural network is trained to perform a given processing on an input image.
[0005] During the method, a set of training images x is provided. These training images x can be labeled as "ground truth" for the output that the student network should produce from these training images x, but this is not required.
[0006] Based on at least one training image x, one or more style-enhanced versions x' are generated. These style-enhanced versions x' have the same semantic content as the original training image x. That is, in the context of the application at hand, the style-enhanced versions x' have the same meaning as the original training image x. The style-enhanced versions x' are stylistically different from the original training image x. That is, the style-enhanced versions x' represent different ways of presenting the same semantic meaning of the original training image x.
[0007] A training image x and an augmented version x' are processed by a teacher neural network on the one hand and a student neural network on the other. This causes the teacher and student neural networks to produce their respective outputs. While producing these outputs, the teacher and student neural networks generate intermediate work products. Specifically, in a network organized into layers or other building blocks, the output of one such layer or building block can be considered an intermediate work product that is passed to the next layer or building block.
[0008] Using a predetermined loss function, the output and / or intermediate work products generated by the student neural network for each image x, x' are then evaluated to what extent they agree with the output and / or intermediate work products generated by the teacher neural network for the same image x, x'. Parameters characterizing the student neural network's behavior are then optimized toward improving the loss function. These parameters may also include parameters related to the architecture of the student network itself. This approach is not only useful for training student neural networks with fixed architectures, but can also be used as a tool for neural architecture search (NAS).
[0009] It has been found that augmenting a training image x by changing its style significantly expands the manifold of images on which the student neural network's behavior is evaluated to see if it corresponds, as expected, to that of the teacher neural network. If this manifold is large enough, then it can be expected that the student neural network's behavior will also be sufficiently consistent with that of the teacher neural network for input images not seen during training of the student neural network. Each new style-changing concept placed on a training image x effectively adds the entire number of training images x back to the final training dataset.
[0010] Furthermore, data augmentation in this way is independent of the specific tasks being solved by the teacher and student neural networks. This means that the method is not limited to neural networks used for, for example, image classification (i.e., assigning a classification score to an image as a whole for one or more classes). This also means that the student neural network can even solve tasks different from the one solved by the teacher neural network.
[0011] Here, determining whether the outputs and / or intermediate work products are "consistent" is not limited to a 1:1 comparison. Such a 1:1 comparison may not even be possible in all cases. Since the student neural network typically has a much smaller architecture than the teacher neural network, the outputs of the student neural network, and especially the intermediate work products, will have different dimensions (i.e., vectors or tensors of different sizes) than the teacher neural network. For example, aggregates and / or statistics can be calculated from the outputs and / or intermediate work products, and these aggregates and / or statistics can be compared.
[0012] By augmenting the training dataset with augmented versions x' of training images x above those training images x, the tendency of the student neural network to overfit to the training images x is reduced. Typically, the dataset of training images x is much smaller than the dataset of training images used to train the teacher neural network. For example, in an application related to the performance of the student neural network in the task of semantically segmenting urban scene images, the main training dataset for this application, Cityscapes, only includes 2795 training images, and they are all related to sunny conditions. Without data augmentation, it would be difficult, if not impossible, to generalize the training of the student model to conditions such as nighttime, rainy, or snowy conditions.
[0013] In a particularly advantageous embodiment, generating an enhanced version x' of a training image x comprises:
[0014] Extract content c from training images x;
[0015] deriving a style s based at least in part on a style source image y; and
[0016] The semantic content c and style s are processed into the sought enhanced version x' by the trained generative model G.
[0017] In this way, the content c from the training image x is mixed with the style s from the style source image y. This means that the new style s applied to the training image x does not need to be explicitly formulated. Instead, it can be implicitly provided in the form of the style source image y. This means that the style source image y can also include multiple simultaneous style changes, and there is no need to separate them.
[0018] In particular, extracting semantic content c from a training image x and / or obtaining style s based on a style source image y can be performed by feeding the training image x, respectively the style source image y, into a trained feature extractor network F, which is configured to determine both content c and style s from the input image. This means that for each of the training image x and the style source image y, a corresponding pair (c, s) of content c and style s is created. Only the content portion c from the pair (c, s) corresponding to the training image x and the style portion s from the pair (c, s) corresponding to the style source image y are used.
[0019] For example, the feature extractor network F can be trained in tandem with the generative model G, so that if the training image First, it is decomposed into a pair (c, s) of content c and style s, and the pair (c, s) is fed into the generative model G, which is close to the original training image Things are reconstructed.
[0020] In a further advantageous embodiment, the style s is selected as an interpolation between a first style s' of the training image x and a second style s" of the style source image y. In this way, based on a single style source image y, multiple style-enhanced versions x' of the same training image x can be generated.
[0021] For example, a training image x might have been taken during the day, so its style s is that of a daytime image. A style source image y might have been taken at night, so its style is that of a nighttime image. Through style interpolation, various degrees of twilight transitions between daytime and nighttime can then be created as new styles and applied to the content c of the training image x to create a new enhanced version x'.
[0022] Specifically, the style of the source image y being different from the style of the training image x can mean that the style of the source image y is different from one or more training images x in at least one aspect, which is different from the content c. As discussed above, it is not necessary to explicitly name this aspect. However, by selecting this aspect, we can actively guide the direction in which the image manifold expands, and in which direction the consistency between the student neural network and the teacher neural network will be checked.
[0023] For example, aspects in which the style source image y differs from all training images x may specifically include one or more of the following:
[0024] The time of day the image was acquired;
[0025] The season of the year when the image was acquired;
[0026] Weather and / or lighting conditions under which the image was acquired;
[0027] Defects and / or disturbances in the image; and
[0028] ●Camera settings for acquiring images.
[0029] These are typical variations that arise in real-world applications where the input image is acquired by at least one sensor and should not cause the student neural network being used in the application to suddenly deviate in its behavior from the teacher neural network.
[0030] In another particularly advantageous embodiment, at least one enhanced version x' of a training image x and / or at least one style source image y is obtained from a trained generative diffusion model D using a textual prompt indicating the desired style of the enhanced version x', respectively, the style source image y. In this way, if a clear concept of the desired stylistic change is available, this can be presented by the trained generative diffusion model D. For example, given a training image x, the generative diffusion model D can be instructed to create an enhanced version x' that still shares the concept of the original training image x, but modified according to the textual prompt. Depending on how strongly adherence to the original training image x is desired, the appearance of the content of the original training image x can be altered by the generative diffusion model D, within the constraints that the content still retains the same semantic meaning in the application at hand. For example, if the original training image x depicts a castle, and the generative diffusion model D is instructed to transform it into a winter image, the number and appearance of the castle's towers may also change, despite this stylistic change. However, the castle will still remain a castle.
[0031] In another particularly advantageous embodiment, feature maps output by an intermediate layer of the corresponding neural network are selected as intermediate work products. For example, these feature maps can be output by a convolutional layer. In particular, feature maps of different sizes can be processed (for example, by evaluating the profiles of the features contained therein) so that the degree to which they are consistent with each other can be checked.
[0032] In another particularly advantageous embodiment, the output of the neural network is selected as logits and / or other unaggregated results output by the neural network. In particular, logits are outputs that have not yet been normalized by a normalization function (such as a softmax function). Furthermore, they have not yet been condensed into a final decision, such as a one-hot vector of classification scores output by the final fully connected layer of the neural network used as a classifier. In this way, even subtle differences between the behavior of the teacher neural network and the student neural network can be captured.
[0033] In another particularly advantageous embodiment, the loss function measures the distance between the outputs and / or intermediate work products generated by the teacher neural network on the one hand and the student neural network on the other hand. For example, the distance can be measured in a common space to which the outputs, respectively the intermediate work products, of the teacher and student neural networks belong. If there is no such common space (for example because the outputs, respectively the intermediate work products, have different sizes), another option is to transform the two outputs, respectively the two work products, into a common work space and measure the distance in this work space.
[0034] For example, if there is a common space, a norm such as the L2 norm can be used as a distance metric. Furthermore, in particular, the architecture of the student neural network can be chosen to include at least one layer or other building block whose output has the same dimensions as the output of the corresponding layer or other building block in the given teacher neural network. In this way, the distance between the outputs of the corresponding layers can be directly measured in the common space. This loss function L KD An example is:
[0035] L KD =‖f t -f s ‖,
[0036] where f t and f s are the intermediate work products output by the teacher and student neural networks, respectively. Here, the index KD stands for “knowledge distillation”.
[0037] In another particularly advantageous embodiment, the Kullback-Leibler divergence is chosen as the distance metric. This distance metric is originally intended to measure the distance between different distributions and therefore does not require that the outputs, respectively the work products, of both the teacher and the student neural network are in the same space. Alternatively, the outputs, respectively the work products, can also be divided by a scalar temperature constant τ in order to smooth the outputs, respectively the work products. This loss function L KD An example is:
[0038] L KD =KL(p t / τ,p s / τ),
[0039] where p t and p s are the outputs or intermediate work products (“predictions”) of the teacher and student neural networks, respectively.
[0040] In another particularly advantageous embodiment, the student neural network is configured to generate outputs for a given task. The training images x are labeled with ground truth values for the given task. The loss function also measures the difference and / or distance between the output generated from each training image x and the ground truth value of the corresponding training image x. task As the task loss, the total loss function L can then be, for example:
[0041] L=L task +L KD .
[0042] In this way, a student neural network can be trained both to behave in a manner that corresponds to that of a teacher neural network and to solve a given task. For example, a given teacher neural network can be a generally trained base model that has been trained to perform rather abstract processing of input images on unlabeled data, while the student neural network can be trained to solve a specific task for which ground truth is available. Thus, instead of training a neural network from scratch for a given task (which is not feasible in many applications), the training of the student neural network can build on the knowledge in the teacher neural network and refine it so that it can solve a specific task with only a small number of training images x.
[0043] In another particularly advantageous embodiment, the loss function also measures the difference and / or distance between the output generated from each augmented image version x' and the ground truth of the training image x to which the augmented version x' relates. Because the augmented image versions x' have the same semantic content as the training image x to which they relate, the ground truth for which the training image x was annotated ("labeled") is also valid for the augmented versions x'. Thus, obtaining augmented versions x' according to the method proposed herein also expands the pool of available training examples, even for supervised training to solve a given task, without incurring additional labeling overhead.
[0044] In another particularly advantageous embodiment, the student neural network is configured as an image classifier, a semantic segmentation model, and / or an object detector. These are typical tasks where changes in image style should not abruptly change the student neural network's response. In this context, it is advantageous that the method presented here for obtaining the enhanced version x' is not limited to classification tasks.
[0045] The ultimate goal of training a student neural network is to use it in a real-world application, particularly on hardware platforms that would not allow the teacher neural network to execute. Therefore, in another particularly advantageous embodiment, an input image acquired using at least one sensor is provided to the trained student neural network. Based on the results of the trained student neural network, an actuation signal is determined. The vehicle, driver assistance system, robot, surveillance system, quality inspection system, and / or medical imaging system is then actuated using the actuation signal. In this way, the likelihood that the corresponding actuated technical system's response to actuation using the actuation signal will be appropriate for the situation characterized by the acquired input image is increased. In particular, sudden changes in image style, such as from daytime to nighttime or from summer to winter conditions, will not trigger a sudden, unexpected change in behavior. For example, when a car travels through a tunnel across the Alps, the conditions at both ends of the tunnel may be completely different.
[0046] The method can be implemented in whole or in part by a computer and embodied in software. Therefore, the present invention also relates to a computer program having machine-readable instructions that, when executed by one or more computers and / or computing instances, cause the one or more computers and / or computing instances to perform the above-described method. Control units for vehicles or robots, as well as other embedded systems capable of executing machine-readable instructions, are also considered computers. Computing instances include virtual machines, containers, or other execution environments that allow for the execution of machine-readable instructions in the cloud.
[0047] The non-transitory machine-readable data carrier and / or download product may include a computer program. A download product is an electronic product that can be sold online and transmitted over a network for immediate fulfillment. One or more computers and / or computing instances may be equipped with the computer program and / or the non-transitory storage medium and / or download product.
[0048] Hereinafter, the present invention is described using the accompanying drawings without any intention of limiting the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The accompanying drawings show:
[0050] Figure 1 is an exemplary embodiment of a method 100 for training a student neural network 2;
[0051] Figure 2 is based on output 1b, 2b ( Figure 2a ) and intermediate work products 1a, 2a( Figure 2b ), an exemplary way of comparing the behavior of the teacher neural network 1 and the student neural network 2;
[0052] Figure 3 is an exemplary data augmentation for training to solve a given task using an unlabeled style source image y. DETAILED DESCRIPTION
[0053] Figure 1 is a schematic flow chart of an example of a method 100 for training a student neural network 2 to adopt the behavior of a given teacher neural network 1. The teacher neural network 1 is trained to perform a given processing on an input image.
[0054] In step 110 , a set of training images x is provided.
[0055] Optionally, the student neural network 2 may be configured to produce an output 2b with respect to a given task according to block 105. The training image x may then be labeled with a ground truth 2b* with respect to the given task according to block 111.
[0056] In step 120 , one or more style-enhanced versions x′ are generated from at least one training image x. These style-enhanced versions x′ have the same semantic content as the original training image x, but are different from the original training image x in style.
[0057] According to block 121, content c may be extracted from a training image x. According to block 122, style s may be derived based at least in part on a style source image y. According to block 123, the semantic content c and style s may then be processed by a trained generative model G into the sought enhanced version x'.
[0058] According to blocks 121a, respectively 122a, the extraction 121, respectively the obtaining 122 may be performed by feeding the training image x, respectively the style source image y, into a trained feature extractor network F configured to determine both the content c and the style s from the input image.
[0059] According to block 124 , the style s may be selected as an interpolation between a first style s′ of the training image x and a second style s″ of the style source image y.
[0060] According to block 125 , the style source image y may differ from the one or more training images x in at least one aspect, the at least one aspect being different from the content c.
[0061] According to block 126 , at least one enhanced version x′ of a training image x and / or at least one style source image y may be obtained from the trained generative diffusion model D using textual cues indicating a desired style of the enhanced version x′, respectively style source image y.
[0062] In step 130, the training image x and the augmented version x' are processed by:
[0063] On the one hand, through the teacher neural network 1, which produces intermediate work products 1a and outputs 1b;
[0064] as well as
[0065] On the one hand, through the student neural network 2, which produces intermediate work products 2a and outputs 2b.
[0066] In step 140, the degree to which the output 2b and / or intermediate work product 2a produced by the student neural network 2 from each image x, x' is consistent with the output 1b and / or intermediate work product 1a produced by the teacher neural network 1 from the same image x, x' is evaluated using a predetermined loss function 3. The result of the loss function 3 is a value 3a.
[0067] According to block 141 , the loss function 3 may measure the distance between the outputs 1 b, 2 b and / or intermediate work products 1 a, 2 a produced by the teacher neural network 1 on the one hand and the student neural network 2 on the other hand. Specifically, according to block 141 a, the Kullback-Leibler divergence may be chosen as a measure of distance.
[0068] According to block 142, if the student neural network 2 is configured to solve a specific task according to block 105, and the training images x are labeled with corresponding ground truth values 2b*, the loss function 3 can also measure the difference and / or distance between the output 2b generated from each training image x and the ground truth value 2b* of the corresponding training image x.
[0069] In particular, according to block 142a, the loss function 3 may also measure the difference and / or distance between the output 2b generated from each augmented image version x' and the ground truth 2b* of the training image x to which the augmented version x' relates.
[0070] In step 150, parameters 2c characterizing the behavior of the student neural network 2 are optimized with the goal of improving the value 3a of the loss function 3. The final optimized state of the parameters 2c is marked with reference symbol 2c* and also characterizes the fully trained state 2* of the student neural network 2.
[0071] exist Figure 1 In the example shown, an input image 4 acquired using at least one sensor 5 is provided to a trained student neural network 2* in step 160. The trained student neural network 2* then generates an output 2b.
[0072] From this output 2b, in step 170, an actuation signal 170a is determined. In step 180, the vehicle 50, the driver assistance system 51, the robot 60, the monitoring system 70, the quality inspection system 80 and / or the medical imaging system 90 are actuated using this actuation signal 170a.
[0073] FIG2 shows an exemplary way of comparing the behavior of a teacher neural network 1 and a student neural network 2 .
[0074] exist Figure 2aIn the example shown, according to step 120 of method 100, an enhanced version x' is created from a training image x. Here, the training image x, showing a vehicle on a road, was taken during daytime, but the new style applied to it is that of a nighttime image. Therefore, the resulting enhanced version x' shows the same vehicle on the same road as in the original training image x, but at night. Both the training image x and the enhanced version x' are provided to both a teacher neural network 1 and a student neural network 2. Teacher neural network 1 produces output 1b, while student neural network 2 produces output 2a. These outputs 1b and 2b are compared using a loss function 3. The snowflake symbol on the square representing teacher neural network 1 indicates that the parameters representing the behavior of teacher neural network 1 remain frozen. The flame symbol on the square representing student neural network 2 indicates that parameters 2c representing the behavior of student neural network 2 are being optimized. That is, based on the value 3a of loss function 3, an update Δ2c to the parameters 2c of student neural network 2 is calculated and fed back to student neural network 2.
[0075] exist Figure 2b In the example shown, as Figure 2a As shown, an enhanced version x' is created from the training image x. Similarly, according to Figure 2a , both the training image x and the augmented version x' are provided to both the teacher neural network 1 and the student neural network 2. However, unlike Figure 2a Instead, the intermediate work products 1a and 2a produced by the teacher neural network 1 and the student neural network 2 are taken into account by the loss function 3. Figure 2a In a similar manner, the update Δ2c of the parameters 2c of the student neural network 2 is calculated and fed back to the student neural network 2. Meanwhile, the parameters of the teacher neural network 1 remain frozen.
[0076] Figure 3 It is shown how the data augmentation given above can also be used for supervised training of a student neural network 2 to solve a given task using unlabeled style source images y. Figure 3 In the example shown, the given task to be solved is semantic segmentation. The training image x and the enhanced version x' are the same as in Figure 2. In addition, the source style image y from which the night style is derived is shown.
[0077] Both the training image x and the source style image y are decomposed into corresponding content c and style s by the trained feature extractor F. The content c derived from the training image x and the style derived from the source style image y are processed into an enhanced version x' by the trained generative model G.
[0078] The training image x is labeled with the corresponding ground truth 2b*. Since the enhanced version x' has the same semantic content, it can also be used to train the student neural network 2 for the task of semantic segmentation. According to the loss function 3 given above and its contribution Ltask To measure task completion.
Claims
1. A method (100) for training a student neural network (2) to adopt the behavior of a given teacher neural network (1), the given teacher neural network (1) being trained to perform a given processing on an input image, the method comprising the steps of: Provide (110) a set of training images x; generating (120) one or more style-enhanced versions x' from at least one training image x, The one or more style-enhanced versions x' o has the same semantic content as the original training image x, but o is different in style from the original training image x; The training image x and the augmented version x' are processed (130) by the teacher neural network (1) on the one hand and the student neural network (2) on the other hand; ≤ evaluating (140) using a predetermined loss function (3) the extent to which the output (2b) and / or intermediate work product (2a) produced by the student neural network (2) from each image x, x' agrees with the output (1b) and / or intermediate work product (1a) produced by the teacher neural network (1) from the same image x, x'; and Optimizing (150) the parameters (2c) characterizing the behavior of the student neural network (2) towards the goal of improving the value (3a) of the loss function (3).
2. The method (100) of claim 1, wherein generating (120) an enhanced version x' of the training image x comprises: Extract (121) content c from the training image x; · obtaining (122) a style s based at least in part on the style source image y; as well as • The semantic content c and style s are processed (123) by the trained generative model G into the sought enhanced version x'.
3. The method (100) of claim 2, wherein the following operations are performed (121a, 122a) by feeding a training image x, respectively a style source image y, into a trained feature extractor network F configured to determine both content c and style s from an input image: Extract (121) semantic content c from the training image x, and / or Obtain (122) a style s based on the style source image y.
4. The method (100) according to any one of claims 2 to 3, wherein the style s is selected (124) as an interpolation between a first style s' of the training image x and a second style s" of the style source image y.
5. The method (100) according to any one of claims 2 to 4, wherein the style source image y differs (125) from the one or more training images x in at least one aspect, the at least one aspect being different from the content c.
6. The method (100) according to claim 4, wherein aspects in which the style source image y differs from all training images x specifically include one or more of the following: The time of day the image was acquired; The season of the year when the image was acquired; Weather and / or lighting conditions under which the image was acquired; Defects and / or disturbances in the image; and Get the camera settings for the image.
7. A method (100) according to any one of claims 1 to 6, wherein at least one enhanced version x' of a training image x and / or at least one style source image y is obtained (126) from a trained generative diffusion model D using a textual cue indicating a desired style of the enhanced version x', respectively the style source image y.
8. The method (100) according to any one of claims 1 to 7, wherein a feature map output by an intermediate layer of the corresponding neural network (1, 2) is selected as the intermediate work product (1a, 2a).
9. The method (100) according to any one of claims 1 to 8, wherein the output (1b, 2b) of the neural network (1, 2) is selected as a logarithmic value and / or other unaggregated results output by the neural network (1, 2).
10. A method (100) according to any one of claims 1 to 9, wherein the loss function (3) measures (141) the distance between the outputs (1b; 2b) and / or intermediate work products (1a; 2a) produced by the teacher neural network (1) on the one hand and by the student neural network (2) on the other hand.
11. The method (100) according to claim 10, wherein the Kullback-Leibler divergence is selected (141a) as the measure of distance.
12. The method (100) according to any one of claims 1 to 11, wherein The student neural network (2) is configured (105) to produce an output (2b) with respect to a given task; The training images x are labeled (111) with the ground truth values (2b*) for the given task; and and • The loss function (3) also measures (142) the difference and / or distance between the output (2b) generated from each training image x and the ground truth (2b*) of the corresponding training image x.
13. The method (100) of claim 12, wherein the loss function (3) further measures (142a) the difference and / or distance between the output (2b) generated from each augmented image version x' and the ground truth (2b*) of the training image x to which the augmented version x' relates.
14. The method (100) according to any one of claims 1 to 13, wherein the student neural network (2) is configured as an image classifier, a semantic segmentation model and / or an object detector.
15. The method (100) according to any one of claims 1 to 14, further comprising: providing (160) an input image (4) acquired using at least one sensor (5) to a trained student neural network (2*); Determining (170) an actuation signal (170a) from the results (2b) of the trained student neural network (2*); and Actuating (180) the vehicle (50), the driver assistance system (51), the robot (60), the monitoring system (70), the quality inspection system (80) and / or the medical imaging system (90) with the actuation signal (170a).
16. A computer program comprising machine-readable instructions which, when executed on one or more computers and / or computing instances, cause the one or more computers and / or computing instances to perform the method (100) of any one of claims 1 to 15.
17. A non-transitory machine-readable data carrier and / or a download product having a computer program as claimed in claim 16.
18. One or more computing instances having the computer program as claimed in claim 16 and / or the machine-readable data carrier and / or the download product as claimed in claim 17.