Device and method for training neuronal networks - Patents.com

Abstract

To enhance performance of a classifier by regularizing a neuron network during training.SOLUTION: A method for training a neural network includes: a step 901 of receiving, by a preceding layer, first layer input (i) in a first layer based on an input signal (x); a step 902 of determining first layer output (z) based on the first layer input (i) and a plurality of parameters (w) of the first layer; a step 903 of determining a first layer loss value (l) based on the first layer output (z), in the step 903, the first layer loss value (l) characterizes degree of dependency between feature maps of the first layer output (z), and the first layer loss value (l) is obtained in an unsupervised fashion; and a step 904 of training the neural network, in the step 904, the training includes adaption of the parameters of the first layer, and the adaption is based on the first layer loss value (l).SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a method for training a neuronal network, a method for using a neuronal network, a method for operating an actuator, a computer program and a machine-readable storage medium, a neuronal network, a control system, and a training system.

[0002] Background technology EP 1 752 362 A1 discloses a technique for unsupervised training of neural networks.

[0003] Advantages of the invention Machine learning based classifiers, such as neural networks, extract knowledge from image data to learn a classification function. To this end, machine learning based classifiers contain multiple parameters that can be adapted to affect classification behavior. In the best performing classifiers, the number of parameters ranges from 10 to 100. 7 Here, performance can be understood as a measure of how well a neuronal network can predict a desired class.

[0004] Especially in safety-critical applications, obtaining the highest possible performance is a high priority. Consider, for example, using a neuronal network as part of an at least partially autonomous vehicle's perception pipeline to detect pedestrians in the vehicle's vicinity. In this scenario, information about the detected pedestrians can be used, for example, to compute a safe trajectory for navigating the vehicle's environment. In the event that a pedestrian is misclassified, e.g., overlooked by the classifier, the system may plan a trajectory that actually passes through the pedestrian. This, in turn, may result in severe injury and / or death.

[0005] An advantage of the method having the features of independent claim 1 is that it regularizes the neuronal network during training, which leads to an increase in the performance of the classifier.

[0006] Regularization is beneficial for training because it reduces overfitting of the neural network, which would otherwise be prone to overfitting due to the large number of parameters, i.e. memorizing certain data points, which in turn would lead to poor performance of the classifier.

[0007] Another advantage of the method is that it does not impose constraints on the use of specific layers in a given structure, as is necessary for example in the case of flow-based models, which allows the classifier to be freely adapted to the characteristics of the training data at hand, which in turn leads to better performance of the classifier.

[0008] Disclosure of the Invention In a first aspect, the present invention provides a method for training a neuronal network, the method comprising: a classifier comprising at least one first layer, the first layer comprising a plurality of filters for providing a first layer output comprising a plurality of feature maps; and further comprising: training the classifier comprising: receiving a first layer input based on an input signal from a previous layer in the first layer; determining a first-layer output based on the first-layer input and a plurality of first-layer parameters; determining a first-layer loss value based on the first-layer output, the first-layer loss value being a measure of dependency between feature maps of the first-layer output, and further, the first-layer loss value is obtained in an unsupervised manner; training the neural network, the training including adapting parameters of a first layer, the adaptation being based on a loss value of the first layer; The present invention relates to a method comprising the steps of:

[0009] A neuronal network is understood to comprise one or more layers of linear and / or nonlinear units used to predict an output signal of a received input signal.

[0010] The layers are organized in a topology where the order of the layers is determined by the flow of information: if an initial layer provides input to another layer, then the other layer is considered to follow the initial layer, and the initial layer precedes the other layer.

[0011] The output provided by a layer may depend on the current values ​​of the corresponding set of parameters, the parameters of a layer being understood to belong to the parameters of a classifier.

[0012] The advantage of this layered design is that different layers provide more abstract representations of the input signal: as is known from the literature, earlier layers of a neuronal network typically extract a low-level representation of the input signal, capable of encoding information such as, for example, edges in the case of image data, while later layers of the classifier can extract higher-level representations, encoding information about, for example, objects, object parts, etc.

[0013] When provided with an input signal, each layer of the classifier may provide to one or more other layers of the classifier an output of the layer that may include multiple feature maps. A feature map can be considered as a representation of the input of a particular layer, obtained using some transformation. For example, in a convolutional layer, the feature map is obtained by applying one or more filters of a given height and width to the input of the layer. The output of such a convolutional layer is as many feature maps as there are filters in the layer. Preferably, these feature maps can be composed of tensors of the same height and width as the feature maps, the number of outputs, i.e. the number of feature maps, determining the depth of the tensor. This tensor can be used as the output of the convolutional layer. Each filter can include multiple parameters that can be adapted during training.

[0014] In general, the neuronal network can receive as input signals data of various modalities, in particular sensor image data such as provided by video, radar, LIDAR and / or ultrasonic sensors, as well as images provided by a thermal camera. The input signals may also include audio data, for example in the form of quantized audio signals or spectral images, as provided by one or more microphones.

[0015] The data contained in the input signal may also be artificially generated. In the case of image data, this can be achieved, for example, by creating a virtual environment in a computer and using a computer-implemented method for rendering images, for example based on a physical model. Alternatively, a generative machine learning model, for example a generative adversarial network or a flow-based model, may be used to obtain synthetic images.

[0016] In the case of audio data, the synthetic data may be generated by composing different audio signals in a computer. Alternatively, it is also possible to envisage synthesizing the audio data by machine learning methods, for example WaveNet.

[0017] The data contained in the input signal can be provided directly by sensors and / or synthetic techniques. Alternatively, the data may also be provided by some storage unit containing pre-recorded or synthetic data.

[0018] The input signal may also include multiple input data, also known as batches of data, in which case the classifier may predict the output signal for each data in the batch.

[0019] The output of the classifier can be seen as a prediction for the input signal, provided based on the classifier's knowledge encoded by its parameters.

[0020] In the context of the present invention, a neuron network is understood to comprise a first layer that can be trained without a teacher. For this purpose, the first layer calculates a first layer loss value that characterizes the dependency between the feature maps of the output of the first layer. The output of the first layer may be obtained from the input of the first layer, for example, by a discrete convolution operation. In this case, the parameters of the first layer can be used as filter weights of the convolution operation.

[0021] Training the first layer based on the first layer loss may be done without the need for a backpropagation algorithm as is common for training classifiers, e.g., neural networks. Instead, the weights may be updated, e.g., by running a gradient descent algorithm using the first layer loss. To this end, the gradient of the first layer loss with respect to the first layer weights may be calculated.

[0022] When trained in the manner described in the first aspect of the present invention, the neuronal network is able to extract information in an unsupervised manner from the training data it is fed (i.e. the data it is trained with). The term unsupervised may be understood as meaning that the classifier does not require additional information from humans, for example in the form of annotations (also known as labels).

[0023] This is advantageous because obtaining labels is a very time-consuming task. To obtain reliable labels, experts must assign labels to training data. In a given amount of time, it is possible to collect more data by simply acquiring data than by acquiring and labeling the same data. Since the training data is used to "teach" a classifier about a particular problem, the more data there is, the more it can learn about the problem. This leads to better classifier performance.

[0024] For example, consider the case of an autonomous vehicle, where the classifier is used as part of a perception pipeline to detect pedestrians around the vehicle. The perception pipeline may comprise sensors and software components. The sensors may be configured to sense information about the vehicle's environment, and the software components may infer higher level information, such as the location of pedestrians in the sensed information. This higher level information may be used, for example, to plan a safe path that does not collide with any of the pedestrians. It may further be envisioned that the training data for this pedestrian detector is obtained from a set of vehicle-collected data by a suitable sensing device, such as a camera, radar, or LIDAR.

[0025] It is possible to assume that there are a large number of rare pedestrians in the data, e.g., pedestrians wearing disguises. Although these pedestrians may be covered in the data, it takes too much time for a human to search all the data for rare pedestrians.

[0026] The current approach in situations like the example above is to sample a random subset of the data designated for labeling, which is later used to train, for example, a pedestrian detector.

[0027] However, this approach to collecting training data is likely to miss a significant amount of rare events, since the subset of data that is labeled is typically small compared to the entire dataset of recorded data.

[0028] However, it can be envisaged that the entire dataset can be used for unsupervised training of the pedestrian detector. The pedestrian detector may then be further trained with a potentially limited amount of training data. Since rare events are much more likely to be covered in a dataset recorded from a group than in a smaller training dataset, the pedestrian detector can also learn information from unlabelled but valuable data. This in turn improves the performance of the pedestrian detector.

[0029] Another advantage of the proposed training method is that it allows training the neuronal network in a way that incorporates information from unlabeled data while still allowing it to be fine-tuned for downstream tasks.

[0030] The advantage of learning individual feature maps in a classifier is that the classifier can infer high-level concepts from the data without being given explicit labels. For example, independent component analysis is known to be able to learn high-level concepts such as objects from input signals. The first aspect can be seen as involving some kind of nonlinear independent component analysis in training the classifier. This allows the classifier to learn nonlinear relationships in the data, which in turn leads to learning highly discriminatory features of the elements contained in the input signal.

[0031] This is particularly advantageous when the classifier trained in this way is fine-tuned for a classification task, because the output layer of the classifier already outputs highly discriminatory features before fine-tuning. This makes the classification task much easier to learn, and therefore reduces the time required for fine-tuning the classifier. This in turn saves computer memory for learning, reduces the number of required calculations, and reduces energy consumption.

[0032] It is understood that reducing the dependency between feature maps in the output of the first layer can be achieved by reducing the loss value of the first layer.

[0033] In another aspect of the invention, training the neural network comprises: determining an output signal from the neural network, the output signal depending on an output of the first layer, the output signal characterizing a classification of the input signal; determining a loss value for the classification based on the output signal; training the neural network, the training including adapting parameters of a first layer, the adaptation being based on a classification loss; It can further be envisaged that the

[0034] By training a neural network in this way, it can be used as a classifier, the term classifier being understood as a device configured to determine one or many class labels for an input signal or a portion thereof.

[0035] The results can then be used to control the device. For example, the classifier may be employed as part of a perception pipeline of an at least partially autonomous vehicle. In this scenario, for example, the classifier may be used to detect whether the vehicle is currently located in a suburban area, a rural area, or on a highway. This information may then be used to activate and / or deactivate certain driving functions, driving maneuvers, and / or speed limits when the vehicle is operating at least partially autonomously.

[0036] In general, a classifier may determine different kinds of class labels. For example, it may be determined whether an object is present or not at a particular location in an image. This technique is known as object detection. Using the example of an at least partially autonomous vehicle, object detection may be used to detect other road participants, in particular pedestrians, other vehicles, and cyclists, for example, to plan a collision-free trajectory through an environment.

[0037] Another common task for which classifiers can be used is semantic segmentation, where each element of an input signal, e.g., each pixel of an input image, is assigned a class label from a classifier. Using the above example, a classifier may be used to classify each pixel of a camera image as belonging to either drivable or non-drivable space. This information can be used alternatively or additionally with respect to information about other road participants to determine a safe trajectory through the environment.

[0038] The output signal of the classifier can be seen as a prediction based on the input signal. This prediction may be incorrect if the model is insufficient for the given data. In this sense, the performance of a classifier may be understood as a measure of how well the classifier can predict the desired class label. When the output of the classifier is used as part of the control of a device, it is very important to achieve the highest possible performance, since incorrect classification may cause undesirable and / or unsafe behavior of the device.

[0039] The advantage of training a neural network with an additional loss for classification is that it can be used as a classifier while performing regularized training. Regularization is known to improve the performance of classifiers. Thus, the method improves the performance of the neural network when used as a classifier compared to simply training the neural network with the training data.

[0040] This is advantageous as higher performance translates into better and / or safer behavior of devices that operate based on the classification of the classifier.

[0041] For example, consider the example of an autonomous vehicle where a classifier is used to detect pedestrians around the vehicle. Information about the detected pedestrians may be used, for example, to plan a collision-free trajectory. If the classifier does not perform well enough, it may miss the pedestrian until the vehicle is close to it. The classifier then detects the pedestrian and applies emergency braking to avoid colliding with the pedestrian.

[0042] When a classifier is configured and / or trained according to the first aspect, it can improve performance and detect pedestrians, which in turn leads to the vehicle being able to plan a collision-free path without the need for emergency braking. Planning a path in this manner can reduce emergency behaviors such as emergency braking, which in turn reduces wear on vehicle components, such as brakes.

[0043] It is understood that by reducing the classification loss value, the classification performance of the classifier can be improved.

[0044] In another aspect of the invention, it can be envisaged to determine a loss value for the first layer based on a change in the distribution of the input of the first layer relative to a desired distribution of the output of the first layer.

[0045] The input of the first layer may be seen as a concrete realization of a first random variable. This random variable may be transformed into a second random variable with certain desired properties. In this case, the output of the first layer will then follow the distribution of the second random variable and thus obtain the desired properties. For example, a desired independence between feature maps may be a desired property of the second random variable.

[0046] Therefore, the advantage of this approach is that the output of the first layer can acquire certain desirable properties, e.g., independence between feature maps, which acts as a regularizer when used to train a neural network for a particular classification task.

[0047] Another advantage is that the first layer loss may be used as an independent loss to train a neural network. This results in a pre-trained neural network. This pre-trained neural network may then be used as a starting point to train a classifier. Since the classifier gets a good initialization by pre-training, the training of the classifier converges at a much higher speed. This allows the classifier to achieve higher performance using less training time.

[0048] In another aspect, a method for training a neural network comprises: Pruning the first layer (L1) filters It can be envisaged that the method further comprises:

[0049] Pruning can be understood as removing certain filters in the layers of a neural network. This has the effect of reducing the amount of parameters of the neural network. When applied during training, this acts as a regularization. In addition, removing the number of parameters reduces the amount of memory required to store the neural network. In addition, because the number of parameters is reduced, the neural network requires fewer computational operations to obtain an output signal.

[0050] This in turn allows the neural network to consume less energy and provide output signals faster. In the case of training, getting the output signal faster may be seen as equivalent to processing more training data in a given time. This in turn allows the neural network to extract more information from the training data, thus improving performance.

[0051] In another aspect, it can be assumed that the first layer loss value characterizes the factoring of the distribution of the first layer output among the feature maps.

[0052] The advantage of this extension is that it allows an efficient way of enforcing independence between feature maps. For example, one can imagine training the outputs of the first layer such that their distribution is a multivariate standard normal distribution. In this way, independence between feature maps may be added by constructing factoring random variables, which in this case can be understood to be the feature maps of the outputs of the first layer.

[0053] In a further embodiment, the loss value of the first layer is determined according to the following formula:

number

[0054] This has the advantage that reducing the loss value is equivalent to reducing the dependency between individual feature maps in the output of the first layer. In addition, if the first layer contains a bijective transformation, as is the case for convolutional and fully connected layers, the first layer can be trained individually without having to backpropagate error gradients through the entire neural network. This effectively reduces the amount of memory required to train the classifier, since each first layer can be trained individually.

[0055] When using neural networks for classification, this stratified training may be applied in a pre-training step before fine-tuning the neural network with a labeled training dataset, which effectively shortens the training time of the entire system, thus further saving memory and energy.

[0056] In another embodiment, A i and / or b. iAnd / or it can be assumed that α is adapted during training of the classifier.

[0057] By training these parameters, the neural network is able to learn more about the characteristics of the data it is provided with, which leads to improved predictive performance of the neural network.

[0058] In another aspect, it can be envisaged that the step of determining the first layer output comprises padding of the first layer input such that the size of the first layer output matches the size of the first layer input along height and / or width and / or depth.

[0059] In general, the output of the first layer does not need to be of the same shape, i.e., height, width, and depth, as the input. However, if layered learning is to be applied, the projection from input to output can preferably be chosen to be bijective. For the first layer, if the shape of the output does not match the shape of the input, the input of the first layer can therefore be padded along the required dimensions to obtain an output of equal size to the original input.

[0060] The advantage of this approach is that it reduces the constraints on the architecture of the neural network, i.e. the composition of its layers, and thus allows the architecture to adapt depending on the characteristics of the data used for training. This leads to improved performance, since the architecture can be tuned with respect to the data.

[0061] Embodiments of the invention will now be described in more detail with reference to the following figures. [Brief description of the drawings]

[0062] [Figure 1] FIG. 1 illustrates a control system with a classifier for controlling actuators in an actuator environment. [Diagram 2] FIG. 1 illustrates a control system for controlling an at least partially autonomous robot. [Diagram 3] FIG. 1 illustrates a control system for controlling a manufacturing machine. [Figure 4] FIG. 1 illustrates a control system for controlling an automated personal assistant. [Diagram 5] FIG. 2 illustrates a control system that controls the access control system. [Figure 6] FIG. 2 is a diagram showing a control system that controls the monitoring system. [Figure 7] FIG. 2 is a diagram showing a control system that controls the imaging system. [Figure 8] FIG. 2 illustrates the first layer of a classifier. [Figure 9] 1 is a flow chart of a training method performed by the training system.

[0063] Description of the embodiments Shown in Fig. 1 is an embodiment of an actuator (10) in an actuator environment (20). The actuator (10) interacts with a control system (40). The actuator (10) and the actuator environment (20) together are referred to as the actuator system. A sensor (30), preferably at equally spaced time points, detects the state of the actuator system. The sensor (30) may comprise a number of sensors. Preferably, the sensor (30) is an optical sensor imaging the environment (20). An output signal (S) of the sensor (30) (or the output signal (S) of each sensor, if the sensor (30) comprises a number of sensors) encoding the detected condition is transmitted to the control system (40).

[0064] Thereby, the control system (40) receives the stream of sensor signals (S). The control system then calculates a set of actuator control commands (A) responsive to the stream of sensor signals (S), which are then transmitted to the actuator (10).

[0065] The control system (40) receives a stream of sensor signals (S) of the sensors (30) at an optional receiving unit (50). The receiving unit (50) converts the sensor signals (S) into input signals (x). Alternatively, in the absence of the receiving unit (50), each sensor signal (S) may be received directly as an input signal (x). The input signal (x) may, for example, be provided as part of the sensor signals (S). Alternatively, the sensor signals (S) may be processed to obtain the input signals (x). The input signals (x) include image data corresponding to an image recorded by the sensors (30). That is, the input signals (x) are provided in accordance with the sensor signals (S).

[0066] The input signal (x) is then fed into a neural network (60).

[0067] The neural network (60) is parameterized by parameters (φ) stored in and provided by a parameter store (St1).

[0068] The neural network (60) determines an output signal (y) from the input signal (x). The output signal (y) includes information that assigns one or more labels to the input signal (x). The output signal (y) is sent to an optional transformation unit (80), which transforms the output signal (y) into a control command (A). The actuator control command (A) is then sent to the actuator (10) to control the actuator (10) accordingly. Alternatively, the output signal (y) may be directly taken as the actuator control command (A).

[0069] The actuator (10) receives the actuator control command (A) and is controlled accordingly to perform an action corresponding to the actuator control command (A). The actuator (10) may comprise control logic that converts the actuator control command (A) into further control commands, which are used to control the actuator (10).

[0070] In further embodiments, the control system 40 may comprise the sensor 30. In further embodiments, the control system 40 may alternatively or additionally comprise the actuator 10.

[0071] In one embodiment, the neural network (60) may be designed to identify lanes on the road ahead, for example by classifying the road surface and markings on said road and identifying lanes as patches of road surface between said markings. Based on the output of a navigation system, a suitable lane can be selected to follow along the selected route, and depending on the current lane and the target lane, the vehicle (100) may decide whether to change lanes or to stay in the current lane. An actuator control command (A) may then be calculated, for example by retrieving a predefined movement pattern from a database corresponding to the identified movement.

[0072] Similarly, upon identifying a road sign or traffic light, corresponding constraints on possible movement patterns of the vehicle (100) depending on the type of identified road sign or the identified state of said traffic light may be obtained, for example from a database, a future path of the vehicle (100) that meets the constraints may be calculated, and actuator control commands (A) may be calculated to steer the vehicle (100) to execute said trajectory.

[0073] Similarly, upon identifying a pedestrian and / or vehicle, an expected future behavior of said pedestrian and / or vehicle may be estimated, and then a trajectory may be selected based on the estimated future behavior to avoid a collision with the identified pedestrian and / or vehicle, and actuator control commands (A) may be calculated to steer the vehicle (100) to execute said trajectory.

[0074] In yet a further embodiment, it may be envisaged that the control system (40) controls the display (10a) instead of the actuator (10).

[0075] Additionally, the control system (40) may include a processor (45) (or multiple processors) and at least one machine-readable storage medium (46) having stored thereon instructions that, when executed, cause the control system (40) to perform a method according to one aspect of the present invention.

[0076] FIG. 2 illustrates an embodiment in which the control system (40) is used to control an at least partially autonomous robot, such as an at least partially autonomous vehicle (100).

[0077] The sensors (30) may include one or more video sensors, one or more radar sensors, one or more ultrasonic sensors, one or more LiDAR sensors, and / or one or more location sensors (such as GPS), some or all of which are preferably, but not necessarily, on-board the vehicle (100).

[0078] Alternatively or additionally, the sensor 30 may comprise an information system for determining the state of the actuator system. One example of such an information system is a weather information system for determining current or future weather conditions in the environment 20.

[0079] For example, using the input signal (x), the neural network (60) may detect, for example, an object in the vicinity of the at least partially autonomous robot. The output signal (y) may include information characterizing where the object is located in the vicinity of the at least partially autonomous robot. Actuator control commands (A) may then be determined according to this information, for example, to avoid a collision with the detected object.

[0080] The actuator (10) is preferably on-board the vehicle (100) and may be provided by the brakes, propulsion system, engine, driveline or steering of the vehicle (100). An actuator control command (A) may be determined to control the actuator (or actuators) (10) to cause the vehicle (100) to avoid a collision with the detected object. The detected object may also be classified according to what the classifier (60) determines to be most likely, e.g., a pedestrian or a tree, and the actuator control command (A) may be determined in response to the classification.

[0081] In further embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving or walking. The mobile robot may be, inter alia, an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. In all of the above embodiments, an actuator command control (A) may be determined to control the propulsion units and / or steering and / or braking of the mobile robot such that the mobile robot may avoid a collision with the identified object.

[0082] In a further embodiment, the at least partially autonomous robot may be provided by a horticultural robot (not shown) that uses sensors (30), preferably optical sensors, to determine the state of the plants in the environment (20). The actuators (10) may control nozzles for liquid application and / or cutting devices, e.g. blades. Depending on the identified species and / or the identified state of the plants, the actuators (10) may determine actuator control commands (A) to apply an appropriate amount of liquid to the plants and / or to cut the plants.

[0083] In yet a further embodiment, the at least partially autonomous robot may be provided by a home appliance (not shown), such as a washing machine, stove, oven, microwave, or dishwasher. A sensor (30), such as an optical sensor, may detect a state of an object being processed by the home appliance. For example, if the home appliance is a washing machine, the sensor (30) may detect a state of laundry in the washing machine. An actuator control command (A) may then be determined depending on the detected material of the laundry.

[0084] 3 shows an embodiment in which a control system (40) is used to control a manufacturing machine (11) (e.g., a punch cutter, cutter, or gun drill) in a manufacturing system (200), such as part of a production line. The control system (40) controls an actuator (10) which controls the manufacturing machine (11).

[0085] The sensor 30 may for example be provided by an optical sensor capturing characteristics of the product 12. The classifier 60 may determine the state of the product 12 from these captured characteristics. The actuator 10 controlling the manufacturing machine 11 may then be controlled in response to the determined state of the product 12 for a subsequent manufacturing step of the product 12. Alternatively, it may be envisaged to control the actuator 10 during the manufacturing of the subsequent product 12 in response to the determined state of the product 12.

[0086] 4 shows an embodiment in which the control system (40) is used to control an automated personal assistant (250). The sensor (30) may be an optical sensor, for example, for receiving video images of the gestures of the user (249). Alternatively, the sensor (30) may also be an audio sensor, for example, for receiving voice commands of the user (249).

[0087] The control system (40) then determines an actuator control command (A) for controlling the automated personal assistant (250). The actuator control command (A) is determined according to a sensor signal (S) of the sensor (30). The sensor signal (S) is transmitted to the control system (40). For example, the neural network (60) may be configured to execute, for example, a gesture recognition algorithm for identifying a gesture made by the user (249). The control system (40) may then determine an actuator control command (A) for transmitting to the automated personal assistant (250). The control system then transmits the actuator control command (A) to the automated personal assistant (250).

[0088] For example, an actuator control command (A) may be determined according to a user gesture recognized and identified by the classifier (60). The control command may include information that causes the automated personal assistant (250) to retrieve information from a database and output the retrieved information in a form suitable for receipt by the user (249).

[0089] In a further embodiment, instead of an automated personal assistant (250), it may be envisaged that the control system (40) controls a home appliance (not shown) which is controlled according to the identified user gestures. The home appliance may be a washing machine, a stove, an oven, a microwave oven or a dishwasher.

[0090] 5 shows an embodiment in which the control system (40) controls an access control system (300). The access control system (300) may be designed to physically control access. The access control system may for example comprise a door (401). The sensor (30) may be configured to detect a relevant scene for deciding whether to grant access or not. For example, the sensor may be an optical sensor for providing image or video data, for example for detecting a person's face. The classifier (60) may be configured to interpret this image or video data, for example by matching the identity with known persons stored in a database, thereby determining the identity of the person. Depending on the interpretation of the neural network (60), for example according to the determined identity, an actuator control signal (A) may then be determined. The actuator (10) may be a lock that allows or does not allow access depending on the actuator control signal (A). Logical, not physical, access control is also possible.

[0091] Illustrated in Fig. 6 is an embodiment in which the control system (40) controls the surveillance system (400). This embodiment is almost identical to the embodiment shown in Fig. 5. Therefore, only the different aspects will be described in detail. The sensor (30) is configured to detect the scene under surveillance. The control system (40) does not necessarily control the actuator (10), but may instead control the display (10a). For example, the neural network (60) may determine a classification of the scene, e.g. whether the scene detected by the optical sensor (30) is suspicious or not. The actuator control signal (A) sent to the display (10a) may then be configured to cause the display (10a) to adjust its display content depending on the determined classification, e.g. to highlight objects determined to be suspicious by the neural network (60).

[0092] 7 shows an embodiment of a control system (40) for controlling an imaging system (500), such as an MRI, X-ray, or ultrasound imaging device. The sensor (30) may be, for example, an imaging sensor. The neural network (60) may determine a classification of all or a portion of the sensed image. An actuator control signal (A) may then be selected according to the classification to control the display (10a). For example, the neural network (60) may interpret a region of the sensed image as potentially anomalous. In this case, an actuator control signal (A) may be determined to cause the display (10a) to display the image and highlight the potentially anomalous region.

[0093] FIG. 8 illustrates an embodiment of a first layer (L1) included in a neural network (60). The first layer (L1) receives first layer inputs (i). The first layer inputs (i) may be provided as tensors of a given height, width, and depth that contain data that can be considered as a representation of the input signal (x) provided to the classifier. Alternatively, the first layer inputs (i) may also contain such representations of multiple input images, in which case the first layer inputs (i) are provided as a batch of representations, preferably composed of tensors of a given batch size, height, width, and depth. Alternatively, the first layer inputs (i) may not have a width dimension, for example if the input signal (x) contains one-dimensional data, such as a representation of audio data.

[0094] If the first layer input (i) has dimensions of height, width, and depth, the first layer output (z) can be obtained by a convolution operation (C) in which the first layer input (i) is discretely convolved with a predetermined amount of filters. The filters may be parameterized by a set of weights (w). Alternatively, if the first layer input (i) does not have a dimension of width, the first layer output (z) can be obtained by matrix multiplication of the first layer input (i) by a matrix (not shown) parameterized by a set of weights (w). This matrix multiplication may be assumed to be similar to processing the first layer input (i) by a fully connected layer to obtain the first layer output (z).

[0095] After obtaining the output (z) of the first layer, a loss value of the first layer is obtained based on the feature map contained in the output (z). The loss value (z) of the first layer may be obtained according to the following formula:

number

[0096] The output (z) of the first layer may then be provided (81) to one or more other layers of the neural network (60), or alternatively, the output of the first layer may be used as the output signal (y) of the neural network (60).

[0097] 9 is a schematic flow diagram of a method for training a neural network (60) with a first layer (L1). In a first step (901), the first layer (L1) receives a first layer input (i) from a second layer of the neural network (60). Alternatively, the first layer input (i) may be provided to the first layer (L1) by an input layer of the neural network (60). For example, the first layer input (i) may be just the input signal (x) of the neural network (60).

[0098] In a second step (902), a first layer output (z) may be determined by the first layer (L1), for example as shown in FIG. 8 and as described in the previous embodiment.

[0099] In a third step (903), a first layer loss value (l) may be obtained based on the first layer output (u), e.g. as shown in FIG. 8 and as described in the previous embodiment.

[0100] In a fourth step (904), the weights (w) of the first layer (L1) are updated, for example, by obtaining the gradient of the first layer loss value (l) with respect to the input signal (x) and adapting the weights (w) according to the negative of the gradient.

[0101] In a fifth step (905), an output signal (y) of the neural network (60) is obtained based on the output (z) of the first layer. This may be done by providing the output (z) of the first layer to a subsequent layer which provides a subsequent output. This subsequent output may be forwarded through the rest of the neural network (60), as is typical for feedforward neural networks. Alternatively, it may be envisaged to just use the output (z) of the first layer as the output signal (y).

[0102] In a sixth step (906), a classification loss value is obtained based on the output signal (y). This may be accomplished, for example, by applying a softmax function to the output signal (y) to obtain a set of probabilities. From these probabilities, a cross-entropy value for the desired classification may be calculated. The cross-entropy loss value may then be used as the classification loss value.

[0103] In a seventh step (907), the weights of the classifier (preferably including the first layer weights (w)) may be updated such that the classification loss value is smaller. This may be achieved by calculating the gradient of the classification loss value with respect to the input signal (x) and updating the weights (preferably including the first layer (L1) weights (w)) according to the negative gradient. Steps 1 (901)-(907) may then be repeated iteratively until a desired number of iterations have been completed. Alternatively, the steps may be repeated until the classification loss value is smaller than a predetermined classification loss threshold.

[0104] In an eighth step (908), the filters in the first layer (L1) may be pruned. This may be achieved, for example, by determining the filter that corresponds to the feature map with the smallest norm, for example, the L2-norm. This filter may then be discarded and the classifier may be fine-tuned to account for changes in the network architecture. This may be achieved, for example, by minimizing the classification loss value from the previous step using a training data set of the input signal (x) until the classification loss value is less than a classification loss threshold. The eighth step (908) may be repeated to remove a predetermined amount of filters from the first layer (L1).

[0105] In a further embodiment (not shown), it may be envisaged to repeat the first step (901), the second step (902) and the third step (903) iteratively with different input signals (x) until a predefined number of iterations is reached. Alternatively, the iterations may be repeated until the first layer loss value (l) falls below a certain first layer loss threshold.

[0106] In both alternatives of this embodiment, steps 4 (904)-8 (908) may then be performed. It is also envisioned that after the iterative execution of steps 1 (901)-3 (903) as described above, training may proceed without updating the first layer loss (l), but only updating the classification loss.

Claims

1. A method for training a neural network (60), comprising: The neural network (60) has a first layer (L １ ) The first layer (L １ ) comprises a plurality of filters providing a first layer output (z) comprising a plurality of feature maps; Furthermore, the training of the neural network (60) comprises: The computer generates a first layer (L) based on an input signal (x) by a preceding layer. １ receiving (901) an input (i) of a first layer in The computer calculates the first layer input (i) and the first layer (L １ determining (902) an output (z) of the first layer based on a number of parameters (w) of the first layer; The method includes: determining (903) a first-layer loss value (l) based on the first-layer output (z), the first-layer loss value (l) characterizing a degree of dependency between the feature maps of the first-layer output (z), and the first-layer loss value (l) is obtained in an unsupervised manner; A step (904) of the computer training the neural network (60), the training including the computer training the first layer (L １ (904) comprising adapting the parameters (w) of the first layer (1) based on the loss value (l) of the first layer; Including, The loss value (l) of the first layer is calculated according to the following formula: [0010] where i is the first layer input (i), C out is the number of feature maps in the first layer output (z), A j , α and b j are pre-defined values, and z j is the j-th feature map of the first layer output (z).

2. The step of training the neural network (60) comprises: a step (905) of the computer determining an output signal (y) of the neural network (60), the output signal (y) depending on the output (z) of the first layer, the output signal (y) characterizing a classification of the input signal (x); determining (906) a classification loss value based on the output signal (y); The computer trains (907) the neural network (60), the training including: １ ) based on classification loss; and The method of claim 1 , further comprising:

3. The method of claim 1 or 2, wherein the first layer loss value (l) characterizes a factoring of the distribution of the first layer output (z) among feature maps.

4. A j and / or b j 4. The method according to claim 1, wherein and / or α is adapted by the computer during training of the neural network (60).

5. 5. The method of claim 1, wherein the step of determining (902) the first layer output (z) comprises the computer padding the first layer input (i) such that a size of the first layer output (z) matches a size of the first layer input (i) along height and / or width and / or depth.

6. The method comprises: The computer １ ) pruning the filters (908); The method of any one of claims 1 to 5, further comprising:

7. A neural network (60) trained by the method of any one of claims 1 to 6.

8. A computer-implemented method for controlling an actuator (10) using a neural network (60) according to claim 7, which recites claim 2, wherein the input signal (x) provided to the neural network (60) is based on a sensor signal (S) including data from a sensor (30) and / or an actuator control signal (A) for controlling the actuator (10) is provided based on the output signal (y) of the neural network (60).

9. The method of claim 8, wherein the actuator (10) controls an at least partially autonomous robot (100) and / or a manufacturing machine (200) and / or an access control system (300).

10. A computer program configured, when executed by a processor (45), to cause the computer to carry out the method according to any one of claims 1 to 6 through all the steps of the method.

11. A machine-readable storage medium (46) having stored thereon the computer program of claim 10.

12. A control system (40) for operating an actuator (10), comprising the neural network (60) of claim 7, and further configured to operate the actuator (10) of claim 8.

13. A training system (140) configured to perform the method of any one of claims 1 to 6.

Images