Method and device for classifying sensor data and determining actuation signals for actuating actuators

Abstract

A method for providing an actuation signal (A) for actuating an actuator (10), wherein the actuation signal (A) is determined based on an output signal (y) of a neural network (60), characterized in that the neural network (60) comprises a scaling layer (S4), wherein the scaling layer maps an input signal (z4) attached to an input of the scaling layer (S4) to an output signal (z5) attached to an output of the scaling layer (S4), so that the mapping corresponds to a projection of the input signal (z4) onto a predeterminable value range, wherein parameters (ρ, c) characterizing the mapping can be predetermined.

Description

Technical Field

[0001] The invention relates to a method for classifying an input signal, a method for providing an actuation signal, a computer program, a machine-readable storage medium and an actuator control system. Background Art

[0002] From "Improving neural networks by preventing co-adaptation of feature detectors" (arXiv preprint arXiv:1207.0580vl, Geoffrey E. Hinton, NitishSrivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov (2012)), a method for training neural networks is known, in which the feature detectors are randomly switched during training. This method is also known by the name "Dropout".

[0003] From "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" (arXiv preprint arXiv:1502.03167v3, Sergey Ioffe, Christian Szegedy (2015)) a method for training neural networks is known, in which the input variables are normalized in the layers for small batches (English: "mini-batch") of training examples. Summary of the invention

[0004] Advantages of the invention

[0005] In contrast, the method having the features of independent claims 1 and 2 has the advantage that the architecture search of the neural network is simplified.

[0006] Advantageous embodiments are the subject matter of the dependent claims.

[0007] Disclosure of the Invention

[0008] With enough training data, so-called “deep learning” methods, i.e. (deep) artificial neural networks, can be used to efficiently determine the relationship between the input space V0 and the output space V kThis can be, for example, the classification of sensor data, especially image data, that is, the mapping from sensor data or image data to categories. The scheme is based on setting k-1 multiple latent spaces V1, ..., V k-1 In addition, we set k multiple mappings between these spaces Each of these mappings f i They are usually called layers. i Typically by having a suitably chosen space W i Weight To parameterize. k multiple layers f i The weights w1, ..., w k Jointly also called weights , and the input space V0 to the output space V k The mapping on , From each (with weight w explicitly shown as a subscript i ) mapping f i The result is: .

[0009] Using the definition in Given a probability distribution D, the task of training a neural network is to determine the weights , so that the expected value of the loss function (Kostenfunktion) L is Φ

[0010]

[0011] is minimized. In this case, the loss function L represents the loss function for w Determine the input variable x D To the output space V k The variable f in w (x D ) and the mapping on the output space V k The actual output variable y in D A measure of the spacing between.

[0012] A "deep neural network" can be understood as a neural network with at least two hidden layers.

[0013] To minimize the expected value Φ, a gradient-based method can be used, which determines the gradient of the weight w The gradient As usual, with the help of training data To approximate, that is, by to approximate, where the index j is chosen from a so-called epoch. In this case, an epoch is the label of an available training data point Arrangement.

[0014] To expand the training dataset, so-called data augmentation (also called augmentation) can be employed. In this case, for each index j from the round, the pair You can choose to enhance , where the input value is enhanced by Substitute input signal x j . In this case the input signal x j The set of typical variations (including the input signal x j itself), the typical variation makes the input signal x j The classification, that is, the output signal of the neural network does not change.

[0015] However, round-based sampling is not completely consistent with the definition from equation (1), because each data point in the variation process of a round is selected exactly once. The definition from equation (1) is based on independently drawn (gezogene) data points. That is, equation (1) assumes that data points are drawn "with replacement (Zuruecklegen)", while round-based sampling performs "without replacement" data points. This may lead to the premise that there is no mathematical convergence proof (because if N instances are drawn from a set N of multiple data points, then the probability of drawing each of these data points exactly once is less than (for N>2), while this probability is always equal to 1 in round-based sampling).

[0016] If data augmentation is used, this statistical effect may be further enhanced because there is a collection of elements, and the visual enhancement function It cannot be ruled out that , . With the help of collection A statistically correct mapping of the enhancement is difficult in this case, since the effect does not have to be true for every input data x j So, for example, rotation may have no effect on a circular object, but it will have a very strong effect on a normal object. The dimensions may be similar to the input data x j This can be problematic for adverse training methods.

[0017] Finally, the number of training data points N is a size that is usually very complex to set. If N is chosen to be too large, the runtime of the training method can be excessively prolonged; if N is chosen to be too small, convergence cannot be guaranteed, since the mathematical proof of convergence is generally based on assumptions that are not then met. In addition, it is unclear at which point in time the training is to be reliably terminated. If in this case a portion of the data points is taken as an evaluation data set and the quality of the convergence is determined with the aid of this evaluation data set, this can lead to an overfitting of the weights w with respect to the data points of the evaluation data set, which, when applied to data that differs from the training data, not only reduces the data difference but can also deteriorate the performance of the network. This can lead to a reduction in the so-called generalizability.

[0018] In order to reduce overfitting, the information stored in the hidden layer can be randomly removed with the help of the "discarding method" mentioned at the beginning.

[0019] In order to improve the randomization of the training process, by using a so-called batch normalization layer, statistical parameters μ and σ for so-called mini-batches can be introduced, which are updated probabilistically during the training process. During inference, the values ​​of these parameters μ and σ are selected as fixed predefinable values, for example, as estimated values ​​from training by extrapolating the exponential backpropagation characteristic.

[0020] If the layer with index i is a batch normalization layer, the belonging weights are not updated in case of gradient descent , that is, it is different from the weight w of the other layers k k To process these weights w i This increases the complexity of the implementation.

[0021] Among other things, the mini-batch size is a parameter that generally influences the training result, for example during a (possibly complex) architecture search, and must therefore be set as well as possible to other hyperparameters.

[0022] In a first aspect of the invention, it is thus provided that the neural network comprises a scaling layer, wherein the scaling layer maps an input signal attached to the input of the scaling layer to an output signal attached to the output of the scaling layer, so that the mapping corresponds to a projection of the input signal onto a predeterminable value range, wherein parameters characterizing the mapping can be predetermined. The value range can in this case be defined by a norm. If this is the case, the scaling layer enables the scale of the input signal to be limited with respect to the norm.

[0023] In this context, “predeterminable” may mean, in particular, that the parameters can be adapted during the training of the neural network as a function of a gradient, wherein the gradient is determined in a conventional manner as a function of an output signal of the neural network and as a function of an associated desired output signal.

[0024] That is, firstly in a training phase, predefinable parameters are adapted according to a method for training a neural network, wherein during training, an adaptation of these predefinable parameters occurs when an input signal of the neural network and an associated expected output signal are fed to the neural network according to an output signal of the neural network, wherein the adaptation of the predefinable parameters occurs according to determined gradients, which are related to the output signal of the neural network and the associated expected output signal.

[0025] Preferably, in an extended solution, it can be provided that the scaling layer maps an input signal attached to the input of the scaling layer to an output signal attached to the output of the scaling layer, so that the mapping corresponds to a projection onto a sphere, wherein the center c and / or radius ρ of the sphere can be fixedly predefined.

[0026] In this case, the mapping can be obtained by the equation Given that the equation has a first norm (N1) and a second norm (N2), the term "norm" is to be understood in this case in a mathematical sense.

[0027] In a refinement that is to be calculated particularly efficiently, it can be provided that the first norm ( N1 ) and the second norm ( N2 ) are selected to be equal.

[0028] Alternatively or additionally, the first norm (N1) may be Norm. This norm can also be calculated particularly efficiently, in particular even if the first norm (N1) and the second norm (N2) are not chosen to be equal.

[0029] Alternatively, it can be arranged that the first norm (N1) is L 1Norm. The first norm is chosen in a way that helps scale the sparsity of the output signal of the layer. This is good for compression of neural networks, for example, because a weight with a value of 0 does not contribute to the output value of its layer.

[0030] Thus, a neural network having such layers can be used in a particularly memory-efficient manner, in particular in combination with compression methods.

[0031] In the described variant for the first norm (N1), it can be advantageously provided that the second norm (N2) is L 2 This makes it possible to implement these methods particularly easily.

[0032] In this case, it is particularly advantageous to solve the equation using the deterministic Newton method. .

[0033] Surprisingly, it has been found that the method is particularly efficient if an input signal having many important (ie strongly weighted) features is attached to the input of the scaling layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Subsequently, the embodiments of the present invention will be described in more detail with reference to the attached drawings. In the drawings:

[0035] Figure 1 The structure of an implementation form of the control system is schematically shown;

[0036] Figure 2 Schematically illustrates an embodiment for controlling an at least semi-autonomous robot;

[0037] Figure 3 An embodiment for controlling a production system is schematically shown;

[0038] Figure 4 An embodiment for controlling a personal assistant is schematically shown;

[0039] Figure 5 An exemplary embodiment for controlling an access system is schematically shown;

[0040] Figure 6 An embodiment for controlling a monitoring system is schematically shown;

[0041] Figure 7 An embodiment for controlling a medical imaging system is schematically shown;

[0042] Figure 8 The training system is schematically shown;

[0043] Fig. 9 The structure of a neural network is schematically shown;

[0044] Fig.10 Schematically illustrates information switching within a neural network;

[0045] Fig.11 The implementation form of the training method is shown in a flow chart;

[0046] Fig.12 An implementation form of the method for estimating a gradient is shown in a flow chart;

[0047] Fig.13 An alternative embodiment of a method for estimating a gradient is shown in a flow chart;

[0048] Fig.14 An embodiment of a method for scaling an estimated gradient is shown in a flow chart;

[0049] Fig.15 An embodiment for implementing a scaling layer within a neural network is shown in a flow chart.

[0050] Fig.16 A method for operating a trained neural network is shown in a flow chart. DETAILED DESCRIPTION

[0051] Figure 1 An actuator 10 is shown interacting with a control system 40 in its environment 20. The actuator 10 and the environment 20 are also collectively referred to as an actuator system. Preferably at regular intervals, the state of the actuator system is detected by means of a sensor 30, which can also be provided by a plurality of sensors. A sensor signal S of the sensor 30 (or each sensor signal S in the case of a plurality of sensors) is transmitted to the control system 40. The control system 40 thus receives a series of sensor signals S. The control system 40 determines therefrom a control signal A, which is transmitted to the actuator 10.

[0052] The sensor 30 is any sensor that detects the state of the environment 20 and transmits it as a sensor signal S. The sensor can be, for example, an imaging sensor, in particular an optical sensor, such as an image sensor or a video sensor, or a radar sensor, or an ultrasonic sensor, or a LiDAR sensor. The sensor can also be an acoustic sensor that, for example, receives structure-borne sound or voice signals. Likewise, the sensor can be a position sensor (such as, for example, a GPS) or a kinematic sensor (such as a uniaxial or multi-axial acceleration sensor). Sensors that characterize the orientation of the actuator 10 in the environment 20 (such as a compass) are also possible. Sensors that detect the chemical composition of the environment 20 (such as a lambda sensor) are also possible. Alternatively or in addition, the sensor 30 can also include an information system that determines information about the state of the actuator system, such as, for example, a weather information system that determines the current or future state of the weather in the environment 20.

[0053] The control system 40 receives a series of sensor signals S of the sensor 30 in an optional receiving unit 50, which converts the series of sensor signals S into a series of input signals x, (alternatively, the individual sensor signals S can also be directly received as input signals x). The input signals x can be, for example, fragments or further processed sensor signals S. The input signals x can, for example, comprise image data or images, or can comprise individual frames of a video recording. In other words, the input signals x are determined from the sensor signals S. The input signals x are fed to the neural network 60.

[0054] The neural network 60 is preferably parameterized by parameters θ, which include, for example, weights w, which are stored in a parameter memory P and are provided by the latter.

[0055] The neural network 60 determines the output signal y from the input signal x. Typically, the output signal y encodes information about the classification of the input signal x. The output signal y is supplied to an optional conversion unit 80, which determines therefrom an actuation signal A, which is supplied to the actuator 10 in order to actuate the actuator 10 accordingly.

[0056] The actuator 10 receives the control signal A, is controlled accordingly, and performs a corresponding action. The actuator 10 may include a control logic device (not necessarily structurally integrated) in this case, which determines a second control signal from the control signal A, with which the actuator 10 is then controlled.

[0057] In other embodiments, the control system 40 includes the sensor 30 . In still other embodiments, the control system 40 also includes the actuator 10 as an alternative or in addition.

[0058] In other preferred embodiments, the control system 40 includes one or more processors 45 and at least one machine-readable storage medium 46, on which instructions are stored. When the instructions are executed on the processor 45, the instructions prompt the control system 40 to implement a method for operating the control system 40.

[0059] In an alternative embodiment, a display unit 10 a is provided as an alternative to the actuator 10 or in addition to the actuator 10 .

[0060] Figure 2 An exemplary embodiment is shown in which the control system 40 is used to control an at least semi-autonomous robot, here an at least partially automated motor vehicle 100 .

[0061] The sensor 30 may be a combination of Figure 1 One of the mentioned sensors is preferably one or more video sensors, preferably arranged in the motor vehicle 100 , and / or one or more radar sensors, and / or one or more ultrasonic sensors, and / or one or more lidar sensors, and / or one or more position sensors (e.g. GPS).

[0062] The neural network 60 can detect, for example, objects in the environment of the at least semi-autonomous robot from the input data x. The output signal y can be information that characterizes where objects are located in the environment of the at least semi-autonomous robot. The output signal A can then be determined based on and / or corresponding to this information.

[0063] The actuator 10 preferably arranged in the motor vehicle 100 may be, for example, a brake device, a drive device or a steering device of the motor vehicle 100. The control signal A may then be determined such that one or more actuators 10 are controlled so that, in particular when objects of a certain category, such as pedestrians, the motor vehicle 100, for example, prevents a collision with an object recognized by the neural network 60. In other words, the control signal A may be determined according to the determined category and / or corresponding to the determined category.

[0064] Alternatively, the at least semi-autonomous robot may also be another mobile robot (not shown), for example, such a mobile robot that moves by flying, swimming, diving or walking. The mobile robot may also be, for example, an at least semi-autonomous lawn mower or an at least semi-autonomous cleaning robot. In these cases, the control signal A may also be determined so that: the drive device and / or the steering device of the mobile robot are controlled so that the at least semi-autonomous robot, for example, prevents a collision with an object recognized by the neural network 60.

[0065] In other alternatives, the at least semi-autonomous robot can also be a gardening robot (not shown) which determines the type or state of plants in the environment 20 using an imaging sensor 30 and a neural network 60. The actuator 10 can then be, for example, a chemical applicator. Depending on the determined type or the determined state of the plant, the control signal A can be determined so that an amount of chemical corresponding to the determined type or the determined state is applied.

[0066] In other alternatives, the at least semi-autonomous robot can also be a household appliance (not shown), in particular a washing machine, a stove, an oven, a microwave oven or a dishwasher. Using a sensor 30 (e.g., an optical sensor), the state of an object processed by a household appliance can be detected, for example, in the case of a washing machine, the state of the laundry in the washing machine can be detected. Then, using a neural network 60, the type or state of the object can be determined and characterized by an output signal y. Then, a control signal A can be determined so that the household appliance is controlled according to the determined type or determined state of the object. For example, in the case of a washing machine, the washing machine can be controlled according to what material the laundry is made of. Then, depending on what material the laundry has been determined to be, the control signal A can be selected.

[0067] Figure 3 The following exemplary embodiment is shown in which the control system 40 is used to control a production machine 11 of a production system 200 by controlling an actuator 10 which controls the production machine 11. The production machine 11 may be, for example, a machine for punching, sawing, drilling and / or cutting.

[0068] The sensor 30 may be a combination of Figure 1 One of the mentioned sensors is preferably an optical sensor which detects, for example, a property of the production product 12. It is possible to actuate the actuator 10 which controls the production machine 11 as a function of the determined property of the production product 12, so that the production machine 11 accordingly carries out the subsequent processing steps of the production product 12. It is also possible for the sensor 30 to determine the property of the production product 12 processed by the production machine 11 and to adapt the actuation of the production machine 11 for the subsequent production product accordingly.

[0069] Figure 4 The following embodiment is shown: In the embodiment, the control system 40 is used to control the personal assistant 250. The sensor 30 can be combined with Figure 1 One of the sensors mentioned. Sensor 30 is preferably an acoustic sensor, which receives a speech signal of user 249. Alternatively or additionally, sensor 30 can also be configured to receive an optical signal, for example a video image of a gesture of user 249.

[0070] Based on the signal of the sensor 30, the control system 40 determines the control signal A for the personal assistant 250, for example, in that the neural network performs gesture recognition. The determined control signal A is then transmitted to the personal assistant 250, and the personal assistant is thus controlled accordingly. The determined control signal A can in particular be selected so that it corresponds to the inferred desired control by the user 249. Based on the gesture recognized by the neural network 60, the inferred desired control can be determined. Based on the inferred desired control, the control system 40 can then select the control signal A for transmission to the personal assistant 250 and / or select the control signal A corresponding to the inferred desired control 250 for transmission to the personal assistant.

[0071] The corresponding control may include, for example, personal assistant 250 calling up information from a database and reproducing it in a form acceptable to user 249 .

[0072] Instead of personal assistant 250 , a domestic appliance (not shown), in particular a washing machine, a stove, an oven, a microwave or a dishwasher, can also be provided in order to be controlled accordingly.

[0073] Figure 5 The following embodiment is shown: In the embodiment, the control system 40 is used to control the access system 300. The access system 300 may include a physical access control device, such as a door 401. The sensor 30 may be combined with Figure 1 One of the mentioned sensors is preferably an optical sensor (e.g. for detecting image data or video data), which is set up to detect a face. The detected image can be interpreted by means of the neural network 60. For example, the identity of a person can be determined. The actuator 10 can be a lock, which releases the access control device or does not release the access control device according to the control signal A, for example, opens the door 401 or does not open the door 401. For this purpose, the control signal A can be selected according to the interpretation of the neural network 60, for example, the control signal A can be selected according to the determined identity of the person. Instead of a physical access control device, a logical access control device can also be provided.

[0074] Figure 6 The following embodiment is shown: In the embodiment, the control system 40 is used to control the monitoring system 400. Figure 5 The embodiment shown in FIG. 4 differs in that a display unit 10 a is provided instead of the actuator 10, which is activated by the control system 40. For example, the neural network 60 can determine whether an object recorded by the optical sensor is suspicious, and the activation signal A can then be selected so that the object is displayed in a color-highlighted manner by the display unit 10 a.

[0075] Figure 7 The following embodiment is shown: in the embodiment, the control system 40 is used to control a medical imaging system 500, such as an MRT device, an X-ray device or an ultrasound device. The sensor 30 can be provided, for example, by an imaging sensor, and the display unit 10a is controlled by the control system 40. For example, the neural network 60 can determine whether the area recorded by the imaging sensor is conspicuous, and then select the control signal A so that the display unit 10a displays the area in a color-contrasted manner.

[0076] Figure 8 An exemplary embodiment of a training system 140 for training a neural network 60 by means of a training method is schematically shown.

[0077] The training data unit 150 determines a suitable input signal x to be fed to the neural network 60. For example, the training data unit 150 accesses a computer-implemented database in which a training data set is stored and selects the input signal x, for example randomly, from the training data set. Optionally, the training data unit 150 also determines an expected, or “actual” output signal y associated with the input signal x. T , the output signal y T is supplied to the evaluation unit 180 .

[0078] The artificial neural network x is designed to determine an associated output signal y from an input signal x supplied to it. The output signal y is supplied to an evaluation unit 180 .

[0079] For example, by comparing the output signal y and the expected output signal y T The relevant loss function , the evaluation unit 180 can characterize the performance of the neural network 60. The loss function To optimize the parameter θ.

[0080] In other preferred embodiments, the training system 140 includes one or more processors 145 and at least one machine-readable storage medium 146, on which instructions are stored. When the instructions are executed on the processor 145, the instructions prompt the control system 140 to implement the training method.

[0081] Fig. 9 By way of example, a possible structure of a neural network 60 is shown, which is provided as a neural network in this exemplary embodiment. The neural network comprises a plurality of layers S1, S2, S3, S4, S5 in order to determine an output signal y from an input signal x supplied at the input of the input layer S1, which output signal y is attached to the output of the output layer S5. In this case, each of the layers S1, S2, S3, S4, S5 is designed to determine an (if necessary multidimensional) output signal z1, z2, z4, z5, y from an (if necessary multidimensional) input signal x, z1, z3, z4, z6 attached to the input of the respective layer S1, S2, S3, S4, S5, which output signal z1, z2, z4, z5, y is attached to the output of the respective layer S1, S2, S3, S4, S5. In particular, in image processing, such output signals are also referred to as feature maps (English: “feature maps”). In this case, it is not necessary that the layers S1, S2, S3, S4, S5 are arranged so that all output signals that enter other layers as input signals enter the next layer from the previous layer. Instead, residual connections (English: "Skip connection") or recursive connections are also possible. Of course, it is also possible that the input signal x enters multiple layers of these layers, or that the output signal y of the neural network 60 is composed of output signals of multiple layers.

[0082] The output layer S5 can be provided by an Argmax layer, for example (i.e. a layer that selects from a plurality of inputs having respectively associated input values ​​the identification of the input whose associated input value is the largest among these input values), for example, one or more of the layers S1, S2, S3 can be provided by a convolutional layer.

[0083] Advantageously, the layer S4 is designed as a scaling layer, which is designed such that an input signal (x) attached to the input of the scaling layer (S4) is mapped to an output signal (y) attached to the output of the scaling layer (S4), so that the output signal (y) attached to the output is a controlled ratio of the input signal (x), wherein parameters characterizing the controlled ratio can be fixedly predefined. An exemplary embodiment of a method that can be performed for the scaling layer S4 is described below in conjunction with Fig.15 Be described.

[0084] Fig.10 The schematic diagram illustrates the information switching within the neural network 60. Here, three multidimensional signals within the neural network 60 are schematically shown, namely the input signal x and the subsequent feature maps z1, z2. The input signal x has in this embodiment The spatial resolution of pixels, the first feature map z1 has The spatial resolution of pixels, the second feature map z2 has In this embodiment, the resolution of the second feature map z2 is smaller than the resolution of the input signal x, but this is not absolutely necessary to be the case.

[0085] Furthermore, a feature, for example a pixel (i, j)3 of the second feature map z2 is shown. If, for example, a function of the second feature map z2 determined from the first feature map z1 is shown via a convolutional layer or a fully connected layer (English: "fully connected layer"), it is also possible that multiple features of the first feature map z1 enter into the determination of the value of the feature (i, j)3. However, it is also possible that only a single feature of the first feature map z1 enters into the determination of the value of the feature (i, j)3.

[0086] In this case, "entry" can advantageously be understood to mean that there is a combination of values ​​of the parameters characterizing the function for determining the second characteristic map z2 from the first characteristic map z1 and the values ​​of the first characteristic map z1, such that the value of the feature (i, j)3 is related to the value of the entered feature. The totality of the entered features is Fig.10 The area is called Be.

[0087] For his part, one or more features of the input signal x enter into the determination of each feature (i, j)2 of the region Be. The set of all features of the input signal x that enter into the determination of at least one feature of the feature (i, j)2 of the region Be is called the receptive field rF of the feature (i, j)3. In other words, the receptive field rF of the feature (i, j)3 includes all those features of the input signal x that directly or indirectly (in other words at least indirectly) enter into the determination of the feature (i, j)3, that is, those features whose values ​​can influence the value of the feature (i, j)3.

[0088] Fig.11 The sequence of a method for training neural network 60 according to one embodiment is shown in a flow chart.

[0089] First (1000), a training data set X is provided, wherein the training data set X includes an input signal x i and the output signal yi The pair (x i , y i ). The learning rate η is initialized, for example, to η=1.

[0090] In addition, for example, when the portion of the method used in step 1100 is Fig.12 In the embodiment illustrated in FIG. 1 , the first set G and the second set N are optionally initialized. If part of the method is to be used in step 1100 Fig.13 In the embodiment illustrated in FIG. 1 , the initialization of the first set G and the second set N may be omitted.

[0091] The initialization of the first set G and the second set N can be performed as follows: the first set G is initialized as an empty set, the first set G comprising those pairs (x i , y i ). By assigning all pairs (x i , y i ), initialize the second set N, which includes those pairs (x i , y i ).

[0092] Now (1100), with the training data set X consisting of the input signal x i and the output signal y i The pair (x i , y i ), estimate the characteristic parameters The gradient of the parameter θ is The embodiment of the method combines Fig.12 Or 13 to describe.

[0093] Next (1200), optionally scaling the gradient g is performed. Fig.14 Be described.

[0094] Next (1300), an adaptation of the learning rate η is optionally performed. In this case, the learning rate η can be reduced, for example, by a predeterminable learning rate reduction factor Dη (for example Dη=1 / 10), as long as the number of traversed rounds is divisible by a predeterminable number of rounds (for example 5), (i.e. ).

[0095] Next (1400), the parameters θ are updated with the aid of the determined and possibly scaled gradient g and the learning rate η. For example, by Substitute the parameter θ.

[0096] Now (1500), it is checked with the aid of a predeterminable convergence criterion whether the method has converged. For example, based on the absolute change in the parameter θ (e.g. between the last two rounds), it can be determined whether the convergence criterion is met or not. For example, exactly when L with respect to the change in all parameters θ between the last two rounds is 2 If the norm is smaller than a predeterminable convergence threshold, then the convergence criterion is met.

[0097] If it is determined that the convergence criteria are met, the parameters θ are accepted as learned parameters and the method ends. If not, a return branch is made to step 1100 .

[0098] Fig.12 An exemplary method for determining the gradient g in step 1100 is illustrated in a flow chart.

[0099] First (1110), a predeterminable number bs of pairs (x) of the training data set X are to be extracted (without replacement), i.e., selected. i , y i ), and assign these pairs (x i , y i ). The predeterminable number bs is also called the batch size (English: "batch size"). The batch B is initialized to an empty set.

[0100] To this end, it is checked (1120) whether the batch size bs is larger than the number of pairs (x i , y i ) number.

[0101] If the batch size bs is not larger than the number of pairs (x i , y i ), then randomly select (1130) from the second set N, that is, select bs multiple pairs (x i , y i ), and these pairs (x i , y i ) is added to batch B.

[0102] If the batch size bs is larger than the number of pairs (x i , y i), then extract (1140), that is, select all pairs of the second set N whose number is marked by s, and add all pairs to batch B, and extract from the first set G, that is, select the remaining (that is, bs-s multiple) pairs, and add the remaining pairs to batch B.

[0103] For all parameters θ, it is optionally determined at step (1130) or (1140) whether the parameters θ are to be changed in this training round. For this purpose, for example, the probability of changing the parameters θ of this layer is specified separately for each layer (S1, S2, ..., S6). For example, this probability can be 50% for the first layer (S1) and can be reduced by 10% in the case of each subsequent layer.

[0104] With the aid of the specified respective probabilities, it can then be decided for each of the parameters θ whether or not said parameter is to be changed.

[0105] For each pair (x i , y i ), now (1155) optionally determines whether to enhance the corresponding input signal x i For each corresponding input signal x to be enhanced i , preferably randomly selecting an enhancement function, and applying the enhancement function to the input signal x i Then, the input signal x is enhanced to such an extent (dermassen) i Replace the original input signal x i If the input signal x i is an image signal, the enhancement function can be given, for example, by rotating it by a predeterminable angle.

[0106] Next (1160), for each pair (x i , y i ), select the corresponding (and if necessary enhanced) input signal x i , and supplies the corresponding (and possibly enhanced) input signal x to the neural network 60 i In this case, the parameters θ of the neural network 60 to be transformed are disabled during the determination of the corresponding output signal, for example by temporarily setting them to the value zero. The corresponding output signal y(x i ) and the corresponding pair (x i , y i ) association. According to this pair (x i , y i ) as the desired output signal y T The output signal y(x i) and the corresponding output signal y i , respectively determine the loss function .

[0107] Next (1165), for all pairs (x i , y i ), jointly determine the complete loss function , and for each parameter of the parameters θ not to be transformed, the corresponding component of the gradient g is determined, for example by means of backpropagation. For each parameter of the parameters θ to be transformed, the corresponding component of the gradient g is set to zero.

[0108] Now, a check ( 1170 ) is performed to determine whether the batch size bs is larger than the number of pairs (x i , y i ) number.

[0109] If it is found that the batch size bs is not larger than the number of pairs (x i , y i ), then (1180) all pairs of (x i , y i ) is added to the first set G, and all pairs (x) of this batch B are removed from the second set N. i ,y i ). Now it is checked (1185) whether the second set N is empty. If the second set N is empty, a new round (1186) begins. For this purpose, the first set G is reinitialized to an empty set and the second set N is newly initialized in such a way that all pairs (x i , y i ) is reallocated to the second set N, and branches to step (1200). If the second set N is not empty, directly branch to step (1200).

[0110] If it is found that the batch size bs is larger than the number of pairs (x i , y i ), then the first set G is newly initialized (1190) in the following way: all pairs (x i , y i ) is assigned to the first set G; the second set N is newly initialized in the following way: all pairs (x i , y i ) is redistributed to the second set N, and then the pairs (x i , y i). Then, a new round begins and is branched to step (1200). This ends this part of the method.

[0111] Fig.13 A further exemplary method for determining the gradient g in step 1100 is illustrated in a flowchart. First, the parameters of the method are initialized (1111). In the following, the mathematical space of the parameters θ is denoted by W. That is, if the parameters θ include np multiple individual parameters, then the space W is a space of np dimensions, for example W= The iteration counter n is initialized to the value n=0, and the first variable m1 is then set to (that is, it is set to a vector of np dimensions), and the second variable is set to (That is, it is set to a matrix of np x np dimensions).

[0112] Next (1121), randomly select pairs (x i , y i ), and if necessary, enhance the i ,y i ). This can happen, for example, so that for the training dataset X, the pair (x i , y i ) for each input signal x i , the number is determined to be Possible enhancements , and the position variable

[0113]

[0114] With each pair (x i , y i ) is associated. Then, if we draw random numbers uniformly , then select the position variable p that satisfies the following inequality chain i :

[0115]

[0116] The index i of the selected pair (x i , y i ), can be randomly selected from possible enhancements Extract the input variable x from the set i Enhancement , and the input variable x i Enhancement Can be applied to the input variable x i , that is to say, through Replace the selected pair (x i , yi ).

[0117] Input signal x i is sent to the neural network 60. i , y i ) as the desired output signal y T The corresponding output signal y(x i ) and the output signal y i , determine the corresponding loss function For the parameter θ, the gradient d associated with it is determined, for example, by back propagation, that is, .

[0118] Next (1131), the iteration counter n, the first variable m1 and the second variable m2 are updated as follows:

[0119] .

[0120] Following (1141), the components of the covariance matrix C Provided as

[0121] .

[0122] Thus, the dot product S is formed using the (vector-valued) first variable m1, i.e.

[0123] .

[0124] It should be understood that in order to determine the dot product S with sufficient accuracy using equation (8), it is not necessary to have both the covariance matrix C and the inverse covariance matrix C -1 It is more memory efficient to determine the next necessary entries of the covariance matrix C during the analysis of equation (8) .

[0125] Next, check (1151) whether the dot product S satisfies the following inequality:

[0126]

[0127] Therein, λ is a predeterminable threshold value which corresponds to the confidence level.

[0128] If the inequality is satisfied, the current value of the first variable m1 is accepted as the estimated gradient g, and a return branch is taken to step ( 1200 ).

[0129] If the inequality is not satisfied, a branch can be returned to step (1121). Alternatively, a check (1171) can also be performed to determine whether the iteration counter n has reached a predeterminable maximum iteration value n. maxIf this is not the case, a return branch is taken to step (1121), otherwise the zero vector is accepted (1181) as the estimated gradient g. , and returns to step (1200). This ends this part of the method.

[0130] This method realizes that m1 corresponds to the determined gradient d in the extracted pair (x i , y i ), and m2 corresponds to the matrix product of the determined gradient d In the extracted pairs (x i , y i ) is the arithmetic mean of .

[0131] Fig.14 An implementation form of the method for scaling the gradient g in step (1200) is shown. Label each component of the gradient g, where Indicate the layer corresponding to the parameter θ, and Marked in The corresponding parameter θ within the layer. If the neural network is as in Fig.10 As shown in the diagram, it is constructed for The layer processes the corresponding feature map For multidimensional input data x, the number l is advantageously associated with the feature in the feature map through the corresponding parameter θ The position in is given.

[0132] Now (1220), for each component of the gradient g , determine the scaling factor For example, the scaling factor Through the The size of the receptive field rF of the l corresponding features of the feature map of the layer is given by the scaling factor Alternatively, it is also possible to It is given by the ratio of the resolution (i.e. the number of features) of the layer compared to the input layer.

[0133] Next (1220), using the scaling factor To scale each component of the gradient g ,that is

[0134] .

[0135] If the scaling factor Overfitting of the parameters θ can be avoided particularly effectively by giving the size of the receptive field rF. Given by the ratio of the resolution φ, this is a particularly efficient approximation of the size of the receptive field rF.

[0136] Fig.15 The diagram illustrates an implementation form of the method implemented by the scaling layer S4.

[0137] The scaling layer S4 is set up to realize the projection of the input signal x attached to the input of the scaling layer S4 onto a sphere with a radius ρ and a center c. This is characterized by a first norm N1(yc) that determines the distance between the center c and the output signal y attached to the output of the scaling layer S4, and a second norm N2(xy) that determines the distance between the input signal x attached to the input of the scaling layer S4 and the output signal y attached to the output of the scaling layer S4. In other words, the output signal y attached to the output of the scaling layer S4 solves the following equation.

[0138]

[0139] Fig.15 a) illustrates a particularly efficient first implementation form for the case where the first norm N1 and the second norm N2 are equal. The first norm and the second norm are hereinafter referred to as To indicate.

[0140] First (2000), an input signal x, a center parameter c and a radius parameter ρ are provided which are appended to the input of a scaling layer S4.

[0141] Next (2100), the output signal y added to the output of the scaling layer S4 is determined as

[0142] .

[0143] This ends this part of the method.

[0144] Fig.15 b) and 15 c) illustrate embodiments for particularly advantageously selected combinations of the first norm N1 and the second norm N2.

[0145] Fig.15 b) A second implementation form is illustrated for the case where, in condition (12) to be satisfied, the first norm By maximum norm is given by, and the second norm By 2-norm To give. To compute the combination of norms particularly efficiently.

[0146] First (3000), similar to step (2000), an input signal x, a center parameter c and a radius parameter ρ are provided which are appended to the input of the scaling layer S4.

[0147] Next (3100), the component y of the output signal y added to the output of the scaling layer S4 i Determined to be

[0148] ,

[0149] Here, i indicates the quantity.

[0150] The method is particularly computationally efficient. This concludes this part of the method.

[0151] Fig.15 c) A third implementation form is illustrated for the case where, in condition (12) to be satisfied, the first norm By 1-norm is given by, and the second norm By 2-norm The combination of these norms results in that as many small components as possible are set to the value zero in the input signal x appended to the input of the scaling layer S4.

[0152] First (4000), similar to step (2000), an input signal x, a center parameter c and a radius parameter ρ are provided which are appended to the input of the scaling layer S4.

[0153] Next (4100), the symbol variable Determined to be

[0154]

[0155] And the component x of the input signal x added to the input of the scaling layer S4 i pass

[0156]

[0157] The auxiliary parameter γ is initialized to the value zero.

[0158] Next (4200), the set N is determined to be , and the spacing measure (Abstandsmass) .

[0159] Then (4300) checks whether the inequality is satisfied.

[0160] .

[0161] If this is the case (4400), then by

[0162]

[0163] Substitute the auxiliary parameter γ and return to step (4200).

[0164] If inequality (16) (4500) is not satisfied, then the component y of the output signal y attached to the output of the scaling layer S4 i Determined as

[0165]

[0166] In this case, the notation In a common way it means

[0167] .

[0168] This part of the method ends here. The method corresponds to Newton's method and is particularly computationally efficient, in particular when a number of the components of the input signal x attached to the input of the scaling layer S4 are important.

[0169] Fig.16 The diagram illustrates an embodiment of a method for operating a neural network 60. First (5000), the neural network is trained using one of the described methods. Next (5100), the control system 40 is operated as described using the neural network 60 trained in this way. The method ends here.

[0170] It should be understood that the neural network is not limited to a feedforward neural network (English: "feedforward neural network"), but the present invention can be applied in the same way to any type of neural network, in particular to a recursive network, a convolutional network (English: "convolutional neural network"), an autoencoder, a Boltzmann machine, a perceptron or a capsule network (English: "Capsule Neural Network").

[0171] The term "computer" includes any device for executing predefinable calculation specifications. These calculation specifications may exist in the form of software, in the form of hardware, or in a hybrid form consisting of software and hardware.

[0172] It should also be understood that these methods may not be implemented entirely in software as described, but may also be implemented in hardware or in a hybrid form consisting of software and hardware.

Claims

1. A method for classifying an input signal by means of a neural network (60), wherein the input signal is determined based on an output signal of a sensor (30), wherein the neural network (60) comprises a scaling layer, wherein the scaling layer maps an input signal x attached to the input of the scaling layer to an output signal y attached to the output of the scaling layer, so that the mapping corresponds to a projection of the input signal x attached to the input of the scaling layer onto a predeterminable value range, characterized in that The parameters ρ, c that characterize the mapping can be predetermined, and the mapping is obtained by the equation with a first norm N1 and a second norm N2 is given by, wherein ρ is the radius of the predeterminable value range and c is the center of the predeterminable value range, wherein the first norm N1 determines the distance of the center from the output signal y attached to the output of the scaling layer, and the second norm N2 determines the distance of the input signal x attached to the input of the scaling layer from the output signal y attached to the output of the scaling layer, The neural network (60) is used to determine the type or state of the object and to characterize the type or state of the object by the output signal y, thereby classifying the input signal of the neural network (60), and the sensor is an imaging sensor, a radar sensor, an acoustic sensor, a position sensor, or a kinematic sensor.

2. The method according to claim 1, wherein the control signal is determined as a function of an output signal of the neural network (60).

3. The method according to claim 2, wherein: The actuator (10) is controlled according to the control signal.

4. The method according to any one of claims 1 to 3, wherein: The first norm N1 and the second norm N2 are selected to be equal.

5. The method according to claim 1, wherein: The first norm N1 is L ∞ Norm.

6. The method according to claim 1, wherein: The first norm N1 is L 1 Norm.

7. The method according to claim 5 or 6, wherein: The second norm N2 is L 2 -norm.

8. The method according to claim 7, wherein: Using the deterministic Newton method, we solve the equation 9. The method according to any one of claims 1 to 3, wherein: Firstly, in a training phase, a predefinable parameter ρ, c is adapted according to a method for training the neural network (60), wherein during training, the predefinable parameter ρ, c is adapted according to an output signal of the neural network (60) when an input signal of the neural network (60) and an associated expected output signal are supplied, wherein the predefinable parameter is adapted according to a determined gradient, which is related to the output signal of the neural network (60) and the associated expected output signal.

10. The method according to any one of claims 1 to 3, wherein: The imaging sensor is an optical sensor.

11. The method according to claim 10, wherein: The optical sensor is an image sensor or a video sensor.

12. The method according to any one of claims 1 to 3, wherein: The sensor is a lidar sensor.

13. The method according to any one of claims 1 to 3, wherein: The acoustic sensor is an ultrasonic sensor.

14. A computer program product comprising instructions, which, when the program is executed by a computer, cause the computer to implement the method according to any one of claims 1 to 13.

15. A machine-readable storage medium (46, 146) having a computer program stored thereon, which, when executed by a computer, implements the method according to any one of claims 1 to 13.

16. An actuator control system (40) which is designed to carry out the method according to any one of claims 2 to 13.

Images