Apparatus and method for training a neural network

Abstract

Apparatus and method for training a neural network. A method for training a neural network (60), the neural network (60) comprising a first layer ( ), the first layer ( ) comprising a plurality of filters to provide a first layer output (z) comprising a plurality of feature maps, the training of the classifier (60) comprising the steps of: receiving (901) a first layer input (i) in the first layer ( ) by a preceding layer, the first layer input (i) being based on an input signal (x); determining (902) a first layer output (z) based on the first layer input (i) and a plurality of parameters (w) of the first layer ( ); determining (903) a first layer loss value (l) based on the first layer output (z), the first layer loss value (l) characterizing a degree of correlation between the feature maps of the first layer output (z), the first layer loss value (l) being obtained in an unsupervised manner; training (904) the neural network (60), the training comprising an adaptation of the parameters (w) of the first layer ( ), the adaptation being based on the first layer loss value (l).

Description

Technical Field

[0001] The present invention relates to methods for training neural networks, methods for using neural networks, methods for operating actuators, computer programs and machine-readable storage media, neural networks, control systems and training systems. Background Technology

[0002] Prior Technology EP 1 752 362 discloses a method for training neural networks in an unsupervised manner.

[0003] Advantages of the present invention Machine learning-based classifiers (e.g., neural networks) extract knowledge from image data to learn classification functions. To this end, they include multiple parameters that can be adapted to influence classification behavior. For the best-performing classifiers, the number of parameters can easily exceed 10. 7 Here, performance can be understood as a measure of how well a neural network can predict the expected class.

[0004] Especially in safety-critical applications, achieving the highest possible performance is crucial. For example, consider using neural networks as part of the perception pipeline for at least partially autonomous vehicles to detect pedestrians near the vehicle. In this scenario, information about detected pedestrians can be used, for example, to calculate safe trajectories for navigating the vehicle's environment. If a pedestrian is incorrectly classified, for example, missed by the classifier, it could cause the system to plan a trajectory directly toward the pedestrian. This could, in turn, lead to serious injury and / or death.

[0005] The advantage of the proposed method is that it regularizes the neural network during training, which leads to improved classifier performance.

[0006] Regularization is beneficial for training because it mitigates overfitting in neural networks. Otherwise, a large number of parameters would make the neural network susceptible to overfitting, i.e., it would try to learn certain data points. Consequently, overfitting erodes the performance of the classifier.

[0007] Another advantage of this method is that it imposes no constraints, unlike processes-based models that, for example, require the use of specific layers in a predefined structure. This allows the classifier to be freely adapted to the characteristics of the training data at hand, which in turn leads to better classifier performance. Summary of the Invention

[0008] In a first aspect, the present invention relates to a method for training a neural network, wherein a classifier includes at least one first layer, wherein the first layer includes a plurality of filters to provide a first layer output including a plurality of feature maps, and further wherein training the classifier includes the following steps: • Receive the first layer input from the previous layer in the first layer, where the first layer input is based on the input signal; The first-layer output is determined based on the first-layer input and multiple parameters of the first layer. • The first-layer loss value is determined based on the first-layer output, where the first-layer loss value is a measure of the correlation between the feature maps of the first-layer output, and further wherein the first-layer loss value is obtained in an unsupervised manner; • Train the neural network, where training includes adapting the parameters of the first layer, where the adaptation is based on the loss value of the first layer.

[0009] A neural network is understood to consist of one or more linear and / or nonlinear units that are used to predict the output signal of a received input signal.

[0010] The layers are organized in a topology, where the order of the layers is determined by the information flow. If an initial layer provides input to another layer, that other layer is considered to follow the initial layer, and the initial layer precedes that other layer.

[0011] The output provided by a layer can be based on the current values ​​of the corresponding parameter set. The parameters of a layer are understood as parameters belonging to the classifier.

[0012] The advantage of this layered design is that different layers provide increasingly abstract representations of the input signal. It is known from the literature that layers appearing early in a neural network typically extract low-level representations of the input signal, which can encode information such as edges in the case of image data, for example, while layers appearing later in the classifier can extract high-level representations encoding information about, for example, objects or parts of objects.

[0013] When provided with an input signal, each layer of the classifier can provide a layer output to one or more other layers of the classifier, where the layer output can include multiple feature maps. A feature map can be viewed as a representation of the input to a layer, obtained using some form of transformation. For example, in a convolutional layer, feature maps are obtained by applying one or more filters with predefined height and width to the layer's input. The output of such a convolutional layer is as many feature maps as there are filters present in that layer. Preferably, these feature maps can be organized as tensors with the same height and width as the feature maps, and the number of outputs, i.e., the number of feature maps, determines the depth of the tensor. This tensor can be used as the output of the convolutional layer. Each filter can include multiple parameters that can be adapted during training.

[0014] Generally, neural networks can receive data from various modalities as input signals, especially sensor image data provided by video, radar, LiDAR, and / or ultrasound sensors, as well as images provided by thermal cameras. Input signals can also include audio data, for example, in the form of quantized audio signals or spectral images, provided by one or more microphones.

[0015] Data contained in the input signal can also be generated synthetically. For image data, this can be achieved, for example, by using computer-based methods for creating virtual environments in a computer and rendering images based on, for example, a physical model. Alternatively, generative machine learning models (e.g., generative adversarial networks) or process-based models can be used to obtain synthetic images.

[0016] For audio data, synthetic data can be generated by combining different audio signals in a computer. Alternatively, it is conceivable that audio data can be synthesized using machine learning methods such as WaveNet.

[0017] The data contained in the input signal can be provided directly by sensors and / or synthesis methods. Alternatively, the data can also be provided by some form of storage unit, which includes previously recorded or synthesized data.

[0018] The input signal can also include multiple input data, which are also called data batches. In this case, the classifier can predict the output signal for each data point in the batch.

[0019] The output of a classifier can be viewed as a prediction about the input signal based on the classifier's knowledge, which is encoded through the classifier's parameters.

[0020] In the context of this invention, a neural network is understood to include a first layer capable of being trained in an unsupervised manner. For this purpose, the first layer computes a first-layer loss value representing the correlation between feature maps characterizing the outputs of the first layer. The outputs of the first layer can be obtained from the inputs of the first layer, for example, through a discrete convolution operation. In this case, the parameters of the first layer can be used as weights for filters in the convolution operation.

[0021] Training the first layer based on the first-layer loss can be performed without backpropagation, as this is common practice for training classifiers, such as neural networks. Instead, the weights can be updated by running, for example, a gradient descent algorithm using the first-layer loss. To do this, the gradient of the first-layer loss with respect to the first-layer weights can be calculated.

[0022] When trained using the method described in the first aspect of the invention, the neural network is able to extract information from the supplied training data (i.e., the data used for training) in an unsupervised manner. The term unsupervised can be understood as a classifier that does not require additional information from humans, such as in the form of annotations (also called labels).

[0023] This is advantageous because acquiring labels is a very time-consuming task. To obtain reliable labels, human experts must allocate them to the training data. Given a fixed amount of time, it's possible to collect more data simply by acquiring it, rather than having to acquire and label the same amount of data. Since the training data is used to "teach" the classifier about a problem, more data allows for more learning on that problem. This leads to increased performance in the classifier.

[0024] For example, consider the case of an autonomous vehicle where a classifier is used as part of a perception pipeline to detect pedestrians around the vehicle. The perception pipeline can include sensors and software components. While sensors can be configured to sense information about the vehicle's environment, the software components can infer higher-level information, such as the location of pedestrians in the sensed information. This higher-level information can be used, for example, to plan a safe route without colliding with any pedestrians. It can be further envisioned that the training data for this pedestrian detector is obtained from a fleet of vehicles that collect data from suitable sensing devices such as cameras, radar, or LiDAR.

[0025] It's conceivable that there are a massive number of rare pedestrians in the data, such as pedestrians dressed in theatrical costumes. While these pedestrians may be included in the data, it would be too time-consuming for humans to review all the data and search for rare pedestrians.

[0026] The current approach for situations like the one in the example above is to sample a random subset of the data, which is then designated for labeling. This data is then later used to train, for example, a pedestrian detector.

[0027] However, this method of collecting training data is very likely to miss a considerable number of rare events because the subset of data to be labeled is usually small compared to the entire dataset of recorded data.

[0028] While it's conceivable that the entire dataset could be used for unsupervised training of the pedestrian detector, the detector could then be further trained with a potentially limited amount of training data. This allows the pedestrian detector to learn information from unlabeled but valuable data, as it's far more likely to cover rare events from a dataset of fleet records than from a smaller training dataset. Consequently, this improves the performance of the pedestrian detector.

[0029] Another advantage of the proposed training method is that it can train neural networks to be fine-tuned for downstream tasks while incorporating information from unlabeled data.

[0030] The advantage of learning independent feature maps in a classifier is that the classifier can infer high-level concepts from the data without being given explicit labels. For example, it is known that independent component analysis can learn high-level concepts, such as objects, from input signals. It can be seen that the first aspect includes a nonlinear form of independent component analysis that serves as the training method for the classifier. This allows the classifier to learn nonlinear relationships in the data, which in turn leads to learning highly discriminative features contained in the elements of the input signal.

[0031] This is particularly advantageous if the classifier trained in this way is fine-tuned for a classification task, because the classifier's output layer already outputs highly discriminative features before fine-tuning. This reduces the time required to fine-tune the classifier, as the classification task becomes much easier to learn. Furthermore, this saves computer memory used for training and reduces the number of necessary computations, which in turn reduces energy consumption.

[0032] It should be understood that reducing the correlation between feature maps in the first layer output can be achieved by reducing the first layer loss value.

[0033] In another aspect of the invention, it can be further envisioned that training the neural network further includes the following steps: • Determine the output signal from the neural network, where the output signal depends on the output of the first layer, and the output signal characterizes the classification of the input signal; • Determine the classification loss value based on the output signal; • Train the neural network, where training includes adapting the parameters of the first layer, where the adaptation is based on the classification loss.

[0034] Training a neural network in this way enables it to be used as a classifier. The term classifier can be understood as a device configured to determine one or more class labels for an input signal or a portion thereof.

[0035] The results can then be used to control devices. For example, a classifier can be used as part of the perception pipeline of a vehicle that is at least partially autonomous. In this scenario, a classifier can be used, for example, to detect whether the vehicle is currently in an urban community, a rural area, or a highway. When the vehicle is operating at least partially autonomously, this information can then be used to activate and / or deactivate certain driving functions, driving maneuvers, and / or speed limits.

[0036] Generally, a classifier can determine class labels for different categories. For example, it can determine whether an object exists at a certain location in an image. This method is called object detection. Using examples of at least partially autonomous vehicles, object detection can be used to detect other road users, particularly pedestrians, other vehicles, and cyclists, in order to, for example, plan collision-free trajectories across the environment.

[0037] Another common task that classifiers can be used for is semantic segmentation. Here, the classifier assigns a class label to each element of the input signal, such as each pixel of an input image. Using the example from above, the classifier can be used to classify each pixel of a camera image as belonging to either drivable or non-drivable space. This information can be used in lieu of or in addition to information about other road users to determine a safe trajectory across the environment.

[0038] The output signal of a classifier can be viewed as a prediction based on the input signal. If the model is inadequate for the given data, the prediction may be incorrect. In this sense, the performance of a classifier can be understood as a measure of how well the classifier can predict the desired class label. If the classification output is used as part of a control device, achieving the highest possible performance is of great importance, as incorrect classification can cause unwanted and / or unsafe behavior of the device.

[0039] The advantage of training a neural network with an additional classification loss is that it can be used as a classifier while performing regularized training. Regularization is known to increase the performance of classifiers. Therefore, this method increases the performance of the neural network when used as a classifier compared to simply training the neural network using the training data.

[0040] This is advantageous because high performance leads to better and / or safer device behavior based on classifier-based classification of device actions.

[0041] For example, consider an autonomous vehicle where a classifier is used to detect pedestrians around the vehicle. Information about detected pedestrians can be used, for example, to plan a collision-free trajectory. If the classifier is not performing well enough, it might miss pedestrians until the vehicle is close enough to them. Then, if the classifier does detect a pedestrian, it will initiate emergency braking to avoid a collision.

[0042] If the classifier is configured and / or trained according to the first aspect, performance increases, and pedestrians can be detected. This, in turn, enables the vehicle to plan a collision-free route without requiring emergency braking. Route planning in this manner allows for reduced emergency maneuvers such as emergency braking, which in turn reduces wear on vehicle components such as brakes.

[0043] It should be understood that the classification performance of a classifier can be increased by reducing the classification loss value.

[0044] In another aspect of the invention, it is conceivable that the first layer loss value is determined based on the change from the distribution of the first layer input to the desired distribution of the first layer output.

[0045] The input to the first layer can be viewed as a concrete realization of a first random variable. This random variable can be transformed into a second random variable with certain expected properties. In this case, the output of the first layer will then follow the distribution of the second random variable and thus obtain the expected properties. For example, the expected independence between feature maps can be an expected property of the second random variable.

[0046] Therefore, the advantage of this method is that it allows the first layer output to acquire certain desirable properties, such as independence between feature maps. This acts as regularization when used to train a neural network for a specific classification task.

[0047] Another advantage is that the first layer loss can be used as an independent loss to train the neural network. This allows for the acquisition of a pre-trained neural network. This pre-trained neural network can then be used as a starting point for training a classifier. With the classifier well initialized from the pre-training, the trained classifier converges at a much higher rate. This allows the classifier to achieve higher performance with less training time.

[0048] In another aspect, it can be envisioned that the method for training the neural network further includes the following steps: • Delete the first layer ( ) filter Reduction can be understood as removing certain filters from each layer of a neural network. This has the effect of reducing the number of parameters in the neural network. If applied during training, this acts as regularization. Furthermore, removing a number of parameters reduces the amount of memory required to store the neural network. Additionally, with a reduced number of parameters, the neural network requires fewer computational operations to obtain the output signal.

[0049] Consequently, this results in the neural network consuming less energy and being able to provide output signals more quickly. For training, obtaining output signals more quickly can be seen as being able to process more training data in a given amount of time. This, in turn, allows the neural network to extract more information from the training data, thus increasing its performance.

[0050] In another respect, it can be envisioned that the first-layer loss value represents the factoring of the distribution of the first-layer output between feature maps.

[0051] The advantage of this extension is that it allows for an efficient way to impose independence between feature maps. For example, one can imagine that the outputs of the first layer are trained such that their distribution is a multivariate standard normal distribution. Thus, due to the fact that the multivariate standard normal distribution factorizes random variables, the independence between feature maps can be increased through construction, in which case the random variables can be understood as feature maps of the first layer outputs.

[0052] In another aspect, it can be envisioned that, in the first layer, the loss value is determined according to the following formula: , , in, It is the number of feature maps in the first layer output. A j , and b j It is a predefined value, and It is the first j Each feature map.

[0053] This has the advantage that reducing the loss value is equivalent to reducing the correlation between individual feature maps in the first layer output. Furthermore, if the first layer includes bijective transformations, as is the case, for example, with convolutional and fully connected layers, then the first layer can be trained independently without having to backpropagate the error gradient through the entire neural network. This effectively reduces the amount of memory required to train the classifier, since each first layer can be trained separately.

[0054] If the neural network is used for classification, this layer-by-layer training can be applied in a pre-training step before fine-tuning the neural network with a labeled training dataset. This pre-training effectively reduces the overall training time of the system, which in turn saves memory and energy.

[0055] In another aspect, it can be envisioned that during the training of the classifier, adaptation... A i and / or b i and / or .

[0056] Allowing these parameters to be trained enables the neural network to learn, and even further, the characteristics of the provided data. This leads to improvements in the predictive performance of the neural network.

[0057] In another aspect, it can be envisioned that the step of determining the first layer output includes padding the first layer input such that the size of the first layer output matches the size of the first layer input along its height and / or width and / or depth.

[0058] Generally, the output of the first layer does not need to have the same shape—i.e., height, width, and depth—as the input. However, if layer-by-layer training should be applied, the projection from the input to the output can preferably be chosen as bijective. If, for the first layer, the output shape does not match the input shape, the first layer input can therefore be padded along the necessary dimensions to obtain an output of equal size to the original input.

[0059] The advantage of this method is that it reduces the constraints on the architecture, specifically the layer configuration of the neural network. This, in turn, allows the architecture to be adapted to the characteristics of the data used for training. This leads to increased performance because the architecture can be tuned relative to the data. Attached Figure Description

[0060] Embodiments of the invention will be discussed in more detail with reference to the following figures. The figures illustrate: Figure 1 A control system, which includes a classifier that controls actuators in its environment; Figure 2 A control system for controlling at least partially autonomous robots; Figure 3 Control systems for manufacturing machines; Figure 4 Control system for an automated personal assistant; Figure 5 A control system that controls access control systems; Figure 6 Control system for monitoring systems; Figure 7 Control system for the imaging system; Figure 8 The first layer of the classifier; Figure 9 A flowchart of the training method implemented by the training system. Detailed Implementation

[0061] Figure 1 The diagram shows an embodiment of an actuator (10) in its environment (20). The actuator (10) interacts with a control system (40). The actuator (10) and its environment (20) will be collectively referred to as the actuator system. At preferably uniformly spaced time points, a sensor (30) senses the condition of the actuator system. The sensor (30) may include a plurality of sensors. Preferably, the sensor (30) is an optical sensor that acquires an image of the environment (20). The output signal (S) of the sensor (30) (or, in the case where the sensor (30) includes a plurality of sensors, the output signal (S) of each sensor) is transmitted to the control system (40), which encodes the sensed condition.

[0062] Therefore, the control system (40) receives a stream of sensor signals (S). It then calculates a series of actuator control commands (A) based on the sensor signal stream S, and then transmits the series of actuator control commands (A) to the actuator (10).

[0063] The control system (40) receives a stream of sensor signals (S) from the sensor (30) in an optional receiving unit (50). The receiving unit (50) transforms the sensor signals (S) into an input signal (x). Alternatively, without a receiving unit (50), each sensor signal (S) can be directly taken as the input signal (x). For example, the input signal (x) can be given as an extract from the sensor signals (S). Alternatively, the sensor signals (S) can be processed to generate the input signal (x). The input signal (x) includes image data corresponding to the image recorded by the sensor (30). In other words, the input signal (x) is provided based on the sensor signals (S).

[0064] The input signal (x) is then passed to the neural network (60).

[0065] The neural network (60) is stored in the parameter storage device ( The parameters provided by it () ) to parameterize.

[0066] The neural network (60) determines an output signal (y) from the input signal (x). The output signal (y) includes information that assigns one or more labels to the input signal (x). The output signal (y) is transmitted to an optional conversion unit (80), which converts the output signal (y) into a control command (A). The actuator control command (A) is then transmitted to the actuator (10) for corresponding control of the actuator (10). Alternatively, the output signal (y) can be directly taken as the actuator control command (A).

[0067] The actuator (10) receives an actuator control command (A), is controlled accordingly, and performs an action corresponding to the actuator control command (A). The actuator (10) may include control logic that transforms the actuator control command (A) into another control command, which is then used to control the actuator (10).

[0068] In another embodiment, the control system (40) may include a sensor (30). In even another embodiment, the control system (40) may alternatively or additionally include an actuator (10).

[0069] In one embodiment, the neural network (60) can be designed to identify lanes on the road ahead, for example by classifying the road surface and markings on the road, and identifying lanes as patches on the road surface between the markings. Based on the output of the navigation system, a suitable lane for tracking the selected path can then be selected, and depending on the current lane and the target lane, it can then be determined whether the vehicle (100) should change lanes or remain in the current lane. The actuator control command (A) can then be calculated, for example, by retrieving predefined motion patterns from a database corresponding to the identified actions.

[0070] Similarly, when identifying road signs or traffic lights, depending on the type of road sign or the state of the traffic light, the corresponding constraints for possible motion modes of the vehicle (100) can be retrieved from, for example, a database, the future path of the vehicle (100) commensurate with the constraints can be calculated, and an actuator control command (A) can be calculated to turn the vehicle (100) in order to execute the trajectory.

[0071] Similarly, when identifying pedestrians and / or vehicles, the expected future behavior of said pedestrians and / or vehicles can be estimated, and based on the estimated future behavior, a trajectory can then be selected to avoid collisions with the identified pedestrians and / or vehicles, and an actuator control command (A) can be calculated to steer the vehicle (100) to execute said trajectory.

[0072] In yet another embodiment, it is conceivable that the control system (40) controls the display (10a) instead of the actuator (10).

[0073] Furthermore, the control system (40) may include a processor (45) (or multiple processors) and at least one machine-readable storage medium (46) on which instructions are stored, which, if executed, cause the control system (40) to perform a method according to one aspect of the invention.

[0074] Figure 2 An embodiment is shown in which the control system (40) is used to control a robot that is at least partially autonomous, such as a vehicle (100) that is at least partially autonomous.

[0075] The sensor (30) may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more location sensors (e.g., GPS). Some or all of these sensors are preferably, but not necessarily, integrated into the vehicle (100).

[0076] Alternatively or additionally, the sensor (30) may include an information system for determining the state of the actuator system. An example of such an information system is a weather information system for determining the current or future state of the weather in the environment (20).

[0077] For example, by using an input signal (x), the neural network (60) can detect objects in the vicinity of at least a partially autonomous robot. The output signal (y) may include information characterizing where the object is located in the vicinity of the at least partially autonomous robot. An actuator control command (A) can then be determined based on this information, for example, to avoid a collision with the detected object.

[0078] The actuators (10) preferably integrated in the vehicle (100) can be provided by the vehicle 100's brakes, propulsion system, engine, drivetrain, or steering system. An actuator control command (A) can be determined to control one or more actuators (10) so that the vehicle (100) avoids collision with the detected object. The detected objects can also be classified according to what a classifier (60) deems most likely to be (e.g., a pedestrian or a tree), and the actuator control command (A) can be determined based on the classification.

[0079] In another embodiment, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving, or walking. This mobile robot may, in particular, be a lawnmower that is at least partially autonomous, or a cleaning robot that is at least partially autonomous. In all the above embodiments, actuator command control (A) may be determined such that the propulsion unit and / or steering device and / or brakes of the mobile robot are controlled so that the mobile robot can avoid collisions with the identified object.

[0080] In another embodiment, at least partially autonomous robotic operation may be provided by a gardening robot (not shown) that uses sensors (30), preferably optical sensors, to determine the state of the plants in the environment (20). Actuators (10) may control nozzles and / or cutting devices, such as blades, for spraying liquid. Depending on the identified variety and / or identified state of the plant, actuator control commands (A) may be determined to cause the actuators (10) to spray an appropriate amount of suitable liquid onto the plant and / or cut the plant.

[0081] In even further embodiments, at least partially autonomous robots may be provided by household appliances (not shown), such as washing machines, stoves, ovens, microwave ovens, or dishwashers. Sensors (30), such as optical sensors, may detect the state of objects to be processed by the household appliance. For example, in the case of a washing machine, sensor (30) may detect the state of the clothes inside the washing machine. Actuator control commands (A) may then be determined based on the detected material of the clothes.

[0082] Figure 3 The embodiment shown is a control system (40) used to control a manufacturing machine (11), such as a stamping tool, cutting tool, or gun drill of a manufacturing system (200), for example, as part of a production line. The control system (40) controls an actuator (10), which in turn controls the manufacturing machine (11).

[0083] The sensor (30) can be provided by an optical sensor that captures, for example, the properties of the manufactured product (12). A classifier (60) can determine the state of the manufactured product (12) from these captured properties. Then, the actuator (10) controlling the manufacturing machine (11) can be controlled depending on the determined state of the manufactured product (12) for subsequent manufacturing steps of the manufactured product (12). Alternatively, it can be envisioned that the actuator (10) is controlled depending on the determined state of the manufactured product (12) during subsequent manufacturing of the manufactured product (12).

[0084] Figure 4 The illustration shows an embodiment in which a control system (40) controls an automated personal assistant (250). The sensor (30) may be an optical sensor, for example, for receiving video images of gestures from the user (249). Alternatively, the sensor (30) may also be an audio sensor, for example, for receiving voice commands from the user (249).

[0085] The control system (40) then determines an actuator control command (A) for controlling the automated personal assistant (250). The actuator control command (A) is determined based on sensor signals (S) from the sensor (30). The sensor signals (S) are transmitted to the control system (40). For example, a neural network (60) can be configured to, for example, implement a gesture recognition algorithm to identify gestures made by the user (249). The control system (40) can then determine the actuator control command (A) for transmission to the automated personal assistant (250). It then transmits the actuator control command (A) to the automated personal assistant (250).

[0086] For example, an actuator control command (A) can be determined based on the identified user gesture recognized by a classifier (60). It can then include information that causes an automated personal assistant (250) to retrieve information from a database and output the retrieved information in a form suitable for the user (249) to receive.

[0087] In another embodiment, it is conceivable that, instead of an automated personal assistant (250), the control system (40) controls a household appliance (not shown) controlled according to an identified user gesture. The household appliance may be a washing machine, stove, oven, microwave oven, or dishwasher.

[0088] Figure 5 The embodiment shown is in which the control system (40) controls the access control system (300). The access control system (300) can be designed to physically control access. For example, it can include a door (401). The sensor (30) can be configured to detect a scene related to deciding whether to grant access. For example, it can be an optical sensor for providing image or video data (e.g., for detecting faces). The classifier (60) can be configured to interpret the image or video data, for example, by matching the identity with known people stored in a database, thereby determining the identity of the person. The actuator control signal (A) can then be determined based on the interpretation of the neural network (60), for example, based on the determined identity. The actuator (10) can be a lock, which grants access based on the actuator control signal (A). Non-physical logical access control is also possible.

[0089] Figure 6 The embodiment shown is an example in which the control system (40) controls the monitoring system (400). This embodiment is similar to... Figure 5 The embodiments shown are largely equivalent. Therefore, only the differences will be described in detail. The sensor (30) is configured to detect the monitored scene. The control system (40) does not necessarily control the actuator (10), but may instead control the display (10a). For example, the neural network (60) may determine the classification of the scene, for example, whether the scene detected by the optical sensor (30) is suspicious. The actuator control signal (A) transmitted to the display (10a) may then be configured, for example, to cause the display (10a) to adjust the content displayed based on the determined classification, for example, to highlight objects that the neural network (60) considers suspicious.

[0090] Figure 7The diagram illustrates an embodiment of a control system (40) for controlling an imaging system (500), such as an MRI apparatus, an X-ray imaging apparatus, or an ultrasound imaging apparatus. The sensor (30) may be, for example, an imaging sensor. A neural network (60) can then determine a classification of all or part of the sensed image. An actuator control signal (A) can then be selected based on this classification, thereby controlling the display (10a). For example, the neural network (60) can interpret a region of the sensed image as a potential anomaly. In this case, an actuator control signal (A) can be determined to cause the display (10a) to display the image and highlight the potentially anomalous region.

[0091] Figure 8 The image shown is the first layer contained in the neural network (60). ) Implementation example. First layer ( The classifier receives a first-layer input (i). The first-layer input (i) can be given as a tensor comprising a predefined height, width, and depth of data, which can be considered a representation of the input signal (x) provided to the classifier. Alternatively, the first-layer input (i) can also comprise representations of multiple input images, in which case the first-layer input (i) is given as a batch representation, preferably organized as a tensor with a predefined batch size, height, width, and depth. Alternatively, for example, if the input signal (x) comprises one-dimensional data, such as a representation of audio data, the first-layer input (i) may not include the width dimension.

[0092] If the first-layer input (i) includes height, width, and depth dimensions, the first-layer output (z) can be obtained through a convolution operation (C), where the first-layer input (i) is discretely convolved with a filter of predefined amounts. The filter can be parameterized by a set of weights (w). Alternatively, if the first-layer input (i) does not include the width dimension, the first-layer output (z) can be obtained through a matrix multiplication of the first-layer input (i) with a matrix (not shown), which is parameterized by the set of weights (w). This matrix multiplication can be conceived as analogous to processing the first-layer input (i) using a fully connected layer to obtain the first-layer output (z).

[0093] Having obtained the first-layer output (z), the first-layer loss value is obtained based on the feature maps contained in the first-layer output (z). The first-layer loss value (z) can be obtained according to the following formula: , , in, It is the number of feature maps in the first layer output (z). A j , andb j It is a predefined value, and It is the first j Each feature map.

[0094] The first layer output (z) can then be provided (81) to one or more other layers of the neural network (60). Alternatively, it can be used as the output signal (y) of the neural network (60).

[0095] Figure 9 The training process includes the first layer ( A schematic flowchart of the method for a neural network (60). In the first step (901), the first layer ( The input (i) of the first layer is received from the second layer of the neural network (60). Alternatively, the input (i) of the first layer can be provided to the first layer by the input layer of the neural network (60). For example, the first layer input (i) can be directly the input signal (x) of the neural network (60).

[0096] In the second step (902), the first layer output (z) can be obtained from the first layer ( ) Determine, for example, Figure 8 As shown in the example, and as described in the previous embodiments.

[0097] In the third step (903), the first-layer loss value (l) can be obtained based on the first-layer output (u), for example, as shown in the figure. Figure 8 As shown in the example, and as described in the previous embodiments.

[0098] In the fourth step (904), the first layer ( The weights (W) of the first layer loss (l) are updated, for example by obtaining the gradient of the first layer loss value (l) with respect to the input signal (x) and adapting the weights (w) according to the negative gradient.

[0099] In the fifth step (905), the output signal (y) of the neural network (60) is obtained based on the first layer output (z). This can be done by providing the first layer output (z) to subsequent layers that provide subsequent outputs. This subsequent output can be forwarded by the rest of the neural network (60), as is common for feedforward neural networks. Alternatively, it can be envisioned that the first layer output (z) is used directly as the output signal (y).

[0100] In step six (906), a classification loss value is obtained based on the output signal (y). This can be achieved, for example, by applying a softmax function to the output signal (y) to obtain a set of probabilities. From these probabilities, the cross-entropy value relative to the desired classification can be calculated. The cross-entropy loss value can then be used as the classification loss value.

[0101] In the seventh step (907), the weights of the classifier (preferably including the weights (w) of the first layer) can be updated to reduce the classification loss value. This can be done by calculating the gradient of the classification loss value with respect to the input signal (x) and updating the weights (preferably including the weights (w) of the first layer) based on the negative gradient. The weights (w) are used to achieve this. Steps (901) to (907) can then be repeated iteratively until the desired number of iterations has been performed. Alternatively, the steps can be repeated until the classification loss value becomes less than a predefined classification loss threshold.

[0102] In step eight (908), the first layer can be deleted ( This can be achieved, for example, by determining the filter corresponding to the minimum norm (e.g., ...). L This is achieved by using a filter on the feature map of the 2-norm. This filter can then be discarded, and the classifier can be fine-tuned to account for changes in the network architecture. This can be achieved, for example, by minimizing the classification loss value from the previous step using the training dataset of the input signal (x) until the classification loss value has become less than the classification loss threshold. Step 8 (908) can be repeated to start from the first layer ( ) Filters that reduce predefined values.

[0103] In another embodiment (not shown), it is conceivable that the first step (901), the second step (902), and the third step (903) are repeated iteratively with different input signals (x) until a predefined number of iterations has been achieved. Alternatively, the iterations can be repeated until the first-layer loss value (l) falls below a certain first-layer loss threshold.

[0104] In two alternatives to this embodiment, steps four (904) to eight (908) can then be performed. It is also conceivable that after the iterative execution of steps one (901) to three (903) as explained above, training can be performed without updating the first layer loss value (l) but only the classification loss value.

Claims

1. A method for training a classifier (60) to classify image data, wherein the classifier (60) includes a first layer ( ), of which the first layer ( The classifier (60) includes multiple filters to provide a first layer output (z) comprising multiple feature maps, wherein training the classifier (60) includes the following steps: • Received by the previous layer (901) First layer ( The first layer input (i) in the image is based on the input image data (x) and is given as a tensor with predefined height, width and depth, the tensor including data as a representation of the input image data (x); • Based on the first layer input (i) and the first layer ( The multiple parameters (w) of (902) determine the first layer output (z); • Determine the first-layer loss value (l) based on the first-layer output (z), where the first-layer loss value (l) characterizes the degree of correlation between the feature maps of the first-layer output (z), and the first-layer loss value (l) is obtained in an unsupervised manner; • Train (904) classifier (60), where training includes the first layer ( The adaptation of the parameters (w) of the first layer is based on the first layer loss value (l).

2. The method of claim 1, wherein training the classifier (60) further comprises the following step: • Determine the output signal (y) of the classifier (60) (905), wherein the output signal (y) depends on the first layer output (z), wherein the output signal (y) characterizes the classification of the input image data (x); • Determine the (906) classification loss value based on the output signal (y); • Train (907) classifier (60), where training includes the first layer ( The adaptation of the parameters (w) of ) is based on the classification loss.

3. The method according to claim 1 or 2, wherein the first layer loss value (l) characterizes the factor decomposition of the distribution of the first layer output (z) between feature maps.

4. The method according to any one of the preceding claims, wherein the first layer loss value (l) is determined according to the following formula (903): ， ， in, i It is the first layer input (i). It is the number of feature maps in the first layer output (z). A j , and b j It is a predefined value, and It is the first layer output (z) of the first layer. j Each feature map.

5. The method according to claim 4, wherein, During the training of classifier (60), adaptation A i and / or b i and / or .

6. The method according to any one of the preceding claims, wherein the step of determining (902) the first layer output (z) includes padding the first layer input (i) such that the size of the first layer output (z) matches the size of the first layer input (i) along the height and / or width and / or depth.

7. The method according to any one of the preceding claims, wherein the method further comprises the following step: • Delete (908) first layer ( () filter.

8. A classifier (60) for classifying image data, wherein the classifier (60) is trained using a method according to any one of the preceding claims.

9. A computer-implemented method for controlling an actuator (10) using a classifier (60) according to claim 8, wherein the input image data (x) provided to the classifier (60) is based on a sensor signal (S) including data from a sensor (30), and / or an actuator control signal (A) for controlling the actuator (10) is provided based on the output signal (y) of the classifier (60).

10. The method of claim 9, wherein the actuator (10) controls at least partially autonomous robot (100) and / or manufacturing machine (200) and / or access control system (300).

11. A computer program product comprising instructions configured to cause a computer to perform the method and all steps thereof according to any one of claims 1 to 10 if the instructions are executed by a processor (45).

12. A machine-readable storage medium (46) thereon storing a computer program product according to claim 11.

13. A control system (40) for operating an actuator (10), wherein the control system (40) includes a classifier (60) according to claim 8, wherein the control system is configured to operate an actuator (10) according to claim 9.

14. A training system (140) configured to perform the method according to any one of claims 1 to 7.

Images