How to initialize a neural network

Abstract

In a computer-implemented method (100) for training a neural network (60), the neural network (60) is configured to determine an output signal (y) based on an input signal (x), the training including training parameters of a depthwise convolutional layer of the neural network (60), the depthwise convolutional layer being initialized based on values ​​drawn from a predetermined probability distribution, the variance of the probability distribution being characterized by the inverse of the square root of the number of filters applied to each depth of an input of the depthwise convolutional layer.

Description

[Technical field]

[0001] The present invention relates to a method for training a neural network, a training system, a computer program and a machine-readable storage medium. [Background technology]

[0002] He et al., "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification," 2015 (https: / / arxiv.org / pdf / 1502.01852.pdf) discloses a method to initialize neural network weights before training.

[0003] Glorot and Bengio, "Understanding the difficulty of training deep feedforward neural networks," AISTATS, 2010 (http: / / jmlr.org / proceedings / papers / v9 / glorot10a / glorot10a.pdf), discloses a method for initializing the weights of a neural network before training.

[0004] Technical background Neural networks serve as the backbone for solving a variety of tasks across a range of technical disciplines. Neural networks utilize data in a process known as training, during which the neural network is adapted to the data at hand (i.e., training data) to solve the desired task. Tasks may include classification, regression analysis, and various subforms of the aforementioned paradigms, such as object detection or semantic segmentation.

[0005] The efficiency or even ability to train such networks often depends on the initialization of the parameters involved. A common approach is that introduced by Xavier Glorot and Yoshua Bengio, sometimes called Glorot initialization or Xavier initialization, where the parameters of linear layers (i.e. layers that perform linear transformations such as fully connected or convolutional layers) are initialized with random values ​​drawn from a random distribution with mean 0 and variance V, where V depends on the particular layer.

[0006] Initializing the parameters of neural networks to appropriate values ​​is a recurring problem in machine learning, as these models are typically trained by selecting a starting point for the parameters (i.e., initializing) and then iteratively updating the parameters according to some loss function. Typical of these approaches are gradient-based optimization methods, such as stochastic gradient descent, or optimization methods based on evolutionary or genetic algorithms.

[0007] Thus, initialization serves an important goal when training neural networks: if the parameters are improperly initialized, training may converge slowly or even not at all. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] He et al., “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,” 2015, https: / / arxiv.org / pdf / 1502.01852.pdf [Non-Patent Document 2] Glorot and Bengio, “Understanding the difficulty of training deep feedforward neural networks,” 2010, AISTATS, http: / / jmlr.org / proceedings / papers / v9 / glorot10a / glorot10a.pdf Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention proposes a new approach for the initialization parameters (also known as weights) of neural networks. [Means for solving the problem]

[0010] In a first aspect, the invention relates to a computer-implemented method for training a neural network, the neural network configured to determine an output signal based on an input signal, the training including training parameters of a depthwise convolutional layer of the neural network, the depthwise convolutional layer being initialized based on values ​​drawn from a predetermined probability distribution, the variance of the probability distribution being characterized by the inverse of the square root of the number of filters applied to each depth of an input of the depthwise convolutional layer.

[0011] In the following, the depthwise convolutional layer is also simply referred to as a “layer”.

[0012] The output signal may be determined based on the input signal by providing the input signal to the neural network, which then routes the input signal through its layers to determine the output signal. The layers may in particular be considered to be located along this information path. The layer may be, for example, the first layer of the neural network, thereby taking the input signal as input and providing the output of the layer to other layers of the neural network. Alternatively, the layer may be a hidden layer of the neural network, i.e. a layer that receives input from other layers of the neural network. The layer may be located at the end of the neural network, thereby functioning as an output layer. The output of the layer may therefore also be provided as an output signal of the neural network. All these embodiments are considered to fall under the term "determining the output signal based on the input signal".

[0013] The layer may be specifically used as part of a depthwise separable convolution.

[0014] A layer thus plays a determining role for determining the output signal of the neural network. During training, a parameter or several parameters of a layer may be adapted to achieve a desired behavior of the neural network, for example to enable the neural network to determine a desired output signal given an input signal.

[0015] The output signal may, for example, characterize the classification and / or the result of a regression analysis given the input signal. From a statistical point of view, the input signal may serve as an independent variable and the output signal as a dependent variable. The training may be performed in a supervised manner, for example by providing a loss function to be minimized, the loss function being configured to characterize the difference or dissimilarity between the output signal determined by the neural network for the input signal and a desired signal related to the input signal. However, the training may also be performed in a semi-supervised or unsupervised manner. For example, the neural network may be configured as a generative adversarial network, with regularization flows and training being performed in a manner known for these types of models.

[0016] Preferably, the input signal characterises a measurement obtained from a sensor.

[0017] Thus, for example, if the input signal includes multiple signals, the input signal may be or include a sensor signal. The sensor may be considered to perform measurements and provide the measurements to the neural network, which determines the output signal based on the sensor signal. Thus, if the input signal characterizes a sensor signal, the output signal may be considered to be an indirect measurement obtained based on the direct measurements of the sensor, i.e., the input signal.

[0018] The measurements obtained from the sensors, i.e. the sensor signals, may contain different types of data: the sensors may be, for example, cameras, lidar sensors, radar sensors, ultrasonic sensors, thermal cameras, piezo sensors or hall sensors.

[0019] The sensor signal may also include multiple sensor signals, for example from multiple sensors of the same type and / or multiple sensors of different types.

[0020] Thus, preferably, the neural network is configured to receive as input an input signal characterizing the sensor signal and to provide an output signal characterizing a classification and / or regression result and / or probability density value of the input signal (e.g. if the neural network is a regularized flow), where the regression result may be understood as the result of a regression analysis using the input signal as an independent variable.

[0021] The neural network may be trained by backpropagation and gradient descent, e.g., stochastic gradient descent. Optionally, the neural network may be trained using an evolutionary or genetic algorithm.

[0022] For training, the neural network is initialized, which may be considered as setting the values ​​of the parameters of the neural network to some specific values. The parameters of the neural network are organized in layers, which means that the initialization of the neural network can also be considered as the initialization of the layers of the neural network. For initialization, only a subset of the parameters of the neural network and / or a subset of the parameters of the layers may be initialized. Preferably, all parameters of the neural network are initialized. The initialization may preferably be performed before training or when resuming training of the neural network.

[0023] For the initialization of the layer, values ​​may be drawn from a predefined probability distribution and the parameters of the layer may be set according to the drawn values. Preferably, the values ​​are used as parameters, but other approaches may be used to select the parameters. For example, the values ​​may be offset by a predefined value before being used as parameters.

[0024] The inventors have found that initializing the parameters of a layer according to the inverse of the square root of the product of the number of inputs of the layer and the number of outputs of the layer leads to faster convergence of the training, especially when the number of inputs and the number of outputs differ by, for example, at least an order of magnitude. Advantageously, the speedup of convergence is due to the proposed initialization allowing the layer to obtain an upscaling factor of exactly 1, i.e. the expected factor of the values ​​at the inputs and at the outputs of the layer is exactly 1. Advantageously, this limits the value of the gradient determined for the layer to a reasonable value, i.e. the gradient is not too small (i.e. gradient vanishing is unlikely to occur) and not too large (i.e. excessive gradient explosion is unlikely to occur).

[0025] Since training always occurs in a finite amount of time, faster training allows the neural network to be trained on more data in a finite amount of time, which results in better generalization of the neural network.

[0026] Preferably, the variance is determined according to the following formula:

number

[0027] Preferably, the neural network may also include multiple depthwise convolutional layers, and each of the multiple depthwise convolutional layers is initialized according to any one of the above embodiments.

[0028] In a preferred embodiment, the predetermined probability distribution is a normal distribution. Preferably, the predetermined probability distribution has an expected value of zero.

[0029] In another aspect, the invention relates to a computer-implemented method for determining an output signal based on an input signal, the method comprising: obtaining a neural network trained by a method according to any one of the embodiments listed above; determining an output signal based on the input signal by the obtained neural network; Includes.

[0030] The step of obtaining a trained neural network may be considered as a step of actually performing the training method. Optionally, the neural network may also be obtained from some other source, which provides the neural network after training according to the embodiment presented above. The neural network may for example be trained by an entity and then provided by the entity, for example via online download.

[0031] Advantageously, this aspect allows for a better determination of the output signal since the neural network has been trained by the improved method described above. The improved training method results in an increased generalization ability, i.e. the neural network is able to more accurately predict the desired output signal.

[0032] The invention further relates to a computer-implemented neural network, the neural network being trained by a method according to the training method presented above.

[0033] The neural network may preferably be used as part of a control system for controlling a technical system. The invention therefore also relates to a control system, the control system being arranged to use the neural network for determining control signals for controlling actuators and / or displays of the technical system.

[0034] Embodiments of the invention will now be described in more detail with reference to the following drawings. [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 illustrates a schematic diagram of a method for training a neural network. [Diagram 2] FIG. 1 illustrates a training system for training a neural network. [Diagram 3] FIG. 2 illustrates a control system. [Figure 4] FIG. 1 illustrates a control system for controlling an at least partially autonomous robot. [Diagram 5] FIG. 1 illustrates a control system for controlling a manufacturing machine. [Figure 6] FIG. 1 illustrates a control system for controlling an automated personal assistant. [Figure 7] FIG. 2 illustrates a control system that controls the access control system. [Figure 8] FIG. 2 is a diagram showing a control system that controls the monitoring system. [Figure 9] FIG. 2 is a diagram showing a control system that controls the imaging system. [Figure 10] FIG. 1 illustrates a control system for controlling a medical analysis system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] FIG. 1 shows an embodiment of a method (100) for training a neural network including at least one depthwise convolutional layer. In a first step (101), the parameters of the neural network are initialized. The parameters of the neural network are organized into layers, and the parameters of the depthwise convolutional layers are initialized with respect to the number of filters to be applied at each depth of the input of the depthwise convolutional layer. Preferably, all layers including the parameters of the neural network are initialized. That is, step (101) may be repeated for each depthwise convolutional layer of the neural network, or for several depthwise convolutional layers.

[0037] In a first step (101), the values ​​of the parameters of the depthwise convolutional layer are preferably drawn from a normal distribution, preferably having an expectation of 0 and a variance of the following formula:

number

[0038] Then, after completing the first step (101) or the first step(s), the training samples may be forwarded through the neural network in a second step (102) of the method. In a third step (103), gradients for the parameters of the neural network may be determined, for example by backpropagation. Then, in a fourth step (104) of the method, the gradients may be used to determine new parameters of the neural network. The gradients may be used in a first order gradient descent method or a second order gradient descent method, such as stochastic gradient descent.

[0039] The second step (102), the third step (103) and the fourth step (104) may be repeated iteratively until a desired amount of iterations is achieved or a predetermined criterion is met, such as, for example, the loss value for the training data set is below a predetermined threshold or the loss value for the validation data set is below a predetermined threshold, etc. When the desired amount of iterations is achieved or the predetermined criterion is met, the method ends.

[0040] 2 illustrates one embodiment of a training system (140) for training a neural network (60) with a training data set (T). Prior to training, the neural network (60) is initialized according to the embodiment of FIG. 1. The training data set (T) includes a number of input signals (x i ), and the training data set (T) includes each input signal (x i ), the input signal (x i ) and the input signal (x i ) and / or the classification of the input signal (x i ) that characterizes the desired regression results for the desired output signal (t i ).

[0041] For training, the training data unit (150) accesses a computer implemented database (St2), which provides a training data set (T). The training data unit (150) selects from the training data set (T) at least one input signal (x i ) and the input signal (x i ) corresponding to the desired output signal (t i ), preferably randomly determined, and the input signal (x i ) to the neural network (60). The neural network (60) receives the input signal (x i ) based on the output signal (y i ) to determine

[0042] The desired output signal (t i ) and the determined output signal (y i ) is sent to the modification unit (180).

[0043] The modification unit (180) then modifies the desired output signal (t i ) and the determined output signal (y i ) to determine new parameters (Φ') for the neural network (60). To this end, the correction unit (180) uses a loss function to determine the desired output signal (t i) and the determined output signal (y i ) The loss function is the output signal (y i ) is the desired output signal (t i ) as the loss function. In the given embodiment, a negative log-likelihood function is used as the loss function. In alternative embodiments, other loss functions are contemplated.

[0044] Furthermore, the determined output signal (y i ) and the desired output signal (t i ) each includes a number of sub-signals, e.g., in the form of a tensor, and the desired output signal (t i ) is the sub-signal of the determined output signal (y i For example, if the neural network (60) is configured for object detection, and the first sub-signal corresponds to a sub-signal of the input signal (x i ) and the second sub-signal characterizes the exact location of the object. i ) and the desired output signal (t i ) comprises a plurality of corresponding sub-signals, a second loss value is preferably determined for each corresponding sub-signal by a suitable loss function, and the determined second loss values ​​are suitably combined, e.g. by a weighted sum, to form the first loss value.

[0045] The refinement unit (180) determines the new parameters (Φ') based on the first loss value. In the given embodiment, this is done using a gradient descent method, preferably stochastic gradient descent, Adam or AdamW. In further embodiments, the training may also be based on evolutionary algorithms or second-order methods for training neural networks.

[0046] In another preferred embodiment, the described training is repeated iteratively for a predetermined number of iteration steps or until the first loss value falls below a predetermined threshold. Alternatively or additionally, the training may be terminated when the average first loss value for the test or validation data set falls below a predetermined threshold. In at least one of the iterations, new parameters (Φ') determined in the previous iteration are used as parameters (Φ) of the neural network (60).

[0047] Additionally, the training system (140) may include at least one processor (145) and at least one machine-readable storage medium (146) containing instructions that, when executed by the processor (145), cause the training system (140) to perform a training method according to one of the aspects of the present invention.

[0048] In a further embodiment (not shown), training of the neural network (60) may also be unsupervised. The neural network may be, for example, a regularized flow, with a loss function that is the training input signal (x i ) may be based solely on the output of the neural network (60) for the desired output signal (t i ), the remainder of the training system (140) does not need to be modified.

[0049] Figure 3 shows an embodiment of an actuator (10) in its environment (20). The actuator (10) interacts with a control system (40). The actuator (10) and its environment (20) are collectively referred to as the actuator system. At preferably equally spaced time points, a sensor (30) senses a state of the actuator system. The sensor (30) may comprise several sensors. Preferably, the sensor (30) is an optical sensor that takes an image of the environment (20). A sensor signal (S) of the sensor (30) (or the sensor signal (S) of each of the sensors if the sensor (30) comprises multiple sensors), encoding the sensed state, is transmitted to the control system (40).

[0050] Thereby, the control system (40) receives the stream of sensor signals (S) and then calculates a set of control signals (A) in response to the stream of sensor signals (S), which are then transmitted to the actuators (10).

[0051] The control system (40) receives a stream of sensor signals (S) of the sensors (30) at an optional receiving unit (50). The receiving unit (50) converts the sensor signals (S) into input signals (x). Alternatively, in the absence of the receiving unit (50), each sensor signal (S) may be regarded as an input signal (x) as it is. The input signal (x) may for example be provided as an excerpt from the sensor signals (S). Alternatively, the sensor signals (S) may be processed to obtain the input signals (x). In other words, the input signals (x) are provided according to the sensor signals (S).

[0052] The input signal (x) is then passed to a neural network (60). The neural network is preferably trained with a training system (140) according to Figure 2. Optionally, the neural network (60) may also be obtained from an external source, for example by downloading the neural network (60) from the internet.

[0053] The neural network (60) is parameterized by parameters (Φ) provided and stored in a parameter store (St1).

[0054] The neural network (60) determines an output signal (y) from the input signal (x). The output signal (y) includes information that assigns one or more labels and / or real values ​​to the input signal (x). The output signal (y) is sent to an optional conversion unit (80) that converts the output signal (y) into a control signal (A). The control signal (A) is then sent to the actuator (10) in order to control the actuator (10) accordingly. Alternatively, the output signal (y) may be considered as the control signal (A) as it is.

[0055] The actuator (10) receives the control signal (A) and is controlled accordingly to perform an action corresponding to the control signal (A). The actuator (10) may include control logic that converts the control signal (A) into a further control signal that is used to control the actuator (10).

[0056] In further embodiments, the control system 40 may include the sensor 30. In yet other embodiments, the control system 40 may alternatively or additionally include the actuator 10.

[0057] In a further embodiment, the control system (40) may be envisaged to control the display (10a) instead of or in addition to the actuator (10).

[0058] Additionally, the control system (40) may include at least one processor (45) and at least one machine-readable storage medium (46) having stored thereon instructions that, when executed, cause the control system (40) to perform a method according to one aspect of the present invention.

[0059] FIG. 4 illustrates an embodiment in which the control system (40) is used to control an at least partially autonomous robot, such as an at least partially autonomous vehicle (110).

[0060] The sensors (30) may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more lidar sensors. Some or all of these sensors are preferably, but not necessarily, integrated into the vehicle (110). Thus, the input signal (x) may be understood as an input image and the neural network (60) as an image classifier.

[0061] The neural network (60) may be configured to detect objects in the vicinity of the at least partially autonomous robot based on an input image (x). The output signal (y) may include information characterizing where objects are located in the at least partially autonomous robot's vicinity. A control signal (A) may then be determined according to this information, for example to avoid collisions with the detected objects.

[0062] Preferably, the actuator (10) integrated in the vehicle (110) may be provided by the brakes, propulsion system, engine, drive train or steering of the vehicle (110). The control signal (A) may be determined such that the actuator (10) is controlled to avoid a collision between the vehicle (110) and the detected object. The detected objects may also be classified according to what the neural network (60) considers to be most likely, e.g. a pedestrian or a tree, and the control signal (A) may be determined in response to the classification.

[0063] Alternatively or additionally, the control signal (A) may also be used to control the display (10a), for example to display the objects detected by the neural network (60). It is also conceivable that the control signal (A) may control the display (10a) to generate a warning signal if a collision between the vehicle (110) and at least one of the detected objects is imminent. The warning signal may be an audible warning and / or a haptic signal, for example a vibration of the steering wheel of the vehicle (110).

[0064] In further embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving or stepping. The mobile robot may in particular be an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. In all the above embodiments, the control signal (A) may be determined such that the propulsion unit and / or steering and / or braking of the mobile robot are controlled so as to avoid a collision between the mobile robot and the identified object.

[0065] In a further embodiment, the at least partially autonomous robot may be provided by a gardening robot (not shown) that uses sensors (30), preferably optical sensors, to determine the condition of the plants in the environment (20). The actuators (10) may control cutting devices such as nozzles and / or blades for spraying liquid. Depending on the identified species of the plant and / or the identified condition, a control signal (A) may be determined to cause the actuators (10) to spray an appropriate amount of an appropriate liquid on the plant and / or to cut the plant.

[0066] In yet another embodiment, the at least partially autonomous robot may be provided by a household appliance (not shown), such as a washing machine, stove, oven, microwave or dishwasher. A sensor (30), such as a light sensor, may detect the state of an object being processed by the household appliance. For example, if the household appliance is a washing machine, the sensor (30) may detect the state of laundry inside the washing machine. The control signal (A) may then be determined depending on the detected material of the laundry.

[0067] 5 illustrates an embodiment in which a control system (40) is used to control a manufacturing machine (11) of a manufacturing system (200), such as a punch cutter, cutter, gun drill, or gripper, for example as part of a production line. The manufacturing machine may include a transport device, such as a conveyor belt or an assembly line, that moves the manufactured product (12). The control system (40) controls actuators (10), which in turn control the manufacturing machine (11).

[0068] The sensor 30 may for example be provided by an optical sensor capturing characteristics of the manufactured product 12. The neural network 60 may therefore be understood as an image classifier.

[0069] The neural network 60 may determine a position of the manufactured product 12 relative to the transport device. The actuator 10 may then be controlled according to the determined position of the manufactured product 12 for a subsequent manufacturing step of the manufactured product 12. For example, the actuator 10 may be controlled to cut the manufactured product at a specific location on the manufactured product itself. Optionally, the neural network 60 may be envisaged to classify the manufactured product as being faulty or exhibiting a defect. The actuator 10 may then be controlled to remove the manufactured product from the transport device.

[0070] 6 shows an embodiment in which the control system (40) is used to control an automated personal assistant (250). The sensor (30) may be, for example, an optical sensor for receiving video images of the gestures of the user (249). Optionally, the sensor (30) may also be, for example, an audio sensor for receiving voice commands of the user (249).

[0071] The control system (40) then determines a control signal (A) for controlling the automated personal assistant (250). The control signal (A) is determined in response to the sensor signal (S) of the sensor (30). The sensor signal (S) is transmitted to the control system (40). For example, the neural network (60) may be configured to execute a gesture recognition algorithm, for example, for identifying a gesture made by the user (249). The control system (40) may then determine a control signal (A) for transmission to the automated personal assistant (250). The control system (40) then transmits the control signal (A) to the automated personal assistant (250).

[0072] For example, the control signal (A) may be determined according to an identified user gesture recognized by the neural network (60). The control signal (A) may include information that causes the automated personal assistant (250) to retrieve information from a database and output the retrieved information in a format suitable for receipt by the user (249).

[0073] In a further embodiment, instead of an automated personal assistant (250), the control system (40) may be envisaged to control a home appliance (not shown) which is controlled according to the identified user gestures. The home appliance may be a washing machine, a stove, an oven, a microwave oven or a dishwasher.

[0074] In Fig. 7 an embodiment is shown in which the control system (40) controls an access control system (300). The access control system (300) may be designed to physically control access. This may include, for example, a door (401). The sensor (30) may be configured to detect a relevant scene in order to decide whether access should be allowed or not. This may be, for example, an optical sensor for providing image or video data, for example for detecting human faces. The neural network (60) may therefore be understood as an image classifier.

[0075] The neural network (60) may be configured to classify the identity of the person, for example by matching the detected face of the person with other faces of known people stored in a database, thereby determining the identity of the person. The control signal (A) may then be determined in response to the classification of the neural network (60), for example according to the determined identity. The actuator (10) may be a lock that opens or closes a door in response to the control signal (A). Optionally, the access control system (300) may be a non-physical logical access control system. In this case, the control signal may be used to control the display (10a) to show information about the identity of the person and / or whether the person is granted access.

[0076] In Fig. 8, an embodiment is shown in which the control system (40) controls the monitoring system (400). This embodiment is almost identical to the embodiment shown in Fig. 7. Therefore, only the different aspects will be described in detail. The sensor (30) is arranged to detect the scene under monitoring. The control system (40) may control the display (10a) but not necessarily the actuator (10). For example, the neural network (60) may determine a classification of the scene, for example whether the scene detected by the optical sensor (30) is normal or whether the scene exhibits an anomaly. The control signal (A) sent to the display (10a) may then be arranged to cause the display (10a) to adjust the displayed content, for example according to the determined classification, for example by highlighting objects that are considered to be anomalous by the neural network (60).

[0077] 9 shows an embodiment of a medical imaging system 500 controlled by a control system 40. The imaging system may be, for example, an MRI device, an X-ray imaging device, or an ultrasound imaging device. The sensor 30 may be, for example, an imaging sensor that acquires at least one image of a patient, for example displaying different types of body tissue of the patient.

[0078] The neural network (60) may then determine a classification of at least a portion of the detected image, which is then used as an input image (x) to the neural network (60). The neural network (60) may therefore be understood as an image classifier.

[0079] The control signal (A) may then be selected according to the classification to control the display (10a) accordingly. For example, the neural network (60) may be configured to detect different types of tissue in the sensed image, for example by classifying tissues displayed in the image as either malignant or benign tissue. This may be done by semantic segmentation of the input image (x) by the neural network (60). The control signal (A) may then be determined to cause the display (10a) to display the different tissues, for example by displaying the input image (x) and coloring different regions of similar tissue type with the same color.

[0080] In further embodiments (not shown), the imaging system (500) may be used for non-medical purposes, for example to determine material properties of a workpiece. In these embodiments, the neural network (60) may be configured to receive an input image (x) of at least a portion of the workpiece and perform a semantic segmentation of the input image (x) to thereby classify a material property of the workpiece. A control signal (A) may then be determined to cause the display (10a) to display the input image (x) and information related to the detected material property.

[0081] 10 shows an embodiment of a medical analysis system (600) controlled by a control system (40). The medical analysis system (600) is provided with a microarray (601), which includes a number of spots (602, also known as features) that are exposed to a medical specimen. The medical specimen may be, for example, obtained from a human or animal specimen, such as a swab.

[0082] The microarray (601) may be a DNA microarray or a protein microarray.

[0083] The sensor 30 is configured to sense the microarray 601. The sensor 30 is preferably an optical sensor, such as a video sensor. The neural network 60 may therefore be understood as an image classifier.

[0084] The neural network (60) is configured to classify the result of the specimen based on the input image (x) of the microarray provided by the sensor (30). In particular, the neural network (60) may be configured to determine whether the microarray (601) indicates the presence of a virus in the specimen.

[0085] The control signal (A) may then be selected to cause the display (10a) to show the results of the classification.

[0086] The term "computer" may be understood to encompass any device for processing predefined computational rules. These computational rules may be in the form of software, hardware, or a mixture of software and hardware.

[0087] In general, the plurality may be understood to be indexed, i.e., each element of the plurality is assigned a unique index, preferably by assigning consecutive integers to the elements in the plurality. Preferably, if the plurality includes N elements, where N is the number of elements in the plurality, the elements are assigned integers from 1 to N. It will also be understood that the elements of the plurality may be accessed by their index.

Claims

1. A computer-implemented method (100) for training a neural network (60), comprising: The neural network (60) is configured to determine an output signal (y) based on an input signal (x); The training includes training parameters for a depthwise convolutional layer of the neural network (60); the depthwise convolutional layer is initialized based on values ​​drawn from a predetermined probability distribution; wherein the variance of the probability distribution is characterized by the inverse of the square root of the number of filters applied to each depth of the input of the depthwise convolutional layer.

2. The method (100) of claim 1, wherein the input signal (x) characterizes a measurement obtained from a sensor (30).

3. The variance is determined according to the following formula: [0010] In the ceremony, K ｗ is the kernel width of the filter of the convolution layer, and K ｈ 3. The method (100) of claim 1 or 2, wherein x is the kernel height of the filters of the convolutional layer, and k is the number of the filters applied to each depth of the input of the depthwise convolutional layer.

4. The method (100) of any one of claims 1 to 3, wherein the predetermined probability distribution is a normal distribution.

5. The method (100) of claim 4, wherein the predetermined probability distribution has an expected value of zero.

6. 1. A computer-implemented method for determining an output signal (y) based on an input signal (x), the method comprising: obtaining a neural network (60) trained by the method of any one of claims 1 to 5; determining, by the obtained neural network (60), the output signal (y) based on the input signal (x); A method comprising:

7. A computer-implemented neural network (60), comprising: A computer implemented neural network (60), wherein the neural network (60) is trained according to the method of any one of claims 1 to 5.

8. A control system (40) configured to use a neural network (60) according to claim 7 for determining control signals (A) for controlling actuators (10) and / or displays (10a) of a technical system (110, 200, 250, 300, 400, 500, 600).

9. A training system (140) configured to carry out the training method of any one of claims 1 to 5.

10. A computer program configured to cause a computer to carry out the method according to any one of claims 1 to 6 together with all of the steps of said method when said computer program is executed by a processor (45, 145).

11. A machine-readable storage medium (46, 146) having stored thereon a computer program according to claim 10.

Images