Method for initializing neural network
By adopting a new initialization method based on deep-by-deep convolution layer in neural networks, using the variance representation of predefined probability distributions, the training convergence problem caused by parameter initialization in the existing technology is solved, and a faster training process and better generalization ability.
Patent Information
- Application Number
- CN202380032435.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-08
- Filing Date
- 2023-04-05
- Publication Date
- 2024-11-22
AI Technical Summary
Existing neural network parameter initialization methods may lead to slow convergence or non-convergence during training, especially when the number of inputs and outputs is significantly different, resulting in inefficiency in training.
Using a new initialization method based on a depth-by-deep convolution layer, the input value of the layer is initialized by extracting values from a predefined probability distribution, where the variance of the probability distribution is characterized by the reciprocal of the square root of the number of filters applied at the input depth, ensuring the input value of the layer and The desired factor for the output value is 1, thereby limiting the gradient to a suitable range.
The training convergence process is accelerated, allowing neural networks to utilize more data for a limited time, thereby improving generalization and prediction accuracy.
Smart Images

Figure CN119013674A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method, a training system, a computer program and a machine-readable storage medium for training a neural network. Prior art
[0002] He et al., “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”, 2015, https: / / arxiv.org / pdf / 1502.01852.pdf discloses a method for initializing the weights of a neural network before training.
[0003] Glorot and Bengio, “Understanding the difficulty of training deepfeedforward neural networks”, 2010, AISTATS, http: / / jmlr.org / proceedings / papers / v9 / glorot10a / glorot10a.pdf discloses methods for initializing the weights of a neural network before training. Background Art
[0005] Neural networks serve as a backbone for solving different tasks across a variety of technological fields. Neural networks utilize data in a process known as training, where during training, the neural network is adapted to the data at hand (i.e., training data) to solve the desired task. The tasks may include classification, regression analysis, and various sub-forms of the aforementioned paradigm, such as object detection or semantic segmentation.
[0006] The efficiency and even the ability to train such networks often depends on the initialization of the parameters involved. A common approach is the one introduced by Xavier Glorot and Yoshua Bengio, which is sometimes called Glorot initialization or Xavier initialization. Here, the parameters of linear layers (i.e., layers that perform linear transformations, such as fully connected layers or convolutional layers) are initialized with random values drawn from a random distribution with mean 0 and variance V, where V depends on the specific layer.
[0007] Initializing the parameters of a neural network to appropriate values is a recurring problem in machine learning, as these models are typically trained by choosing a starting point for the parameters (i.e., initialization), and then iteratively updating the parameters according to some loss function. Typical approaches to these methods are gradient-based optimization, such as stochastic gradient descent or optimization methods based on evolutionary or genetic algorithms.
[0008] Therefore, initialization serves a key goal in training a neural network. If the initialization of the parameters is not appropriate, the training may converge slowly or even not converge at all.
[0009] The present invention proposes a new method for initializing the parameters (also called weights) of a neural network. Summary of the invention
[0010] In a first aspect, the present invention relates to a computer-implemented method for training a neural network, wherein the neural network is configured to determine an output signal based on an input signal, and wherein the training comprises training parameters of a depth-wise convolutional layer of the neural network, wherein the depth-wise convolutional layer is initialized based on values extracted from a predefined probability distribution, wherein the variance of the probability distribution is characterized by the inverse of the square root of a number of filters applied at each depth of the input to the depth-wise convolutional layer.
[0011] In the following, the depthwise convolutional layer will also be simply referred to as “layer”.
[0012] The output signal may be determined based on the input signal by providing the input signal to a neural network, which in turn forwards the input signal through its various layers to determine the output signal. The layers may in particular be considered to be placed along this information path. For example, a layer may be the first layer of a neural network, thereby taking the input signal as input and providing the output of the layer to other layers of the neural network. Alternatively, a layer may be a hidden layer of the neural network, i.e. a layer that receives input from another layer of the neural network. A layer may be placed at the end of a neural network, thereby acting as an output layer. Thus, the output of the layer may also be provided as an output signal of the neural network. All of these embodiments are considered to fall within the scope of the term "determining the output signal based on the input signal".
[0013] The layer can especially be used as part of a depthwise separable convolution.
[0014] Therefore, the layer plays a decisive role in determining the output signal of the neural network. During training, the parameters or some parameters of the layer can be adapted in order to achieve the desired behavior of the neural network, for example, to allow the neural network to determine the desired output signal given an input signal.
[0015] The output signal can, for example, characterize the results of a classification and / or regression analysis given an input signal. In statistics, the input signal can be used as an independent variable, wherein the output signal can be used as a dependent variable. Training can be performed in a supervised manner, for example, by providing a loss function that should be minimized, wherein the loss function is configured to characterize the difference or dissimilarity between an output signal determined by the neural network for the input signal and an expected signal about the input signal. However, training can also be implemented in a semi-supervised or unsupervised manner. For example, a neural network can be configured to generate an adversarial network or a normalization process, and training of these types of models can be implemented using known methods.
[0016] Preferably, the input signal is representative of measurements obtained from the sensor.
[0017] For example, if the input signal comprises a plurality of signals, the input signal may therefore be or include a sensor signal. The sensor may be considered to perform a measurement and provide the measurement result to the neural network, which in turn determines the output signal based on the sensor signal. If the input signal represents a sensor signal, the output signal may therefore be considered to be an indirect measurement result obtained based on the direct measurement result of the sensor (i.e., the input signal).
[0018] The measurements obtained from the sensor (ie the sensor signal) may include different kinds of data. The sensor may be, for example, a camera, a lidar sensor, a radio radar sensor, an ultrasonic sensor, a thermal camera, a piezoelectric sensor or a Hall sensor.
[0019] The sensor signal may also include multiple sensor signals, for example from multiple sensors of the same type and / or from multiple sensors of different types.
[0020] Preferably, the neural network is thus configured to accept an input signal as input, wherein the input signal represents the sensor signal, and to provide an output signal representing a classification and / or regression result and / or a probability density value of the input signal (e.g., in the case where the neural network is a normalized process). The regression result may be understood as the result of a regression analysis using the input signal as an independent variable.
[0021] The training of the neural network can be implemented by means of back propagation and gradient descent (eg, stochastic gradient descent). Alternatively, an evolutionary algorithm or a genetic algorithm can also be used to train the neural network.
[0022] For training, the neural network is initialized, wherein initializing the neural network can be considered as setting the parameter values of the neural network to certain specific values. The parameters of the neural network are organized into layers, which means that initializing the neural network can also be considered as initializing the layers of the neural network. For initialization, only a subset of the parameters of the neural network and / or a subset of the parameters of the layers can be initialized. Preferably, all parameters of the neural network are initialized. Initialization can preferably be performed before training or when resuming the training of the neural network.
[0023] For initialization of the layer, individual values are extracted from a predefined probability distribution, and the parameters of the layer can be set according to the extracted values. Preferably, these values are used as parameters, but other methods may also be chosen. For example, these values may be offset by predefined values before being used as parameters.
[0024] The inventors have found that initializing the parameters of a layer according to the inverse of the square root of the product of the number of inputs to the layer and the number of outputs of the layer leads to faster convergence of the training, especially when the number of inputs and the number of outputs are significantly different (e.g., at least one order of magnitude different). Advantageously, the acceleration of convergence is due to the fact that the proposed initialization allows the layer to obtain an amplification factor of exactly 1. That is, the expected factor of the input values and the output values of the layer is exactly 1. Advantageously, this limits the gradient values determined for the layer to suitable values, i.e., the gradient is not too small (i.e., not close to a vanishing gradient) and is not too large (i.e., not close to an exploding gradient).
[0025] Since training is always performed in a limited amount of time, faster training results in the neural network being able to train with more data in a limited amount of time, which in turn results in better generalization of the neural network.
[0026] Preferably, the variance is determined according to the following formula:
[0027]
[0028] Where K w is the kernel width of the filter in the convolutional layer, K h is the kernel height of the filter of the convolutional layer, and k is the number of filters applied at each depth of the input of the depth-wise convolutional layer.
[0029] Preferably, the neural network may also include a plurality of depth-wise convolutional layers, wherein each of the plurality of depth-wise convolutional layers is initialized according to any one of the embodiments listed above.
[0030] In a preferred embodiment, the predefined probability distribution is a normal distribution.Preferably, the expected value of the predefined probability distribution is zero.
[0031] In another aspect, the present invention relates to a computer-implemented method for determining an output signal based on an input signal, wherein the method comprises the following steps:
[0032] ● obtaining a neural network that has been trained using a method according to any of the previously listed embodiments;
[0033] • Determine the output signal based on the input signal with the aid of the obtained neural network.
[0034] Obtaining an already trained neural network may be considered as actually performing the training method. Alternatively, the neural network may also be obtained from some other source, wherein the source provides the neural network after training according to the previously presented embodiments. The neural network may, for example, be trained by an entity and then provided by the entity, for example by online downloading.
[0035] Advantageously, this aspect allows for a better output signal to be determined, since the neural network has been trained using the improved method discussed above. The improved training method results in an improvement in generalization capability. In other words, the neural network is able to more accurately predict the desired output signal.
[0036] The invention also relates to a computer implemented neural network, wherein the neural network is trained using a method according to the previously presented training method.
[0037] The neural network can preferably be used as part of a control system for controlling a technical system.The invention therefore also relates to a control system, wherein the control system is configured to use a neural network to determine control signals for controlling actuators and / or displays of a technical system. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Embodiments of the present invention will be discussed in more detail with reference to the following drawings. The drawings show:
[0039] Figure 1 A method for training a neural network is schematically shown;
[0040] Figure 2 A training system for training neural networks;
[0041] Figure 3 Control systems;
[0042] Figure 4 A control system controls the at least partially autonomous robot;
[0043] Figure 5 Control systems control manufacturing machines;
[0044] Figure 6 The control system controls the automated personal assistant;
[0045] Figure 7Control systems control access control systems;
[0046] Figure 8 Control system controls monitoring system;
[0047] Fig. 9 The control system controls the imaging system;
[0048] Fig.10 The control system controls the medical analysis system. DETAILED DESCRIPTION
[0049] Figure 1 An embodiment of a method (100) for training a neural network comprising at least one depthwise convolutional layer is shown. In a first step (101), the parameters of the neural network are initialized. The parameters of the neural network are organized into layers, and the parameters of the depthwise convolutional layer are initialized with respect to the number of filters applied at each depth of the input of the depthwise convolutional layer. Preferably, all layers comprising the parameters of the neural network are initialized. That is, step (101) may be repeated for each depthwise convolutional layer or some depthwise convolutional layers of the neural network.
[0050] In the first step (101), the values of the parameters of the depthwise convolutional layer are preferably drawn from a normal distribution, where the expected value of the normal distribution is preferably 0 and the variance is
[0051]
[0052] Where K w is the kernel width of the filter in the convolutional layer, K h is the kernel height of the filter of the convolutional layer, and k is the number of filters applied at each depth of the input of the depth-wise convolutional layer.
[0053] After completing the first step (101) or the first steps, then, in a second step (102) of the method, the training sample can be forwarded through the neural network. In a third step (103), the gradients with respect to the parameters of the neural network can be determined, for example, by back propagation. Then, in a fourth step (104) of the method, the gradients can be used to determine new parameters of the neural network. The gradients can be used in a first-order gradient descent method (such as stochastic gradient descent) or in a second-order gradient descent method.
[0054] The second step (102), the third step (103) and the fourth step (104) can be iteratively repeated until the desired number of iterations is achieved, or until a predefined criterion is met, for example, the loss value with respect to the training data set is equal to or lower than a predefined threshold, or the loss value with respect to the validation data set is equal to or lower than a predefined threshold. If the desired number of iterations is achieved, or if the predefined criterion is met, the method ends.
[0055] Figure 2 An embodiment of a training system (140) for training a neural network (60) with the aid of a training data set (T) is shown. Figure 1 The training data set (T) includes a plurality of input signals (x) for training the neural network (60). i ), where the training data set (T) also includes for each input signal (x i ) of the expected output signal (t i ), the expected output signal corresponds to the input signal (x i ) and characterizes the input signal (x i ) and / or about the input signal (x i ) is the expected regression result.
[0056] For training, the training data unit (150) accesses a computer-implemented database (St2) which provides a training data set (T). The training data unit (150) preferably randomly determines at least one input signal (x i ) and corresponds to the input signal (x i ) of the expected output signal (t i ), and the input signal (x i ) is transmitted to the neural network (60). The neural network (60) is based on the input signal (x i ) determines the output signal (y i ).
[0057] The expected output signal (t i ) and the determined output signal (y i ) is transmitted to the modification unit (180).
[0058] Then, based on the desired output signal (t i ) and the determined output signal (y i ), the modification unit (180) determines new parameters (Φ′) of the neural network (60). For this purpose, the modification unit (180) uses a loss function to compare the desired output signal (t i ) and the determined output signal (y i The loss function determines a first loss value that characterizes the determined output signal (y i ) deviates from the expected output signal (t i ) How far away. In the given embodiment, the negative log-likelihood function is used as the loss function. In alternative embodiments, other loss functions can also be conceived.
[0059] Furthermore, it is conceivable that the determined output signal (y i ) and the desired output signal (t i ) each includes multiple sub-signals (e.g., in the form of tensors), where the desired output signal (t i ) corresponds to the determined output signal (y i ). For example, it is conceivable that the neural network (60) is configured for object detection, and the first sub-signal represents the input signal (x i ) and the second sub-signal characterizes the exact position of the object. i ) and the desired output signal (t i ) comprises a plurality of corresponding sub-signals, a second loss value is preferably determined for each corresponding sub-signal by means of a suitable loss function, and the determined second loss values are suitably combined to form the first loss value, for example by means of a weighted summation.
[0060] The modification unit (180) determines a new parameter (Φ′) based on the first loss value. In a given embodiment, this is done using a gradient descent method (preferably stochastic gradient descent, Adam or AdamW). In other embodiments, the training may also be based on an evolutionary algorithm or a second-order method for training neural networks.
[0061] In other preferred embodiments, the described training is iteratively repeated for a predefined number of iteration steps or iteratively repeated until the first loss value falls below a predefined threshold. Alternatively or additionally, it is also conceivable that the training is terminated when the average first loss value for the test or validation data set falls below a predefined threshold. In at least one of the iterations, the new parameters (Φ′) determined in the previous iteration are used as parameters (φ) of the neural network (60).
[0062] In addition, the training system (140) may include at least one processor (145) and at least one machine-readable storage medium (146) containing instructions, which, when executed by the processor (145), cause the training system (140) to perform a training method according to one of the aspects of the present invention.
[0063] In another embodiment (not shown), the training of the neural network (60) may also occur unsupervised. The neural network may, for example, be a normalized process, and the loss function may be based solely on the training input signal (x i ) is the output of the neural network (60). However, in addition to the desired input-output signal (t i ), the rest of the training system (140) does not need to be changed.
[0064] Figure 3 An embodiment of an actuator (10) in its environment (20) is shown. The actuator (10) interacts with a control system (40). The actuator (10) and its environment (20) will be collectively referred to as the actuator system. At preferably evenly spaced points in time, a sensor (30) senses a condition of the actuator system. The sensor (30) may include several sensors. Preferably, the sensor (30) is an optical sensor that captures an image of the environment (20). An output signal (S) of the sensor (30) (or, in the case where the sensor (30) includes multiple sensors, an output signal of each sensor) encoding the sensed condition is transmitted to the control system (40).
[0065] Thus, the control system (40) receives a stream of sensor signals (S). The control system (40) then calculates a series of control signals (A) from the stream of sensor signals (S), which are then transmitted to the actuator (10).
[0066] The control system (40) receives a stream of sensor signals (S) from the sensor (30) in an optional receiving unit (50). The receiving unit (50) transforms the sensor signals (S) into input signals (x). Alternatively, in the absence of a receiving unit (50), each sensor signal (S) can be used directly as an input signal (x). The input signal (x) can be given, for example, as an extract of the sensor signal (S). Alternatively, the sensor signal can be processed to generate the input signal (x). In other words, the input signal (x) is provided based on the sensor signal (S).
[0067] Then, the input signal (x) is passed to the neural network (60). Preferably, the input signal (x) has been Figure 2 Alternatively, the neural network (60) may also be obtained from an external source (eg, by downloading the neural network (60) from the Internet).
[0068] The neural network (60) is parameterized by parameters (φ) stored in and provided by a parameter storage device (St1).
[0069] The neural network (60) determines an output signal (y) based on the input signal (x). The output signal (y) includes information that assigns one or more labels and / or true values to the input signal (x). The output signal (y) is transmitted to an optional conversion unit (80), which converts the output signal (y) into a control signal (A). The control signal (A) is then transmitted to the actuator (10) in order to control the actuator (10) accordingly. Alternatively, the output signal (y) can be directly used as the control signal (A).
[0070] The actuator (10) receives the control signal (A), is controlled accordingly and performs an action corresponding to the control signal (A). The actuator (10) may include control logic that transforms the control signal (A) into another control signal, which is then used to control the actuator (10).
[0071] In further embodiments, the control system (40) may include the sensor (30). In still further embodiments, the control system (40) may alternatively or additionally include the actuator (10).
[0072] In still further embodiments, it is contemplated that the control system (40) controls the display (10a) (instead of or in addition to controlling the actuator (10)).
[0073] Furthermore, the control system (40) may include at least one processor (45) and at least one machine-readable storage medium (46) on which instructions are stored which, if executed, cause the control system (40) to perform a method according to an aspect of the present invention.
[0074] Figure 4 An embodiment of a control system (40) for controlling an at least partially autonomous robot (eg, an at least partially autonomous vehicle (110)) is shown.
[0075] The sensors (30) may include one or more video sensors and / or one or more radio radar sensors and / or one or more ultrasonic sensors and / or one or more lidar sensors. Some or all of these sensors are preferably, but not necessarily, integrated into the vehicle (110). Thus, the input signal (x) may be understood as an input image and the neural network (60) may be understood as an image classifier.
[0076] The neural network (60) may be configured to detect objects in the vicinity of the at least partially autonomous robot based on an input image (x). The output signal (y) may include information characterizing the location of the object in the vicinity of the at least partially autonomous robot. A control signal (A) may then be determined based on the information, for example to avoid a collision with the detected object.
[0077] The actuator (10), preferably integrated in the vehicle (110), may be provided by the brakes, propulsion system, engine, transmission system or steering device of the vehicle (110). A control signal (A) may be determined so that the actuator (10) is controlled so that the vehicle (110) avoids a collision with a detected object. The detected object may also be classified according to what the neural network (60) considers the detected object to be most likely to be (e.g. a pedestrian or a tree), and the control signal (A) may be determined according to the classification.
[0078] Alternatively or additionally, the control signal (A) may also be used to control a display (10a), for example, for displaying objects detected by the neural network (60). It is also conceivable that the control signal (A) may control the display (10a) so that the display (10a) generates a warning signal if the vehicle (110) is close to colliding with at least one of the detected objects. The warning signal may be a warning sound and / or a tactile signal, for example, a vibration of the steering wheel of the vehicle (110).
[0079] In a further embodiment, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving or walking. The mobile robot may in particular be an at least partially autonomous lawnmower, or an at least partially autonomous cleaning robot. In all the above-described embodiments, a control signal (A) may be determined so that a propulsion unit and / or a steering device and / or a brake of the mobile robot are controlled such that the mobile robot can avoid a collision with the identified object.
[0080] In a further embodiment, the at least partially autonomous robot may be provided by a gardening robot (not shown) which uses a sensor (30), preferably an optical sensor, to determine the state of plants in the environment (20). The actuator (10) may control a nozzle for spraying a liquid and / or a cutting device, such as a blade. Depending on the identified type of plant and / or the identified state, a control signal (A) may be determined to cause the actuator (10) to spray the plant with an appropriate amount of an appropriate liquid and / or to cut the plant.
[0081] In yet another embodiment, the at least partially autonomous robot may be provided by a household appliance (not shown), such as, for example, a washing machine, a stove, an oven, a microwave or a dishwasher. A sensor (30) (e.g., an optical sensor) may detect the state of an object to be processed by the household appliance. For example, in the case where the household appliance is a washing machine, the sensor (30) may detect the state of the clothes inside the washing machine. The control signal (A) may then be determined based on the material of the detected clothes.
[0082] Figure 5 An embodiment of a control system (40) for controlling a manufacturing machine (11) (e.g., a punch cutter, a cutter, a barrel drill, or a gripper) of a manufacturing system (200) (e.g., as part of a production line) is shown. The manufacturing machine may include a transport device that moves the manufactured product (12), such as a conveyor belt or an assembly line. The control system (40) controls the actuator (10), which in turn controls the manufacturing machine (11).
[0083] The sensor (30) may be given by an optical sensor that captures, for example, properties of the manufactured product (12). Thus, the neural network (60) may be understood as an image classifier.
[0084] The neural network (60) may determine the position of the manufactured product (12) with respect to the transport device. The actuator (10) may then be controlled for subsequent manufacturing steps of the manufactured product (12) according to the determined position of the manufactured product (12). For example, the actuator (10) may be controlled to cut the manufactured product at a specific location of the manufactured product itself. Alternatively, it is conceivable that the neural network (60) classifies whether the manufactured product is broken or exhibits a defect. The actuator (10) may then be controlled to remove the manufactured product from the transport device.
[0085] Figure 6 An embodiment is shown in which the control system (40) is used to control an automated personal assistant (250). The sensor (30) may be a light sensor, for example, for receiving a video image of a gesture of a user (249). Alternatively, the sensor (30) may also be an audio sensor, for example, for receiving a voice command of a user (249).
[0086] The control system (40) then determines a control signal (A) for controlling the automated personal assistant (250). The control signal (A) is determined based on the sensor signal (S) from the sensor (30). The sensor signal is transmitted to the control system (40). For example, the neural network (60) can be configured to, for example, execute a gesture recognition algorithm to recognize a gesture made by the user (249). The control system (40) can then determine the control signal (A) for transmission to the automated personal assistant (250). It then transmits the control signal (A) to the automated personal assistant (250).
[0087] For example, the control signal (A) may be determined based on a recognized user gesture resolved by the neural network (60). It may include information that causes the automated personal assistant (250) to retrieve information from a database and output the retrieved information in a form suitable for receipt by the user (249).
[0088] In a further embodiment, it is conceivable that, instead of the automated personal assistant (250), the control system (40) controls a household appliance (not shown) that is controlled according to the recognized user gesture. The household appliance may be a washing machine, a stove, an oven, a microwave or a dishwasher.
[0089] Figure 7An embodiment of a control system (40) controlling an access control system (300) is shown in FIG. The access control system (300) may be designed to physically control access. For example, it may include a door (401). The sensor (30) may be configured to detect a scene relevant to a decision whether access is authorized. For example, it may be an optical sensor for providing image or video data in order to, for example, detect a person's face. Thus, the neural network (60) may be understood as an image classifier.
[0090] The neural network (60) may be configured to classify the identity of a person, for example, by matching the detected face of the person with other faces of known persons stored in a database, thereby determining the identity of the person. The control signal (A) may then be determined based on the classification of the neural network (60) (e.g., based on the determined identity). The actuator (10) may be a lock that opens or closes a door based on the control signal (A). Alternatively, the access control system (300) may be a non-physical, logical access control system. In this case, the control signal may be used to control a display (10a) to show information about the identity of the person and / or whether the person is given access.
[0091] Figure 8 An embodiment of a control system (40) controlling a monitoring system (400) is shown in FIG. Figure 7 . Therefore, only the different aspects will be described in detail. The sensor (30) is configured to detect a scene under surveillance. The control system (40) does not necessarily control the actuator (10), but may instead control the display (10a). For example, the neural network (60) can determine the classification of the scene, for example, whether the scene detected by the optical sensor (30) is normal, or whether the scene exhibits an abnormality. Then, the control signal (A) transmitted to the display (10a) can, for example, be configured to cause the display (10a) to adjust the displayed content according to the determined classification, for example, highlighting objects that are considered abnormal by the neural network (60).
[0092] Fig. 9 An embodiment of a medical imaging system (500) controlled by a control system (40) is shown in FIG. The imaging system may be, for example, an MRI device, an x-ray imaging device or an ultrasound imaging device. The sensor (30) may be, for example, an imaging sensor that captures at least one image of a patient, for example showing different types of body tissue of the patient.
[0093] The neural network (60) may then determine a classification of at least part of the sensed image. Thus, at least part of the image is used as an input image (x) for the neural network (60). Thus, the neural network (60) may be understood as an image classifier.
[0094] A control signal (A) may then be selected based on the classification to control the display (10a). For example, the neural network (60) may be configured to detect different types of tissue in the sensed image, for example, by classifying the tissue displayed in the image as malignant or benign tissue. This may be accomplished by semantic segmentation of the input image (x) by the neural network (60). The control signal (A) may then be determined to cause the display (10a) to display different tissues, for example, by displaying the input image (x) and coloring different areas of the same tissue type with the same color.
[0095] In other embodiments (not shown), the imaging system (500) may be used for non-medical purposes, for example, to determine material properties of a workpiece. In these embodiments, the neural network (60) may be configured to receive an input image (x) of at least a portion of the workpiece and perform semantic segmentation on the input image (x) to classify the material properties of the workpiece. A control signal (A) may then be determined to cause the display (10a) to display the input image (x) and information about the detected material properties.
[0096] Fig.10 An embodiment of a medical analysis system (600) controlled by a control system (40) is shown in FIG. The medical analysis system (600) is equipped with a microarray (601), wherein the microarray comprises a plurality of points (602, also referred to as features) that have been exposed to a medical specimen. The medical specimen may be, for example, a human specimen or an animal specimen, for example, a specimen obtained from a swab.
[0097] The microarray (601) can be a DNA microarray or a protein microarray.
[0098] The sensor (30) is configured to sense the microarray (601). The sensor (30) is preferably an optical sensor (such as a video sensor). Therefore, the neural network (60) can be understood as an image classifier.
[0099] The neural network (60) is configured to classify the results of the specimen based on the input image (x) of the microarray provided by the sensor (30). In particular, the neural network (60) can be configured to determine whether the microarray (601) indicates the presence of a virus in the specimen.
[0100] Then, the control signal (A) can be selected so that the display (10a) shows the result of the classification.
[0101] The term "computer" may be understood to cover any device for processing predefined computing rules. These computing rules may be in the form of software, hardware or a combination of software and hardware.
[0102] Generally, a plurality may be understood to be indexed, i.e., each element in the plurality is assigned a unique index, preferably by assigning consecutive integers to the elements contained in the plurality. Preferably, if the plurality comprises N elements, where N is the number of elements in the plurality, an integer from 1 to N is assigned to each element. It may also be understood that the elements in the plurality may be accessed by their index.
Claims
1. A computer-implemented method (100) for training a neural network (60), wherein the neural network (60) is configured to determine an output signal (y) based on an input signal (x), and wherein training comprises training parameters of a depthwise convolutional layer of the neural network (60), wherein the depthwise convolutional layer is initialized based on values extracted from a predefined probability distribution, wherein the variance of the probability distribution is characterized by the inverse of the square root of the number of filters applied at each depth of the input of the depthwise convolutional layer.
2. The method (100) of claim 1, wherein the input signal (x) represents a measurement result obtained from a sensor (30).
3. The method (100) according to claim 1 or 2, wherein the variance is determined according to the following formula: Where K w is the kernel width of the filter in the convolutional layer, K h is the kernel height of the filter of the convolutional layer, and k is the number of filters applied at each depth of the input to the depthwise convolutional layer.
4. The method (100) according to any of the preceding claims, wherein the predefined probability distribution is a normal distribution. The method (100) according to claim 4, wherein the expected value of the predefined probability distribution is zero.
6. A computer-implemented method for determining an output signal (y) based on an input signal (x), wherein the method comprises the following steps: ● obtaining a neural network (60) that has been trained using a method according to any one of claims 1 to 5; - Determining the output signal (y) based on the input signal (x) by means of the obtained neural network (60).
7. A computer-implemented neural network (60), wherein the neural network (60) is trained using the method according to any one of claims 1 to 5.
8. Control system (40), wherein the control system (40) is configured to use a neural network (60) according to claim 7 to determine control signals (A) for controlling actuators (10) and / or displays (10a) of a technical system (110, 200, 250, 300, 400, 500, 600).
9. A training system (140) configured to perform the training method according to any one of claims 1 to 5.
10. A computer program configured to cause a computer to perform the method according to any one of claims 1 to 6 and all its steps when the computer program is executed by a processor (45, 145).
11. A machine-readable storage medium (46, 146) on which a computer program according to claim 10 is stored.
