Apparatus and method for training a classifier using a reversible factorization model

Abstract

Apparatus and method for training a classifier using a reversible factorization model. Training the classifier comprises the steps of: • determining a first training input signal; • determining, based on the first training input signal, a first latent representation comprising a plurality of factors by means of a reversible factorization model, wherein the reversible factorization model is characterized by: - a plurality of functions, wherein any one function from said plurality of functions is continuous and almost everywhere continuously differentiable, wherein said function is further configured to accept as input the training input signal or at least one factor provided by another function from said plurality of functions, and wherein said function is further configured to provide at least one factor; • determining a second latent representation by adapting at least one factor of the first latent representation; • determining, based on the second latent representation, a second training input signal by means of the reversible factorization model; • training the classifier based on the second training input signal.

Description

Technical Field

[0001] The present invention relates to methods for obtaining adversarial input signals, methods for using and / or training classifiers, methods for evaluating the robustness of said classifiers, and methods for operating actuators, computer programs and machine-readable storage media, classifiers, control systems, and training systems. Existing technology

[0002] Jens Behrmann, Will Grathwohl, Ricky TQ Chen, David Duvenaud, and Jörn-Henrik Jacobsen's "Invertible Residual Networks" (https: / / arxiv.org / abs / 1811.00995v3, May 18, 2019) discloses a method for inverting neural networks.

[0003] Advantages of the present invention

[0004] Especially in the case of deep neural networks, classifiers typically require a large amount of labeled training data. Preferably, the training data is selected such that the classifier learns variations in the data that become invariant or equivalent to these variations. For example, when a classifier is configured to classify images, the classifier's generally preferred features will be invariant to changes in image contrast.

[0005] To train a classifier to be invariant or equivariant to certain changes, data augmentation can be used. The goal of data augmentation is to obtain at least one new training dataset by applying some form of transformation to existing training data, preferably such that the transformed training data still shares the same desired classification as the original training data; that is, the transformation is label-preserving. This takes into account obtaining more training data, which allows the classifier to learn more variations of the data it should be invariant or equivariant to, leading to better classification accuracy.

[0006] However, existing augmentation methods typically only consider a fairly limited number of invariants that can be learned. The method with the features of independent claim 1 has the advantage of being able to augment training data based on low-level factors such as brightness, noise, or contrast, and high-level factors such as the rotation of objects in a given scene. This method considers presenting the classifier with more factor data to which the classifier should be invariant or equivariant, which in turn further improves the classification accuracy of the classifier. Summary of the Invention

[0007] In a first aspect, the present invention relates to a computer-implemented method for training a classifier, wherein the classifier is configured to determine an output signal characterizing a classification of an input signal, wherein training the classifier includes the following steps:

[0008] ● Determine the first training input signal;

[0009] ● A first latent representation comprising multiple factors is determined based on a first training input signal using a reversible factorization model, wherein the reversible factorization model is characterized by:

[0010] - A plurality of functions, wherein any one of the plurality of functions is continuous and nearly everywhere continuously differentiable, wherein the function is further configured to accept a training input signal or at least one factor provided by another of the plurality of functions as input, and wherein the function is further configured to provide at least one factor, wherein the at least one factor is either provided as at least part of a potential representation or as at least part of the input of another of the plurality of functions, wherein there exists an inverse function corresponding to the function, the inverse function being continuous and nearly everywhere continuously differentiable, and being configured to determine the input of the layer based on the at least one factor provided from the function;

[0011] ● Determine the second latent representation by adapting at least one factor of the first latent representation;

[0012] ● The second training input signal is determined based on the second latent representation using an invertible factorization model;

[0013] ● Train the classifier based on the second training input signal.

[0014] The input signal may include or consist of at least one image. Images may be acquired from sensors such as camera sensors, LiDAR sensors, radar sensors, ultrasonic sensors, or thermal cameras. Images may also be acquired from simulated renderings, such as virtual environments created in a computer. Digital rendering of the images is also conceivable. In particular, the input signal may also include images of different modalities or images from different sensors of the same type. For example, the input signal may include two images acquired from a stereo camera. As another example, the input signal may also include images from a camera sensor and a LiDAR sensor.

[0015] The input signal may also include audio signals, such as recordings from a microphone or audio data obtained from computer simulation or computer digital generation. Audio signals may characterize speech recordings. Alternatively or additionally, audio signals may characterize recordings of audio events, such as alarm sounds, alerts, or other notification signals.

[0016] The training input signal can be obtained from a database storing the training input signals of the classifier. Alternatively, the training input signal can be obtained directly from the sensor.

[0017] The output signal can assign at least one label to the input signal that characterizes the classification of the input signal. For example, if the input signal includes an image, the output signal can characterize a single class to which the classifier believes the image belongs. Alternatively or additionally, the output signal can characterize the classification and location of objects in the image; that is, the classifier can be configured for object detection. Alternatively or additionally, the classifier can assign the class to which the classifier believes that pixel belongs to each pixel in the image; that is, the classifier can be configured for semantic segmentation.

[0018] It is also conceivable that if the input signal includes audio data, the output signal can assign the class to which the classifier believes the audio data belongs to the input signal. Alternatively or additionally, the output signal can also characterize audio events included in the audio signal and / or their location within the audio signal.

[0019] A reversible factorization model can be understood as a machine learning model. It takes an input signal as input and provides a latent representation, i.e., multiple factors, as output. Because each function in a reversible factorization model is invertible, the input signal can be reconstructed from the latent representation. Viewing a reversible factorization model as part of a neural network can be understood as the reverse propagation of the latent representation to obtain the input signal. A reversible factorization model can also be understood as a generative model.

[0020] A function derived from multiple functions and its corresponding inverse function can be understood as one or more layers of an invertible factorization model. The function can be used to pass the input of that layer forward to obtain its output, and the inverse function can be used to pass the input backward, i.e., to obtain the input from the output of that layer. By definition, the functions of an invertible model, and therefore the invertible model itself, are completely invertible. That is, if a latent representation is obtained from an invertible factorization model of an input signal, the input signal can be accurately reconstructed from the latent representation.

[0021] In a reversible factorization model, the information flow determines the order of its functions. If the first function provides factors to the second function, then the first function is considered to precede the second function, and the second function is considered to follow the first function.

[0022] Surprisingly, the inventors discovered that by designing a reversible factorization model, at least some factors of the latent representation are disentangled. A factor can be understood as disentangled if, starting from any given input signal, a certain direction in the vector space of the input signal changes only that factor while leaving all other factors relatively unchanged. Therefore, disentanglement can be understood as a continuous measurement of the factors. Thus, a factor can be considered disentangled if a change in direction corresponding to the input signal in the vector space does not change another factor by more than a predefined threshold.

[0023] Adapting the first latent representation can be understood as changing at least one factor of the first latent representation to achieve the desired effect. For example, if the first training input signal should be enhanced by contrast transformation, the factors of the contrast-related first latent representation can be changed to obtain a second latent representation. The second latent representation can then be processed by an invertible factorization model to determine the second training input signal.

[0024] It is also conceivable to modify more than one factor of the first latent representation, for example, to enhance the first training input signal with a change relative to more than one.

[0025] It can be further imagined that the classifier includes a reversible factorization model, and the output signal is determined based on the latent representation of the input signal, wherein the latent representation is determined by means of the reversible factorization model.

[0026] Deentangled factors are advantageous because they allow for the decomposition of the input signal, i.e., separation of the input signal into semantic aspects. For example, a first factor can change if the color of an object in an image included in the input signal changes. A second factor might change if the contrast of the image changes. A third factor might change based on the number of tires visible in the image. This is advantageous because the internal classifier bases its decisions on the factors. For example, if the internal classifier relies on the first and second factors to classify the input signal as containing a car, this indicates a strong bias in the classifier towards color and contrast, two semantic features that do not necessarily allow for the conclusion that the image contains a car. In this case, the classifier's user can identify this inappropriate relationship between the factors and the classification and train the classifier with new data to eliminate the classifier's bias towards color and contrast. Therefore, deentangled factors allow for direct insight into the internal workings of the classifier and can guide the user to train the classifier for better classification performance.

[0027] Another advantage of the unentangled factors provided by reversible factorization models is that they allow for robust evaluation of the semantic features, or factors, of the input signal. Because the factors are robust, the classifier's classification is also more robust and therefore more accurate after training.

[0028] Another advantage of reversible factorization models is that the latent representation can include factors from different functions derived from the reversible factorization model. For example, one can imagine multiple functions including at least one function that provides a first factor to the latent representation and a second factor to another function derived from multiple functions.

[0029] This is advantageous because the functions of the invertible model are able to extract increasingly more abstract information from the input signal. For example, a function closer to an input signal including an image can extract factors related to the image's color or contrast, while a function closer to the latent representation can extract high-level factors related to, for example, the number of objects in the image. Similarly, when the input signal includes an audio signal, a function closer to the input signal can extract factors related to the amplitude or frequency of the audio signal, while a factor closer to the latent representation can extract high-level factors related to, for example, the number of speakers that can be heard in the audio signal.

[0030] Compared to other generative models (i.e., autoencoders) that provide their latent representation based on the output of a single function, providing factors from different functions of the invertible factorization model to the latent representation allows functions closer to the latent representation to avoid carrying information about low-level factors (e.g., color, contrast, amplitude) through the invertible factorization model if these are irrelevant to the function, i.e., if high-level factors do not need to be determined. Thus, the capacity of the invertible factorization model is not burdened by having to carry unnecessary information through the model, which in turn allows functions closer to the latent representation to extract more high-level factors, as the model has the ability to focus on high-level factors. Furthermore, high-level factors enable the classifier to base its classification on more reliable information, thereby improving the classifier's classification accuracy.

[0031] The semantic aspects involved in determining the factors of a latent representation can be accomplished, for example, by means of an ablation test. For instance, a user can provide data where each data point differs from the others only in a single aspect. For example, a user can provide multiple images, each obtained from a single image by applying a random contrast transformation. Thus, all images show the same content and differ only in the aspect of "contrast." The user can then determine the factors of the latent representation that change with respect to the contrast, i.e., with respect to multiple images. This factor of the latent representation thus carries semantic information about the contrast in a given input image. The same approach can also be chosen for higher-level aspects, such as the number of objects visible in the image.

[0032] Alternatively, the semantic aspects of a factor can be determined by identifying the latent representation for a given input signal, the factors that change the latent representation, and the new input signal obtained from the changed latent representation.

[0033] It can be further imagined that a classifier is trained based on a first difference between a determined output signal and a desired output, wherein the determined output signal is determined for the training input signal, and the desired output signal characterizes the desired classification of the training input signal.

[0034] Because invertible models and internal classifiers can be configured to represent chain-like mathematical functions, they can be trained, for example, through backpropagation, like neural networks. This can be achieved by: determining the output signal of the input signal, determining the determined output signal and a first difference between the determined output signal and the input signal, determining the gradients of multiple parameters of the classifier with respect to the difference, and updating the parameters based on the gradients.

[0035] Surprisingly, the authors found that when training multiple parameters of the invertible factorization model on the difference, even more factors become unentangled. This further helps users identify semantic entities in the input signal of the classifier based on its decisions, and also further improves the classification accuracy of the classifier.

[0036] It can be further envisioned that the multiple functions of the reversible factorization model include at least one function configured to provide two factors according to the following formula.

[0037] ,

[0038] in It is the input of this function. z 1 is the first factor. z 2 is the second factor. It is an encoder for an automatic encoder, and It is the decoder of the autoencoder, and further, the inverse function corresponding to this function is given by the following equation.

[0039] .

[0040] The advantage of this method is that the encoder allows for the compression of information contained in its input. Therefore, the extracted factors convey a higher level of information, possessing all the advantages mentioned above.

[0041] Both encoders and / or decoders can be neural networks.

[0042] It can be further imagined that the training method includes additional steps to train the encoder and / or decoder based on the following loss function.

[0043] ,

[0044] in It is the input of the function, and the summation is over all the squared elements of the subtraction.

[0045] The inventors discovered that the method further unentangles the factors provided by the reversible factorization model, and thus allows the training of the classifier to become invariant to even more factors.

[0046] It can be further envisioned that the multiple functions of the reversible factorization model include at least one function configured to provide three factors according to the following formula.

[0047] ,

[0048] in It is the input of the function. z 1 is the first factor and the sum of the inputs. The result of application group standardization z 2 is the expected value of the second factor and the group standardization. z 3 is the third factor, and the group standardization and the standard deviation of the group standardization further depend on the scaling parameter. and shift parameters The inverse of the function is given by the following equation.

[0049] .

[0050] The advantage of this function is that group normalization smooths the gradients of multiple parameters in the invertible factorization model, which in turn leads to faster classifier training. Given the same amount of time, faster training allows the classifier to be trained with more input signals, thus improving its performance.

[0051] One can further imagine that multiple functions include at least one function that provides two factors according to the following formula.

[0052] ,

[0053] in It is the input of the function. z 1 is the first factor. z 2 is the second factor, and ReLU It is the linear unit of rectification, and further, the inverse function is given by the following equation.

[0054] .

[0055] The advantage of using this function in a reversible factorization model is that it transforms the model into a nonlinear one. The reversible factorization model can therefore extract factors in a nonlinear manner. Thus, it can identify factors that have a nonlinear relationship with the input signal. Since high-level factors typically have a nonlinear relationship with the input signal, this method allows for the unentanglement of these factors, which in turn allows for the acquisition of more unentangled factors. As explained above, this leads to higher classification accuracy in the classifier.

[0056] It can be further imagined that training the classifier includes the steps of training the reversible factorization model by adapting multiple parameters of the reversible factorization model according to the following loss function.

[0057] ,

[0058] in z It is a potential representation. This describes applying a reversible factorization model to an input signal to obtain a latent representation, and... The paper describes the application of a reversible factorization model to the latent representation in order to obtain the input signal.

[0059] Multiple parameters can be included in the trainable parameters of the function of the reversible factorization model, such as encoder weights or group-normalized shift or scaling parameters.

[0060] The inventors discovered that training the reversible factorization model in this way further increases the deentanglement of factors, which leads to an increase in the classification accuracy of the classifier as discussed above.

[0061] It can also be envisioned that training the classifier further includes the step of training a reversible factorization model using a neural architecture search algorithm, wherein the objective function of the neural architecture search algorithm is based on the disentanglement of multiple factors of the latent representation provided by the reversible factorization model.

[0062] This method is advantageous because it has more entangled potential representations and even further improves the classification accuracy of the classifier.

[0063] Training a reversible factorization model using neural architecture search (also known as NAS) can be understood as determining the architecture of the reversible factorization model, specifically functions from multiple functions of the reversible model, such that training a classifier leads to the best possible unentanglement of the factors.

[0064] Since the function (and its corresponding inverse function) can be understood as a layer of a neural network, the invertible factorization model can be trained according to any NAS algorithm.

[0065] The goal of applying the NAS algorithm can be understood as finding a reversible factorization model that maximizes the disentanglement of its factors. Therefore, any known disentanglement metric can be used as the objective function of the NAS algorithm, such as betaVAE, FactorVAE, mutual information difference, or DCI disentanglement.

[0066] Preferably, a NAS algorithm capable of multi-objective optimization, such as LEMONADE, is chosen. This is preferred because the architecture of the invertible factorization model can be trained to optimize both the disentanglement of the latent representation and the classification accuracy of the classifier. However, single-objective NAS algorithms are also possible. For these algorithms, the disentanglement metric can be used as the objective function.

[0067] One can further imagine controlling the device based on the output signal of the classifier.

[0068] This is advantageous because the increased sorting performance leads to better operation of the equipment. Attached Figure Description

[0069] Embodiments of the invention will be discussed in more detail with reference to the following figures. The figures illustrate:

[0070] Figure 1 It is a classifier;

[0071] Figure 2 It is a control system that includes a classifier that controls the actuators in its environment;

[0072] Figure 3 It is a control system that controls at least partially autonomous robots;

[0073] Figure 4 It is a control system for controlling manufacturing machines;

[0074] Figure 5 It is a control system for controlling automated personal assistants;

[0075] Figure 6 It is a control system that controls access control systems;

[0076] Figure 7 It is a control system for the control and monitoring system;

[0077] Figure 8 It is the control system for the imaging system;

[0078] Figure 9 It is a control system for the medical analysis system;

[0079] Figure 10 It is a training system used to train classifiers. Detailed Implementation

[0080] Figure 1 An embodiment of a classifier (60) for determining an output signal (y) that characterizes an input signal (x) is shown, wherein the classifier includes a reversible factorization model (61) and an internal classifier (62).

[0081] The input signal (x) is provided to the reversible factorization model and processed by multiple functions (F1, F2, F3, F4). The first function (F1) accepts the input signal as input and provides a first factor (z1) to the latent representation (z) and a second factor to the second function (F2). Based on the second factor, the second function (F2) determines a third factor. The third factor is provided to the third function (F3). Based on the third factor, the third function (F3) provides a fourth factor (z3) to the latent representation (z) and a fifth factor to the fourth function (F4). Based on the fifth factor, the fourth function (F4) then provides a sixth factor (z3) and a seventh factor (z4) to the latent representation (z).

[0082] The latent representation (z) is then forwarded to an internal classifier (62), which is configured to determine an output signal (y) based on the latent representation (z). In this embodiment, a linear classifier, such as a logistic regression classifier, is used as the internal classifier (62). In other embodiments, other internal classifiers (62) may also be used.

[0083] In even other embodiments, the reversible factorization model (61) may include functions different from those described in this embodiment.

[0084] Figure 2 The diagram illustrates a control system (40) for controlling the actuator (10) based on the output signal (y) of a classifier (60). At preferably uniformly spaced time points, a sensor (30) senses the condition of the actuator system. The sensor (30) may include several sensors. Preferably, the sensor (30) includes an optical sensor, such as a camera, for capturing images of the environment (20). Alternatively or additionally, the sensor (30) may also include an audio sensor, such as a microphone. The output signal (S) of the sensor (30) (or, in the case where the sensor (30) includes multiple sensors, an output signal (S) for each sensor) is transmitted to the control system (40), which encodes the sensed condition.

[0085] Therefore, the control system (40) receives a stream of sensor signals (S). It then calculates a series of control signals (A) based on the stream of sensor signals (S) and transmits the series of control signals (A) to the actuator (10).

[0086] The control system (40) receives a stream of sensor signals (S) from the sensor (30) in an optional receiving unit (50). The receiving unit (50) transforms the sensor signals (S) into an input signal (x). Alternatively, in the absence of a receiving unit (50), each sensor signal (S) can be directly taken as the input signal (x). The input signal (x) can be given, for example, as an extract from the sensor signals (S). Alternatively, the sensor signals (S) can be processed to generate the input signal (x). In other words, the input signal (x) is provided based on the sensor signals (S).

[0087] The input signal (x) is then passed to the classifier (60).

[0088] The classifier (60) is stored in the parameter storage device ( St 1) and the parameters provided by it ( Parameterization.

[0089] The classifier (60) determines the output signal (y) based on the input signal (x). The output signal (y) is transmitted to an optional conversion unit (80), which converts the output signal (y) into a control signal (A). The control signal (A) is then transmitted to the actuator (10) to control the actuator (10) accordingly. Alternatively, the output signal (y) can be directly used as the control signal (A).

[0090] The actuator (10) receives the control signal (A), is controlled accordingly, and performs an action corresponding to the control signal (A). The actuator (10) may include control logic that transforms the control signal (A) into another control signal, which is then used to control the actuator (10).

[0091] In another embodiment, the control system (40) may include a sensor (30). In even another embodiment, the control system (40) may alternatively or additionally include an actuator (10).

[0092] In yet another embodiment, it is conceivable that a control system (40) replaces the actuator (10) or controls the display device (10a) in addition to the actuator (10).

[0093] Furthermore, the control system (40) may include at least one processor (45) and at least one machine-readable storage medium (46) thereon storing instructions which, if executed, cause the control system (40) to perform the method according to aspects of the invention.

[0094] Figure 3 An embodiment is shown in which a control system (40) is used to control a robot that is at least partially autonomous, such as a vehicle (100) that is at least partially autonomous.

[0095] The sensor (30) may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors. Some or all of these sensors are preferably, but not necessarily, integrated into the vehicle (100). Thus, the input signal (x) can be understood as an input image, and the classifier (60) can be understood as an image classifier.

[0096] The image classifier (60) can be configured to detect objects near at least part of the autonomous robot based on the input image (x). The output signal (y) may include information characterizing that the object is located near at least part of the autonomous robot. A control signal (A) can then be determined based on this information, for example, to avoid collisions with the detected objects.

[0097] The actuator (10), preferably integrated in the vehicle (100), can be provided by the vehicle's (100) brakes, propulsion system, engine, drivetrain, or steering. A control signal (A) can be determined to cause the actuator (10) to be controlled so that the vehicle (100) avoids collisions with detected objects. The detected objects can also be classified according to what an image classifier (60) deems most likely to be—e.g., a pedestrian or a tree—and the control signal (A) can be determined based on this classification.

[0098] Alternatively or additionally, the control signal (A) may also be used to control the display (10a), for example, to display objects detected by the image classifier (60). It is also conceivable that if the vehicle (100) approaches and collides with at least one detected object, the control signal (A) may control the display (10a) to generate a warning signal. The warning signal may be an audible warning and / or tactile signal, such as vibration of the vehicle's steering wheel.

[0099] The factors (z1, z2, z3, z4) obtained from the input image (x) can, for example, be displayed on a display (10a). The driver of the vehicle (100) will thus be given insight into which factors (z1, z2, z3, z4) cause the image classifier (60) to determine whether an object is present or absent near the vehicle (100). If the classification is based on at least one incorrect or unwise factor (z1, z2, z3, z4), the driver can, for example, take over manual control of the vehicle (100).

[0100] In even further embodiments, factors (z1, z2, z3, z4) may also be transmitted to the vehicle operator. The operator does not necessarily have to be inside the vehicle (100), but may be located outside the vehicle (100). In this case, the factors may be transmitted to the operator by means of the vehicle's (100) wireless transmission system, such as by means of a cellular network. If the classification is based on at least one erroneous or unwise factor (z1, z2, z3, z4), the operator may, for example, take over manual control of the vehicle or issue a control signal (A) that causes the vehicle (100) to be in a safe state, such as pulling over to the side of the road on which the vehicle (100) is traveling.

[0101] In even further embodiments, the sensor (30) may be a microphone or may include a microphone. The classifier (60) can then be configured, for example, to classify audio events near the vehicle (100) and control the vehicle (100) accordingly. For example, if the classifier (60) can detect the siren of a police car or fire truck. The conversion unit (80) can then determine a control signal (A) causing the vehicle (100) to open a rescue lane on its current path.

[0102] In some embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving, or walking. The mobile robot may, in particular, be a lawnmower that is at least partially autonomous, or a cleaning robot that is at least partially autonomous. In these additional embodiments, a control signal (A) may be determined such that the propulsion unit and / or steering and / or braking of the mobile robot are controlled, enabling the mobile robot to avoid collisions with the identified object. In these additional embodiments, the factors (z1, z2, z3, z4) may also be transmitted to an operator via wireless transmission, such as a cellular network.

[0103] In another embodiment, at least partially autonomous robotic operation may be provided by a gardening robot (not shown) that uses sensors (30), preferably optical sensors, to determine the state of the plants in the environment (20). Actuators (10) may control nozzles for spraying liquid and / or cutting devices such as blades. Depending on the type and / or state of the plant markings, control signals (A) may be determined to cause the actuators (10) to spray the appropriate amount of liquid onto the plant and / or cut the plant.

[0104] In even further embodiments, at least partially autonomous robots may be provided by household appliances (not shown), such as washing machines, stoves, ovens, microwave ovens, or dishwashers. Sensors (30), such as optical sensors, can detect the state of the object to be processed by the household appliance. For example, in the case of a washing machine, sensor (30) can detect the state of the clothes inside the washing machine. A control signal (A) can then be determined based on the detected material of the clothes.

[0105] Figure 4 An embodiment is shown, wherein a control system (40) controls a manufacturing machine (11), such as a stamping tool, cutting tool, gun drill, or jig, of a manufacturing system (200) that is, for example, part of a production line. The manufacturing machine may include transport equipment, such as a conveyor belt or assembly line, for moving manufactured products (12). The control system (40) controls an actuator (10), which in turn controls the manufacturing machine (11).

[0106] The sensor (30) can be provided by an optical sensor that captures, for example, the characteristics of the manufactured product (12). The classifier (60) can therefore be understood as an image classifier.

[0107] The image classifier (60) can determine the position of the manufactured product (12) relative to the transport equipment. The actuator (10) can then be controlled depending on the determined position of the manufactured product (12) for subsequent manufacturing steps of the manufactured product (12). For example, the actuator (10) can be controlled to cut the manufactured product at a specific location on the manufactured product itself. Alternatively, it is conceivable that the image classifier (60) classifies whether the manufactured product is broken or exhibits defects. The actuator (10) can then be controlled to remove the manufactured product from the transport equipment.

[0108] The factors (z1, z2, z3, z4) obtained from the input image (x) can, for example, be displayed on a display (10a). The operator of the manufacturing system (200) will thus be given insight into which factors (z1, z2, z3, z4) cause the image classifier (60) to determine the output signal (y). If the classification is based on at least one incorrect or unwise factor (z1, z2, z3, z4), the operator can, for example, take over the manual control of the manufacturing system (200).

[0109] Figure 5 An embodiment is shown, in which a control system (40) controls an automated personal assistant (250). The sensor (30) may be an optical sensor, such as a video image of a user's (249) gesture. Alternatively, the sensor (30) may also be an audio sensor, such as a voice command from the user (249).

[0110] The control system (40) then determines a control signal (A) for controlling the automated personal assistant (250). The control signal (A) is determined based on sensor signals (S) from the sensor (30). The sensor signals (S) are transmitted to the control system (40). For example, a classifier (60) can be configured to, for example, implement a gesture recognition algorithm to identify gestures made by the user (249). The control system (40) can then determine the control signal (A) to be transmitted to the automated personal assistant (250). It then transmits the control signal (A) to the automated personal assistant (250).

[0111] For example, a control signal (A) can be determined based on a user gesture identified by a classifier (60). This can include information that causes an automated personal assistant (250) to retrieve information from a database and output that retrieved information in a form suitable for the user (249) to receive.

[0112] In another embodiment, it is conceivable that, instead of the automated personal assistant (250), the control system (40) controls a household appliance (not shown) controlled according to an identified user gesture. The household appliance may be a washing machine, stove, oven, microwave oven, or dishwasher.

[0113] Figure 6 An embodiment is shown, in which the control system (40) controls the access control system (300). The access control system (300) can be designed to physically control access. For example, it can include a door (401). The sensor (30) can be configured to detect scenarios related to deciding whether access is permitted. For example, it can be an optical sensor for providing image or video data, such as for detecting a person's face. The classifier (60) can therefore be understood as an image classifier.

[0114] An image classifier (60) can be configured to classify a person's identity, for example, by matching the detected person's face with other faces of known people stored in a database. A control signal (A) can then be determined based on the classification by the image classifier (60), for example, according to the determined identity. An actuator (10) can be a lock that opens or closes a door depending on the control signal (A). Alternatively, the access control system (300) can be a non-physical, logical access control system. In this case, the control signal can be used to control a display (10a) to show information about the person's identity and / or whether the person has been granted access.

[0115] The factors (z1, z2, z3, z4) obtained for the input image (x) can be displayed, for example, on a display (10a). Thus, the operator of the access control system (300) will be given insight into which factors (z1, z2, z3, z4) cause the image classifier (60) to determine the output signal (y). If the classification is based on at least one incorrect or unwise factor (z1, z2, z3, z4), the operator can, for example, deny access.

[0116] Figure 7 An embodiment is shown, wherein the control system (40) controls the monitoring system (400). This embodiment is largely consistent with... Figure 6 The embodiments shown are identical. Therefore, only the differences will be described in detail. The sensor (30) is configured to detect the monitored scene. The control system (40) does not necessarily control the actuator (10), but may instead control the display (10a). For example, the image classifier (60) may determine the classification of the scene, for example, whether the scene detected by the optical sensor (30) is normal or whether the scene exhibits anomalies. The control signal (A) transmitted to the display (10a) may then be configured, for example, to cause the display (10a) to adjust the displayed content based on the determined classification, for example, highlighting objects that the image classifier (60) considers abnormal.

[0117] Figure 8 An embodiment of a medical imaging system (500) controlled by a control system (40) is shown. The imaging system may be, for example, an MRI device, an X-ray imaging device, or an ultrasound imaging device. The sensor (30) may be, for example, an imaging sensor that captures at least one image of the patient, thereby displaying, for example, different types of the patient's body tissues.

[0118] The classifier (60) can then determine the classification of at least a portion of the sensed image. Thus, at least a portion of the image is used as the input image (x) to the classifier (60). The classifier (60) can therefore be understood as an image classifier.

[0119] The control signal (A) can then be selected based on the classification, thereby controlling the display (10a). For example, the image classifier (60) can be configured to detect different types of tissue in the sensed image, for example, by classifying the tissue displayed in the image as malignant or benign tissue. This can be accomplished by semantic segmentation of the input image (x) by the image classifier (60). The control signal (A) can then be determined to cause the display (10a) to display different tissues, for example, by displaying the input image (x) and coloring different regions of the same tissue type with the same color.

[0120] The factors (z1, z2, z3, z4) obtained for the input image (x) can be displayed, for example, on a display (10a). The operator of the imaging system (500) will thus be given insight into which factors (z1, z2, z3, z4) cause the image classifier (60) to determine the output signal (y). If the classification is based on at least one incorrect or unwise factor (z1, z2, z3, z4), the operator can, for example, further examine the input image (x).

[0121] In another embodiment (not shown), the imaging system (500) can be used for non-medical purposes, such as determining the material properties of a workpiece. In these embodiments, an image classifier (60) can be configured to receive an input image (x) of at least a portion of the workpiece and perform semantic segmentation of the input image (x) to classify the material properties of the workpiece. A control signal (A) can then be determined to cause a display (10a) to show the input image (x) and information about the detected material properties.

[0122] Figure 9 An embodiment of a medical analysis system (600) controlled by a control system (40) is shown. The medical analysis system (600) is equipped with a microarray (601), wherein the microarray includes multiple points (602, also referred to as features) that have been exposed to a medical sample. The medical sample may be, for example, a human sample or an animal sample obtained from a swab.

[0123] The microarray (601) can be a DNA microarray or a protein microarray.

[0124] The sensor (30) is configured to sense a microarray (601). The sensor (30) is preferably an optical sensor, such as a video sensor. The classifier (60) can therefore be understood as an image classifier.

[0125] The image classifier (60) is configured to classify the results of the sample based on the input image (x) of the microarray supplied by the sensor (30). In particular, the image classifier (60) can be configured to determine whether the microarray (601) indicates the presence of a virus in the sample.

[0126] Then, a control signal (A) can be selected to make the display (10a) show the classification results.

[0127] The factors (z1, z2, z3, z4) obtained for the input image (x) can be displayed, for example, on a display (10a). Thus, the operator of the analysis system (600) will be given insight into which factors (z1, z2, z3, z4) cause the image classifier (60) to determine the output signal (y). If the classification is based on at least one incorrect or unwise factor (z1, z2, z3, z4), the operator can, for example, further examine the sample or issue another test or another swab.

[0128] Figure 10 An embodiment of a training system (140) for training a classifier (60) of a control system (40) using a training dataset (T) is shown. The training dataset (T) includes multiple input signals (x) used to train the classifier (60). i ), wherein the training dataset (T) further includes, for each input signal (x) i ), corresponding to the input signal (x) i ) and characterize the input signal (x) i The expected output signal (y) of the classification i ).

[0129] For training purposes, the training data unit (150) accesses a computer-implemented database (St2) that provides a training dataset (T). The training data unit (150) preferably randomly determines at least one input signal (x) from the training dataset (T). i ) and corresponding to the input signal (x) i The expected output signal (y) i ), and the input signal (x) i ) is transmitted to classifier (60).

[0130] The classifier (60) determines the input signal (x) using a reversible factorization model (61). i The latent representation (z) of ) is then determined based on the input signal (x). i Which feature of the latent representation (z) will be modified to alter one or more desired factors to adapt to the latent representation (z). For example, a contrast-related factor could be adapted. The adapted latent representation is then backpropagated through a reversible factorization model to determine the second input signal (z). Based on the second input signal ( The classifier (60) then determines the determined output signal. ).

[0131] Desired output signal (y) i ) and a defined output signal ( ) is transmitted to the modification unit (180).

[0132] Based on the desired output signal (y) i ) and a defined output signal ( ), modify unit (180) and then determine the new parameters for classifier (60) ( For this purpose, the modification unit (180) uses a loss function to reduce the desired output signal (y). i ) and a defined output signal ( The loss function is used to determine the output signal that has been determined. ) and the desired output signal (y i The first loss value is the value that deviates from the given loss function. In the given embodiment, the negative log-likelihood function is used as the loss function. In alternative embodiments, other loss functions are also conceivable.

[0133] Furthermore, it is conceivable that a definite output signal ( ) and the desired output signal (y i Each of these components comprises multiple sub-signals, for example, in the form of a tensor, where the desired output signal (y) is... i The sub-signal of ) corresponds to a defined output signal ( The first sub-signal represents the object relative to the input signal (x). For example, it is conceivable that the classifier (60) is configured for object detection, and the first sub-signal represents the object relative to the input signal (x). i The probability of occurrence of a portion of the signal is given by the first sub-signal, and the second sub-signal characterizes the exact location of the object. If the determined output signal ( ) and the desired output signal (y i If the signal includes multiple corresponding sub-signals, then a second loss value is preferably determined for each corresponding sub-signal by means of a suitable loss function, and the determined second loss value is appropriately combined, for example by means of weighted sum, to form a first loss value.

[0134] The modification unit (180) determines new parameters based on the first loss value. In the given embodiment, this is accomplished using a gradient descent method, preferably stochastic gradient descent, Adam, or AdamW.

[0135] In other preferred embodiments, the described training iteratively repeats a predefined number of iteration steps, or iteratively repeats until a first loss value falls below a predefined threshold. Alternatively or additionally, it is conceivable that training terminates when the average first loss value relative to the test or validation dataset falls below a predefined threshold. In at least one iteration, new parameters (determined in the previous iteration) are used... ) is used as a parameter of classifier (60) ).

[0136] In even further embodiments, the classifier is trained based on a neural architecture search algorithm, wherein at least several functions (F1, F2, F3, F4) of the reversible factorization model (61) of the classifier (60) are determined based on the neural architecture search algorithm. In this embodiment, the LEMONADE algorithm is used as the neural architecture search algorithm. However, other neural architecture search algorithms are conceivable in other embodiments. The DCI deentanglement and classification accuracy of the image classifier (60) can be selected as the target of the LEMONADE algorithm.

[0137] Furthermore, the training system (140) may include at least one processor (145) and at least one machine-readable storage medium (146) containing instructions that, when executed by the processor (145), cause the training system (140) to perform a training method according to one aspect of the invention.

[0138] The term "computer" can be understood to encompass any device used to process predefined computational rules. These computational rules can be in the form of software, hardware, or a combination of both.

[0139] Generally, "multiple" can be understood as being indexed, that is, preferably by assigning consecutive integers to the elements contained in the multiple, with each element in the multiple being assigned a unique index. Preferably, if the multiple has N There are elements, among which N If the number of elements in a plurality is a given number, then the elements are assigned a number from 1 to 1. N Integers. It can also be understood that multiple elements can be accessed through their indices.

Claims

1. A computer-implemented method for training a classifier (60), wherein the classifier (60) is configured to determine an output signal (y) characterizing a classification of an input signal (x), wherein the input signal (x) includes an image signal or an audio signal, and the output signal (y) assigns at least one label characterizing a classification of the input signal (x) to the input signal (x), wherein training the classifier (60) includes the following steps: ● Determine the first training input signal (x) from the training dataset (T). i The training dataset (T) also includes the expected output signal (yi) for each input signal (xi). ● Using the reversible factorization model (61) based on the first training input signal (x) i ) Determine the first latent representation (z) comprising multiple factors (z1, z2, z3, z4), wherein the reversible factorization model (61) is characterized by: - Multiple functions (F1, F2, F3, F4), wherein each of the multiple functions (F1, F2, F3, F4) is continuous and almost everywhere continuously differentiable, wherein the functions (F1, F2, F3, F4) are further configured to receive a training input signal (x). i The input is either a factor (z1, z2, z3, z4) provided by another function (F1, F2, F3, F4) among the plurality of functions (F1, F2, F3, F4), and each function (F1, F2, F3, F4) is further configured to provide at least one factor (z1, z2, z3, z4), wherein the at least one factor (z1, z2, z3, z4) is either provided as at least a part of the latent representation (z) or provided as at least a part of the input of another function (F1, F2, F3, F4) among the plurality of functions, wherein the plurality of factors (z1, z2, z3, z4) of the latent representation (z) are... z2, z3, z4) are provided by at least two different functions (F1, F2, F3, F4) among the plurality of functions (F1, F2, F3, F4) of the invertible decomposition model (61), wherein the corresponding functions (F1, F2, F3, F4) among the plurality of functions (F1, F2, F3, F4) of the invertible decomposition model (61) are invertible, wherein for each corresponding function (F1, F2, F3, F4), there exists a corresponding inverse function, wherein each inverse function is continuous, almost everywhere continuously differentiable, and is configured to determine the input of the corresponding function (F1, F2, F3, F4) based on at least one factor (z1, z2, z3, z4) provided from the function (F1, F2, F3, F4); ● Determine the second latent representation by changing at least one factor (z1, z2, z3, z4) of the first latent representation (z); ● The second training input signal is determined based on the second latent representation using backpropagation with the aid of an invertible factorization model; ● The output signal is determined based on the second input signal using a classifier (60). ) and the determined output signal ( The classifier is trained by comparing the input signal with the corresponding expected output signal (yi) and training the classifier based on the second training input signal.

2. The method of claim 1, wherein the classifier (60) comprises a reversible factorization model, and the output signal (y) is determined based on the latent representation (z) of the input signal (x), wherein the latent representation (z) is determined by means of the reversible factorization model (61).

3. The method according to claim 1, wherein the plurality of functions (F1, F2, F3, F4) of the reversible factorization model (61) includes at least one function (F1, F2, F3, F4) that provides a first factor (z1, z2, z3, z4) to the latent representation (z) and provides a second factor (z1, z2, z3, z4) to another function (F1, F2, F3, F4) from the plurality of functions (F1, F2, F3, F4).

4. The method according to any one of the preceding claims, wherein the plurality of functions (F1, F2, F3, F4) of the reversible factorization model (61) includes at least one function (F1, F2, F3, F4), said at least one function (F1, F2, F3, F4) being configured to provide two factors according to the following formula. ， in It is the input of the function (F1, F2, F3, F4). z 1 is the first factor. z 2 is the second factor. It is an encoder for an automatic encoder, and It is the decoder of the autoencoder, and further within it, The inverse function corresponding to the stated function is given by the following equation. 。 5. The method of claim 3, wherein the method further comprises the step of training an encoder and / or decoder based on the following loss function. ， in It is the input of the function, and the summation is over all the squared elements of the subtraction.

6. The method according to any one of claims 1-3, wherein the plurality of functions (F1, F2, F3, F4) of the reversible factorization model (61) includes at least one function (F1, F2, F3, F4), said at least one function (F1, F2, F3, F4) being configured to provide three factors according to the following formula. ， in It is the input of the function (F1, F2, F3, F4). z 1 is the first factor and the sum of the inputs. The result of application group standardization z 2 is the expected value of the second factor and the group standardization. z 3 is the third factor, and the group standardization and the standard deviation of the group standardization further depend on the scaling parameter. and shift parameters The inverse of the function is given by the following equation. 。 7. The method according to any one of claims 1-3, wherein the plurality of functions (F1, F2, F3, F4) of the reversible factorization model (61) includes at least one function (F1, F2, F3, F4), said at least one function (F1, F2, F3, F4) providing two factors according to the following formula. ， in It is the input of the function (F1, F2, F3, F4). z 1 is the first factor. z 2 is the second factor, and ReLU It is the linear unit of rectification, and further within it, The inverse function of (F1, F2, F3, F4) is given by the following equation. 。 8. The method according to any one of claims 1-3, wherein training the classifier (60) comprises the step of training the reversible factorization model (61) by adapting a plurality of parameters of the reversible factorization model according to the following loss function. ， in z It is a potential representation. This describes applying a reversible factorization model to an input signal (x) to obtain a latent representation (z), and The application of the reversible factorization model (61) to the latent representation (z) is described in order to obtain the input signal (x).

9. The method according to any one of claims 1-3, wherein training the classifier (60) further comprises the step of training a reversible factorization model (61) by means of a neural architecture search algorithm, wherein the objective function of the neural architecture search algorithm is based on the maximum untangling of multiple factors (z1, z2, z3, z4) of the latent representation (z) provided by the reversible factorization model (61).

10. The method according to any one of claims 1-3, wherein the device (100, 200, 250, 300, 400, 500, 600) is controlled based on the output signal.

11. A training system (140) configured to perform the method according to any one of claims 1 to 10.

12. A computer program product configured such that, if executed by a processor (45, 145), it causes a computer to perform the method and all steps thereof according to any one of claims 1 to 10.

13. A machine-readable storage medium (46, 146) thereon storing a computer program product according to claim 12.

Images