Apparatus and method for training a classifier - Patent application

Abstract

To optimize performance of a class-classifier for performance metrics which cannot be decomposed.SOLUTION: In a training system 140, a computer-implemented method trains a class-classifier 60, particularly a binary class-classifier, for class-classifying input signals (xi), and a non-decomposable metric measures an alignment between class-classifications (yi) corresponding to the input signals (xi) of a set of training data and corresponding predicted class-classifications of the input signals obtained from the class-classifier. The method includes the steps of: providing weighting factors that characterize how the non-decomposable metric depends on a plurality of terms by a confusion matrix of the class-classifications(yi) and the predicted class-classifications; and training the class-classifier 60 according to the provided weighting factors.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a method for training a classifier, a method for using the classifier, a computer program and machine-readable storage medium, a control system, and a training system.

[0002] prior art Zhan Shi, Xinhua Zhang, and Yaoliang Yu, "Bregman divergence for stochastic variance reduction: Saddle-point and adversarial prediction," Advances in Neural Information Processing Systems, 2017, pp. 6033-6043, discloses a machine learning method for optimizing the F1-score metric by applying an adversarial prediction technique that uses marginalization to reduce the optimization for an exponential full-size conditional distribution to polynomial-size marginal distributions.

[0003] Hong Wang, Wei Xing, Kaiser Asif, and Brian Ziebart, Adversarial prediction games for multivariate losses, Advances in Neural Information Processing Systems, 2015, pp. 2710-2718, disclose the use of a dual oracle technique to train classifiers for several performance metrics.

[0004] Advantages of the Invention While the accuracy metric is the most common evaluation measure, many applications require the use of more complex evaluation metrics that do not lend themselves to additive decomposition into per-sample measures, i.e., cannot be expressed as the sum of the contributions of the individual samples in the evaluation dataset.

[0005] In real-world applications, the performance of machine learning algorithms is preferably measured by an evaluation metric that is specifically customized for the problem of interest, and such evaluation metrics are often non-decomposable metrics.

[0006] For example, in an optical inspection task, which may be used to automatically inspect whether or not an item is being produced according to specifications in an industrial manufacturing environment, examples of such non-decomposable metrics are precision, recall, specificity, or F β It may be desirable to use a score.

[0007] For example, suppose a "1" label means "OK" and a "0" means "Not OK." Precision, or the ratio of true positives to predicted positives, measures how many "1" labels are actually correct. An end-of-line test with 100% precision would result in no defective products being shipped. A machine trained for high precision could be expected to trust all "1" labels and perform (potentially very expensive) retests only on products labeled as "0."

[0008] Recall, or the ratio of true positives to actual positives, measures how many "OK" cases are actually correctly labeled. If recall is very high, it might be possible to forgo costly retesting of items labeled as "0" and immediately discard them.

[0009] Specificity, or the ratio of true negatives to actual negatives, measures how many "not OK" cases are correctly labeled as "0." If specificity is high, one can expect to immediately ship all items labeled as "1."

[0010] F βcan be thought of as a smooth interpolation between precision (β=0) and the harmonic mean between precision and recall (β=1), satisfying both the goals of high precision and high recall.

[0011] These metrics are also important in other applications. For example, in tasks that rely on semantic segmentation of incoming video images (e.g., pedestrian detection in autonomous vehicles or suspicious object detection in video surveillance systems), F1 is an important optimization objective. This is because recall is typically higher than precision for objects that occupy a large portion of the image, while the opposite is true for small objects. Penalizing the simultaneous occurrence of low recall and low precision improves the resulting segmentation.

[0012] For an automated personal assistant that uses a classifier to interpret user commands, it may be desirable for the automated personal assistant to correctly recognize as many actual commands as possible, in which case a high precision rate may be a desirable evaluation metric.

[0013] In the case of an access control system that may grant access depending on the output of a classifier, where it may be desirable to not grant access to unauthorized persons, high specificity may be desirable in such a system.

[0014] Furthermore, F β The use of metrics is useful for classification tasks involving imbalanced datasets. In the medical field, for example, when evaluating images from an imaging system, recall, specificity, and information availability are preferred metrics to ensure good classification performance.

[0015] In other words, optimizing according to such non-decomposable metrics in training has great practical value. However, training algorithms for non-decomposable metrics are not widely used in practical applications, especially in current machine learning applications that rely on the expressive power of deep architectures, which are typically trained using gradient-based methods. Instead of being trained to optimize the evaluation metric of interest, they can be trained to minimize the cross-entropy error, in the hope of indirectly optimizing the non-decomposable metrics as well.

[0016] A method having the features of independent claim 1 has the advantage that a gradient-based learning procedure can be used to optimize the performance of the classifier for a wide range of non-decomposable performance metrics, resulting in improved performance for these metrics.

[0017] Further improvements are set out in the dependent claims.

[0018] DISCLOSURE OF THE INVENTION In a first aspect, the present invention provides a computer-implemented method for training a classifier, in particular a binary classifier, for classifying input signals to optimize performance according to a non-decomposable metric, the non-decomposable metric measuring the consistency between classifications corresponding to input signals in a training dataset and corresponding predicted classifications of said input signals obtained from said classifier (in other words, the metric measures how well the classifications match with the corresponding predicted classifications), said method comprising: - providing weighting coefficients that characterize how the indecomposable metric depends on a number of terms from a confusion matrix between the classification and the predicted classification; - training said classifier in response to said provided weighting coefficients. The present invention has the advantage that it is possible to automatically perform classifier optimization for a wide range of non-decomposable metrics in response to provided weighting coefficients.

[0019] The non-decomposable metric is assigned a weighting factor a j ,b j ,f j ,g j Formula using

number

[0020] It has been found that a non-decomposable metric that can be written in this form allows for efficient training of the classifier depending on the weighting coefficients provided.

[0021] Note that the dependencies on entries PN and AN are redundant and will be ignored below.

[0022] Examples of such metrics are shown in the following table: [Table 2] is.

[0023] Preferably, the optimization is performed as an adversarial prediction method, i.e. by finding an equilibrium, more specifically a Nash equilibrium, of a two-player game between a first player (predictor) and a second player (adversary), where the first player tries to find a first classification corresponding to (all) input values ​​of the data, and the second player tries to find a second classification corresponding to (all) input values ​​of the data, and the first player tries to maximize and the second player tries to minimize the expected value of the metric by which the confusion matrix is ​​evaluated based on the first and second classifications, the second classification being subject to a moment matching constraint.

[0024] In mathematics, the adversarial prediction is called

number

number

number

number

[0025] The adversarial player determines the conditional probability that the expected value of a feature matches the empirical feature statistics.

number

number

[0026] where Φ represents the feature vector of the classifier. For example, if the classifier is provided by a neural network, Φ is the input to the last fully connected layer, which acts as a linear classifier for the features. The feature function is additive, i.e., Φ(x,y)=Σ i Φ(x i ,y i ) for simplicity, in the following we use Φ(x i ,y i = 0) = 0 (otherwise we can replace the feature extractor φ with φ', with Φ'(x,0) = 0 and Φ'(x,1) = Φ(x,1) - Φ(x,0).

[0027] The boundary condition of the minimax optimization is called a "moment matching constraint." In other words, the "moment matching constraint" means that the empirical expected value of the feature vector Φ on the test data is such that the empirical distribution of the input signal (x) is

number

number

[0028] It should be noted that while this description focuses on binary classifiers, it can be readily applied to training general classifiers. To this end, it can be envisioned to create a general classifier using multiple binary classifiers. For example, it can be envisioned to create a single binary classifier for each class into which the general classifier classifies input data, where each binary classifier must determine whether the input data belongs to the associated class. Alternatively, it can be envisioned to arrange the total number of target classes in a tree-like classification with binary branches, where multiple binary classifiers are used.

[0029] Performing this optimization with the adversarial prediction method is a way to robustly maximize a performance metric against an adversary. In other words, the resulting classifier is more robust. We show how to apply this adversarial prediction framework to classification problems to optimize an indecomposable metric during training.

[0030] For example, the optimization may be performed by finding the optimal value of the Lagrangian multiplier corresponding to the moment matching constraint, where the trained parameters of the fully connected layer of the binary classifier are set equal to the optimal value of the Lagrangian multiplier (note that the Lagrangian multiplier is vector-valued). This exploits the strong duality of the convex-concave saddle point problem:

number

[0031] It has been found that the parameters of the fully connected layer can be conveniently optimized by setting them equal to the optimal values ​​of the Lagrange multipliers, subject to moment matching constraints.

[0032] To solve this efficiently, preferably the expectation is calculated based on marginal probabilities of the first classification and / or the second classification, where the marginal probabilities represent the marginal probabilities that the classification of a given input value is equal to a predetermined classification and that the sum of all classifications is equal to a predetermined sum.

[0033] In other words, the marginal probability of the first classification

number

number

number

number

[0034] Similarly, for the corresponding marginal probability of the adversary,

number

number

number

[0035] The total marginal probability is

number

number

[0036] To understand how this contributes to the solution of equation (2), consider the interior minimax problem of equation (2), i.e.,

number

[0037] Using the notation above, the exponentially sized conditional probability

number

number

number

[0038] Some metrics (e.g., precision, recall, F-measure, sensitivity, specificity) perform special cases to avoid division by zero. For metrics that involve true positives, the special cases are usually

number

number

[0039] where:

number

number

number

[0040] Each column P (:,k) but

number

number

number

number

[0041] As a very efficient way of solving the optimization, we can imagine solving the two-player game by solving a linear program for only one of the two players. This is possible because we can convert the inner minimizer for Q in equation (5) into

number

[0042] In some machine learning settings, it may be desirable to optimize a performance metric subject to constraints on other metrics, which arise when there are trade-offs between different performance metrics.

[0043] For example, a machine learning system may wish to optimize the precision of its predictions, subject to the condition that the recall is greater than a pre-definable threshold. In these tasks, the adversarial prediction formula may be:

number

number

number

[0044] Therefore, according to a further aspect of the invention, the optimization of the performance according to the non-decomposable metric is further subject to an inequality constraint on the expectation of a second metric measuring the consistency between the classification and the predicted classification.

[0045] This too,

number

[0046] where μ l is a constant, and B (l) is a matrix, both of which are analytically defined by the l-th metric constraint and the ground truth label.

[0047] Embodiments of the present invention will now be described in more detail with reference to the following figures. [Brief explanation of the drawings]

[0048] [Figure 1] FIG. 1 illustrates a control system with a classifier for controlling actuators in an environment. [Figure 2] FIG. 1 illustrates a control system for controlling an at least partially autonomous robot. [Figure 3]FIG. 1 illustrates a control system for controlling a manufacturing machine. [Figure 4] FIG. 1 illustrates a control system for controlling an automated personal assistant. [Figure 5] FIG. 1 illustrates a control system for controlling the access control system. [Figure 6] FIG. 1 is a diagram illustrating a control system that controls the monitoring system. [Figure 7] FIG. 2 is a diagram illustrating a control system that controls the imaging system. [Figure 8] FIG. 1 illustrates a training system for training a classifier. [Figure 9] FIG. 1 illustrates an exemplary structure of a classifier. [Figure 10] FIG. 2 is a flow chart diagram of a training method performed by the training system. [Figure 11] FIG. 1 is a flow chart diagram of one embodiment of this training method.

[0049] Description of the embodiment 1 shows an embodiment of an actuator 10 in its environment 20. The actuator 10 interacts with a control system 40. The actuator 10 may be a technical system that is able to receive actuator control commands A and to act according to said received actuator control commands A. The actuator 10 and its environment 20 together are called an actuator system. Sensors 30, preferably at equally spaced distances, detect the state of the actuator system. The sensor 30 may comprise several sensors. Preferably, the sensor 30 is an optical sensor that images the environment 20. An output signal S of the sensor 30 (or of each sensor, if the sensor 30 comprises several sensors) encoding the detected situation is sent to the control system 40.

[0050] Thereby, the control system 40 receives the stream of sensor signals S. The control system then calculates a series of actuator control commands A in response to the stream of sensor signals S, which are then transmitted to the actuator 10.

[0051] The control system 40 receives a stream of sensor signals S from the sensors 30 via an optional receiving unit 50. The receiving unit 50 converts the sensor signals S into input signals x. Alternatively, in the absence of a receiving unit 50, each sensor signal S may be received directly as an input signal x. The input signal x may, for example, be provided as part of the sensor signals S. Alternatively, the sensor signals S may be processed to obtain the input signals x. The input signals x may include image data corresponding to images recorded by the sensors 30, or may include audio data, for example if the sensors 30 are audio sensors. In other words, the input signals x may be provided in accordance with the sensor signals S.

[0052] The input signal x is then sent to a classifier 60, for example an image classifier, which may for example be provided by an artificial neural network.

[0053] The classifier 60 is parameterized by parameters ξ stored in and provided by a parameter store St1.

[0054] The classifier 60 determines an output signal y from the input signal x. The output signal y includes information that assigns one or more labels to the input signal x. The output signal y is sent to an optional transformation unit 80, which transforms the output signal y into control commands A. The actuator control commands A are then sent to the actuator 10 to control the actuator 10 accordingly. Alternatively, the output signal y may be directly captured as the control commands A.

[0055] The actuator 10 receives the actuator control command A and is controlled accordingly to perform an action corresponding to the actuator control command A. The actuator 10 may comprise control logic that converts the actuator control command A into a further control command, which is then used to control the actuator 10.

[0056] In further embodiments, the control system 40 may comprise a sensor 30. In still further embodiments, the control system 40 may alternatively or additionally comprise an actuator 10.

[0057] In still further embodiments, it may be envisaged that the control system 40 controls the display 10 a instead of the actuator 10 .

[0058] Additionally, the control system 40 may include a processor 45 (or multiple processors) and at least one machine-readable storage medium 46 having stored thereon instructions that, when executed, cause the control system 40 to perform a method according to one aspect of the present invention.

[0059] FIG. 2 illustrates an embodiment in which a control system 40 is used to control an at least partially autonomous robot, such as an at least partially autonomous vehicle 100.

[0060] The sensors 30 may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more location sensors (such as GPS). Some or all of these sensors are preferably, but not necessarily, onboard the vehicle 100. Alternatively or additionally, the sensors 30 may include an information system for determining the state of the actuator system. One example of such an information system is a weather information system that determines current or future weather conditions in the environment 20.

[0061] For example, using input signal x, classifier 60 may detect, for example, an object in the vicinity of the at least partially autonomous robot. Output signal y may include information characterizing where the object is located in the vicinity of the at least partially autonomous robot. Control commands A may then be determined according to this information, for example, to avoid a collision with the detected object.

[0062] The actuator 10 is preferably onboard the vehicle 100 and may be provided by the brakes, propulsion system, engine, drivetrain, or steering of the vehicle 100. An actuator control command A may be determined to control the actuator (or actuators) 10 to cause the vehicle 100 to avoid a collision with the detected object. Alternatively, the detected object may be classified according to what the classifier 60 determines to be most likely, such as a pedestrian or a tree, and the actuator control command A may be determined according to the classification.

[0063] In one embodiment, the classifier 60 may be designed to identify lanes on the road ahead, for example, to classify the road surface and markings on the road and identify lanes as the road surface portions between the markings. Based on the output of the navigation system, the classifier 60 may then select a target lane suitable for following the selected route, and depending on the current lane and the target lane, it may then determine whether the vehicle 10 should change lanes or stay in the current lane. A control command A may then be calculated, for example, by retrieving from a database a predetermined movement pattern corresponding to the identified action.

[0064] Similarly, once a road sign or traffic light is identified, corresponding constraints on the possible movement patterns of the vehicle 10 depending on the identified type of road sign or the identified state of the traffic light may then be obtained, for example from a database, a future path for the vehicle 10 that meets the constraints may be calculated, and the actuator control command A may be calculated to steer the vehicle to execute the trajectory.

[0065] Similarly, once a pedestrian and / or vehicle is identified, a future behavior of the pedestrian and / or vehicle may be estimated, and based on the estimated future behavior, a trajectory may then be selected to avoid a collision with the pedestrian and / or vehicle, and the actuator control command A may be calculated to steer the vehicle to execute the trajectory.

[0066] In further embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move by, for example, flying, swimming, diving, or walking. The mobile robot may be, inter alia, an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. In all of the above embodiments, actuator control commands A may be determined to control the propulsion unit and / or steering and / or brakes of the mobile robot, so that the mobile robot may avoid a collision with the identified object.

[0067] In a further embodiment, the at least partially autonomous robot may be provided by a gardening robot (not shown) that uses sensors 30, preferably optical sensors, to determine the state of plants in the environment 20. The actuator 10 may be a nozzle for spraying chemicals. Depending on the identified species and / or the identified state of the plants, an actuator control command A may be determined to cause the actuator 10 to spray the appropriate amount of chemical on the plants.

[0068] In yet a further embodiment, the at least partially autonomous robot may be provided by a home appliance (not shown), such as a washing machine, stove, oven, microwave, or dishwasher. A sensor 30, such as an optical sensor, may detect the condition of an object being processed by the home appliance. For example, if the home appliance is a washing machine, the sensor 30 may detect the condition of the laundry in the washing machine based on images. An actuator control signal A may then be determined according to the detected material of the laundry.

[0069] 3 shows an embodiment in which a control system 40 is used to control a manufacturing machine 11 (e.g., a punch cutter, a cutter, a gun drill, or a gripper) of a manufacturing system 200, for example as part of a production line. The control system 40 controls an actuator 10 which controls the manufacturing machine 11.

[0070] The sensor 30 may be, for example, an optical sensor that captures characteristics of the manufactured products 12. The classifier 60 may determine the state of the manufactured products 12 from these captured characteristics, for example, whether said products 12 are defective. The actuator 10 controlling the manufacturing machine 11 may then be controlled in response to the determined state of the manufactured products 12 for subsequent manufacturing steps of the manufactured products 12. Alternatively, it may be envisaged that the actuator 10 is controlled during the manufacturing of subsequent manufactured products 12 in response to the determined state of the manufactured products 12. For example, the actuator 10 may be controlled to select products 12 identified as defective by the classifier 60 and sort the products into predetermined bins where they can be re-inspected before being discarded.

[0071] 4 shows an embodiment in which the control system 40 is used to control an automated personal assistant 250. The sensor 30 may be an optical sensor, for example, for receiving video images of the gestures of the user 249. Alternatively, the sensor 30 may also be an audio sensor, for example, for receiving voice commands of the user 249 as audio signals.

[0072] The control system 40 then determines an actuator control command A for controlling the automated personal assistant 250. The actuator control command A is determined according to the sensor signal S of the sensor 30. The sensor signal S is transmitted to the control system 40. For example, the classifier 60 may be configured to execute a gesture recognition algorithm to identify, for example, a gesture made by the user 249. The control system 40 may then determine an actuator control command A for transmitting to the automated personal assistant 250. The control system then transmits said actuator control command A to the automated personal assistant 250.

[0073] For example, the actuator control command A may be determined according to the user gesture identified and recognized by the classifier 60. The control command may include information that causes the automated personal assistant 250 to retrieve information from a database and output the retrieved information in a form suitable for receipt by the user 249.

[0074] In a further embodiment, instead of the automated personal assistant 250, it may be envisaged that the control system 40 controls an appliance (not shown) that is controlled according to the identified user gestures. The appliance may be a washing machine, a stove, an oven, a microwave, or a dishwasher.

[0075] FIG. 5 illustrates an embodiment in which a control system controls an access control system 300. The access control system may be designed to physically control access. The access control system may, for example, comprise a door 401. The sensor 30 is configured to detect a relevant scene for determining whether access is permitted. The sensor may, for example, be an optical sensor for providing image or video data for detecting a person's face. The classifier 60 may be configured to interpret this image or video data, for example, by matching the identity with known persons stored in a database, thereby determining the identity of the person. Depending on the interpretation of the classifier 60, an actuator control signal A may then be determined, for example, according to the determined identity. The actuator 10 may be a lock that allows or denies access depending on the actuator control signal A. Logical, rather than physical, access control is also possible.

[0076] FIG. 6 illustrates an embodiment in which the control system 40 controls the surveillance system 400. This embodiment is substantially identical to the embodiment illustrated in FIG. 5. Therefore, only the different aspects will be described in detail. The sensor 30 is configured to detect the scene under surveillance. The control system controls the display 10a, but not necessarily the actuator 10. For example, the machine learning system 60 may determine a scene classification, e.g., whether the scene detected by the optical sensor 30 is suspicious. The actuator control signal A sent to the display 10a may then be configured to cause the display 10a to adjust its display content in response to the determined classification, e.g., to highlight objects determined to be suspicious by the machine learning system 60.

[0077] 7 illustrates an embodiment of a control system 40 for controlling an imaging system 500, such as an MRI, X-ray, or ultrasound imaging device. The sensor 30 may be, for example, an imaging sensor. The machine learning system 60 may determine a classification of all or a portion of the sensed image. An actuator control signal A may then be selected according to this classification to control the display 10a. For example, the machine learning system 60 may interpret a region of the sensed image as potentially abnormal. In this case, the actuator control signal A may be determined to cause the display 10a to display the image and highlight the potentially abnormal region.

[0078] 8 illustrates an embodiment of a training system 140 for training a classifier 60. A training data unit 150 determines the input signal x to be sent to the classifier 60. For example, the training data unit 150 may access a computer-implemented database St2 in which at least one set T of training data is stored. The at least one set T includes a set of training data for the input signal x. i and the corresponding desired output signal y i The desired output signal y i is sent to evaluation unit 180. The set of training data T may be the complete set of training data, or it may be a selected batch of training data if training is performed in batches.

[0079] The classifier 60 is a classifier for the input signal x i output signal from

number

number

[0080] The modification unit 160 determines updated parameters ξ' in response to input from the evaluation unit 180. The updated parameters ξ' are sent to the parameter storage St1 to replace the current parameters ξ.

[0081] Additionally, the training system 140 may include a processor 145 (or multiple processors) and at least one machine-readable storage medium 146 having stored thereon instructions that, when executed, cause the control system 140 to perform a method according to one aspect of the present invention.

[0082] 9 shows an exemplary structure of a classifier 60, in this embodiment the classifier is given by a neural network parameterized by parameters or weights ξ. Input data x is fed to an input layer 61, processed and then sent successively to hidden layers 62 and 63. The output of layer 63 is a feature map Φ. If classifier 60 is a convolutional neural network, layers 61, 62 and 63 comprise at least one convolutional layer. The parameters that parameterize layers 61, 62 and 63 are called w. The feature map Φ is calculated by the parameters ξ f The output Φ T ·ξ f is the output Φ T ·ξ f and an argmax function that selects the classification label y corresponding to the highest softmax score as the output signal of the classifier 60.

[0083] 10 is a flow chart diagram outlining an embodiment of a training method for training classifier 60 that may be performed by training system 140. In a first step (1000), the Lagrangian multiplier values ​​θ are initialized, for example randomly or as a predetermined value, for example 0. The parameters ξ of fully connected layer 64 are f is set equal to the Lagrange multiplier value θ. The parameter a characterizing the metric as defined in equation (M) i,b i ,f i ,g i Similarly, a data set T is provided. Optionally, parameters characterizing the constraints as given in equation (7) are also provided.

[0084] Then, in (1010), the optimal value Q of the optimization problem written as an interior minimax problem of equation (5) (or (7) if constraints are imposed) is found. * In addition, the matrix Ψ is calculated. Details of this calculation are described in connection with FIG. 11.

[0085] Next, in (1020), increment dθ=-Ψ(Q *T 1-y T ) is calculated, where y T =(y1,...y n ) T is a vector containing the class classifications of the training dataset.

[0086] Then, at 1030, it is checked whether the method has converged, for example by checking whether the absolute value of the increment dθ is less than a predetermined threshold.

[0087] If the method converges, the algorithm is stopped and training is complete (1060).

[0088] Otherwise, in optional step 1040, the increment to dθ is adjusted to the parameter ξ of the fully connected layer 64. f is taken as an increment to parameter w and backpropagated through the rest of the network, i.e., through layers 63, 62, and 61, to obtain an increment dw to parameter w, and the method continues at step (1050). Alternatively, parameter w can remain constant, and the method branches from step (1030) directly to step (1050).

[0089] In step (1050), the parameters θ and ξ f and w is θ←θ+dθ w←w+dw ξ f ←θ will be updated as.

[0090] The method then continues to step (1010) and repeats until the method ends at step (1060).

[0091] Figure 11 shows the optimal value Q of the interior minimax problem described by equation (5) (or (7)) in step (1010). * FIG. 1 is a flowchart diagram of a method for calculating

[0092] First, in (2010), the base n×n matrices D, E, and F are

number

[0093] Next, in (2020), Z(Q) is Z(Q)=QD T +diag(1,…,1 / n) 2 Q11 T E T 11 T -diag(1,…,1 / n)QE T 11 T -diag(1,…,1 / n)Q11 T E T +QE T +diag(1,…,1 / n)F T diag(1,…,1 / n)Q11 T It is provided as a symbolic expression such as

[0094] Then, in (2030), the representation Z'(Q) is provided, which is linearly transformed from Z(Q) by Z'(Q) = Z(Q) · diag(1,...,n).

[0095] Furthermore, if there is no need to implement the special cases defined in equations (S1) and (S2), c(Q) is calculated as c(Q) = 0. If one wishes to implement (S1), Z(Q) is calculated as follows: (diag(1,…,1 / n)11 T Q T -Id)diag(1,…,1 / n)11 T and c(Q) is increased by c(Q)=1-1 T diag(1,...,1 / n)Q1, where Id is the nxn dimensional identity matrix.

[0096] When (S2) is performed, Z(Q) is augmented by a matrix E of dimension n×n, which is the matrix that nn is 0 everywhere except at position (n,n), where it is set to

[0097] Then, in (2040), all input signals x in the data set T are i is the feature vector Φ1(x i ) are propagated through a classifier (60). Ψ :,i =Φ1(x i ) (where n is the number of data samples in the dataset T and m is the number of features) and a matrix W, W=Ψ T θ1 T It is calculated as:

[0098] When solving equation (7), the output value obtained from the classifier (60) is also

number

[0099] Next, when solving equation (5), in (2050), Q * but,

number

[0100] When solving Equation (7), for each constraint in Equation (7), matrix B (i) and scalar μ i are defined by calculating

Number

[0101] for l = Σ l y l for each constraint i, and when neither (S1) nor (S2) is implemented for any i,

Number

[0102] When (S1) is implemented, as long as l = Σ l y l > 0, the above expression remains the same. When l = 0, the above variables become B (i) = -diag(1, …, 1 / n)11 T μ i = 1 are set as such.

[0103] When (S2) is implemented, as long as l = Σ l y l < n, the above expression (before the special case of S1) remains the same. When l = n, μ i = 0 and B(i) and this matrix is ​​0 everywhere except at position (n,n), where it is 1.

[0104] Next,

number

[0105] This ends the method.

[0106] The term "computer" covers any device for processing certain computational instructions, which may be in the form of software, hardware, or a mixture of software and hardware.

[0107] It is further understood that the procedures may not be implemented entirely in software as described, but may also be implemented in hardware or a mixture of software and hardware.

Claims

1. The input signal (x) is scaled to optimize performance according to a non-decomposable metric. ｉ ), wherein the non-decomposable metric is a function of the input signal (x) of a training dataset. ｉ ) corresponding to the classification (y ｉ ) and the corresponding predicted class classification of the input signal obtained from the binary classifier (60). [Equation 1] and measuring the consistency between - the indecomposable metric is the classification (y ｉ ) and the predicted class classification [Equation 2] The weighting coefficient a characterizes how the confusion matrix depends on multiple terms TP, TN, PP, PN, AP, and AN. ｊ , b ｊ , f ｊ , g ｊ providing a - the provided weighting coefficient a ｊ , b ｊ , f ｊ , g ｊ training the binary classifier (60) according to Including, The indecomposable metric can be expressed as [Equation 3] where a ｊ and b ｊ is a scalar value, and f ｊ and g ｊ is a function, TP, TN, PP, PN, AP, and AN are the entries of the confusion matrix, i.e., TP=“true positive”, TN=“true negative”, PP=“predicted positive”, PN=“predicted negative”, AP=“actual positive”, and AN=“actual negative”, The optimization of the performance according to the indecomposable metric is performed by combining the classifications (y i ) and the predicted classifications (y i ). [Equation 4] and further subject to an inequality constraint on the expected value of a second metric that measures the consistency between

2. The optimization is performed by finding an equilibrium for a two-player game between a first player (P) and a second player (Q), where the first player (P) is given input signals (x ｉ ) the first class classification corresponding to [Equation 5] and the second player (Q) tries to find the input values ​​(x ｉ ) the second classification corresponding to [Equation 6] and the first classification [Equation 7] and the second classification [Equation 8] The first player (P) tries to maximize and the second player (Q) tries to minimize the expected value of the metric on which the confusion matrix is ​​evaluated, and the second classifier [Equation 9] The method of claim 1 , wherein {right arrow over (x)} is subject to a moment matching constraint.

3. 3. The method of claim 2, wherein the optimization is performed by finding an optimal value of a Lagrangian multiplier (θ) corresponding to the moment matching constraint, and wherein trained parameters of a fully connected layer (64) of the binary classifier (60) are set equal to the optimal value of the Lagrangian multiplier (θ).

4. The expected value is the first classification [Equation 10] and / or said second classification [0011] Marginal probability of [0012] The method of claim 3, wherein the calculation is based on

5. 5. The method of claim 4, wherein the optimization comprises solving the two-player game by solving a linear program in only one of the two players (P, Q).

6. 1. A computer-implemented method for using a binary classifier (60) to classify a sensor signal, the method comprising: - training said binary classifier (60) by the method of any one of claims 1 to 5; - receiving a sensor signal (S) containing data from a sensor (30); - determining an input signal (x) dependent on said sensor signal (S); - feeding said input signal (x) to said binary classifier (60) to obtain an output signal (y) characterizing the classification of said input signal (x); A method comprising:

7. 10. A computer-implemented method for using a binary classifier (60) trained by the method of any one of claims 1 to 5 to provide an actuator control signal (A) for controlling an actuator (10), comprising all the steps of the method of claim 6, - determining said actuator control signal (A) in response to said output signal (y).

8. The method of claim 7, wherein the actuator (10) controls an at least partially autonomous robot (100) and / or a manufacturing machine (200) and / or an access control system (300).

9. A computer program product configured, when executed by a processor (45, 145), to cause the computer to carry out the method of any one of claims 1 to 8 by all of the steps of said method.

10. A machine-readable storage medium (46, 146) having stored thereon the computer program of claim 9.

11. A control system (40) configured to carry out the method of any one of claims 6 to 8.

12. A training system (140) configured to perform the method of any one of claims 1 to 5.

Images