METHOD FOR FOCUSING A CLASSIFICATION DEVICE, METHOD AND SYSTEM FOR THIS

Description

[0001] The present invention relates to a method for developing a classification device. It also relates to an observation method, a classification device, and an observation system.

[0002] The present invention lies in the field of developing classification devices using neural networks learned by using a machine learning technique.

[0003] Machine learning is referred to by many different terms, such as the English term "machine learning," the term "automatic learning," the term "artificial learning," or even the term "statistical learning." Machine learning involves using data to train a classification system.

[0004] This leads to obtaining neural networks capable of predicting a class and estimating a confidence in that prediction.

[0005] However, this confidence is not robust to the extent that for input data that is rare in the set of learned data or different from the learned data, this confidence can easily be misled.

[0006] A similar process is also known from the article by Sravanti Addepalli et al. entitled "DeGAN: Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier" dated December 27, 2019.

[0007] Therefore, there is a need for a process for developing a classification device that allows for obtaining more robust estimates of the predictive confidence of the classification device.

[0008] To that end, the description describes a method according to claim 1.

[0009] According to particular embodiments, the process has one or more of the characteristics of claims 2 to 6, taken individually or in all technically possible combinations.

[0010] The description also describes an observation system according to claim 7.

[0011] Features and advantages of the invention will become apparent from the following description, given solely by way of non-limiting example, and made with reference to the accompanying drawings, in which: there figure 1 is a schematic representation of an observation system, and the figure 2 is a schematic representation of an example of the implementation of a process for developing a classification device that is part of the observation system of the figure 1 .

[0012] The observation system 10 is a system specifically designed to observe the environment.

[0013] The observation system 10 includes a sensor 12 and a classification device 14.

[0014] Sensor 12 is designed to acquire a signal from the environment.

[0015] Any type of signal can be considered here, including audio and visual signals.

[0016] In addition, the signal can come from multiple sensors if needed.

[0017] For the following, it is assumed that sensor 12 is an illustrative camera.

[0018] The classification device 14 is a device designed to classify the signals coming from the sensor 12, which are thus input signals for the classification device.

[0019] These signals are audio, radio, or image signals.

[0020] The classification device 14 takes as input an input signal from at least one sensor and predicts at output the membership of a class among a set of predefined classes and estimates a confidence value in this prediction of membership.

[0021] For example, the image presented by sensor 12 is a dog with a confidence value of 70%.

[0022] The class "dog" is one class among a set of classes where each class is an animal, for example, a dog, a cat, or a fish.

[0023] In the following, the number of classes is denoted n.

[0024] The classification device 14 is, in this case, a physical device implementing a neural network.

[0025] The physical device is preferably an embedded device such as a programmable logic circuit, like an FPGA (field-programmable gate array), a CPU (central processing unit), a GPU (graphical processing unit), or a SoC (system on a chip) integrating, for example, DSPs (digital signal processors). The physical device can also be a combination of the components mentioned.

[0026] A neural network is notably defined in the book "Deep Learning", Ian Goodfellox, Yoshua Bengio, Aaron Courville, The MIT Press, November 18, 2016, chapter 6.

[0027] In operation, the observation system 10 implements an observation process comprising two stages.

[0028] In the first step, sensor 12 acquires an input signal, here in the example, an image of the environment.

[0029] Then, in the second step, the classification device 14 classifies the acquired signal according to predefined classes.

[0030] Here, classification device 14 will give the name of the animal contained in the image.

[0031] In the example provided, the classification achieved is of better quality because the classification device 14 was developed using a specific development process which will now be described with reference to the figure 2 .

[0032] Such a development process is implemented by computer, and preferably by a calculator.

[0033] The development process seeks to optimize the neural network of the classification device 14.

[0034] For the purposes of this discussion, this neural network is a target neural network, the one whose performance we are trying to improve. The term "C classification neural network" will be used throughout to refer to this neural network.

[0035] The development process includes a reception step, a first training step of the C classification neural network, a data synthesis step, a second training step of the C classification neural network and a training step of a generative adversarial neural network.

[0036] The development process is an iterative process in the sense that the training and synthesis steps are repeated until a first performance criterion on the predictions of the C classification neural network is met, as well as a second performance criterion on the confidence values ​​of the predictions of the C classification neural network is met.

[0037] In this case, the first performance criterion consists of requiring that the ratio of the number of correct predictions to the total number of predictions exceeds a chosen value, or that this same ratio has remained stable for a chosen number of iterations (or epochs as will be explained later).

[0038] The second performance criterion is, for its part, twofold: on the one hand, the process seeks to give accurate predictions with a good confidence value for known data and on the other hand to give very low confidence values ​​for data that the C classification neural network is not able to understand.

[0039] During the reception stage, the computer receives a database in which each piece of data gives the associated class for an input signal.

[0040] The database is thus a training dataset for the classification task, in distinct classes, the number of classes being irrelevant.

[0041] This dataset consists of pairs ( X , y ) i Or X i is data derived from a sensor signal (for example, it could be an image) and y i is a numerical encoding of the true data class X i .

[0042] Typically, the encoding is achieved by a "1 out of n" encoding or in English "on-hot encoding".

[0043] To return to a previous example of three classes n = 3, (1, 0, 0) corresponds to class 0 "dog", (0, 1, 0) to class 1 "cat", and (0, 0, 1) to class 2 "fish". The class is then obtained by the index of the largest number in the vector y i = ( y ij ) 0≤ j <n .

[0044] During the initial training stage, the C classification neural network is learned, as schematically illustrated by the solid features on the figure 2 .

[0045] During the learning (or training) of a neural network, the neural network is learned from a training dataset. A neural network is considered learned when all of its free parameters are adjusted in a learning procedure to best solve the problem posed, according to a defined criterion, on a set of training data for which the desired output of the neural network can be known (in the case of supervised learning).

[0046] During this learning process, the various free parameters of the network are thus determined.

[0047] Here, the first training is implemented with the received database.

[0048] Initially, this database is divided into three distinct sets: a training set, a validation set, and a test set. The proportion of the number of elements in each of the previous sets relative to the total number of elements in the received database is, for example, 70%, 15%, and 15%.

[0049] More precisely, a dataset is extracted from the database. This data constitutes a set of true data, denoted VD in the figure 2 .

[0050] Furthermore, a maximum number of epochs is decided beforehand. An epoch is defined by the use of all the different data from the entire training dataset. We denote it N e the epoch number, which is a large number, for example N e = 10,000.

[0051] For each era, the data from the training database are grouped into different sets in a random manner to form what are called data sets or more commonly "batches" of data according to the corresponding English term.

[0052] The number of pairs in the batch is denoted N in the following.

[0053] For all the data X in the batch, the class prediction obtained by the classification neural network C is calculated. This prediction is denoted ŷ .

[0054] During this training, a learning criterion is used.

[0055] The learning criterion corresponds to an evaluation of the error in determining the determination function.

[0056] The learning criterion is often referred to as the "loss function".

[0057] In the example described, the loss function is specific and is denoted first cost function FC 1.

[0058] This function is the soft cross entropy applied to the classes predicted by the classification system and to the classes in the database. The term "soft cross entropy" is more often referred to by its corresponding English name.

[0059] Thus, the calculator calculates the following mathematical function: FC 1 y , y ^ = 1 log n 1 N ∑ i = 0 N − 1 ∑ j = 0 n − 1 − y ij × log σ ij ^ Or : FC 1 ( y, ŷ ) denotes a cost function to be optimized, y the actual value of the class, and σ ij ^ is the normalized exponential function (more often referred to as "softmax") associated with the prediction y ij ^ .

[0060] The free parameters of the C classification neural network are then updated by exploiting the value of the first cost function FC 1 in a backpropagation gradient technique.

[0061] In parallel or after, a generative adversarial neural network synthesizes data during the data synthesis stage, as illustrated very schematically on the figure 2 with dotted lines.

[0062] More specifically, the generative adversarial neural network generates from random numbers (denoted NA on the figure 2 ) data intended to deceive the classification system.

[0063] Such random numbers NA can be drawn according to different distributions, for example according to a normal distribution or according to a uniform distribution.

[0064] Such data could be classified as "false data" and are marked FD on the figure 2 .

[0065] A generative adversarial neural network is more often referred to by the acronym GAN, which refers to the corresponding English name of "Generative Adversarial Network").

[0066] The generative adversarial neural network therefore constitutes an attack neural network denoted A.

[0067] To fool the C classification neural network, each synthesized data point is a pair of a value corresponding to a signal and a value corresponding to a distribution of the classification over all classes, and the number of synthesized data points corresponds to the number of pairs in the batch used during the first training step.

[0068] More specifically, in the example the distribution among all the classes of each synthesized data point is y i ˜ = 1 n 0 ≤ j < n For example, in the example of a dog, cat, and fish, y i ˜ = 1 3 , 1 3 , 1 3 This allows for equiprobability across all classes.

[0069] During the second training stage of the C classification neural network, the computer implements a second training.

[0070] This second training is implemented on the synthesized FD data.

[0071] More specifically, a batch is extracted from the entire dataset, the same as in the case of the first training session, but this is not mandatory.

[0072] More specifically, the synthesized FD data can be generated from the current state of the adversary network A, or from one of the randomly selected backups of its previous state. The term "state" here refers to the parameter values ​​at the relevant iteration.

[0073] The distribution between the use of the current state and a previous state can be done randomly: for example, 90% of the time the current state and 10% of the time a previous state randomly chosen from all those available.

[0074] This choice to randomly use different states of the generative adversary network increases the diversity of attacks subjected to the C classification network.

[0075] For all the data X from the batch, a batch is generated X̃ via the A-type attack neural network from NA random numbers. Similarly, a vector of the true classes of synthetic data is produced. ỹ The class prediction obtained by the C classification neural network on the synthetic data is calculated. X̃ This prediction is also noted ŷ .

[0076] He then used at this second training stage a second cost function FC 2 which is also a soft cross entropy applied to the classes predicted by the classification device and to the actual classes of the synthesized data.

[0077] Thus, the calculator calculates the following mathematical function: FC 2 y ˜ , y ^ = 1 log n 1 N ∑ i = 0 N − 1 ∑ j = 0 n − 1 − y ˜ ij × log σ ij ^ Or : FC 2 ( y, ŷ ) denotes a cost function to be optimized, and ỹ the actual value of the class which is here such that y ˜ ij = 1 n 0 ≤ j < n .

[0078] The free parameters of the C classification neural network are then updated by exploiting the value of the second cost function FC 2 in a backpropagation gradient technique.

[0079] During the training stage of the generative adversarial neural network, the computer now trains the generative adversarial neural network A.

[0080] From the random data NA, the generative adversarial neural network A will learn to generate values X̃ corresponding to a signal. The generative adversarial neural network A generates as many as the number of pairs in the batch used during the second training step.

[0081] The computer then has the C classification neural network calculate the value of its prediction. ŷ based on the values X̃ .

[0082] He then used at this training stage a cost function which will be denoted cost function FC A.

[0083] The cost function FC A involves a cost function on the ability to deceive the neural network C, to be confident about at least one of the classes.

[0084] According to the example described, a first sub-function with cost SL 1 is used for this purpose, with the objective of measuring the extent to which the classification network C was misled by the data X̃ and may have falsely given a confident classification.

[0085] Put another way, the first cost sub-function SL 1 depends, for each prediction made, on the extent to which the network C was deceived.

[0086] In this case, the first cost sub-function SL 1 uses the logit values ​​associated with each prediction.

[0087] Such a cost sub-function can be written mathematically, for example, as: SL 1 y ^ = 1 log n 1 N ∑ i = 0 N − 1 − max p ij 0 ≤ j < n + log ∑ j = 0 n − 1 exp p ij Or : p̂ ij is the value of the logit associated with the prediction ŷ ij .

[0088] Alternatively, instead of using the maximum of the logits, it would be conceivable to use the difference in value between the maximum and the maximum value of the remaining values.

[0089] In addition, in the specific case described, the FC A cost function also involves a cost function on the diversity of the synthesized data, that is, a cost function favoring the diversity of the synthesized data.

[0090] In this context, "diversity" means that each synthesized data point is unique. Uniqueness is assessed here for each element of the pair. In other words, two synthesized data points are not unique if at least one of the elements of the pair differs.

[0091] According to the example described, the cost function on the diversity of the synthesized data is the sum of a second sub-cost function SL 2 characterizing the diversity of values ​​corresponding to a signal and a third sub-cost function SL 3 characterizing the diversity of values ​​corresponding to a class.

[0092] For example, the second cost sub-function SL 2 can be mathematically written as follows: SL 2 R X ˜ = 1 ∑ i = 0 N ˜ − 1 ∑ j = 0 i − 1 R i − R j l ¯ ∑ i = 0 N ˜ − 1 ∑ j = 0 i − 1 R i − R j l ¯ 1 + X ˜ i − X ˜ j l ¯ Or : l is an integer that gives the norm, for example l = 2, A gives the average of A, R denotes a vector whose each component is a random number NA, and Ñis a subset of the synthesized data, for example the first 10 synthesized data.

[0093] Such a formulation of the second cost subfunction SL 2 makes it possible to force the generative adversarial neural network A to learn from X i ˜ different and not always the same.

[0094] Furthermore, it is used via Ñ a limitation allowing diversity to be achieved with a controlled computational cost.

[0095] Using similar notation, a third sub-function of cost SL 3 can also be proposed, relating to the diversity of the classification, which can be mathematically written as follows: SL 3 R , y ^ = 1 ∑ i = 0 N ˜ − 1 ∑ j = 0 i − 1 R ι − R j l ¯ ∑ i = 0 N ˜ − 1 ∑ j = 0 i − 1 R ι − R j l ¯ 1 + ∑ k = 0 n − 1 − σ ιk ^ × log σ jk ^

[0096] Similar to the case of the second sub-function with cost SL 2, such a formulation makes it possible to force the generative adversarial neural network A to learn very varied classes while maintaining a controlled computational cost.

[0097] In the example described, the FC cost function A used for training the generative adversarial neural network A is a composite of the three previous cost sub-functions.

[0098] As an illustration, these three sub-functions can be composed by an addition weighted by weights.

[0099] It comes like this: FC A y , y ^ , R , X ˜ = λ 1 SL 1 + λ 2 SL 2 + λ 3 SL 3

[0100] In this example, the values λ 1, λ 2 and λ 3 are weights that allow the importance of the aforementioned cost sub-functions to be modulated.

[0101] For an optimization that takes into account all the aforementioned cost sub-functions with equal importance, it suffices to set the following equality: λ 1 = λ 2 = λ 3 = 1 3 .

[0102] The free parameters of the generative adversarial neural network A are then updated by exploiting the value of the global cost function in a backpropagation gradient technique.

[0103] As explained above, the synthesis and training steps are repeated for all batches of an era, and for each era.

[0104] At the end of each epoch, the state of the best-performing generative adversary network A is saved, as well as the state at the end of the epoch. These saves can be used in the training stage of the classification network C on synthetic data.

[0105] These steps are repeated until a criterion is met.

[0106] The criterion is, for example, that the values ​​of each cost function FC 1, FC 2 and FC A reach a desired value on the validation basis.

[0107] In another example, the criterion is that a budget in calculation time is reached.

[0108] A mixed criterion can also be considered, taking into account the two previous criteria, for example via a weighting.

[0109] This yields a learned generative adversarial neural network A and a learned classification network C.

[0110] Such a process thus makes it possible to obtain the desired C classification neural network, that is to say a C classification neural network exhibiting a better quality confidence estimate while retaining a strong predictive capacity.

[0111] Indeed, the confidence estimate is more robust against attacks (here those generated by the generative adversarial neural network) and against samples outside the distribution of the database used as a training basis.

[0112] This improved robustness is achieved in particular by improving the diversity of data produced by the generative adversarial neural network.

[0113] Moreover, the development process remains relatively easy to implement since only two neural networks C and A are used and in particular no discriminator network.

[0114] This also has implications for the inference phase, since only the C classification neural network needs to be deployed. Specifically, it is not necessary to port both the discriminator network and the C classification neural network; only the C classification neural network needs to be ported.

[0115] This implies that in production, in the figure 1 , classification device 14 physically implements classification network C.

[0116] This advantage is obtained for all types of physical implementations and in particular, for constrained physical implementations.

[0117] Thus, the classification device 14 has improved classification properties, which improves the quality of observation of the observation system 10.

Claims

1. A method for observing an environment by means of an observation system (10), the observation system (10) comprising a classification device (14), the classification device (14) being configured to receive as input an input signal originating from at least one sensor (12) and to output a prediction of membership of a class from among a set of predefined classes together with a confidence value associated with said membership prediction, the input signal being an image, an audio signal, or a radio signal, the method comprising: - a phase for improving the performance of said classification device (14), comprising the following steps: - receipt of a database in which each data item provides the associated class for an input signal, - first training of the classification system, with the first training implemented using the received database, - data synthesis by a generative adversarial neural network configured to generate data, from random numbers, intended to deceive the classification device, - a second training of the classification device, the second training being carried out on the synthesised data, and - training of the generative adversarial neural network, the training of the generative adversarial neural network being carried out using the random numbers and using a cost function based on the ability to deceive the classification device, the cost function based on the ability to deceive the classification device using the values of the logits associated with each prediction, - improving the quality of the confidence estimate provided by said classification device by iterating the training and synthesis steps until a first performance criterion relating to the predictions and a second performance criterion relating to the confidence values of the predictions of the classification device are satisfied; - a use phase of said improved classification device via said improvement phase, said use phase comprising the steps of: - acquisition of an input signal by at least one sensor (12), and - classification of the acquired signal into predefined classes using the classification device (14).

2. A method according to claim 1, in which, during the step of training the generative adversarial neural network, a cost function is also used on the diversity of the synthesised data.

3. A method according to claim 2, in which each synthesised data item is a pair of a value corresponding to a signal and one corresponding to a class, the cost function on the diversity of the synthesised data is the sum of a cost sub-function characterising the diversity of the values corresponding to a signal and a cost sub-function characterising the diversity of the values corresponding to a class.

4. A method according to claim 3, wherein, during the step of training the generative adversarial neural network, an overall cost function is used as a weighted sum of a cost function based on the ability to deceive the classification device and of the two cost sub-functions.

5. A method according to any one of claims 1 to 4, wherein during the synthesis step, the true class of each synthesised data item follows a uniform distribution over all possible classes.

6. A process according to any one of claims 1 to 5, in which, during the first training step, a soft cross-entropy is used as the cost function, applied to the classes predicted by the classification device and to the classes of the database, and, during the second training step, a soft cross-entropy is used as the cost function, applied to the classes predicted by the classification device and to the classes of the database.

7. An observation system (10) comprising a classification device (14) taking as input a signal from at least one sensor and predicting as output membership of a class from a set of predefined classes and a confidence value in this membership prediction, the input signal being an image, an audio signal or a radio signal, the observation system being characterised in that it is configured to implement: - a phase for improving the performance of said classification device (14), comprising the following steps: - receipt of a database in which each data item provides the associated class for an input signal, - first training of the classification system, with the first training implemented using the received database, - data synthesis by a generative adversarial neural network configured to generate data, from random numbers, intended to deceive the classification device, - a second training of the classification device, the second training being carried out on the synthesised data, and - training of the generative adversarial neural network, the training of the generative adversarial neural network being carried out using the random numbers and using a cost function based on the ability to deceive the classification device, the cost function based on the ability to deceive the classification device using the values of the logits associated with each prediction, - improvement in the quality of the confidence estimate presented by the said classification device by repeating the training and synthesis steps until a first performance criterion on the predictions and a second performance criterion on the confidence values of the predictions of the classification system are met; - a use phase of said improved classification device via said improvement phase, said use phase comprising the steps of: - acquisition of an input signal by at least one sensor (12), and - classification of the acquired signal into predefined classes using the classification device (14).