Method for robustifying multi-layer neural networks by hybridization with MPC predictive control against an adversarial attack, using a reference signature database

Abstract

The invention relates to a method for improving the robustness of a multi-layer neural network against an antagonistic attack, comprising, in the inference phase and for an inference data item (108) injected into the neural network: S50) extracting a set of characteristic values ​​(1071) for a plurality of layers (1011) of the multi-layer neural network (100), for each inference data item (108) injected into the neural network, S60) predicting a class (104) of the inference data item (108); S70) determining a current signature for the inference data item (108), the current signature comprising: the class of the inference data item (108); a current vector (300); a current score (400) calculated from the values ​​of the current vector; S80) validate or not the inference data (108) and the associated class (104), depending on the position of the current score in the distribution of the reference scores associated with the predicted class. Figure for the abstract: Fig.5.

Description

Title of the invention: Method for robustifying multi-layer neural networks by hybridization with MPC predictive control against an antagonistic attack, using a database of reference signatures Technical field

[0001] The present invention relates to a method and system for constructing a reference database from a multi-layer neural network, as well as to a method and system for improving the robustness of a multi-layer neural network against an adversarial attack.

[0002] Aircraft avionics offers several geolocation equipment. The most common are those using satellite positioning signals (GPS) or equipment for monitoring the variation in the inertia of the vehicle, referring to a particular geographical location and then proposing a geographical position by integrating the variations in the inertia of the vehicle.

[0003] In the case of very small aircraft such as drones, this equipment is too bulky to be combined during the mission. Avionics therefore favors GPS-type equipment to provide the drone's position.

[0004] On the other hand, the reliability of GPS equipment is not sufficient to ensure the overall level of operational safety, and to achieve it, it is known to add position recognition equipment by capturing and analyzing geographic images by comparing them to a set of images whose geo-referencing is known.

[0005] Image comparison is considered by the use of shape class machine learning and by correlation between classes of georeferenced images. Machine learning uses artificial neural networks and in particular convolutional networks (CNN).

[0006] During a learning phase, each aerial shot is classified in order to assign it terrestrial coordinates. During the inference phase, a new photograph is taken, and the neural network assigns a class to the new photograph, which makes it possible to georeference the image. The acquisition of the same area from different shooting angles then makes it possible to determine the position of the aircraft, according to a technique well known to those skilled in the art.

[0007] However, neural networks can be subject to malicious attacks, particularly using a so-called adversarial technique. An adversarial or adversarial attack is a method of making small changes to objects in such a way that the machine learning model begins to misclassify them. Neural networks are known to be vulnerable to such attacks. The search for adversarial methods historically began in the field of image recognition. It has been shown that minor changes to images, such as the addition of insignificant noise, can lead to significant changes in classifier predictions and even completely confuse the models.

[0008] For example, the glare of a camera by a laser, a bird in the background of the acquired image, a camera defect (iridescence for example), snow or even rain can significantly disrupt the predictions of the classifier.

[0009] Thus, the use of neural networks to perform geolocation is not satisfactory because it does not allow the production of the level of confidence equivalent to that necessary in the aeronautical field.

[0010] The invention aims to overcome the aforementioned drawbacks. Summary of the invention

[0011] An object of the invention is therefore a method for constructing a database of reference signatures from a multi-layer neural network, comprising steps consisting of: S10) extract a set of characteristic values ​​for a plurality of layers of the multi-layer neural network, for each training data injected into the neural network, S20) predict a class of the training data; S30) calculate an average (MxLi) between the sets of characteristic values ​​for the layers of the same rank, the average being calculated for the training data having the same class; S40) determining a reference signature for each training data, the reference signature comprising: the class of the training data; - a reference vector, composed of as many values ​​as there are layers to be extracted, each value of the reference vector being calculated from a distance between the average of the sets of characteristic values ​​for said layer for the predicted class and the set of characteristic values ​​of said layer for the training data; - a reference score based on a metric calculated from the values ​​of the reference vector; the reference signature database being composed of the reference signatures.

[0012] Advantageously, the multi-layer neural network is of the CNN type, the set of characteristic values ​​being, for each layer, the median matrix of the activation maps of the layer, and in which, in step S10), the median matrices are averaged for all the training data so as to obtain a 2D matrix per layer for each class.

[0013] Advantageously, in step S10), the training data comes from a subset of training data correctly predicted by the neural network, the training data of said subset then being sorted according to their probability of belonging to one of the classes, and selected by taking a predefined proportion of the sorted subset.

[0014] The invention also relates to a method for improving the robustness of a multi-layer neural network against an antagonistic attack, comprising, in the inference phase and for an inference data item injected into the neural network: S50) extracting a set of characteristic values ​​for a plurality of layers of the multi-layer neural network, for each inference data item injected into the neural network, S60) predicting a class of the inference data item; S70) determine a current signature for the inference data, the current signature comprising: - the class of the inference data; - a current vector, each value of the current vector corresponding to a layer and being calculated from a distance between the set of characteristic values ​​of said layer for the inference data and the average of the sets of characteristic values ​​for said layer for the training data having the same class, said average of the sets of characteristic values ​​for said layer for the training data having the same class being obtained in accordance with the aforementioned method of constructing a reference signature database; - A current score calculated from the values ​​of the current vector; S80) validate or not the inference data and the associated class, depending on the position of the current score in the distribution of the reference scores associated with the predicted class, the reference scores being obtained in accordance with the aforementioned method of constructing a database of reference signatures.

[0015] Advantageously, in step S80), the inference data and the associated class are validated if the current score is located in the p percentiles of the distribution of the reference scores associated with the predicted class, where p is a predefined threshold.

[0016] Advantageously, the predefined threshold is determined so as to maximize the rate of true positives and / or to minimize the rate of false positives, the rate of true positives and the rate of false positives being respectively calculated with inference data having is subjected to an adversarial attack and the inference data is correctly predicted by the neural network.

[0017] Advantageously, p=95.

[0018] Advantageously, each of the distances is a distance L1 or a distance L2.

[0019] Advantageously, the number of extracted layers is defined as a function of a residual error rate of the current signature.

[0020] Advantageously: - 10 layers are extracted for a residual error rate lower than 103; - 17 layers are extracted for a residual error rate lower than 105; - 24 layers are extracted for a residual error rate lower than 107; - 33 layers are extracted for a residual error rate lower than 109.

[0021] The invention also relates to a computer program product comprising code instructions for implementing the aforementioned method when executed by a computer.

[0022] The invention also relates to a computer-readable storage medium storing instructions which, when executed by a computer, involve the computer implementing the aforementioned method.

[0023] The invention also relates to a system for constructing a reference database from a multi-layer neural network, the system being configured to: - extracting a set of characteristic values ​​for a plurality of layers of the multi-layer neural network, for each training data injected into the neural network, - predict a class of the training data; - calculate an average between the sets of characteristic values ​​for the layers of the same rank, the average being calculated for the training data having the same class; - determine a reference signature for each training data, the reference signature comprising: - the class of the training data; - a reference vector, composed of as many values ​​as there are layers to be extracted, each value of the reference vector being calculated from a distance between the average of the sets of characteristic values ​​for said layer for the predicted class and the set of characteristic values ​​of said layer for the training data; - a reference score based on a metric linked to the values ​​of the reference vector; the reference signature database being composed of the reference signatures.

[0024] The invention also relates to a system for improving the robustness of a multi-layer neural network with respect to an antagonistic attack, said system being configured to: S50) extract a set of characteristic values ​​for a plurality of layers of the multi-layer neural network, for each inference data injected into the neural network, S60) predict a class of the inference data; S70) determine a current signature for the inference data, the current signature comprising: - the class of the inference data; - a current vector, each value of the current vector corresponding to a layer and being calculated from a distance between the set of characteristic values ​​of said layer for the inference data and the average of the sets of characteristic values ​​for said layer for the training data having the same class, said average of the sets of characteristic values ​​for said layer for the training data having the same class being obtained with the aforementioned system for constructing a database of reference signatures; - A current score calculated from the values ​​of the current vector; S80) validate or not the inference data and the associated class, depending on the position of the current score in the distribution of the reference scores associated with the predicted class, the reference scores being obtained with the aforementioned system for constructing a database of reference signatures. Description of the figures

[0025] Other characteristics, details and advantages of the invention will emerge on reading the description given with reference to the appended drawings given by way of example.

[0026] [Fig.l] represents a diagram of an example of an artificial neural network.

[0027] [Fig.2] illustrates the different steps of the methods according to the invention.

[0028] [Fig.3] illustrates an example of calculating the average between sets of values characteristics for layers of the same rank, in the reference signature database.

[0029] [Fig.4] illustrates an example of calculation of the reference vector, in the reference signature database.

[0030] [Fig.5] illustrates the treatment of a normal case in inference.

[0031] [Fig.6] illustrates the processing of an abnormal case in inference.

[0032] [Fig.l] represents a general diagram of an artificial neural network 100, for example a fully connected neural network also called a dense network.

[0033] Generally, each intermediate layer of a network can be totally or partially connected to another layer.

[0034] A neural network is conventionally composed of several layers (125, 126, 127, 128) of interconnected neurons. The network comprises at least one input layer 125 and one output layer 128 and several intermediate layers (126, 127) also called hidden layers. The neurons 129 of the input layer 125 receive input data. The input data may be of different natures depending on the intended application. For example, the input data are pixels of an image.

[0035] A neural network has the general function of learning to solve a given problem, for example a classification or regression problem. A neural network is, for example, used in the field of image classification or image recognition or more generally the recognition of characteristics which can be visual, audio, textual.

[0036] Each neuron of a layer is connected, by its input and / or its output, to a few neurons of the previous or following layer for convolutional neural networks (hereinafter called CNN for “Convolution Neural Network”), or even to all the neurons of the previous or following layer (for dense or “fully connected” neural networks).

[0037] The connections between two neurons 129-130 of two successive layers are made through artificial synapses 131. Thus, the neuron is just a function that makes the weighted sum of its inputs then applies a simple activation function (for example forces the result to zero if it is negative, and leaves it as is if it is positive). This result then serves as an input for other neurons. The synapse is therefore a weight, a coefficient applied to each input. Alternatively, it is possible to implement the neural network in hardware, for example via memristive devices.

[0038] The coefficients of the synapses are optimized using a neural network learning mechanism. The learning mechanism aims to train the neural network to solve a defined problem. This mechanism comprises two distinct phases, a first phase of data propagation from the input layer to the output layer and a second phase of back-propagation of errors from the output layer to the input layer with, for each layer, an update of the weights of the synapses.

[0039] The errors calculated from the output neurons at the end of the data propagation phase are linked to the problem to be solved. This generally involves determining a value called logit (a prediction).

[0040] During the first phase of data propagation, training data, for example reference images or image sequences, are provided as input to the neurons of the input layer and propagated in the network. Each neuron implements, during this first phase, a function for integrating the received data which consists of calculating a sum of the received data weighted by the coefficients of the synapses.

[0041] In the case of a CNN, convolution is defined as the displacement of a filter (matrix of coefficients, the "synapses") "above" a given layer (matrix). Thus, each neuron performs a matrix product between the filter and the area of ​​the previous layer (sub-matrix of neurons) located under the filter. Each neuron then propagates this result to the neurons of the next layer via other convolutions. Depending on the neuron models chosen, the integration function it performs can vary.

[0042] The error between the output value of the neurons 132 and an expected value or a target value that corresponds to the final state of the neurons of the output layer that one wishes to obtain in relation to the training input data and the problem to be solved by the network is calculated. For example, if the network has to solve a classification problem, the neuron that corresponds to the class that it is supposed to identify in the input data must be the one that has the maximum logit among all the other neurons (corresponding to the other classes).

[0043] More generally, for each particular problem to be solved (classification, regression) one or more cost function(s) or objective function to be optimized (by minimization or maximization) is associated. One objective of the error back-propagation phase is to search for the network parameters that minimize the error.

[0044] The objective function depends on the coefficients of the neural network.

[0045] During the second phase of error back-propagation, the error is calculated from the neurons of the output layer 128; this error is a function of the weights (coefficients) of all the layers of the model. Then this error is recalculated locally for each layer taking into account the coefficients specific to each layer (error back-propagation). Then, the coefficients of each layer (and of each neuron) are updated so as to reduce the final overall error.

[0046] The process continues for each layer of neurons until the last layer.

[0047] The input data of the network can be organized in the form of matrices. The network can be configured to generate as output a score or classification information or any other type of output suitable according to the problem to be solved.

[0048] For example, if the input data are images or features extracted from images or image sequences, a neural network can be configured to recognize the objects extracted from the images and classify them according to different predefined classes or categories. The problem solved by the network is then a classification problem.

[0049] [Fig.2] illustrates the main steps of the method for constructing a reference signature database, and the main steps of the method for improving the robustness of a multi-layer neural network against an antagonistic attack in accordance with the invention.

[0050] These steps are also illustrated by Figures 3 and 4 (for the construction of a reference signature database) and by Figures 5 and 6 (for the method of improving the robustness of a multi-layer neural network, in inference).

[0051] A training data 102 is injected into the neural network 100. The training data can be any type of signal for which the neural network can be trained, in particular a 2D image, a 3D image, an audio file.

[0052] In Figures 3 to 6, the data are 2D images, the task being to individually recognize each digit of the image. This task can be extended to any type of signal (in particular images to be georeferenced).

[0053] It is essential that the neural network includes several layers. The neural network can be a convolutional neural network (CNN), or any other type of multi-layer network.

[0054] An additional layer can be added to the output of the neural network, in particular for an image search application for geolocation purposes, for example the NetVLAD layer, inspired by the "Vector of Locally Aggregated Descriptors" image representation commonly used in image search.

[0055] For each training data 102 (in the figures, the training data is a character representing a number from 0 to 9), a set of characteristic values ​​(1061, 1062, 1063, 1064, 1065) is extracted from the layers (1011, 1012, 1013, 1014, 1015) of the neural network (step S10).

[0056] For example, the set of characteristic values ​​1061 is extracted from the layer 1011. For each training data, the characteristic value of a layer is the median matrix of the activation maps of said layer.

[0057] In the case where the multi-layer neural network is of the CNN type, the set of characteristic values ​​(1061, 1062, 1063, 1064, 1065) comprises activation maps. For each layer and each training data, the median 2D matrix of the activation maps. Then for each layer, we calculate the average 2D matrix of the median matrices for all the training data of a given class, so as to obtain an average 2D matrix per layer for each class.

[0058] The dimension of 2D matrices can be reduced by using local sensitive hashing as described in particular in the article “Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions” (A. Andoni and P. Indyk, Foundations of Computer Science, 2006. FOCS'06. 47th Annual IEEE Symposium on. IEEE, 2006, pp. 459-468).

[0059] In a second step S20), the class of the training data is predicted.

[0060] It may be noted that the final prediction of the neural network is not composed of a single prediction but rather of a vector comprising a probability (or score) for each class. The final prediction of the neural network is generally the class with the highest probability.

[0061] According to an advantageous embodiment, only the correctly predicted training data 102 are selected, i.e. those for which the maximum probability is associated with the correct expected class. Once all the correct data have been obtained, they are sorted according to their maximum probabilities in descending order, i.e. from the “best prediction” to the “worst”. It can then be defined to construct the base only with a predefined percentage of the total number of training data 102, from the aforementioned sorting, in descending order. The percentage can be defined empirically, by carrying out series of measurements of true positives and / or false positives (TP / FP) with different percentages, retaining the percentage which will have given the best results in terms of TP / FP measurements.

[0062] This allows the reference database to be built with very reliable data.

[0063] In a third step S30), the training data 102 for which the same class has been predicted are identified as such. In [Fig.3], this corresponds for example to all the characters for which the same digit has been identified.

[0064] For these data having the same predicted class, an average between the sets of characteristic values ​​is calculated, for layers of the same rank. For example for layer 1011, the average is calculated between the sets of characteristic values ​​1061 for different training data 102 for which the same figure has been predicted.

[0065] In the particular case where the multi-layer neural network is of the CNN type, each set of characteristic values ​​(1061, 1062, 1063, 1064, 1065) is the median 2D matrix of the activation maps. Then the median matrices are averaged so as to obtain a 2D matrix per layer for each class. In step S30), the 2D matrices are therefore averaged for each layer, for the training data having the same class.

[0066] In [Fig.3], we thus find an average M1L1 for layer 1011 and class 1 (corresponding to the number “1” recognized by the neural network), an average M1L2 for layer 1012 and class 1, an average M1L3 for layer 1013 and class 1, an average M1L4 for layer 1014 and class 1, and an average M1L5 for layer 1015 and class 1. By generalizing, we obtain averages MxLi, x being the class, and i the layer.

[0067] [Fig.4] illustrates step S40) of the method for constructing the reference signature database. For this step, the same training data as those that were injected for steps S10) to S30) are again injected into the neural network, and the class is predicted for this second pass.

[0068] For each training data, the neural network predicts the class of the training data.

[0069] Then, for each training data, a comparison is carried out, layer by layer, between the set of characteristic values ​​of the layer and the average of the sets of characteristic values ​​for the layer calculated during step S30).

[0070] The result of the comparison forms a reference vector 200, composed of as many values ​​(201, 202, 203, 204, 205) as there are layers (1011, 1012, 1013, 1014 and 1015) to be extracted.

[0071] In the case where the multi-layer neural network is of the CNN type, each value of the reference vector 200 is calculated by comparing the average 2D matrix of the median matrices of the class for this layer and the median 2D matrix for this layer (median 2D matrix specific to the training data).

[0072] The distance, making it possible to measure each value (201, 202, 203, 204, 205) of the reference vector 200, can be a distance L1 or a distance L2.

[0073] In particular, the distance L2 can be defined as follows:

[0074] value = (average 2D matrix of the median matrices of the class for this layer)2 - (median 2D matrix for this layer)2.

[0075] A reference score 210 is calculated for each reference data. The current score is a metric calculated from the values ​​of the current vector. According to one embodiment, the reference score 210 is calculated by squaring each value (201, 202, 203, 204, 205) of the reference vector 200, then adding the squared values. This is an L2 distance, but the reference score 210 could also be calculated with an L1 distance, by adding the absolute values ​​of the values ​​(201, 202, 203, 204, 205) of the reference vector 200, or using another L2 distance. The best results are obtained by summing the squares.

[0076] Thus, the reference signature, for each current training data, comprises the class of the training data, the reference vector, and the reference score. The reference signature database 220 is composed of the reference signatures.

[0077] In [Fig.4], the reference signature database 220 comprises the association of each class 104 with the reference vectors and the reference scores 210 of the training data for which this class is predicted. This is another way of structuring the reference signature database 220.

[0078] [Fig.5] illustrates the process of improving the robustness of a multi-layer neural network against an antagonistic attack, in the inference phase.

[0079] The main steps are illustrated by [Fig.2].

[0080] The first step S50) consists of extracting a set of characteristic values ​​(1071, 1072, 1073, 1074, 1075) for a plurality of layers (101) of the multi-layer neural network 100, for each inference data 108 injected into the neural network.

[0081] The sets of characteristic values ​​are defined in the same way as for the training phase.

[0082] In a step S60), the class of the inference data 108 is predicted.

[0083] Step S70) consists of determining a current signature 109 for the inference data 108. The current signature comprises:

[0084] - the class of the inference data 108;

[0085] - a current vector 300, calculated in the same way as the reference vector. Each value (301, 302, 303, 304, 305) of the current vector 300 corresponds to a layer (respectively 1011, 1012, 1013, 1014, 1015) and is calculated from a distance between the set of characteristic values ​​(1071, 1072, 1073, 1074, 1075) of said layer for the inference data 108 and the average (M4L1, M4L2, M4L3, M4L4, M4L5) defined previously.

[0086] - A current score 400 based on a metric calculated from the values ​​of the current vector, in the same way as during the construction of the reference signature database.

[0087] For example, for the first layer 1011, the set of characteristic values ​​1071 is compared to the average M4L1, which provides a value 301 of the current vector 300.

[0088] According to one embodiment, the current score 400 is calculated by squaring each value (301, 302, 303, 304, 305) of the current vector 300, then adding the squared values. This is an L2 distance, but the current score 400 could also be calculated with an Ll distance, by summing the absolute values ​​of the values ​​(301, 302, 303, 304, 305) of the current vector 300.

[0089] Thus, the current signature is composed of the class of the inference data, the current vector and the current score.

[0090] In a step S 80), a search is carried out in the reference signature database, in order to determine the position of the current score in the distribution of the reference scores associated with the predicted class. The reference scores are extracted from the reference signature database.

[0091] According to one embodiment, it is determined in which percentile this current score is located in the distribution of reference scores 210 associated with the predicted class.

[0092] If the current score 400 is within the p percentiles of the distribution of reference scores 210 associated with the predicted class, where p is a predefined threshold, the inference data, and the predicted class for the inference data, are validated: the inference data is considered to be normal. [Fig.5] illustrates such a case. Conversely, if the current score 400 is not within the p percentiles, the inference data is considered to be corrupted, as illustrated in [Fig.6]. In [Fig.6], the number "7" was classified as a "2", which resulted in a calculation of the current score outside the distribution of reference scores associated with the class "2".

[0093] The threshold is empirical, and can be determined so as to maximize the true positive rate and / or minimize the false positive rate, the true positive rate and the false positive rate being calculated respectively with inference data that has been subject to an adversarial attack and with "clean" data for which the neural network makes correct predictions. It may be advantageous to use a threshold p=95.

[0094] The method according to the invention does not require training a neural network. It only requires storing a database of reference signatures, as well as the database of the averages of each layer of each class (MxLi, where x is a class index, and i a layer index), which stores the average of each layer for each class (MxLi) calculated in the phase of construction of the reference signature database.

[0095] The databases are small in size, and can therefore be carried on aircraft such as drones.

[0096] For example, with a Resnet50 / Imagenet type neural network (1,200,000 trained images, 1000 classes), the reference signature database has a size equal to 87 MB (1,200,000 x 19 (17 reference scores + 1 class + 1 reference score) x 4 bytes (32-bit floating point format) / (1024*1024)).

[0097] The layer averages database has a size equal to 648 MB (17 x 100 x 100 x 1000 (100 x 100 corresponds to the size of the internal representations) x 4 bytes / (1024*1024)).

[0098] Furthermore, the method according to the invention does not require updating the architecture of the neural network. Indeed, it is sufficient to add, externally, a module for extracting sets of characteristic values.

[0099] Finally, the proximity search between the current signature and the reference signatures is a comparison which does not have a high algorithmic cost.

[0100] Thus, the method according to the invention does not add more computing time than that necessary to compare the reference signature and the current signature produced in real time. The vector extraction time can be made negligible by using graphics processing units having a parallel processing architecture (GPU).

[0101] The number of layers extracted for the training data and for the inference data can be defined according to a tolerated residual error rate.

[0102] Indeed, it is possible to calculate a signature word corresponding to a correlation between the current vector and the reference vector closest to the current vector. This signature word includes L elements corresponding to the number of layers extracted.

[0103] In the case where, for each layer, the correlation is either zero or one, the signature word represents 2L possible values. By convention, the identity signature word is a word of L bits of value 1. Thus, the error word represents the number of correlated erroneous values ​​which represents 2X possible values.

[0104] TS is defined as follows: TS = number of errors / total number of values ​​= 2X / 2L

[0105] Assuming similarity (no bits of the word at zero, 0 errors, x=0) and in relation to the size of the network: number of layers TS 10 9.8E-04 <103 17 7.6E-06 <105 24 6.0E-08 <107 33 l.2E-10 <109

[0106] TS is the error rate of the signature, which quantifies the reliability of the entire correlation system. The distance is a criterion for valuing each bit of the error rate TS by calculation: each bit of the vector is valued at one when the distance between the reference signature and the current signature is zero or at zero when the distance is infinite (or greater than a threshold).

[0107] Thus, ten layers are extracted for a residual error rate less than 10 3. Seventeen layers are extracted for a residual error rate less than 105. Twenty-four layers are extracted for a residual error rate less than 107. Thirty-three layers are extracted for a residual error rate less than 109.

[0108] When developing a system using neural networks, the number of extracted layers indicates which criticality level (or DAL for "Development Assurance Level") the system is closest to. Indeed, the Eurocae ED-12 standards (equivalent to the RTCA DO-178 standard) and Eurocae ED-80 standards (equivalent to the RTCA DO-254 standard) set the safety conditions applicable respectively to critical avionics software and to avionics electronic equipment.

[0109] The standards include four levels of criticality (classified from A to D), defined as follows:

[0110] Level A: A defect in the system or subsystem under study can cause a catastrophic problem - Flight safety or compromised landing - Aircraft crash (residual error rate less than 109).

[0111] Level B: A fault in the system or subsystem studied can cause a dangerous problem resulting in serious damage or even the death of some occupants (residual error rate less than 107).

[0112] Level C: A fault in the system or subsystem under study may cause a major problem resulting in a malfunction of the vital equipment of the device (residual error rate less than 105).

[0113] Level D: A defect in the system or subsystem studied may cause a minor problem with no effect on flight safety (residual error rate less than 103).

[0114] Thus, it is possible to define a number of layers to be extracted according to the expected level of criticality. Furthermore, the number of layers to be extracted can be adapted to the complexity of the system using the neural network, which allows an adjustment of the computational effort and the consumption of the associated resources according to the context of use of the solution (embedded or not).

[0115] According to one embodiment, only the layers 101 that are the least correlated with each other, among all the layers of the multi-layer neural network, are extracted. This makes it possible to use only the layers that are likely to vary greatly from one training data to another.

[0116] Having a verifiable confidence level for pattern recognition in an image makes it possible to link the geographic information in that image to the aircraft's position with a quantifiable confidence level, similar to that provided by satellite navigation systems (GPS).

[0117] It is therefore possible to control the position of the aircraft from images, or a succession of images, taken directly in operation and in coherence with the dynamics of the aircraft.

[0118] The invention has been described for input data of the 2D image type. It can be extended to any type of input data, for example voice, text, any image, or any type of data for which a class must be predicted by a neural network structured in layers.

[0119] The invention may be implemented as a computer program comprising instructions for its execution. The computer program may be recorded on a recording medium readable by a processor.

[0120] Reference to a computer program that, when executed, performs any of the functions described above, is not limited to an application program running on a single host computer. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instruction) that can be used to program one or more processors to implement aspects of the techniques described herein.

[0121] The computing means or resources may in particular be distributed ("Cloud computing"), possibly using peer-to-peer technologies. The software code may be executed on any suitable processor (e.g., a microprocessor) or processor core or a set of processors, whether provided in a single computing device or distributed among several computing devices (e.g., as may be accessible in the device environment).

[0122] The executable code of each program allowing the programmable device to implement the processes according to the invention can be stored, for example, in the hard disk or in read-only memory. Generally speaking, the program(s) may be loaded into one of the storage means of the device before being executed. The central unit can control and direct the execution of the instructions or portions of software code of the program(s) according to the invention, instructions which are stored in the hard disk or in the read-only memory or in the other aforementioned storage elements.

[0123] The invention can be implemented on a computing device based, for example, on an embedded processor. The processor can be a generic processor, a specific processor, an application-specific integrated circuit (also known as ASIC for “Application-Specific Integrated Circuit”) or an in situ programmable gate array (also known as FPGA for “ Field-Programmable Gate Array"). The computing device may use one or more dedicated electronic circuits or a general-purpose circuit.

[0124] The technique of the invention can be carried out on a reprogrammable computing machine (a processor or a microcontroller for example) executing a program comprising a sequence of instructions, or on a dedicated computing machine (for example a set of logic gates such as an FPGA or an ASIC, or any other hardware module).

[0125] The invention can also be implemented in a location system embedded in a mobile carrier further comprising one or more image acquisition devices.

[0126] The position of the mobile carrier calculated by the invention can be returned to a user via a human-machine interface.

Claims

1. Claims A computer-implemented method of constructing a database of reference signatures from a multi-layer neural network, with a view to making the database available for georeferencing images in inference, comprising steps of: S10) extracting a set of characteristic values (1061, 1062, 1063, 1064, 1065) for a plurality of layers (1011, 1012, 1013, 1014, 1015) of the multi-layer neural network (100), for each training data (102) injected into the neural network, the training data (102) being an image to be georeferenced; S20) predicting a class of the training data (102), the class of the training data comprising a georeference of the image to be georeferenced; S30) calculating an average (MxLi) between the sets of characteristic values (1061, 1062, 1063, 1064, 1065) for the layers of the same rank, the average (MxLi) being calculated for the training data (102) having the same class; S40) determining a reference signature for each training data (102), the reference signature comprising: - the class of the training data (102); - a reference vector (200), composed of as many values (201, 202, 203, 204, 205) as there are layers (1011, 1012, 1013, 1014, 1015) to be extracted, each value (201, 202, 203, 204, 205) of the reference vector (200) being calculated from a distance between the average (MxLi) of the sets of characteristic values for said layer for the predicted class and the set of characteristic values (1061, 1062, 1063, 1064, 1065) of said layer for the training data (102); - a reference score (210) based on a metric calculated from the values (201, 202, 203, 204, 205) of the reference vector; the reference signature database being composed of the reference signatures.

2. The method of claim 1, wherein the multi-layer neural network is of the CNN type, the set of characteristic values (1061, 1062, 1063, 1064, 1065) being, for each layer, the median matrix of the activation maps of the layer, and wherein, in step S10), the median matrices are averaged for all the training data so as to obtain a 2D matrix per layer for each class (102).

3. 3. Method according to one of the preceding claims, in which, in step S10), the training data (102) come from a subset of training data (102) correctly predicted by the neural network (100), the training data (102) of said subset then being sorted according to their probability of belonging to one of the classes (104), and selected by taking a predefined proportion of the sorted subset.

4. A computer-implemented method for improving the robustness of a multi-layer neural network against an adversarial attack, for the purpose of georeferencing images, comprising, in the inference phase and for an inference datum (108) injected into the neural network, the inference datum (108) being an image to be georeferenced: S50) extracting a set of characteristic values (1071, 1072, 1073, 1074, 1075) for a plurality of layers (1011, 1012, 1013, 1014, 1015) of the multi-layer neural network (100), for each inference datum (108) injected into the neural network, S60) predicting a class (104) of the inference datum (108), the class of the data training comprising georeferencing of the image; S70) determining a current signature for the inference data (108), the current signature comprising: - the class of the inference data (108);- a current vector (300), each value (301, 302, 303, 304, 305) of the current vector (300) corresponding to a layer and being calculated from a distance between the set of characteristic values (1071, 1072, 1073, 1074, 1075) of said layer for the inference data (108) and the average (ML1, ML2, ML3, ML4, ML5) of the sets of characteristic values for said; layer for training data (102) having the same class, said average of the sets of characteristic values for said layer for training data (102) having the same class being obtained in accordance with the method for constructing a reference signature database according to one of claims 1 to 3; - A current score (400) calculated from the values of the current vector; S80) validating or not the inference data (108) and the associated class (104), depending on the position of the current score in the distribution of the reference scores associated with the predicted class, the reference scores being obtained in accordance with the method for constructing a reference signature database according to one of claims 1 to 3.

5. The method of claim 4, wherein, in step S80), the inference data (108) and the associated class (104) are validated if the current score is within the p percentiles of the distribution of the reference scores associated with the predicted class, where p is a predefined threshold.

6. 6. The method of claim 5, wherein the predefined threshold is determined so as to maximize the true positive rate and / or minimize the false positive rate, the true positive rate and the false positive rate being respectively calculated with inference data having been the subject of an adversarial attack and inference data correctly predicted by the neural network.

7.

8. Method according to one of claims 5 or 6, in which p=95.

8. Method according to one of claims 1 to 7, in which each of the distances is a distance L1 or a distance L2.

9. 9. Method according to one of the preceding claims, in which the number of extracted layers (101) is defined as a function of a residual error rate of the current signature.

10. 10. Method according to claim 9, in which: - 10 layers (101) are extracted for a residual error rate less than 103; - 17 layers (101) are extracted for a residual error rate less than 105; - 24 layers (101) are extracted for a residual error rate less than 107; - 33 layers (101) are extracted for a residual error rate less than 109.

11. 11. Computer program product comprising code instructions for implementing the method according to one of claims 1 to 10 when executed by a computer.

12. 12. A computer-readable storage medium storing instructions which, when executed by a computer, cause the computer to carry out the method of one of claims 1 to 10.

13. 3System (113) for constructing a reference database from a multi-layer neural network with a view to making the database available for georeferencing images in inference, the system being configured to: - extract a set of characteristic values (1061, 1062, 1063, 1064, 1065) for a plurality of layers (1011, 1012, 1013, 1014, 1015) of the multi-layer neural network (100), for each training data (102) injected into the neural network, the training data (102) being an image to be georeferenced, - predict a class of the training data (102), the class of the training data comprising a georeferencing of the image to be georeferenced; - calculating an average (MxLi) between the sets of characteristic values (1061, 1062, 1063, 1064, 1065) for the layers of the same rank, the average (MxLi) being calculated for the training data (102) having the same class;- determining a reference signature for each training data (102), the reference signature comprising: - the class of the training data (102); - a reference vector (200), composed of as many values (201, 202, 203, 204, 205) as there are layers (1011, 1012, 1013, 1014, 1015) to be extracted, each value (201, 202, 203, 204, 205) of the reference vector (200) being calculated from a distance between the average (MxLi);

14. sets of characteristic values for said layer for the predicted class and the set of characteristic values (1061, 1062, 1063, 1064, 1065) of said layer for the training data (102); - a reference score (210) based on a metric linked to the values (201, 202, 203, 204, 205) of the reference vector; the reference signature database being composed of the reference signatures. 4System for improving the robustness of a multi-layer neural network (115) against an adversarial attack, with a view to georeferencing images, said system being configured to: S50) extract a set of characteristic values (1071, 1072, 1073, 1074, 1075) for a plurality of layers (1011, 1012, 1013, 1014, 1015) of the multi-layer neural network (100), for each inference data (108) injected into the neural network, the inference data (108) being an image to be georeferenced, S60) predict a class (104) of the inference data (108), the class of the training data comprising a georeferencing of the image; S70) determining a current signature for the inference data (108), the current signature comprising: - the class of the inference data (108); - a current vector (300), each value (301, 302, 303, 304, 305) of the current vector (300) corresponding to a layer and being calculated from a distance between the set of characteristic values (1071, 1072, 1073, 1074, 1075) of said layer for the inference data (108) and the average (MxLi) of the sets of characteristic values for said layer for the training data (102) having the same class, said average of the sets of characteristic values for said layer for the training data (102) having the same class being obtained with the system for constructing a reference signature database according to claim 13; - A current score (400) calculated from the values of the current vector; S80) validate or not the inference data (108) and the associated class (104), depending on the position of the current score in the distribution of the reference scores associated with the predicted class, the reference scores being obtained with the system for constructing a database of reference signatures according to claim 13.

Images