METHOD FOR TRAINING A NEURAL NETWORK, CLASSIFICATION METHOD AND CORRESPONDING COMPUTER PROGRAM

Abstract

METHOD FOR TRAINING A NEURAL NETWORK, CLASSIFICATION METHOD AND CORRESPONDING COMPUTER PROGRAM S A method for training (10) a neural network (15) for data classification includes providing (120) an initial training set (50) comprising pairs of matrices (51), each comprising an input matrix (51A) and an initial label matrix (51B) of which each term is an initial class number (num_i) of a term of the input matrix; training (130) from the initial set (50) training sets (65), final validation sets (66) and intermediate validation sets (67); and training (135) the neural network (15) with these sets. The formation of said sets includes the conversion of each initial label matrix (51B) into an intermediate label matrix (51C) comprising an intermediate class number (num_int_i) for each initial class number (num_i).The intermediate class number of a majority class (num_int_maj) is zero. Figure for the abbreviation: 3.

Description

Title of the invention: METHOD FOR TRAINING A NEURAL NETWORK, CLASSIFICATION METHOD AND CORRESPONDING COMPUTER PROGRAM

[0001] The present invention relates to a method for training a neural network configured for the classification of input data, in particular images, a method for classifying input data using such a neural network and the corresponding computer program.

[0002] In the field of artificial intelligence, and in particular of automatic analysis of data such as image or sound files, it is known to train neural networks by deep learning, in order to generate a classification model.

[0003] In a known manner, the classification model is carried out on a neural network, presenting an initial architecture chosen in terms of number of layers, number of neurons per layer and types of activation functions.

[0004] The values ​​of the parameters of the neural network, in particular the synaptic weights and any biases, are adjusted during an adaptation phase of the neural network, also called the learning phase.

[0005] The learning phase is carried out by means of a set of digital input files and respective annotations, including information relating to the classes to be assigned to these files.

[0006] The annotation of the input files provided for the learning phase is carried out by an operator, on the basis of a set of classes that the operator chooses or that is provided to him.

[0007] The classification model obtained at the end of the learning phase allows the automatic classification of new input files during a subsequent use phase of the classification model, also called the inference phase.

[0008] Input file annotation can be carried out at different scales.

[0009] It is thus possible to assign a single class to the entirety of an input file to be classified, for example, to each image provided as input to the neural network. Typically, in the field of image analysis, any image that contains a car will in this case be assigned the class car.

[0010] It is also possible to process each elementary data point in the input file in such a way as to assign a class to each of these elementary data points. In the case where the input file is an image, a class will thus be assigned to each pixel of the image. In this alternative, the classification is called segmentation.

[0011] Segmentation is said to be "semantic" if no distinction is made between different instances of the same class. Segmentation is said to be "instance-based" if, on the contrary, it allows the identification of individual components belonging to one and the same instance of a given class.

[0012] For example, if the input files are images and the classes to be assigned are the road class and the car class, instance segmentation will allow the pixels corresponding to a given car to be separated from the pixels belonging to other cars, unlike semantic segmentation.

[0013] The learning phase for semantic classification requires processing large volumes of data, since each input file is in bijection with the corresponding annotation.

[0014] Thus, in the example of image-type files, if the input images include L*H pixels, L being the width of the image and H its height, expressed as a function of the characteristic dimension of a pixel, the corresponding annotations include L*H class values, each pixel being able to include several sub-pixels if the sensor includes several recording channels.

[0015] It is therefore understood that it is most often not possible for the learning phase to place all the annotations of the training dataset in a RAM associated with a processor on which the learning phase is implemented, nor a fortiori all the input images themselves.

[0016] Consequently, the processor must repeatedly retrieve the information corresponding to the annotations from a mass memory, for which the read and write access time is much higher than for RAM.

[0017] The learning phase is therefore particularly time-consuming in the case of classification by segmentation, whether it is semantic or instance segmentation.

[0018] One object of the invention is then to propose a method for training a neural network for the purpose of classifying files by segmentation for which the cost in time, and optionally in memory, is reduced, including when no a priori simplification of the input data is possible.

[0019] To this end, the invention relates to a method for training a neural network configured for the classification of matrix input data comprising: a) the initialization of a neural network, b) the provision of an initial training dataset comprising a plurality of matrix pairs, each matrix pair comprising an input matrix containing input data and a respective initial label matrix of the same dimensions as the input matrix and where each term is an initial class number representing a class of a respective term of the input matrix, said class being selected from a set of predetermined classes, c) training from the initial set, a training set, a final validation set and an intermediate validation set, d) iterative training of the neural network using the training set and the intermediate validation set, and e) the final validation of the trained neural network using the final validation set, characterized in that: Training c) of the training sets, final validation and intermediate validation includes: i) determining, for each class in the set of predetermined classes, a frequency indicator representative of the total number of occurrences of that class in the set of terms of the initial label matrices of the initial set, ii) selecting a majority class, which is the class with the highest frequency indicator, iii) assigning each class a respective intermediate class number such that the intermediate class number of the majority class is equal to zero, and (iv) the formation of an intermediate training set from the initial set, by converting each initial label matrix into an intermediate label matrix whose terms are the intermediate class numbers corresponding to the initial class numbers of the initial label matrix, the intermediate label matrices being stored in memory using an intermediate data structure different from an initial data structure used for storing the initial label matrices in memory, said training, final validation and intermediate validation sets being formed from the intermediate set.

[0020] Determining the majority class allows the conversion of the initial label matrices into intermediate label matrices, without any prior assumptions about the distribution of classes in the training set being necessary.

[0021] Unlike the input matrices and the initial label matrices, the intermediate label matrices in many cases have a significant proportion of nuisance terms, so that they can be stored using a specifically adapted intermediate data structure.

[0022] In particular, data structures adapted to the representation of so-called sparse matrices can be used as an intermediate data structure.

[0023] The intermediate label matrices therefore have a lower, or even much lower, memory cost than the initial label matrices. Consequently, converting the initial label matrices into intermediate label matrices makes it possible in many cases to store in RAM all the intermediate label matrices necessary for the iterative training of the neural network.

[0024] Unlike a conventional training method, the processor can then read the data relating to the labels directly from RAM instead of having to read them from storage memory.

[0025] It is therefore understood that the method according to the invention allows for a time saving in the iterative training step that is on the order of the product NE*B*Ataccess of the number of training epochs NE, the number B of input matrices used in each epoch, and the difference between the access time to storage memory and the access time to RAM. This time saving is clearly very significant.

[0026] In addition, the process also allows a saving in reading time of relevant information in the label matrices, this time being shorter for the initial label matrices than for the intermediate label matrices thanks to the use of the intermediate data structure.

[0027] According to other advantageous aspects of the invention, the method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0028] - the method includes, before the iterative training of the neural network:

[0029] c2) a step of selecting a RAM exchanging data with a processor used for the implementation of the process having sufficient memory space for storing information relating to intermediate label matrices and storing this information in said RAM for use during iterative training;

[0030] - during iterative training, at least one set of label matrices estimated intermediates is obtained at the output of the neural network in response to the input of the neural network of a set of input matrices chosen from the intermediate set, and is compared either directly with the respective intermediate label matrices, or with the initial label matrices after inverse conversion of the estimated intermediate label matrices into respective estimated initial label matrices, said label matrices being stored in RAM;

[0031] - the RAM is chosen at the selection stage from among a RAM memory dedicated to processor and VRAM memory optimized for graphics data handling;

[0032] - VRAM memory is chosen in preference to RAM memory;

[0033] - if the step of selecting a RAM includes, if necessary: * the determination of a set of meta-classes from the set of predetermined classes based on the determined majority class, * the formation of secondary label matrices whose terms are the meta-class numbers corresponding to the initial class numbers of the initial label matrix, and * the reiteration of steps i), ii) and iii) from the secondary label matrices before step iv) of formation of the intermediate set (68);

[0034] - the input matrices are previously obtained using an image sensor digital or by means of a digital sound sensor;

[0035] - the intermediate data structure is specifically adapted for matrices hollow.

[0036] The invention also relates to a method of classifying data by segmentation using a neural network obtained at the end of the training process as described above.

[0037] The invention finally relates to a computer program product comprising software instructions which, when executed by a computer, implement the training method as defined above.

[0038] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0039] [Fig-1] [Fig.1] schematically represents an electronic device of classification on which the process according to the invention can be implemented.

[0040] [Fig.2] [Fig.2] schematically represents a particular example of a neural network implemented by the electronic classification device of [Fig.1].

[0041] [Fig.3] [Fig.3] represents in the form of a logic diagram the training method according to the invention for training the neural network of [Fig.2] and the classification method implemented at the end of the training method.

[0042] The invention relates to a method for training 10 a neural network 15 implemented by means of an electronic classification device 20.

[0043] The electronic classification device 20 is described with reference to [Fig.1].

[0044] The electronic classification device 20 comprises, for example, at least one processor 25 exchanging data with at least one mass storage 30 and at least one random access memory 35.

[0045] The processor 25 is for example chosen from a central processing unit (abbreviated CPU; in English Central Processing Unit) and / or a graphics processing unit (abbreviated GPU; in English Graphics Processing Unit).

[0046] The mass memory 30 is a non-volatile memory, with a storage capacity suitable for implementing the training method 10 and optionally for the subsequent classification of at least one matrix input data to be classified 45.

[0047] The mass memory 30 is preferably optimized in terms of processor access time 25 to the data it stores for reading, and optionally for writing.

[0048] As an example, the mass storage 30 is of the SSD type (in English, "Solid State Drive").

[0049] The mass memory 30 is configured to receive and store software instructions to be executed by the processor 25 for the implementation of the training method 10 which will be described later, and optionally for the classification of at least one matrix input data to be classified 45.

[0050] The mass storage 30 is further configured to receive and store an initial set 50 of training data and optionally at least one input data to be classified 45.

[0051] The matrix input data to be classified 45 comprises information relating to a physical object obtained by means of a sensor and stored electronically by means of a matrix data structure.

[0052] Each matrix input data to be classified 45 is a matrix of dimension Ni*N2 with Ni and N2 integers greater than or equal to 1 and such that at most one of Ni and N2 is equal to 1, each term of the matrix comprising C subterms, C denoting a number of sensor channels. Each term is therefore a singlet in the cases where C is equal to 1 and a multiplet in the cases where C is strictly greater than 1.

[0053] Each matrix input data to be classified 45 is obtained by means of a suitable digital sensor, for example a digital image sensor or a digital sound sensor.

[0054] In a first embodiment, the matrix input data to be classified 45 is stored in mass memory 30 in a matrix image file of dimension Ni*N2. Ni, respectively N2, represents in this case an integer number of pixels along a longitudinal direction of the digital image sensor by means of which the file was obtained, Ni and N2 being strictly greater than 1. Each term of the corresponding input matrix then encodes, for example in the form of one or more floating-point numbers, for example a triplet of floating-point numbers, (in English, "float"), a color level on one or more channels, for example three channels.

[0055] In this first embodiment, the number of channels C of the image sensor is, for example, equal to 1 for a grayscale image, to 3 for an RGB color image, or even greater than three for a multispectral, hyperspectral, or ultraspectral sensor. In the case of an RGB color image, each pixel therefore comprises three subpixels, so that each term of the matrix input data to be classified 45 is a triplet of floating-point numbers. The matrix format is, for example, chosen from among the JPEG, PNG, GIF, TIF, and BMP formats.

[0056] In a second embodiment, the matrix input data to be classified 45 is stored in mass memory 30 in a matrix-format audio file of dimension Ni*N2. In a particular embodiment, one of Ni and N2 is then equal to 1, the terms of the corresponding matrix, reduced to a row vector or a column vector, representing a sample of a digital signal obtained by means of a digital sound sensor. The position of any term of the matrix in this matrix then represents an acquisition time of the sample of the sound signal represented by that term.

[0057] In this second embodiment, the number of channels C of the sound sensor is for example equal to 1 or strictly greater than 1. In the case of a sound acquired by means of a multichannel sound sensor, each element of the matrix data 45 to be classified therefore comprises as many sub-terms as channels, so that each term of the matrix input data to be classified 45 is a multiplet of floating numbers.

[0058] The matrix data to be classified 45 is, for example, a medical image obtained on a patient using a medical imaging device such as a 2D digital radiography device, a scanner or a magnetic resonance imaging device.

[0059] Alternatively, the matrix data to be classified 45 is an image file of a portion of space intended to be traversed by an autonomous vehicle or one equipped with a driving assistance system.

[0060] The initial set 50 of training data comprises a plurality of matrix pairs 51.

[0061] Each pair of matrices 51 comprises an input matrix 51A comprising input data and a respective initial label matrix 51B of the same dimension as the input matrix 51A.

[0062] The data format used to store the input matrices 51A in mass memory 30 is preferably the same as that of the matrix input data to be classified 45.

[0063] Thus, in the case of image classification, or respectively of audio files, the input matrices 51A are preferably stored in memory 30 in the form of image files, respectively audio files, of the same format as that used for the matrix input data to be classified 45.

[0064] Each initial label matrix 5 IB is of the same dimension Ni*N2 as the respective input matrix 51A.

[0065] Each term of the initial label matrix 5 IB is an initial class number num_i, representing a respective class of the corresponding term in the input matrix 51A selected from a set {Ci} of N predetermined classes Ci.

[0066] The classes C; are chosen a priori by an operator who proceeds to annotate the input matrices 51A upstream of the training process 10 to generate the initial label matrices 51B.

[0067] Each initial label matrix 5IB is therefore formed at the end of an annotation step of the segmentation annotation type of a respective input matrix 51A, that is to say by assigning a class Ci to each term of the respective input matrix 51A, the annotation step being carried out by an operator upstream of the implementation of the training process 10.

[0068] The classes C; are therefore each represented by a respective class number num_i, preferably a positive integer.

[0069] The initial label matrices 51B are stored in mass memory 30 by means of an appropriate matrix data structure, for example the numpy ndarray type available in the Numpy library of the Python language or the torch.Tensor type available in the PyTorch library of the Python language.

[0070] The RAM 35 is a volatile memory configured to temporarily store data from the mass storage 30 for the implementation of the training method 10 and optionally for the subsequent classification of at least one input data to be classified 45.

[0071] The RAM 35 includes, for example, at least one RAM (Random Access Memory) type memory and / or one VRAM (Video Random Access Memory) type memory.

[0072] The RAM 35 is advantageously chosen according to the nature of the input data.

[0073] In particular, in the case where the matrix input data to be classified 45 is an image, the RAM 35 is preferably of the VRAM type.

[0074] The classification device 20 is configured to implement the training method 10 of the neural network 15, an example of which is shown in [Fig.2].

[0075] The neural network 15 comprises an ordered succession of layers 55-i (i an integer between 1 and k, with k greater than or equal to three) of neurons 60, each of which takes its inputs from the outputs of the previous layer.

[0076] More specifically, each layer 55-i comprises respective neurons 60, taking their inputs from the outputs of the neurons 60 of the preceding layer 55-(il) as appropriate, or receiving the input data for the first layer 55-1.

[0077] In the example of [Fig.2], the neural network 15 comprises an input layer 55-1, two hidden layers 55-2 and 55-3 and an output layer 55-4.

[0078] Alternatively, more complex neural network structures 15 can be envisaged. In this case, a given layer 55-i can be linked to a layer 55-j further away than the immediately preceding layer 55-(i-1).

[0079] Each neuron 60 is also associated with an operation, that is to say a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0080] Each layer 55-i is connected to the other layers 55-j by one or more synapses 65.

[0081] A synaptic weight is associated with each synapse 65, and each synapse forms a link between two respective neurons 60. Each synaptic weight is, for example, a real number or a complex number.

[0082] Each neuron 60 is specific to: - perform a weighted sum of the value(s) received from the neurons 60 of the previous layer, each value then being multiplied by the respective synaptic weight of the corresponding synapse, then - apply an activation function, typically a non-linear function, to said weighted sum, and - deliver at the output of said neuron 60 a value resulting from the application of the activation function.

[0083] The activation function allows for the introduction of non-linearity in the processing carried out by each neuron 60. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.

[0084] As an optional addition, each neuron 60 is also capable of applying an additive factor, also called a bias, to the output of the activation function. The value delivered at the output of said neuron 60 is then the sum of the bias value and the value resulting from the activation function.

[0085] The training method 10 aims to adjust the synaptic weights, including any biases, so that the neural network 15 performs the classification of the matrix input data 45 with the lowest possible error rate.

[0086] The training method 10, described with reference to [Fig. 3], comprises: a) the initialization 110 of the neural network 15, b) the provision 120 of the initial set 50 of training data comprising a plurality of matrix pairs 51, each matrix pair 51 comprising an input matrix 51A and a respective initial label matrix 51B of the same dimension as the input matrix 51A and each term of which is the initial class number num_i representing the class Q of the respective term of the input matrix 51A selected from the set {CJ of predetermined classes, c) training 130 from the initial set 50, a training set 65, a final validation set 66 and an intermediate validation set 67, d) iterative training 135 of the neural network 15 using the training set 65 and the intermediate validation set 67, and e) the final validation 140 of the neural network 15 trained using the final validation set 66.

[0087] Initialization 110 includes: - the choice of a type of neural network 15, including the choice of the different 55-i layers and synapses 65 that link their respective neurons 60, - the choice of the activation functions of the different neurons 60, and - the initialization of synaptic weights, for example in a random manner, and where appropriate biases, of a neural network 15.

[0088] At the end of the initialization step 110, an initial neural network 15_in is thus formed, the representative information of which is stored in the RAM 35 for the purpose of subsequent steps.

[0089] The supply 120 of the initial set 50 includes the storage of the initial set 50 on the mass storage 30.

[0090] Initialization 110 and supply 120 can be carried out in any order or simultaneously.

[0091] The training 130 of the training sets 65, the intermediate validation set 67 and the final validation set 66 is performed by the processor 25 and comprises: i) the determination 130-A for each class C; of the set {CJ of predetermined classes of a frequency indicator £ representative of the total number of occurrences of this class C; in the set of terms of the initial label matrices 51B of the initial set 50, by reading these terms in mass memory 30. ii) the 130-B selection of a majority class Cmaj, which is class C; whose respective frequency indicator (f;) (hereafter designated as fmaj) is the highest (or any one of these classes if there are several), iii) the assignment 130-C to each of the classes C; of a respective intermediate class number num_int_i, such that the intermediate class number num_int_maj of the majority class Cmaj is equal to zero, (iv) the formation 130-D of an intermediate training set 68 from the initial set 50, by converting each initial label matrix 51B into an intermediate label matrix 51C whose terms are the intermediate class numbers num_int_i corresponding to the initial class numbers num_i of the initial label matrix 51B, the intermediate label matrices 51C being stored in memory by means of an intermediate data structure different from the initial data structure used for the in-memory storage of the initial label matrices, said training sets 65, intermediate validation sets 67 and final validation sets 66 being formed from the intermediate set 68.

[0092] For determination 130-A, the set of terms of the initial label matrices 51B are read by the processor 25 in the mass memory 30, so as to count the total number of occurrences of each of the classes Q among these terms.

[0093] The frequency indicator £ of the class Ci is for example equal to the total number of occurrences of this class in the set of terms of the initial label matrices 51 B, or equal to the ratio of this total number and the number of terms of the set of initial label matrices 51 B.

[0094] For selection 130-B, the processor 25 then searches for the frequency indicator(s) f with the highest value. The class, or any of the corresponding classes, is then identified as the majority class Cmaj, with frequency indicator fmaj.

[0095] The processor 25 then proceeds to the allocation 130-C of the intermediate class numbers num_int_i.

[0096] The intermediate class numbers num_int_i are preferably integers.

[0097] The intermediate class numbers num_int_i are in bijection with the numbers of initial class num_i, under the constraint that the intermediate class number num_int_maj of the majority class is chosen equal to zero.

[0098] This bijection allows us to define an operation of conversion of an initial label matrix 51B into an intermediate label matrix 51C and conversely, the operation of inverse conversion of an intermediate label matrix 51C into an initial label matrix 51B.

[0099] The processor 25 can then proceed to the formation 130-D of the intermediate set 68. For this, each initial label matrix 51B is associated with an intermediate label matrix 51C, whose terms are the class numbers intermediate num_int_i corresponding to the initial class numbers num_i present at the same position in the initial label matrix 51B.

[0100] It is therefore understood that the initial label matrices 51B and intermediate 51C have the same dimension Ni*N2.

[0101] On the other hand, the intermediate label matrices 51C contain a proportion of nuisance terms, corresponding to the majority class Cmaj. This proportion is in many cases significant, notably greater than 30%, 40%, 50%, 60%, 70%, 80%, 90%, or even more.

[0102] By way of example, in the context of semantic segmentation of road images, the proportion of harmful terms is commonly between 95% and 99%.

[0103] Consequently, the intermediate data structure used for storing the intermediate matrices 51C in memory can be chosen taking into account the frequency indicator fmaj of the majority class Cmaj, and therefore the expected proportion of zeros, and the intermediate data structure is thus different from the initial data structure used for the initial label matrices 51B.

[0104] The intermediate data structure is chosen so that the memory cost of the set of intermediate label matrices 51C is less than the memory cost of the set of initial label matrices 51B.

[0105] In particular, the intermediate data structure is advantageously chosen from among data structures suitable for sparse matrices, for which only information relating to non-nuisance terms is explicit in memory, information relating to nuisance terms being implicit.

[0106] By way of example, a sparse matrix structure such as the scipy.sparse.coo_array structure from the SciPy library of the Python language can be used as an intermediate data structure, the initial data structure being, for example, the numpy array type available in the Numpy library of the Python language.

[0107] The intermediate training set 68 therefore comprises a plurality of matrix pairs, each comprising an input matrix 51A and a respective intermediate label matrix 51C. The intermediate label matrices 51C are advantageously stored in RAM 35.

[0108] The processor 25 then proceeds to form the training set 65, the final validation set 66 and the intermediate validation set 67 from the intermediate set 68, in a manner known to the person skilled in the art.

[0109] The training set 65 and the intermediate validation set 67 are used for the iterative training 135 of the initial neural network 15_in.

[0110] The training set 65 and intermediate validation set 67 are preferably independent of each other.

[0111] The training set 65 and the final validation set 66 are preferably independent of each other.

[0112] The final validation set 66 and the intermediate validation set 67 are preferably independent of each other.

[0113] Alternatively, the final validation sets 66 and intermediate validation sets 67 are not disjoint, or even identical.

[0114] In the following, the reference signs of the matrices relating to the training set 65 are followed by "_ent". Similarly, in the following, the reference signs of the matrices relating to the intermediate validation set 67 are followed by "_VI"; and the reference signs of the matrices relating to the final validation set 66 are followed by "_VF".

[0115] In a known manner, the iterative training 135 comprises a plurality of iterations, called epochs Ep. The number of epochs Ep is denoted in the following by NE, p being an integer ranging from 1 to NE.

[0116] At each ep Ep, the input matrices 51 A_ent of the training set 65 are read by the processor 25 in the mass memory 30 and provided as input to the neural network 15 in its configuration for the current ep Ep, so as to generate for each an estimated training label matrix 51D_ent, as output of the neural network 15 in its configuration for the current ep Ep.

[0117] Each epoch Ep optionally includes a step of dividing the training set 65 into K batches of B input matrices 51A_ent. The batches of input matrices 51A_ent are in this case provided successively to the neural network 15, each once in each epoch Ep. Each epoch Ep comprises K intermediate iterations, so that at the end of an epoch Ep, each input matrix 51A_ent from the training set 65 has been received as input to the neural network 15 once.

[0118] Alternatively, the division step is performed with replacement at each epoch Ep, so that at the end of a given epoch Ep, some 51A_ent input matrices may have been selected from several batches, and / or some 51A_ent input matrices may never have been selected.

[0119] In a particular embodiment, the division step is performed only once for all epochs Ep.

[0120] The terms of an estimated training label matrix 5lD_ent are the intermediate class numbers num_int_i estimated by the neural network 15 in its configuration for the current epoch Ep in response to the provision of the input matrix 51A_ent on the input layer 55-1.

[0121] At the end of each ep, a first distance d(5 lC_ent, 5 lD_ent) between each of the estimated training label matrices 51D_ent and the respective intermediate label matrix 51C_ent is evaluated by the processor 25 and the value of a cost function, configured to measure a distance between the set of estimated label matrices 51D and the respective intermediate label matrices 51C, is calculated on the basis of the first distances d(51C_ent, 51D_ent) thus evaluated.

[0122] The first distance d(5lC_ent, 5lD_ent) is for example the cross entropy distance.

[0123] Advantageously, each estimated training label matrix 5 lD_ent is erased from RAM 35 once the first respective distance d(51C_ent, 51D_ent) has been obtained.

[0124] Finally, at the end of each epoch Ep, the synaptic weights, and optionally the biases, are adjusted according to the value taken by the cost function. The adjustment of the synaptic weights and optionally the biases is carried out in a manner known to those skilled in the art, for example according to the backpropagation of the gradient method, to provide an updated neural network 15.

[0125] The updated neural network 15 is validated before the implementation of the next Ep+i epoch by means of the intermediate validation set 67.

[0126] For this purpose, the input matrices 51A_VI of the intermediate validation set 67 are read by the processor 25 in the mass memory 30 and provided as input to the neural network 15 in its configuration for the current epoch Ep, so as to generate for each one at the output of the neural network 15 an estimated intermediate validation label matrix 51D_VI.

[0127] A performance indicator of the neural network, configured to allow monitoring of the evolution of the classification performance of the neural network over successive epochs, is evaluated at the end of each ep on the basis of the set of estimated intermediate validation label matrices 51D_VI and the respective intermediate label matrices 51C, in particular so as to allow the transition to the next epoch Ep+i, according to the principle of early stopping or not and / or to adapt one or more hyperparameters of the neural network 15 used during the initialization 110 of this neural network 15.

[0128] In particular, the performance indicator calculated for the intermediate validation set 67 is compared to the same performance indicator calculated for the training set 65 at the same epoch Epainsi as to one or more of the performance indicators calculated for the intermediate validation set 67 at previous epochs.

[0129] The initialized neural network 15_in is used for the first epoch Eb Each epoch Ep after the first is carried out using the updated neural network 15 from the previous epoch Ep i.

[0130] At the end of the last epoch Ene, the trained neural network 15_ent is obtained.

[0131] For the iterative training 135, at least a part, advantageously the whole, of the intermediate label matrices 51C_ent, 51C_VI of the training set 65 and of the intermediate validation set 67 is, if this has not already been done, loaded into RAM 35 at the beginning of the first epoch Ei and kept in RAM 35 until the end of the last epoch Ene.

[0132] This is made possible by the choice of the intermediate data structure, the available space in RAM 35 not generally being sufficient for the initial label matrices 51B but being sufficient for the intermediate label matrices 51C.

[0133] The memory cost of the set of intermediate label matrices 51C_ent, 51C_VI of the training set 65 and the intermediate validation set 67 is in fact in many cases lower, or even much lower, than that of the set of corresponding initial label matrices 51B_ent, 51B_VI, due to the proportion of nuis terms.

[0134] It should be noted that it is generally not possible to carry out an analogous operation on the input matrices 51A, for which there is no reason why a particular value of a term should have a significantly higher frequency of occurrence than other values.

[0135] The evaluation of the initial distances and, ultimately, the calculation of the cost function at the end of each epoch Ep for the training step 135 according to the invention have a lower time cost than prior art learning methods. In particular, according to the invention, the evaluation of the initial distances and the calculation of the cost function only require readings of information stored in the RAM 35, whereas in prior art methods, the initial label matrices 51B must be read from the mass memory 30.

[0136] If we denote Ataccess the difference between the access time t_mass to mass memory 30 and the access time t_live to RAM 35, the order of magnitude of the time saving for the training step 135 according to the invention is on the order of K*NE*B*Ataccess, from which we must deduct the time cost of the determination steps 130-A, selection 130-B and allocation 130-C, which is on the order of K*B*t_mass, generally negligible compared to K*NE*B*Ataccess

[0137] Moreover, the evaluation of the first distances can be performed on the intermediate data structure and can therefore be optimized in time compared to a calculation of distance made on the initial data structure, so that the time saving is even higher.

[0138] Alternatively, the evaluation of each first distance includes a prior step of inverse conversion of the respective estimated intermediate training label matrix 51D into a respective estimated initial label matrix 51D' and of the respective intermediate label matrix 51C into the respective initial label matrix 51B, on the basis of the bijection between the initial class numbers num_i and the intermediate class numbers num_int_i, only the corresponding first distance being kept in RAM 35 until the end of the calculation of the cost function.

[0139] In this case, the time cost of the reverse conversion is on the order of K*B*t_mass.

[0140] It is therefore understood that the training method 10 according to the invention allows for a time saving greater than K*(NE-2)*B*Ataccess. Bearing in mind that: - the number of learning epochs NE is generally greater than 10, frequently greater than 100 or even 1000, - the number K*B of 51A input matrices is frequently greater than a thousand, - t_mass is on the order of a few tens of milliseconds, t_live is on the order of a few nanoseconds to a few tens of nanoseconds, Ataccess itself therefore being on the order of a few tens of milliseconds,

[0141] the time saving is significant, and can reach a relative factor of two or more, and in absolute terms, a few seconds to a few tens of seconds.

[0142] Optionally, the training method 10 includes a memory selection step 150, before the iterative training step 135, for example between the allocation 130-C and the training 130-D of the intermediate set 68.

[0143] The memory selection step 150 includes an evaluation of a memory cost 150-A of the set of intermediate label matrices 51C of the training set 65 and the intermediate validation set 67, and optionally of the final validation set 66 on the basis of the frequency indicator fmaj of the majority class Cmaja as well as the number of corresponding input matrices 51A and the dimension Ni*N2 of these input matrices 51A.

[0144] The memory selection step 150 then includes the search 150B for one or more available RAM 35s based on this memory cost.

[0145] If a random access memory 35 exchanging data with a processor used to implement the method has sufficient memory space for storing information relating to the intermediate label matrices 51C, this information are retrieved into RAM 35 as previously described for their use during iterative training 135.

[0146] In a particular embodiment, the RAM 35 is chosen from a RAM memory dedicated to the processor 25 and a VRAM memory optimized for graphics data management.

[0147] In this case, VRAM memory may be chosen in preference to RAM memory when VRAM memory is available, especially if the processor 25 is of the GPU type.

[0148] Optionally, if no RAM 35 allows the storage of the set of intermediate label matrices 51C corresponding to the input matrices 51A whose memory cost has been evaluated, the memory selection step 150 may include a step of selecting the largest subset of the training set 65 allowing the implementation of the determination step 135 as described above on the basis of a training set with reduced memory cost.

[0149] Alternatively, if no RAM 35 has sufficient available storage space, the memory selection step 150 optionally includes: - determining a set {MCk} of MCk metaclasses from the set {Ci} of predetermined classes Ci, based on the determined majority class Cmaj, and - the formation of secondary label matrices whose terms are the meta-class numbers corresponding to the initial class numbers of the initial label matrix, and - the reiteration of the determination steps 130-A, selection 130-B and allocation 130-D from the secondary label matrices before the training step 130-D.

[0150] The set {MCk} of MCk meta-classes has a smaller dimension than the set {Ci} of predetermined Ci classes, which allows us to determine a majority meta-class MCmaj whose frequency indicator is greater than that of the majority class, and thus reduce the memory cost, at the cost of a less fine annotation.

[0151] Finally, the processor 25 carries out the final validation 140 of the trained neural network 15_ent thus obtained using the final validation set 66.

[0152] For the final validation 140, the set of intermediate label matrices 51C_VF of the final validation set 66 are, if possible and if not already done, retrieved into RAM 35.

[0153] Optionally, the intermediate label matrices 51C_VF of the entire training set 68 are loaded back into RAM 35 at the beginning of the first epoch Eb

[0154] For final validation 140, an estimated final validation label matrix 51F is estimated using the trained neural network 15_ent for each input matrix 51A_VF of the final validation set 66.

[0155] For this purpose, the input matrices 51A of the final validation set 66 are read by the processor 25 in the mass memory 30, so as to generate for each one an estimated final validation label matrix 51D_VF, estimated by the trained neural network 15_ent.

[0156] A classification performance indicator of the trained neural network 15_ent is evaluated by the processor 25 on the same principle as for the intermediate validation set 67, on the basis of the set of estimated final validation label matrices 51D_VF and the respective intermediate label matrices 51C_VF.

[0157] The performance indicator may be based on a metric identical or different from that used for intermediate validations.

[0158] The performance indicator can then be compared to a predetermined setpoint value so as to validate or invalidate the trained neural network 15_ent for subsequent use in an inference phase.

[0159] Optionally, the training method 10 according to the invention includes a step of cutting one or more portions of the input matrices 51A of the training set 68 on the basis of the majority class Cmaj.

[0160] The class-aware cropping step (Cmaj) involves searching for and cutting portions of the input matrices 51A corresponding to rare classes, i.e., classes whose frequency indicator f is much lower than the majority frequency indicator fmaj, for the purpose of training these rare classes. This search and cutting are particularly fast in the training method 10 according to the invention, since the intermediate data structure includes these rare classes explicitly and without dilution by the majority class C.

[0161] The invention also relates to a data classification method 170 by segmentation using the trained neural network 15_ent obtained at the end of the training method 10 as described above for the classification of at least one matrix input data 45.

[0162] As can be seen in [Fig.2], in a first substep 170A, the matrix data 45 is provided on the input layer 55-1 of the trained neural network 15_ent and an intermediate label matrix response 70 is obtained at output 55-p of the trained neural network 15_ent.

[0163] In a second substep 170B, the intermediate label matrix response 70 is converted into the initial label matrix response 75 on the principle of the inverse conversion described previously.

[0164] The invention finally relates to a computer program product comprising software instructions which, when executed by a computer, implement the training method 10 or classification method 170 as described above.

Claims

1. Demands Method for training (10) a neural network (15) configured for classifying matrix input data (45) comprising: a) the initialization (110) of a neural network (15), b) the provision (120) of an initial set (50) of training data comprising a plurality of matrix pairs (51), each matrix pair (51) comprising an input matrix (51A) comprising input data and a respective initial label matrix (51B) of the same dimension as the input matrix (51A) and of which each term is an initial class number (num_i) representing a class (Ci) of a respective term of the input matrix (51A), said class being selected from a set ({Ci}) of predetermined classes, c) the training (130) from the initial set (50), a training set (65), a final validation set (66) and an intermediate validation set (67), d) the iterative training (135) of the neural network (15) using the training set (65) and the intermediate validation set (67), and e) the final validation (140) of the trained neural network (15_ent) using the final validation set (66), characterized in that: training c) of the training sets (65), final validation (66) and intermediate validation (67) includes: i) the determination (130-A) for each class (Ci) of the set ({C;}) of predetermined classes of a frequency indicator (f;) representative of the total number of occurrences of this class (Ci) in the set of terms of the initial label matrices (51B) of the initial set (50), ii) the selection (130-B) of a majority class (Cmaj), which is the class with the highest frequency indicator (fmaj), iii) the assignment (130-C) to each of the classes (Ci) of a respective intermediate class number (num_int_i) such that the intermediate class number of the majority class (num_int_maj) is equal to zero, and (iv) the formation (130-D) of an intermediate training set (68) from the initial set (50), by converting each initial label matrix (51B) into an intermediate label matrix (51C) whose terms are the intermediate class numbers (num_int_i) corresponding to the initial class numbers (num_i) of the initial label matrix (51B), the intermediate label matrices (51C) being stored in memory by means of an intermediate data structure different from an initial data structure used for the in-memory storage of the initial label matrices (51B), said training sets (65), final validation set (66) and intermediate validation set (67) being formed from the intermediate set (68).

2. Method (10) according to claim 1, comprising prior to the iterative training (135) of the neural network (15): c2) a selection step (150) of a RAM (35) exchanging data with a processor (25) used for the implementation of the method (10) having sufficient memory space for storing information relating to the intermediate label matrices (51C) and storing this information in said RAM (35) for use during the iterative training (135).

3. Method (10) according to claim 2, wherein during the iterative training (135), at least one set of estimated intermediate label matrices (51D) is obtained at the output of the neural network (15) in response to the input of the neural network of a set of input matrices (51A) chosen from the intermediate set (68), and is compared either directly with the respective intermediate label matrices (51C) or with the initial label matrices (51B) after inverse conversion of the estimated intermediate label matrices (51D) into respective estimated initial label matrices (51D'), said label matrices being stored in RAM (35).

4. Method (10) according to claim 2 or claim 3, wherein the RAM (35) is chosen at the selection step (150) from a RAM memory dedicated to the processor and a VRAM memory optimized for graphics data management.

5. Method (10) according to claim 4, wherein VRAM memory is chosen in preference to RAM memory.

6. A method (10) according to any one of claims 2 to 5, wherein the step of choosing a RAM includes, if necessary: ​​- determining a set ({MCk}) of meta-classes (MCk) from the set ({Ci}) of predetermined classes on the basis of the majority class (Cmaj) determined, - forming secondary label matrices whose terms are the meta-class numbers (MCk) corresponding to the initial class numbers (num_i) of the initial label matrix (51 A), and - repeating steps i), ii) and iii) from the secondary label matrices before step iv) of forming (130-D) the intermediate set (68).

7. Method (10) according to any one of the preceding claims wherein the input matrices (51 A) are previously obtained by means of a digital image sensor or by means of a digital sound sensor.

8. Method (10) according to any one of the preceding claims wherein the intermediate data structure is specifically adapted for sparse matrices.

9. Method for classifying (170) data by segmentation using a neural network (15_ent) obtained at the end of the training method (10) according to any one of the preceding claims.

10. Product computer program comprising software instructions which, when executed by a computer, implement the method (10) according to any one of claims 1 to 8.

Images