Data classification method, associated method and devices
The data classification process addresses catastrophic forgetting by initializing and partitioning neural networks with unannotated data, using K-means and softmax, allowing efficient learning of new classes without manual annotation, thereby improving robustness and reducing model update times.
Patent Information
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- THALES SA
- Filing Date
- 2024-12-18
- Publication Date
- 2026-06-19
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Data classification method, method and associated devices
[0001] The present invention relates to a data classification method. It also relates to a method for recognizing objects present in an environment. The invention also relates to associated devices.
[0002] The invention falls within the framework of online learning, where it is desired that an artificial intelligence model be able to adapt dynamically to its environment.
[0003] Incremental class learning attempts to solve this challenge by allowing a model, already trained on a certain number of classes (base classes), to continually learn new classes from a dynamic data stream (incremental step),
[0004] However, when the model is trained with data from the new classes, it tends to forget the old classes. This is because the model's weights are updated with information from the new classes, and by default, the model does not retain knowledge of the previous classes.
[0005] This phenomenon, known as catastrophic forgetting, is resolved by implementing one or more of the following techniques:
[0006] - a data-based technique in which backup or generation of examples from previous classes.
[0007] - a technique for modifying the model weights or the training procedure to counter the bias introduced by the new classes.
[0008] - a technique involving a modification of the model architecture.
[0009] However, these techniques rely on the strong constraint that the datasets at each incremental stage are perfectly annotated by a human.
[0010] This assumption is unrealistic in practice, because in real-world scenarios, new datasets are often unknown and unannotated.
[0011] There is therefore a need for a classification process enabling a neural network to learn new classes from new data, without external supervision, limiting the phenomenon of catastrophic forgetting.
[0012] To this end, the description relates to a data classification process, the process being implemented by computer, the classification process comprising:
[0013] - a learning phase of a neural network, to obtain a network of learned neurons, the learning phase including a step of:
[0014] - initialization involving the reception of an initial neural network specific to classify a dataset into an initial number of classes,
[0015] - receiving an unannotated dataset and a number of new classes at identify,
[0016] - for each data point in the received dataset, application of the neural network initial data point, to obtain a projection of each data point into a representation space,
[0017] - partitioning each data projected in the representation space into one plurality of distinct partitions, each partition grouping the projected data exhibiting a distance less than a threshold from the other projected data of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified,
[0018] - assigning an identical label to all projected data that are part of a same score,
[0019] - for each partition, calculation of a confidence index for the prediction of the label assigned to each projected data point of said partition,
[0020] - deduction of the class of each data point from the classes of the neural network initial, the assigned label and a comparison of the confidence index to a reference confidence index, and
[0021] - training the neural network to classify the unannotated dataset based on the deduced classes, the neural network thus trained being the learned neural network,
[0022] - an inference phase, the inference phase comprising a step of:
[0023] - receiving data to be classified, and
[0024] - application of the learned neural network to the received data to obtain a classification of received data.
[0025] According to other advantageous aspects of the invention, the classification method comprises one or more of the following features, taken individually or in all technically possible combinations:
[0026] - the learning phase is iterated k times, k being a natural number greater than or equal to to 2.
[0027] - the learning phase is iterated several times, the classification process including, in addition:
[0028] - a step of establishing a dictionary during the first implementation of a learning phase, said dictionary listing the associations between the classes received and the labels established during the first implementation of a learning phase,
[0029] - a dictionary update step at each learning phase, and
[0030] - a step of measuring a performance level of the neural network learned in depending on the dictionary update, the said measurement step being implemented during the inference phase.
[0031] - the partitioning step is implemented by a K-means algorithm.
[0032] - the confidence indices calculated during the calculation step are calculated from of a distance matrix comprising the distances between a center of a partition and the projected data forming part of said partition.
[0033] - the confidence indices calculated during the calculation step are calculated in applying a softmax function on a vector comprising the inverse of the distances provided by the distance matrix.
[0034] The description also describes a method for recognizing objects present in an environment, the recognition method being implemented by an observation system comprising an environment observation sensor and a computer, the recognition method comprising a step of:
[0035] - acquisition by the sensor of data relating to the environment,
[0036] - implementation by the computer of the inference phase of a process of classification of environmental data, the classification process being as described above, to obtain a classification of environmental data whose classes correspond to the presence or absence of the objects to be recognized, and
[0037] - recognition by the computer of the presence of objects in the environment by based on the classification obtained.
[0038] The description also relates to a computer specifically designed for classifying data, the computer being designed for:
[0039] - receive data to be classified,
[0040] - apply a neural network learned on the received data to obtain a classification of received data,
[0041] the neural network having been learned by implementing a learning process comprising:
[0042] - an initialization comprising the reception of an initial neural network specific to classify a dataset into an initial number of classes,
[0043] - receiving an unannotated dataset and a number of new classes to identify
[0044] - for each piece of data in the received dataset, an application of the neural network initial data point, to obtain a projection of each data point into a representation space,
[0045] - a partitioning of each data point projected into the representation space in a plurality of distinct partitions, each partition grouping the projected data exhibiting a distance less than a threshold from the other projected data of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified,
[0046] - an assignment of an identical label to all projected data forming part of the same score,
[0047] - for each partition, a calculation of a confidence index for the prediction of the label assigned to each projected data point of said partition,
[0048] - a deduction of the class of each data point from the classes of the network of initial neurons, the assigned label, and a comparison of the confidence index to a reference confidence index, and
[0049] - training the neural network to classify the unannotated dataset according to the classes deduced, the neural network thus trained being the learned neural network.
[0050] The description also relates to a computer designed to recognize objects present in an environment, the computer being designed to:
[0051] - receive environmental data, data relating to the environment having been acquired by a sensor,
[0052] - apply a neural network learned on the received data to obtain a classification of environmental data, the classes of which correspond to the presence or absence of objects to be recognized.
[0053] - recognize the presence of objects in the environment based on the classification obtained
[0054] the neural network having been learned by implementing a learning process comprising:
[0055] - an initialization comprising the reception of an initial neural network specific to classify a dataset into an initial number of classes,
[0056] - receiving an unannotated dataset and a number of new classes to identify
[0057] - for each piece of data in the received dataset, an application of the neural network initial data point, to obtain a projection of each data point into a representation space,
[0058] - a partitioning of each data point projected into the representation space in a plurality of distinct partitions, each partition grouping the projected data exhibiting a distance less than a threshold from the other projected data of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified,
[0059] - an assignment of an identical label to all projected data forming part of the same score,
[0060] - for each partition, a calculation of a confidence index for the prediction of the label assigned to each projected data point of said partition,
[0061] - a deduction of the class of each data point from the classes of the network of initial neurons, the assigned label, and a comparison of the confidence index to a reference confidence index, and
[0062] - a neural network training method for classifying the unannotated dataset according to the classes deduced, the neural network thus trained being the learned neural network.
[0063] The description also relates to an environmental observation system comprising:
[0064] - an environmental observation sensor, the sensor being suitable for acquiring environmental data, and
[0065] - a calculator as described above.
[0066] In this description, the expression "specific to" means interchangeably "suitable for", "adapted to" or "configured for".
[0067] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:
[0068] - [Fig. 1] [Fig. 1] is a schematic representation of an observation system, and
[0069] - [Fig.2] [Fig.2] is a flowchart of an example of the implementation of a method of classifying data relating to the environment observed by the observation system of the [Fig.1].
[0070] An observation system 10 is schematically represented in [Fig.1].
[0071] The observation system 10 is suitable for observing an environment 12.
[0072] In this case, the observation system 10 is suitable for recognizing objects 14 in environment 12.
[0073] As a particular example, for a monitoring application, the observation system 10 is suitable for knowing whether animals 14 are present in the environment 12 and for recognizing the animal in question.
[0074] The observation system 10 includes a sensor 16 and a computer 18.
[0075] The sensor 16 is suitable for acquiring data relating to the environment 12.
[0076] By "environmental data 12", it is understood that environmental measurement signals 12 are understood.
[0077] The sensor 16 is thus an environmental observation sensor 12.
[0078] For example, the sensor 16 is a sensor suitable for obtaining an image of the environment 12 in any spectral band.
[0079] According to another example, the sensor 16 is a sensor suitable for recording a sound signal present in the environment 12.
[0080] It should be noted that the observation system 10 can include as many sensors 16 as desired, thus facilitating recognition.
[0081] The computer 18 is suitable for implementing the inference phase of a data classification process relating to the environment 12 which will be described later.
[0082] This implementation leads to obtaining a classification of the data relating to the environment whose classes correspond to the presence or absence of the objects to be recognized.
[0083] By way of example, environmental data are classified into three classes, namely a "cat" class, a "house" class and an "airplane" class.
[0084] The computer 18 is designed to recognize the presence of objects in the environment based on the classification obtained, that is to say that the computer 18 gives two pieces of information, the presence of an object and its identification.
[0085] The calculator 18 is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the calculator and / or memories into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices or storage devices.
[0086] As specific examples, the calculator 18 is implemented in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specifies Integrated Circuit).
[0087] Alternatively, when the method is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium (not shown). The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.
[0088] An example of implementation of the environmental data classification process 12 is now described with reference to the flowchart in [Fig.2].
[0089] The classification process is implemented by the computer 18.
[0090] According to the example described, the classification process includes a learning phase PI and an inference phase P2.
[0091] The PI learning phase is a learning phase of a neural network allowing a learned neural network to be obtained.
[0092] Before explaining the steps of the PI learning phase, some non-limiting generalities on neural networks are given.
[0093] A neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of one or more other layers, in particular from the previous layer.
[0094] More precisely, each layer includes neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.
[0095] Alternatively, more complex neural network structures can be envisaged with a layer that can be linked to a layer further away than the immediately preceding layer.
[0096] Each neuron is also associated with an operation, that is to say a type of processing, to be carried out by said neuron within the corresponding processing layer.
[0097] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a link between two neurons. It is often a real number, which takes on both positive and negative values. In some cases, the synaptic weight is a complex number.
[0098] Each neuron is designed to perform a weighted sum of the value(s) received from the neurons of the preceding layer, each value being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the preceding layer, and then to apply an activation function, typically a non-linear function, to said weighted sum, and to deliver at the output of said neuron, in particular to the neurons of the next layer connected to it, the value resulting from the application of the activation function. The activation function introduces non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, and the Heaviside function are examples of activation functions.
[0099] As an optional complement, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.
[0100] The PI learning phase includes an initialization step E30, a reception step E32, an application step E34, a partitioning step E36, an allocation step E38, a calculation step E40, a deduction step E42 and a training step E44.
[0101] During the initialization step E30, the computer 18 receives an initial neural network suitable for classifying a dataset into an initial number of classes.
[0102] According to one embodiment, the computer 18 obtains the initial neural network at the initialization step E30.
[0103] In such a case, the computer 18 receives a neural network architecture (a neural network whose hyperparameters are either unknown or have been initialized to a value suitable for another task) and an annotated dataset.
[0104] The computer 18 then implements an initial training of the neural network architecture on the basis of the annotated dataset.
[0105] The computer 18 thus obtains an initial neural network capable of classifying the received annotated dataset as well as a list including the classes relating to the annotated dataset.
[0106] The number of classes in the list is denoted P for the remainder of the description.
[0107] For example, the computer 18 receives during the initialization step E30 a neural network architecture to be trained as well as 15 images of size 32 pixels x 32 pixels, including 5 images of cats, 5 images of houses and 5 images of airplanes.
[0108] By implementing learning, the computer 18 obtains an initial neural network capable of classifying the 15 images into 2 different classes, including a "cat" class and a "house" class, as well as the list of classes ("cat" and "house").
[0109] Alternatively, computer 18 receives directly this initial neural network and the list of classes (“cat” and “house”).
[0110] During the E32 reception step, the computer 18 receives an unannotated dataset and a number of new classes to be identified.
[0111] For the remainder, the number of new classes to be identified is named C.
[0112] C is an integer greater than or equal to 1.
[0113] Following the example, during the reception step E32, the computer 18 receives 10 images Jb J2, ... and Ji0, of size 32 pixels x 32 pixels, and 2 classes.
[0114] During the application step E34, for each data item in the data set received at the reception step E32, the computer 18 applies the initial neural network to the data item to obtain a projection of each data item into a representation space.
[0115] For example, the computer 18 successively applies the initial neural network to each piece of data in the dataset received during the reception step E32.
[0116] Thus, for each data, by this application of the neural network, the computer 18 forms a first vector of dimension Ni containing the parameters of said data, and applies a reduction operation to this first vector allowing to obtain a second vector, of dimension N2 < Ni, containing only the parameters relevant to the classification of this data.
[0117] The computer 18 then projects this second vector into a representation space of dimension N2 containing the set of second vectors relating to the data of the dataset received during the reception step E32 and to the annotated dataset received during the initialization step E30.
[0118] The calculator 18 thus obtains at the output of the application step E34 a representation space of the second vectors relating to the set of data received during the previous steps.
[0119] Following the example, during the application step E34, the computer 18 applies the initialized neural network to each of the 10 images Jb J2, ..Ji0.
[0120] The application of the initial neural network then gives, for the first image, a vector including for example the red-green-blue values (more commonly called RGB values in reference to the corresponding English name Red-Green-Blue) of each pixel contained in the image Ji.
[0121] The first vector is thus of dimension Ni= 3072.
[0122] The application of the initial neural network then induces a reduction operation to this first vector, keeping for example the RGB values of the central 8x8 pixels only, and thus delivers a second vector Vj of dimension N2 = 192.
[0123] The application of the initial neural network then leads to projecting this second vector into the representation space p192 comprising the 15 second vectors relating to the 15 images received during the initialization step E30.
[0124] The neural network thus applies these operations to the 9 remaining images J2, ..., Jio and delivers a representation space p192 comprising the 15 second vectors relating to the images received during the initialization step as well as the 10 second vectors Vj, V2, ..., V10 relating to the images received during the reception step E32.
[0125] During the partitioning step E36, the computer 18 performs a partitioning of each data projected in the representation space into a plurality of distinct partitions.
[0126] Each partition groups the projected data presenting a distance less than a threshold from the other projected data of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified.
[0127] More specifically, the computer 18 receives the representation space delivered by the initial neural network and partitions the part of said representation space comprising the second vectors projected during the application step E34, by applying a K-means algorithm, into a number of partitions equal to P + C.
[0128] A K-means algorithm is a data partitioning technique more often referred to by the corresponding English name "K-Means".
[0129] In the example described, the K-means algorithm performs this partitioning by calculating the distances in the representation space between each second vector, and by grouping, into M+C distinct partitions, the sets of second vectors exhibiting the smallest mutual distances.
[0130] The K-means algorithm also provides as output a distance matrix M, comprising the distances, in the representation space, between each second vector and the center of its partition.
[0131] The calculator 18 thus obtains, at the output of the partitioning step E36, the P+C partitions, the set of second vectors included in each partition, as well as the distance matrix M.
[0132] Following the example, during the partitioning step E36, the K-means algorithm partitions the part of p192 comprising the 10 second vectors Vb V2, ..., Vio relating to the images received during the reception step E32 into 4 distinct partitions Pb P2, P3 and P4.
[0133] To do this, the K-means algorithm calculates, for example, the distance in p192 between each pair of second vectors, by calculating, for example, for the pair of second vectors V1= (Ru, Gu, Bu, R1>2, Gj>2, B1>2, ..., R1>64, Gj>64, B1>64) and V2= (R2>j, G2ji, B2ji, R22, G2j2, B22, ..., R2j64, G2j64, B2j64), the absolute values IRij - R2jl, IGij - G2jjl, IBij - B2jl, for all j in [ 1; 64]], and then calculates the sum Si>2= R1J_ r2jÎ + | g1j- G2j| + IB1J- B2jj
[0134] The K-means algorithm therefore calculates the sums Sm>n relating to the second vectors Vm and Vn for all distinct m and n in [ 1; 10], and groups in the partitions Pb P2P3 and P4 the second vectors such that the set of sums relating to 2 second vectors is minimal in each of the partitions Ph P2P3 and P4.
[0135] Advantageously, a sum relative to 2 second vectors is small if the 2 second vectors are close in p192, and large if the 2 second vectors are far apart in p192.
[0136] Thus, the partitions Pb P2 P3 and P4 group the second vectors comprising similar parameters, and therefore similar RGB values.
[0137] In addition, the K-means algorithm provides the distance matrix M = (mb m2, ..., mio) comprising the distances between Vb V2, ... and V10 and the center of the partition in which they are contained.
[0138] For example, if the first partition Pi centered at Ci contains Vb ... V5 and the second partition P2 centered at C2 contains V6, ..., Vio, the distance matrix M established
[0150] VKGN, € softmax: ■ using the K-means algorithm provides the distances m2, ..., m5 between Ci and Vb, Ci and V2, Ci and V5, and m6, m7, mi0 between C2 and V6, C2 and V7, ... C2 and Vio.
[0139] Thus, in the example described, at the output of the partitioning step E36, the computer 18 provides, for example, the information (Vb V2, ..., V5) e Pj and (V6, V7, ..., Vio) e p2 as well as the distance matrix M.
[0140] During the assignment step E38, an identical label is assigned to all projected data that are part of the same partition.
[0141] Following the example, the calculator 18 advantageously assigns, during the assignment step E38, an identical index, for example 1, to all vectors (Vj, V2) − Pb and a distinct identical index, for example 2, to all vectors (V3, V4) − P2 and so on.
[0142] The calculator 18 thus provides as output, for example, a set of pairs {(Vj, 1), (V2, 1), (V3, 2), (V4, 2), ..., (Vio, 4)} allowing a second vector of the representation set to be associated with its label assigned during the assignment step E38.
[0143] Calculator 18 thus has a list of associations between each second vector and its assigned label.
[0144] During the calculation step E40, for each partition, the calculator 18 calculates a confidence index of the prediction of the label assigned to each projected data of said partition.
[0145] More specifically, during a calculation step, the calculator 18 calculates a matrix I comprising confidence indices relating to the assignment of labels implemented during the assignment step E38.
[0146] For each second vector, its confidence index is calculated as a function of its distance in the representation space from the center of its partition, said distance being provided by the distance matrix M.
[0147] The calculator 18 then applies a softmax function to a matrix N comprising the inverse of the distances provided by the distance matrix M.
[0148] The softmax function is a generalization of the logistic function. It converts a vector of K real numbers into a probability distribution over K choices.
[0149] More precisely, the softmax function transforms a vector Z = (Zp ..., ) into a vector of K strictly positive real numbers with a sum of 1. The softmax function can be written mathematically as follows: PR- ]0; 1]A Vz€Pk, Vie [0;K-IL sof tmax(z) . = k 1
[0151] The calculator 18 then associates each confidence index included in matrix I with its second vector and its label provided by the list of associations received.
[0152] Thus, the calculator 18 delivers at the output of the calculation step E40 a set of associations between each second vector, their assigned label and their calculated confidence index.
[0153] Following the example, the calculator 18 receives during the calculation step E40 the matrix M = (mb m2, ..., mio) and the set of pairs {(Vb 1), (V2, 1), (V3, 2), (V4, 2), ..., (V10, 4)}.
[0154] The calculator 18 then establishes the matrix N = (l / mb l / m2, ..., 1 / mio), calculates I = softmax(N) = (ib i2, ..., iio), and associates each confidence index with its pair included in the set of pairs.
[0155] Thus, at the output of the calculation step E40, the calculator 18 provides for example a set of triplets {(Vb 1, ii), (V2, 1, i2), (V3, 2, i3), (V4, 2, i4), ..., (Vio, 4, i10)}, in which a triplet includes the second vector, the assigned label and the calculated confidence index.
[0156] During the deduction step E42, the calculator 18 deduces the class of each data point from the classes of the initial neural network, the assigned label and a comparison of the confidence index to a reference confidence index.
[0157] More specifically, during this deduction step E42, the calculator 18 receives the set of associations established during the calculation step E40, as well as the list of initial classes.
[0158] The calculator 18 then compares the confidence indices included in the set of associations with a reference confidence index, and establishes a set of confident associations, comprising only the associations between the second vectors and their assigned label, comprising a confidence index higher than the reference confidence index.
[0159] Calculator 18 then adds to the received class list the distinct labels included in the set of trusted associations.
[0160] Thus, at the output of the deduction step E42, the computer 18 delivers the set of confident associations between the second vectors and their assigned label, as well as a list of classes including the classes developed during the initialization step, and the distinct labels included in the set of confident associations established during the deduction step E42.
[0161] Following the example, the calculator 18 receives during the deduction step the set of triplets {(Vb 1, ii), (V2, 1, i2), (V3, 2, i3), (V4, 2,14), ..., (V10, 4, iio)}, and the list of classes (“cat”, “house”).
[0162] The calculator 18 reads each of the confidence indices ib i2, ..., ii0 included in the set of received triplets, and for any j in [1; 10 J, checks if the condition ij > iref is met, iref being the reference confidence index.
[0163] For each j in [ 1; 10], if the condition is met, the pair (Vj, L), L being the label assigned to the second vector Vj, is added to a set of trusting associations.
[0164] For example, the condition is satisfied for all j in U 1; 10], except for j = 5 and j = 10.
[0165] Thus, the computer 18 develops the set of trusting associations {(Vi, 1), (V2, 1), (V3, 2), (V4, 2), ... (Vio, 4)}, reads the labels included in this set, and adds the distinct labels to the class list ("cat", "house"), in order to form the class list ("cat", "house", 1, 2).
[0166] At the output of the deduction step E52, the information processing unit thus delivers the set {(Vi, 1), (V2, 1), (V3, 2), (V4, 2), ... (Vio, 4)} and the list of classes ("cat", "house", 1,2)
[0167] According to a particular example, an operator then simply identifies classes 1 and 2 to assign them a realistic label such as "dog" and "man" to give a concrete example.
[0168] In this case, the class list becomes (“cat”, “house”, “dog”, “man”).
[0169] During training step E44, the computer 18 trains the neural network to classify the unannotated dataset according to the inferred classes.
[0170] The neural network thus trained is the learned neural network.
[0171] The PI learning phase just described therefore makes it possible to obtain a neural network capable of recognizing classes that it does not know.
[0172] This allows for incremental learning of the neural network to give it the ability to learn as it goes along.
[0173] For this reason, it is advantageous for the PI learning phase to be implemented several times.
[0174] According to a simple embodiment, the learning phase is iterated k times, k being a natural number greater than or equal to 2.
[0175] This embodiment is represented in [Fig.2], the learning phase being repeated as long as a counter i, incremented at each implementation of a learning phase, is less than k.
[0176] The number k can in particular be parameterized by an operator according to his knowledge for the application envisaged for the neural network to be learned.
[0177] According to a more elaborate embodiment, the learning phase is repeated as long as the performance level of the learned neural network is lower than a reference performance level.
[0178] Advantageously, the performance level is obtained using a dictionary.
[0179] To obtain the performance level, the process includes, for example, an establishment step, an update step and a measurement step.
[0180] During the establishment step, the computer 18 establishes a dictionary during the first implementation of a learning phase, said dictionary listing the associations between the classes received and the labels established during the first implementation of a learning phase.
[0181] During the update step, calculator 18 updates the dictionary at each learning phase.
[0182] During these steps, the computer 18 applies to the learned neural network a Hungarian algorithm, which implements a correspondence between the labels and the classes of the unannotated dataset.
[0183] Each correspondence between a label and a class is then listed in the dictionary, and said dictionary makes it possible to measure the performance of the classification: for each data in the unannotated dataset, its assigned label is read and the dictionary thus makes it possible to associate said data with a class.
[0184] The computer 18 therefore allows external supervision to check whether each data has been correctly classified by the learned neural network, and to establish a performance value corresponding to a ratio of the number of correctly classified data to the number of data included in the received dataset.
[0185] During the measurement step, the calculator 18 measures a performance level of the learned neural network as a function of the dictionary update.
[0186] More specifically, in the example described, the dictionary is applied to the labels obtained to obtain predicted classes and a performance metric is calculated that represents the difference between the classes predicted after application of the dictionary and the actual classes.
[0187] Such a metric is, for example, the metric measuring the rate of correct predictions out of all predictions to be made. Such a metric is more often referred to by the corresponding English term "accuracy," which literally means "precision."
[0188] Advantageously, in the example described, learning is performed only once per PI learning phase, so that the measurement step is preferably implemented during the P2 inference phase.
[0189] As shown in [Fig.2], the inference phase P2 includes a reception step E46 and an application step E48.
[0190] During the E46 reception step, the control unit 18 receives data from the sensor 16.
[0191] Then, during the application step E48, the computer 18 then applies the learned neural network to the received data to obtain a classification of the received data.
[0192] In the example described, this classification has the advantage of including both the initial classes (“cat”, “house”) and the new classes (“dog”, “man”).
[0193] The process just described thus makes it possible to automatically generate quality annotations, thanks to a selection based on a confidence criterion, which can replace human annotations.
[0194] The method also makes it possible to use incremental learning methods efficiently with pseudo-labels (those corresponding to new classes), while limiting the loss of precision.
[0195] Robustness is also improved with the use of a confidence index threshold and a static dictionary in certain embodiments.
[0196] Finally, the process proposes an evaluation technique adapted to unsupervised incremental learning problems.
[0197] The described process is therefore a classification process enabling a neural network to learn new classes from new data, without external supervision, limiting the phenomenon of catastrophic forgetting.
[0198] Put another way, in the general deployment context, the method makes it possible to implement incremental learning techniques without having to manually annotate all new data at each iteration. This reduces the model update time from several days to a few minutes, thus eliminating the need for exhaustive manual annotation.
[0199] Such a process is advantageous for many applications.
[0200] Among these applications, we can cite the designation of targets via a satellite or a designation pod for a satellite application, the recognition of new cars or road signs for the field of autonomous driving, the identification and recognition of new risky behaviors in security and surveillance systems as well as the identification of new diseases for the field of medical imaging.
[0201] Other alternative or complementary embodiments may be considered for the data classification process just described.
[0202] For example, a data augmentation technique can be used to enrich the unannotated dataset.
[0203] It is also interesting to regenerate the pseudo-labels to improve performance.
Claims
1. Demands Data classification method, the method being implemented by computer, the classification method comprising: - a learning phase (LP) of a neural network, to obtain a learned neural network, the learning phase (LP) comprising a step of: — initialization (E30) involving the reception of an initial neural network designed to classify a dataset into an initial number of classes, — receipt (E32) of an unannotated dataset and a number of new classes to be identified, — for each piece of data in the received dataset, application (E34) of the initial neural network on the data, to obtain a projection of each piece of data into a representation space, — partitioning (E36) of each data projected in the representation space into a plurality of distinct partitions, each partition grouping the projected data presenting a distance less than a threshold from the other projected data of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified, — assignment (E38) of an identical label to all projected data that are part of the same partition, — for each partition, calculation (E40) of a confidence index for the prediction of the label assigned to each projected data point of said partition, — deduction (E42) of the class of each data point from the classes of the initial neural network, the assigned label and a comparison of the confidence index to a reference confidence index, and — training (E44) of the neural network to classify the unannotated dataset according to the deduced classes, the neural network thus trained being the learned neural network, - an inference phase (P2), the inference phase (P2) comprising a step of: — receipt (E46) of data to be classified, and — application (E48) of the learned neural network on the received data to obtain a classification of the received data.
2. A data classification method according to claim 1, wherein the learning phase (PI) is iterated k times, k being a natural number greater than or equal to 2.
3. A data classification method according to claim 1, wherein the learning phase (LP) is iterated several times, the classification method further comprising: - a step of establishing a dictionary during the first implementation of a learning phase (LP), said dictionary listing the associations between the classes received and the labels established during the first implementation of a learning phase (LP), and - a step of updating the dictionary at each learning phase (LP), and wherein the method also comprises a step of measuring a performance level of the learned neural network as a function of the dictionary update.
4. A data classification method according to any one of claims 1 to 3, wherein the partitioning step (E36) is implemented by a K-means algorithm.
5. A data classification method according to any one of claims 1 to 4, wherein the confidence indices calculated during the calculation step (E40) are calculated from a distance matrix comprising the distances between a center of a partition and the projected data forming part of said partition.
6. A data classification method according to claim 5, wherein the confidence indices calculated during the calculation step (E40) are calculated by applying a softmax function to a vector comprising the inverse of the distances provided by the distance matrix.
7. A method for recognizing objects (14) present in an environment (12), the recognition method being implemented by an observation system (10) comprising an environment observation sensor (16) and a computer (18), the recognition method comprising a step of: - acquisition by the sensor (16) of data relating to the environment (12), - implementation by the computer (18) of the inference phase (P2) of a method for classifying the data relating to the environment (12), the classification method being according to one any of claims 1 to 6, to obtain a classification of the data relating to the environment (12) whose classes correspond to the presence or absence of the objects (14) to be recognized, and - recognition by the computer (18) of the presence of the objects (14) in the environment (12) according to the classification obtained.
8. A computer (18) suitable for classifying data, the computer (18) being suitable for: - receiving data to be classified, - applying a learned neural network to the received data to obtain a classification of the received data, the neural network having been learned by implementing a learning process comprising: — an initialization including the reception of an initial neural network suitable for classifying a dataset into an initial number of classes, — the reception of an unannotated dataset and a number of new classes to be identified, — for each data point in the received dataset, an application of the initial neural network to the data point, to obtain a projection of each data point into a representation space, — a partitioning of each data point projected into the representation space into a plurality of distinct partitions,Each partition groups together the projected data that are less than a threshold from the other projected data in the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified; — the assignment of an identical label to all projected data belonging to the same partition; — for each partition, the calculation of a confidence index for the prediction of the label assigned to each projected data item in said partition; — the deduction of the class of each data item from the classes of the initial neural network, the assigned label, and a comparison of the confidence index to a reference confidence index; and — the training of the neural network to classify the unannotated dataset according to the deduced classes, the neural network thus trained being the learned neural network.
9. A computer (18) capable of recognizing objects (14) present in an environment (12), the computer (18) being capable of: - receiving data relating to the environment (12), the data relating to the environment (12) having been acquired by a sensor (16) - applying a learned neural network to the received data to obtain a classification of the data relating to the environment (12) whose classes correspond to the presence or absence of the objects (14) to be recognized, - recognizing the presence of the objects (14) in the environment (12) according to the classification obtained, the neural network having been learned by implementing a learning process comprising: — an initialization including the reception of an initial neural network capable of classifying a dataset into an initial number of classes, - the reception of an unannotated dataset and a number of new classes to be identified, — for each piece of data in the received dataset,an application of the initial neural network to the data, to obtain a projection of each data point into a representation space, — a partitioning of each projected data point into the representation space into a plurality of distinct partitions, each partition grouping the projected data points having a distance less than a threshold from the other projected data points of the same partition, the number of partitions being equal to the sum of the initial number of classes and the number of classes to be identified, — an assignment of an identical label to all the projected data points belonging to the same partition, — for each partition, a calculation of a confidence index for the prediction of the label assigned to each projected data point of said partition, — a deduction of the class of each data point from the classes of the initial neural network, the assigned label and a comparison of the confidence index to a reference confidence index,and — training the neural network to classify the unannotated dataset according to the inferred classes, the neural network thus trained being the learned neural network. 20
10. An observation system (10) of an environment (12) comprising: - an environment (12) observation sensor (16), the sensor (16) being adapted to acquire data relating to the environment (12), and - a computer (18) according to claim 8 or 9.