Method for actively training a classification model, model, associated devices and vehicle.

Abstract

Method for training a classification model (201), the method comprising a repetition of the following first steps: Determination of a statistical distribution of labels (203) of labeled data from training data (202); Determination, by the classification model (201), for each data point in the training data (202), of a probable class for said data point and an indicator of the uncertainty of the probable class, the probable class having a frequency and a count in the statistical distribution; Selection of data to be labeled from the training data (202), the method being characterized in that the selection of the data to be labeled is carried out using the indicator of uncertainty for each data point in the training data (202) and the frequency or count, in the statistical distribution, of the probable class for each data point in the training data (202). Figure for the abstract: Figure 1

Description

Title of the invention: Method for actively training a classification model, associated model, devices and vehicle.

[0001] The invention relates to artificial intelligence, and in particular to classification models.

[0002] We know active learning algorithms, where instead of labeled data (in other words; labeled or annotated) being imposed or directly provided, it is the learning algorithm which requests the labeling of the data which seems to it to be the most relevant.

[0003] For example, the data to be labeled is the data that has the greatest uncertainty.

[0004] However, using uncertainty as a criterion for choosing data to label may lead to a class being under-represented in the labeled data and, consequently, the quality of the predictions made by the classification model, for this class, being poor.

[0005] To overcome this drawback, the invention relates to a method for training a classification model (machine learning or automatic learning), the method being characterized in that it comprises a repetition of the following first steps: • Determination of a statistical distribution (in other words; a table of numbers and frequencies) of labels (in other words: annotations or labels. The labels are classes) of data (previously) labeled (in other words: annotated or labeled) among training data (the statistical distribution comprising, for each possible class (at the output of the classification model), a frequency and / or a number of said each class in the labeled data), and • Determination, by the classification model (when implemented or executed), for each data item (in particular, each unlabeled data item) of the training data, of a probable class of said each data item and of an indicator of an uncertainty of the probable class (from the training data), the probable class having a frequency and a size in the statistical distribution, • Then, selection of data to be labeled among the training data (unlabeled), from the uncertainty indicator of each data item in the training data and the frequency or the number, in the statistical distribution, of the probable class of each data item in the training data.

[0006] By taking into account, for the selection of the data to be labeled, the frequency or the number, in the statistical distribution, of the probable class of each data item of the training data, the invention makes it possible to avoid a class being under-represented in the labels of the labeled data, and the predictions made by the classification model being, consequently, poor for this class.

[0007] For example, in a manner known in the field of active learning, during the step of determining the probable class of said each data item and an indicator of an uncertainty of the probable class, a probability is associated with each possible class, for said each data item. The probable class is the one having a higher probability.

[0008] For example: • Training data with the highest entropy are considered to have the greatest uncertainty, or • The training data with the smallest differences between the two highest probability classes are considered to have the greatest uncertainty, or • Training data with the lowest and highest probabilities are considered to have the highest uncertainty.

[0009] In particular: • If the uncertainty indicator of each data item in the training data is the same (i.e.: identical), the data selection step to label selects the data items in the training data with a lower frequency or the lowest sample size, and / or • If the frequency or size of each data item in the training data is the same (i.e., identical), the labeling data selection step selects the data item in the training data with the highest uncertainty indicator,

[0010] Conventionally in the field of active learning, the classification model can then be trained from the data to be labeled, once labeled, and from the labeled data.

[0011] Thus, (for each repetition of the first steps) the first steps can be followed by the following second steps: • Sends a labeling request for the data to be labeled, (then) • Receiving labels in response to sending the labeling request (and updating the labeled training data from the labels), for example, from a human-machine interface (entered by a human operator who determines the labels), (then) • At least one supervised training step (e.g., multiple steps training, because, for example, supervised training can be repeated until the model reaches a predetermined threshold of convergence) of the classification model from the labeled training data.

[0012] Supervised learning is well known to those skilled in the art. For example, when the classification model is an artificial neural network (in other words: an electronic neural network), supervised learning consists of modifying the synaptic weights between the neurons of the artificial neural network as a function of the labeled data at the input of the network, by gradient descent.

[0013] Also, prior to the first implementation of the first steps, the training data can be received, and then the following preliminary steps can be implemented: • Determination, by the classification model, for each data item in the training data, of a probable class of said each data item and an indicator of an uncertainty of the probable class (from the training data), then • Selection of data to be labeled among the training data (unlabeled), from the uncertainty indicator of each data of the training data,

[0014] For example, the method may comprise repeating a sequence comprising the preliminary steps, then the first steps, then the second steps, with different training data at each repetition.

[0015] The number of repetitions may depend on the volume of training data available.

[0016] However, the method may comprise only two implementations of the first steps, for example to quickly train a classification model from a relevant example.

[0017] According to one embodiment, the step of selecting the data to be labeled is implemented from a relevance indicator of (or for) each data item of the training data (unlabeled), following (and the method may comprise a step of calculating the relevance indicator): Ip = li xf(Eff), where • Ip is the said indicator of relevance of each data, • li is the uncertainty indicator of each data, • f is a (not strictly) decreasing function, • Eff is the said effective (of the probable class of each data in the statistical distribution).

[0018] Alternatively, other ways of selecting data to be labeled can be implemented based on the uncertainty indicator of each data item in the training data and the frequency or size, in the statistical distribution, of the probable class of each data item in the training data. For example, Ip = li + f(Eff) or Ip = li xf(Eff)2. One could also choose to label among the training data whose probable classes are the least frequent those with the highest level of uncertainty.

[0019] For example, the decreasing function f is written: f(Eff) = Max (1 - Eff / Eru, Eps), where • Max being a maximum function (producing the maximum between its two inputs), • Eff is a count of the probable class of each data in the statistical distribution, • Eru is a probable class size in the statistical distribution if the statistical distribution had been uniform, • Eps, is a positive constant (strictly) less than 0.001, for example equal to 10-6.

[0020] Since Eff / Eru is a multiple of the frequency freq ( freq x nbr = (Eff / Eru) ), nbr being the number of classes in the distribution, Max (1 - Eff / Eru, Eps) is also a function of the frequency freq.

[0021] For example, the selection step selects the data from the training data having the greatest relevance indicator, for example, the n data from the training data having the greatest relevance indicator, n being a strictly positive integer.

[0022] The invention also relates to a computer program comprising instructions executable by a microprocessor or a microcontroller or a computer, for implementing the steps of the method according to the invention, when it is executed by the microprocessor or the microcontroller or the computer.

[0023] The method according to the invention can be implemented by a computer device. The invention therefore also relates to a computer device configured to implement the steps of the method according to the invention.

[0024] The invention also relates to a classification model (machine learning or automatic learning) trained from steps comprising the steps of the method according to the invention (in other words: trained by the implementation of the method according to the invention).

[0025] For example, the classification model may be an artificial neural network (in other words: an electronic neural network).

[0026] This may be a convolutional or transformer-based electronic neural network for vision. Alternatively, it may be another type of classification model.

[0027] For example, the classification model is a data structure capable of being implemented and / or stored by an electronic device, for example the device electronics comprising a microprocessor capable of implementing the artificial neural network.

[0028] Once trained, the classification model can be installed in an electronic device to be implemented by the electronic device. The invention therefore also relates to an electronic device comprising the classification model and capable of implementing (or executing) the classification model, as well as a motor vehicle comprising the electronic device.

[0029] For example, the motor vehicle comprises a camera, the classification model being configured to classify each pixel of images received from the camera (as input to the classification model) when implemented by the electronic device.

[0030] For example, the classification model thus implements a semantic segmentation of the images received from the camera. The classes can identify, for example: cars, heavy goods vehicles, the roadway, a pedestrian, a cyclist, vegetation and / or buildings.

[0031] The characteristics and advantages of the computer program, the computer device, the classification model, the electronic device, and the motor vehicle are identical to those of the method according to the invention (without it being necessary to repeat them here).

[0032] When the computing device, classification model, electronic device, or motor vehicle (or other element) is "configured to" (or "capable of") performing or implementing a step or operation, this implies, for example, that the element comprises means for performing the step or operation. The means preferably comprise electronic means, for example a computer program, data in memory, specialized electronic circuits, wired or wireless connections, a microprocessor and / or a microcontroller.

[0033] Other characteristics and advantages of the present invention will appear more clearly on reading the detailed description which follows, comprising embodiments of the invention given as non-limiting examples and illustrated by the appended drawings, in which: • [Fig.l] represents a computer device, and a classification model, according to an embodiment of the invention, • [Fig.2] represents an electronic device, and a motor vehicle according to an embodiment of the invention. • [Fig.3] represents an implementation of the method according to the invention, according to a example of realization, by the computer device of [Fig.l].

[0034] In [Fig.2], certain elements are, of course, seen through transparency.

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Detailed description of an exemplary embodiment of the invention, with reference to figures 1 to 3. [Fig.l] represents a device 200. The device 200 comprises: • In memory, a neural network 201 (in other words: an artificial neural network), • 202 training data, • Labeled data 203 among training data 202. • A 204 man-machine interface. In step S00, [Fig.3], the device 200 determines a statistical distribution of the labels of the labeled data 203. The "Class", "Sample" and "Frequency" columns in Table 1 below give an example of such a statistical distribution. [Tables 1] Class Effective Frequency f(Eff) Class 1 80 0.4 0.000001 Class 2 40 0.2 0.000001 Class 3 35 0.175 0.125000 Class 4 15 0.075 0.625000 Class 5 10 0.05 0.750000 Class 6 7 0.035 0.825000 Class 7 6 0.03 0.850000 Class 8 5 0.025 0.875000 Class 9 2 0.01 0.950000 Class 10 0 0 1.000000 In step S10, the classification model 201, implemented by the device 200, determines, in a conventional manner in the field of active learning algorithms, for each of the training data 202, a probable class of said each of the data and an indicator of an uncertainty of the probable class, the probable class having a frequency and a size in the statistical distribution represented in table 1, In step S20, the device 200 calculates, for each of the training data 202, the value Ip = li xf(Eff), where • Ip is the said relevance indicator, • li is the uncertainty indicator, • f is a (not strictly) decreasing function, • Eff is the said effective.

[0041] The value f(Eff) is constant for each probable class and is written Max (1 - Eff / 20, 106), where Max is a maximum function.

[0042] Table 1 also shows the value of f(Eff) for each class.

[0043] In step S30, the device 200 then selects from the training data 202, as data to be labeled, the 10 data having the maximum Ip value.

[0044] In step S40, the device 200 displays, via the human-machine interface 204, a request for labeling the data to be labeled and receives, in response, labels from the human-machine interface 204, entered by a human operator who determines the labels.

[0045] In step S50, the device 200 updates the labeled data 203 from the labels received in step S40.

[0046] In step S60, the device 200 trains the neural network 201, in a manner known in active learning, in a supervised manner, from the training data 202, and the labeled data 203.

[0047] Step S50 may be repeated until the neural network converges.

[0048] Steps S00 to S60 may also be repeated for different training data 202.

[0049] The device 200 may be a computer device comprising for example a microprocessor and a computer program for implementing steps S10 to S50.

[0050] According to a first example, the neural network 201 is a convolutional artificial neural network, for example as described in the article below: • Chen et al., “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation”, in European Conference on Computer Vision, 2018.

[0051] According to a second example, this is a transformer-based neural network for vision, for example as described in the article below: • Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”, in International Conference on Learning Representations, 2021.

[0052] In step S70, the neural network 201 can be loaded and stored in a microprocessor 110 of a motor vehicle 100, as illustrated [Fig.2].

[0053] The motor vehicle 100 comprises, for example, a camera 120 connected to the microprocessor 110. The neural network 201 may, in this case, be capable of classifying each pixel of the images received from the camera 120, as input to the neural network 201, to indicate whether it is a pixel of a car, a heavy goods vehicle, the roadway, a pedestrian, a cyclist, vegetation and / or a building.

[0054] For example, the microprocessor 110 can control the motor vehicle 100 (e.g., its speed, direction, acceleration, a display inside the motor vehicle) from the classification thus produced by the neural network 201.

[0055] As a variant, of course, the neural network can of course have other applications.

Claims

Claims

1. A motor vehicle (100) comprising a camera, an electronic device comprising a classification model (201) configured to classify each pixel of images received from the camera, when implemented by the electronic device (110), the classification model being trained from steps comprising the repetition of the following first steps: • Determination of a statistical distribution (S00) of labels (203) of labeled data among training data (202), • Determination (S 10), by the classification model (201), for each data item of the training data (202), of a probable class of said each data item and an indicator of an uncertainty of the probable class, the probable class having a frequency and a size in the statistical distribution, • Selection of data to be labeled (S30) among the training data (202),the method being characterized in that the selection of the data to be labeled is implemented from the uncertainty indicator of each data item of the training data (202) and the frequency or the number, in the statistical distribution, of the probable class of each data item of the training data (202).,

2. Motor vehicle (100) according to the preceding claim in which the step of selecting the data to be labeled (S30) is implemented from a following relevance indicator, of each data item of the training data: Ip = li xf(Eff), where • Ip is said relevance indicator, • li is the uncertainty indicator, • f is a decreasing function, • Eff is said effective.

3. Motor vehicle (100) according to the preceding claim in which the decreasing function is written: f(Eff) = Max (1 - Eff / Eru, Eps), where Max being a maximum function, Eru is a probable class size in the statistical distribution if the statistical distribution had been uniform, Eps, is a positive constant (strictly) less than 0.001.

Images