Training method, classification method and system implemented by computer and computer readable recording medium

Abstract

The present invention relates to a training method, a classification method and a system implemented by a computer and a computer-readable recording medium. A computer-implemented method for training a classifier (Φη) comprises: S10) training a pre-model (ΦΘ) to learn a pre-task so as to minimize the distance between the output of a source sample via the pre-model (ΦΘ) and the output of a corresponding transformed sample via the pre-model (ΦΘ), wherein the transformed sample is a sample obtained by applying a transformation (T) to the source sample; S20) determining a neighborhood (NXi) of a sample (Xi) of a data set (SD) in an embedding space; S30) using a second training criterion to train a classifier (Φη) to predict the corresponding estimated probability Φη of the sample (Xi) belonging to each cluster (Cj) j (Xi), j=1...C, the second training criterion tends to: - maximize the possibility that the sample and its neighbors (Xj) of the neighborhood (NXi) belong to the same cluster; and - force the sample to be distributed over multiple clusters.

Description

Technical Field

[0001] The present disclosure relates to training methods to learn parametric models for classifying images, or more generally, samples, without the use of ground-truth annotations. Background Art

[0002] Artificial neural networks are able to extract information from large datasets. Increasing the amount of training data generally improves their performance and robustness.

[0003] As a result of this development, the training datasets required to train neural networks have grown exponentially over the past few years. As a result, there is an increasing need to train neural networks completely or at least partially in an unsupervised manner to reduce the need for ground truth annotations. This requires a particular focus on classification neural networks that are used to classify samples (input data) between a certain number of categories.

[0004] Two approaches have been proposed to perform unsupervised learning.

[0005] Representation learning methods use self-supervised learning to generate feature representations only from images using a pre-designed task (called a pre-task). Then, based on another task, a second step for fine-tuning the network parameters is performed. Unfortunately, this second step requires an annotated dataset (i.e., ground truth data). As an alternative, if such an annotated dataset is not available to perform the second step, offline clustering techniques like K-means can be applied to the learned feature representations. However, this may lead to clustering degradation and there is no guarantee that the obtained clusters are semantically meaningful.

[0006] The second category of methods includes end-to-end learning paths that combine feature learning with clustering. However, many of these methods are sensitive to initialization conditions and / or tend to converge to degenerate solutions.

[0007] References

[0008] [1]Gidaris,S.,Singh,P.,Komodakis,N.:Unsupervised representationlearning by predicting image rotations.In:ICLR(2018)

[0009] [2]Wu, Z., Xiong, Y., Yu, SX, Lin, D.: Unsupervised feature learning via non-parametric instance discrimination, in CVPR 2018.

[0010] [3]Chang,J.,Wang,L.,Meng,G.,Xiang,S.,Pan,C.:Deep adaptive imageclustering,in ICCV(2017).

[0011] [4]Gidaris,S.,Singh,P.,Komodakis,N.:Unsupervised representationlearning by predicting image rotations.In:ICLR(2018).

[0012] [5]Ji,X.,Henriques,J.F.,Vedaldi,A.:Invariant information clusteringfor unsupervised image classification and segmentation.In:ICCV(2019).

[0013] [6]Misra,I.,van der Maaten,L.:Self-supervised learning of pretext-invariant representations.arXiv preprint arXiv:1912.01991(2019).

[0014] [7]Michael Gutmann,Aapo Noise-contrastive estimation:Anew estimation principle for unnormalized statistical models.Proceedings oftheThirteenth International Conference on Artificial Intelligence andStatistics,PMLR 9:297-304,2010.

[0015] [8] DeVries Terrance, and Graham W. Taylor.: "Improved regularization of convolutional neural networks with cutout." arXiv preprint arXiv:1708.04552(2017).

[0016] [9] Ekin D.Cubuk, Barret Zoph, Jonathon Shlens, Quoc V.Le.: "RandAugment: Practical automated data augmentation with a reduced search space", arXiv:1909.13719. Summary of the invention

[0017] In view of the shortcomings of existing methods, a first object of the present disclosure is to propose a training method to learn a parametric model for classifying samples in a completely or at least largely unsupervised mode, which does not depend on initialization conditions and exhibits high performance in classification tasks.

[0018] Therefore, a computer-implemented training method for training a classifier is proposed.

[0019] Here, the transformed samples are samples obtained by transforming the source samples (marked as T), wherein the source samples are data of the source data set.

[0020] The training method includes:

[0021] S10) using a first training criterion to train a pre-model (ΦΘ) based on a source data set (SD) to learn a pre-task, wherein the first training criterion tends to minimize the distance between the output of the source sample via the pre-model (ΦΘ) and the output of the corresponding transformed sample via the pre-model (ΦΘ) for each source sample of the source data set;

[0022] S20) for at least one sample (Xi) in the samples of the source data set (SD), determining a neighborhood (NXi) of the at least one sample (Xi);

[0023] Wherein, for the at least one sample, the neighborhood (NXi) of the at least one sample (Xi) includes the K nearest neighbor values ​​of the sample (Xi), K is an integer, K≥1, and the K nearest neighbor values ​​of the sample (Xi) are the K samples Xj whose minimum distance in the data set is between ΦΘ(Xi) and ΦΘ(Xj);

[0024] S30) Use the second training criterion to train the classifier (Φη) to predict the corresponding estimated probability Φη of the sample (Xi) belonging to each cluster (Cj) j (Xi), j=1...C, the second training criterion:

[0025] - tends to maximize the probability that the sample and the neighbors (Xj) of the sample (Xi) belonging to the neighborhood (NXi) of the sample belong to the same cluster; and

[0026] - Tends to force samples to be distributed across multiple clusters.

[0027] In the above description, the transformed sample (also called augmentation) of the source sample Xi is the sample T(Xi) obtained by applying a transformation T to the source sample; the transformation modifies the source sample without destroying (or not overly destroying) its semantic content. For example, when the source sample is an image, the augmentation method commonly used by data scientists to increase the size of their datasets can be obtained by rotating, shaking, cropping, etc. the source sample.

[0028] Since the first training criterion includes terms based on the output of the source samples via the pre-model (ΦΘ) and the output of the corresponding transformed samples via the pre-model (ΦΘ), it should be understood that the data set used in step S10 includes not only the source samples of the source data set, but also the transformed samples obtained by transforming one of these source samples.

[0029] A classifier is a parametric model or function implemented by a program executed by a computer. The classifier can be implemented in particular by an artificial neural network. The term "computer" here includes any form of data processing system, including one or more processing units located in one or more locations, which are configured to process data together to provide an output. For example, it can be a personal computer, a smart phone, a server cluster located in different locations, a processing unit of any device with a certain data processing capability (such as a "smart oven"), etc.

[0030] A sample is any information data. It can be an image, for example, but it can also be any very broad type of information data, such as an audio or video recording, a text paragraph, a 3D point cloud, etc. It can also be a combination of multiple data.

[0031] The sample may be an "initial image" output by a camera. It may also be any image based on such an initial image, and may be obtained by preprocessing the initial image. The sample may be, for example, a sub-image of the initial image.

[0032] In this specification, a sub-image of an initial image is an image obtained by cropping the initial image, and thus includes fewer rows and / or fewer columns than the initial image.

[0033] The number of probabilities determined by the classifier (C) is the number of coordinates of the probability vector output by the classifier. This number C can be viewed as the number of clusters Cj, where j ranges from 1 to C. Therefore, the coordinate Φη j (Xi)(j=1...C) constitutes the prediction generated by the classifier, which contains the corresponding estimated probability that the sample Xi belongs to each cluster Cj. Therefore, the function of the classifier is to classify the samples in each cluster, that is, to perform the clustering function.

[0034] In the above definition, the statement that the second training criterion "tends to maximize the possibility that the sample and the neighboring values ​​(Xj) of the sample (Xi) belonging to the neighborhood (NXi) of the sample belong to the same cluster" means that the second training criterion is configured so that when the classifier Φη is trained in step S30, the weight η of the classifier is set to maximize the possibility that the sample and the neighboring values ​​(Xj) of its neighborhood (NXi) belong to the same cluster.

[0035] For a sample, the highest prediction of the sample is the highest coordinate of the prediction vector Φη(Xi), called Φη max (Xi). Its arrival index is marked as i max The samples with the same highest prediction index form a cluster.

[0036] Therefore, maximizing the probability that a sample Xi and a neighbor value Xj of its neighborhood NXi belong to the same cluster means maximizing the highest probability Φη of Xi max The index of (Xi) and the highest probability Φη of Xj max The probability that the indices of (Xj) are the same is maximized.

[0037] The proposed method mainly includes two training steps. In the first step S10, the pre-trained model is trained to learn semantically meaningful features in a fully self-supervised manner. These features are actually learned by training the pre-trained model to perform the pre-trained task. The pre-trained task usually does not require annotated data to train the neural network and has been widely explored in the literature. Examples of pre-trained tasks can be found in references [2] or [6], where the pre-trained model is trained to perform instance discrimination.

[0038] In order to ensure that the pre-model does not attach too much importance to small details of the sample, in step S10, the first training criterion tends to minimize the distance between the output of the source sample via the pre-model and the output of the corresponding transformed sample via the pre-model for each source sample of the source data set.

[0039] To this end, a variety of transformations T may be used within the scope of the present disclosure. The transformation is preferably selected so as not to change the semantic content of the sample. For example, if the sample is an image, the transformation may be a rotation, an affine or perspective transformation, coloring, image restoration, etc.

[0040] Once the pre-model has been trained in step S10 , the neighborhoods of all samples of the data set may be calculated in step S20 .

[0041] Then, in a second training step S30, a classifier Φη is trained based on these neighborhoods.

[0042] Advantageously, the entire method is based on learning. In particular, after the training step S10, it can be implemented without using the K-means method.

[0043] In some embodiments where the samples are images, during the training step S10, the training criteria are set so that the images are weighted at the region level. By doing so, more importance can be assigned to certain regions of the image.

[0044] This can be done, for example, if it has been observed that the most relevant regions of an image are generally located in a particular portion of the image. For example, in an autonomous driving application, in order to detect unusual objects left on the road, the most interesting region of an image acquired by a front camera of a vehicle is the lower half of the image representing the vicinity of the vehicle, rather than the upper half of the image which generally represents the more distant surroundings of the vehicle. Therefore, the first training criterion can be biased to give more importance to the lower half of the image than to the upper half of the image.

[0045] In some embodiments, for the sample under consideration, the first training criterion includes a term that increases when the difference between the prediction ΦΘ(Xi) for the sample under consideration (Xi) and the prediction ΦΘ(T(Xi)) for the corresponding transformed sample (T(Xi)) increases.

[0046] When training the pre-model (ΦΘ) in step S10, for at least one sample (Xi) of the training data set, the first training criterion tends to minimize the distance between the sample (Xi) and the transformed sample (T(Xi)), which is marked as:

[0047] d(ΦΘ(Xi),ΦΘ(T(Xi)))

[0048] Where T is the transformation.

[0049] Therefore, the training forces the pre-model to be invariant with respect to the transformation. This property is important for ensuring that the pre-model can effectively determine / identify semantically close neighborhoods in step S20. Otherwise, there is a risk that the pre-model is fixed to low-level features or even the pre-task itself, and converges to a solution that results in determining a neighborhood that does not correctly group semantically close samples at step S20.

[0050] In some embodiments, the second training criterion comprises the sum:

[0051]

[0052] in,

[0053] f is an increasing continuous function, such as a logarithm;

[0054] <,> is the dot product;

[0055] D is the data set used to train the classifier in step S30; and

[0056] |D| is the number of samples in the dataset.

[0057] By considering this training criterion, the training algorithm forces the classifier to output substantially the same prediction for a sample Φη(Xi) and its neighborhood Φη(Xj), where Xj is a member of the neighborhood of Xi. Therefore, this training criterion tends to learn the classifier to classify samples from the neighborhood in the same cluster.

[0058] Additionally, in order to prevent the training performed in step S30 from converging to a degenerate solution, in some embodiments, the second training criterion includes the sum:

[0059]

[0060] in,

[0061] c is the index of the coordinate of the predicted Φη(Xi) output by the classifier, and C is a set containing all c indices;

[0062] g is a function that measures the distance between the distribution of the predictions on each cluster and the expected distribution of the predictions;

[0063] For example, the function g can be defined as the Kullback-Leibler divergence or entropy.

[0064] D is the data set used to train the classifier in step S30; and

[0065] |D| is the number of samples in the dataset.

[0066] In the above expression, when the function g is defined as entropy, it can be defined by the following formula:

[0067] g(Φη ′c )=Φη′ c logΦη′ c

[0068] The summation proposed above for the second training criterion tends to force the classifier to distribute its predictions over all clusters (over all indices of its output) rather than over a single cluster.

[0069] As a result of the training classifier performed in step S30, a set of weights η is obtained for the classifier Φη, which enables the classifier to correctly perform its classification function.

[0070] Furthermore, it has been found that the performance of the classifier can be further improved by performing additional fine-tuning operations.

[0071] After the classifier has been trained in step S30, so-called "high confidence samples" (marked with X) can be identified. HC i). These high confidence samples are samples where the prediction (for a coordinate) output by the classifier has a very high value (usually close to 1) above a predetermined threshold. These high confidence samples can be used to further train the classifier to improve its classification performance.

[0072] In order to utilize this property, the additional training step S40 may be iteratively performed one or more times.

[0073] In this case, in some embodiments, the training method further includes performing the following step S40 at least once to fine-tune the classifier:

[0074] S40) training a classifier (Φη) based on a third training criterion (Λ), the third training criterion being configured to max (Xi)) is higher than the predetermined threshold (Thr) of high confidence samples (X hc i) for each considered sample (Xi), so that the considered sample belongs to the prediction (Φη) for the considered sample (Xi) max (Xi)) the maximum coordinate (Φη max The probability (Φη) of the cluster (Cj) represented by (Xi) j (Xi)) is maximized.

[0075] When the training step S40 has been performed, the prediction (or confidence level) Φη(Xi) may be recalculated for each sample Xi.

[0076] Advantageously, after the classifier Φη has been trained by performing one or more steps S40, the confidence level of its predictions increases overall. That is, the classifier learns to classify samples better and better - or at least with higher and higher certainty.

[0077] After each training step S40, the set of high confidence samples is updated based on the new value of the weight η. Therefore, in the next iteration, step S40 is performed with a different loss function and usually for an increased number of samples (because the number of samples that meet high confidence samples tends to increase).

[0078] The fine-tuning step S40 described above can be called a self-labeling step, because in its loss function, the penalty for the prediction considered is calculated based on the prediction of the classifier itself. The loss function is equivalent to considering labeling each high confidence sample (determined by the classifier itself) as belonging to the class with the highest probability (Φη) of the sample under consideration. max The cluster Cj of index j of (Xi)).

[0079] In some variations of the above embodiment, the training method further includes performing at least one fine-tuning step S40; when it is determined that the number of high-confidence samples is no longer increasing, the fine-tuning step (S40) is stopped. That is, when the number reaches a stable state (which means that despite the execution of the training step S40, the classifier Φη no longer improves its performance of classifying samples into clusters), the algorithm can be stopped.

[0080] As the fine-tuning steps S40 are repeated, clusters (indicated by high confidence samples) become increasingly important and more relevant. Thus, these self-labeling steps S40 allow the classifier Φη to gradually correct itself as it gradually becomes more certain, thereby adding more samples to the clusters. The fine-tuning steps even gradually correct errors that may have initially appeared in the data set, thereby reclassifying sample neighbors that may have initially been incorrectly grouped into the same cluster as the sample.

[0081] Amplification function:

[0082] As described above, in step S10, the sample or the amplification of the transformed sample is used for training. To this end, this amplification is added to the training data set used in step S10. This amplification can also be added to the training data set used to train the classifier in step S30 and / or step S40.

[0083] As described above, the transformed samples are obtained based on the source samples. Thus, for example, the dataset used in step S10 may (possibly only) consist of samples of an initial dataset called the "source dataset" and transformed samples of part or all of the source samples that are amplifications of these source samples.

[0084] Different methods can be used to augment the dataset to improve the efficiency of the training steps S10, S30 and / or S40. For example, when the sample is an image, in one or more of these training steps, the training dataset may include augmentations obtained by applying one of the following functions to the source image: horizontal flip, slight rotation, light change, black and white (converting a color image to a grayscale image), dithering, etc.

[0085] In some embodiments, step S30 is repeated multiple times.

[0086] For example, the data set may be changed during different iterations of step S30, in particular by adding augmentations to the data set.

[0087] Furthermore, in some embodiments, during steps S30 and / or S40, additional samples, referred to as "strong amplifications," may be added to the data set.

[0088] A sample called "strongly augmented" is a transformed sample obtained by applying a transformation to the sample that modifies the sample content more deeply than the transformation of regular augmentation. For example, when the sample is an image, strong augmentation can be obtained using one or more of the following transformations: auto-contrast, equalization, rotation, solarization, color, contrast, brightness, sharpness, shearing, translation, toning. More information about strong augmentation can be found in reference [9]. After performing the transformation or these transformations, a clipping function is usually applied to obtain the transformed sample. Reference [8] describes the clipping function.

[0089] For example, in some embodiments, the high confidence sample X HC i's strong amplification SA(Xi) is added to the data set used in step S40; and

[0090] The third training criterion is configured to make the strong augmentation SA(Xi) and the high confidence sample X HC The probability that i belongs to the same cluster is maximized.

[0091] In other words, in step S40, the third training criterion is configured such that during training, the classifier will learn to classify (as much as possible) the considered high confidence samples X HC i and its strong augmentation are classified into the same cluster (i.e., for the high confidence sample X considered HC i’s classifier predicts Φη(X HC i) will have the same rank as the highest coordinate for the considered high confidence sample X HC i's strongly augmented classifier predicts Φη(SA(X HC i)) have the same rank as the highest coordinates.

[0092] In this way, overfitting can be avoided.

[0093] Advantageously, the training method according to the present disclosure allows efficient training of classifiers in a completely unsupervised mode. It has been tested on datasets such as CIFAR10, STL10 and even Imagenet and has outperformed many current state-of-the-art methods in unsupervised learning.

[0094] The method may be supplemented with some semi-supervised training to take into account prior knowledge about the samples. This prior knowledge may be taken into account in different ways: in the neighborhood definition in step S20, and / or during the training phase of the classifier in steps S30 and / or S40.

[0095] Therefore, in some embodiments, the method includes, in one or more of steps S20, S30 and S40, considering the prior knowledge that multiple samples (referred to as similar samples) constitute a cluster (the acquisition of this prior knowledge is independent of steps S10, S20 and S30); thereby, in step S20, when defining the neighborhood, the similar samples are considered to be neighboring values, and / or in step S30 or S40, a classifier must be trained to classify similar samples into the same cluster.

[0096] In some embodiments, in step S20, for a set of at least two or more similar samples that have been determined to belong to the same cluster, a neighborhood is determined for each similar sample, the neighborhood at least including another (some) of the similar samples. In this case, for each similar sample, for example, a neighborhood of the sample can be specially prepared to include all other similar samples that are considered to belong to the same cluster.

[0097] The set of one or more similar samples may then be included in the dataset used to train the classifier in step S30 , and in step S30 , the neighborhood of the similar samples determined as described above may be used.

[0098] In addition, in some embodiments, in step S30, for at least one set of two or more similar samples in the data set that have been determined to belong to the same cluster, the second training criterion is configured to tend to maximize the likelihood that the similar samples in the considered set of similar samples belong to the same cluster.

[0099] Similarly, in some embodiments, in step S40, for at least one set of two or more similar samples in the data set that have been determined to belong to the same cluster, the third training criterion is configured to tend to maximize the likelihood that the similar samples in the considered set of similar samples belong to the same cluster.

[0100] Therefore, when the prior knowledge of clusters is considered in steps S20, S30 or S40 as described above, we have the prior knowledge that each similar sample of the cluster belongs to the same cluster, and in the case of the considered training steps S30 and / or S40, the training criterion is specifically modified or prepared to include a term that tends to maximize the likelihood that similar samples in any considered set of similar samples belong to the same cluster.

[0101] For example, for some samples that are not (or have not yet become) high-confidence samples, this can be accomplished by manually assigning these samples to appropriate clusters. Even in this case, the proposed method is economically attractive because it significantly reduces the number of samples that need to be annotated by automatically determining appropriate annotations for high-confidence samples.

[0102] Classification systems and methods

[0103] As part of this disclosure, a classification method is proposed.

[0104] The method comprises a step D) of classifying the sample using a classifier trained by one of the above training methods.

[0105] This classification approach can be used in many different contexts.

[0106] In some applications, the sample is data based on one or more images acquired by one or more cameras; the classifier is trained to classify such samples. The one or more cameras are, for example, one or more cameras of a vehicle. The data based on the one or more images can of course be an image, but can also be a composite image obtained by stitching together multiple images, or a processed image obtained by pre-processing one or more images (e.g., cropping an image), etc.

[0107] In some embodiments, when the sample (Xi) comprises one or more acquired images, the classification method further comprises step B:

[0108] B) generating at least one sample sub-image based on the sample; and

[0109] In step D, the sample is classified by classifying the at least one sample sub-image.

[0110] In these embodiments, at least a sub-image is generated to perform the classification step D. In some embodiments, the sample is based on data of one or more images acquired by one or more cameras, and the classification method comprises the following steps:

[0111] A. acquiring data based on output of one or more cameras or non-camera sensors; and

[0112] C. Detect whether the acquired data includes abnormal data; in addition, step D is performed only when it has been detected in step C that the acquired data includes abnormal data.

[0113] Here, non-camera sensors refer to sensors that are not cameras; for example, lidar or radar.

[0114] A variety of methods can be used to detect abnormal information when providing the acquired data. For example, in the case where the LiDAR is constantly photographing the road in front of the vehicle, the abnormal information can be information showing that an abnormal object has appeared on the road (for example, a 3D point that is significantly higher than the road normal), usually an object abandoned on the road or dropped on the road. In this case, the above classification method is very useful for determining the type of object that has appeared on the road and triggering appropriate actions, in particular, ignoring the object (if the object is an empty plastic bag blown onto the road by the wind) or taking any measures to avoid hitting the object (if the object is a heavy stone dropped on the road).

[0115] In some embodiments, where the acquired data includes one or more acquired images, the classification method further comprises step B:

[0116] B) generating at least one data sub-image based on the acquired data;

[0117] In step C, abnormal data is detected in the at least one data sub-image.

[0118] In these embodiments, a sub-image is generated to perform at least the detection step C.

[0119] In some embodiments of the above classification method, in step D, the sample is classified by classifying the at least one data sub-image.

[0120] In the above-mentioned embodiment of generating sub-images, preferably, multiple sub-images are generated (based on the initial image), and the samples are classified in step D, and / or abnormal data is detected in step C (as the case may be) based on the multiple sub-images.

[0121] In certain embodiments, the steps of any of the above methods are determined by computer program instructions.

[0122] Therefore, the present disclosure also includes a system (classifier training system or sample classification system), which includes one or more processors and a memory, wherein the memory stores instructions, and when one or more processors execute the instructions, the instructions cause the one or more processors to use one of the training methods defined above to train a classifier, or use one of the classification methods defined above to classify samples.

[0123] The present disclosure also provides a computer program, which is stored on a computer-readable storage medium and is suitable for execution on one or more processors. The program includes instructions, and when the program is run on one or more processors, the instructions are suitable for executing the steps of one of the methods defined above.

[0124] The computer program may use any programming language and may be in the form of source code, object code, or an intermediate code between source code and object code, such as a partially compiled form or any other desired form.

[0125] The present disclosure also provides a computer-readable recording medium, preferably a non-volatile medium, comprising instructions of the computer program as defined above.

[0126] The recording medium may be an entity or device capable of storing a program. For example, the medium may include a storage device such as a read-only memory (ROM), such as a compact disc (CD) ROM or a microelectronic circuit ROM, or may actually be a magnetic recording device, such as a floppy disk or a hard disk.

[0127] Alternatively, the recording medium may be an integrated circuit in which the program is incorporated, the circuit being suitable for executing one of the methods discussed or for use in the execution of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] The present invention may be better understood, and numerous other objects and advantages thereof will become apparent to those skilled in the art, by reference to the accompanying drawings, in which like reference numerals in different figures represent similar elements, wherein:

[0129] Figure 1 is a flow chart illustrating an exemplary training method according to an embodiment of the present disclosure;

[0130] Figure 2 A classification system according to an embodiment of the present disclosure is shown;

[0131] Figure 3 is a flowchart illustrating an exemplary classification method according to an embodiment of the present disclosure; and

[0132] Figure 4 is a flowchart illustrating an exemplary classification method according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0133] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings.

[0134] Training Methods

[0135] First, an exemplary training method for training a classifier Φη to output a prediction for a sample Xi will be presented, which constitutes the first embodiment of the present disclosure. Figure 1 , the steps of the method are schematically shown.

[0136] S0) Preparation

[0137] To implement the method, an initial dataset of samples (images in this embodiment) is required, called source dataset SD. This dataset preferably includes a variety of samples (images) that are large enough to fully train a classifier to recognize and distinguish different individually identifiable clusters of the dataset.

[0138] Choose a pre-task. For example, in the case of images, the pre-task usually consists of trying to learn visual features of the image.

[0139] One or more transformations T are selected. The transformations are those that should not change the predictions made by the previous model.

[0140] The chosen transformation should not change the semantics of the image. For example, it could flip the image or turn it to grayscale, etc.

[0141] The pre-model and classifier must be defined.

[0142] In this embodiment, the front model ΦΘ and the classifier Φη are both neural networks. More specifically, each of them includes a standard ResNet-18 backbone network as its core, and one or more layers are added downstream of the core neural network.

[0143] The weights of the pre-model ΦΘ are labeled Θ, while the weights of the classifier Φη are labeled η.

[0144] The pre-trained model ΦΘ has a standard ResNet-18 backbone as its core neural network. One or more layers are added downstream of this core neural network to adapt the output of the pre-trained model to the chosen pre-trained task.

[0145] For example, if the previous task is an instance discrimination task, the added layer can be a linear layer that outputs the required number of features.

[0146] The core of the classifier Φη can be the same neural network as the core neural network of the previous model (a ResNet-18 backbone in this example).

[0147] As for the front model, one or more layers are added downstream of the core neural network. These additional layers are configured to make the classifier output a number of C outputs, which represent the probability that the input samples belong to C clusters respectively.

[0148] In order to constrain the classifier Φη to output probabilities, the most downstream layer of the classifier is usually a normalization layer, which ensures that the sum of the outputs is always constant. For example, the most downstream layer can be a softmax layer.

[0149] The number of clusters C, i.e. the number of coordinates in the output of the classifier, must be defined during the preparation of the classifier. Typically, this number is initially defined based on a rough estimate of the number of clusters expected to be found in the dataset. If unsure, it is preferred to initially define the preprocessor and the classifier with a large number of clusters (called "overclustering") and then enforce a uniform distribution of clusters. In practice, if it is observed that some clusters never correspond to the highest coordinates (labeled Φ, respectively) in the predictions of the preprocessor or the classifier, then the number of clusters should be increased. max Θ or Φ max η), then the number of clusters can be reduced.

[0150] S10) Train the pre-trained model ΦΘ

[0151] Once the above elements are prepared, the first training step S10 is performed: training the pre-model ΦΘ to perform the pre-task.

[0152] The training is performed using a first training criterion suitable for learning the pre-task. For example, the training criterion may be defined based on references [2], [6], or [7].

[0153] In addition, a first training criterion is defined such that when a transformation is applied to the samples, it is further ensured that the predictions made by the previous model remain substantially unchanged.

[0154] With this purpose in mind, in this embodiment, during the training step S10, the weights Θ of the pre-model ΦΘ are modified so that for all samples of the source data set, an attempt is made to minimize the distance between the prediction for the sample and the prediction for the corresponding transformed sample. This distance is denoted as:

[0155] d(ΦΘ(Xi),ΦΘ(T(Xi)))

[0156] The first training criterion includes the above value in addition to other items used to train the pre-model to learn the pre-task. Due to this value, the first training criterion tends to minimize the distance between the output of the source sample Xi via the pre-model ΦΘ and the output of the corresponding transformed sample T(Xi) via the pre-model ΦΘ among the samples of the source data set SD.

[0157] S20) Determine the neighborhood (NXi)

[0158] Then, in step S20, the trained pre-model ΦΘ is used to determine the K nearest neighbor values ​​of each sample Xi of the data set.

[0159] Here K is an integer at least equal to 1. Setting the number K of neighbors in the set NXi to at least 1 (preferably more, such as at least 5 or at least 10) is important to better capture the differences between each cluster.

[0160] At least step S22 and optionally step S24 may be performed to determine a neighborhood NXi of the sample.

[0161] In step S22, the neighborhood NXi is determined based on the prediction of the pre-model ΦΘ.

[0162] To this end, in step S22, the K nearest neighbor values ​​of each sample Xi of the data set are identified in the embedding space ΦΘ.

[0163] In other words, for each sample Xi of the dataset D, determine the neighborhood NXi, or the set of neighboring values ​​NXi of the sample Xi, which includes the neighborhood NXi of the dataset D. ΦΘ (Xi,Xj)The smallest K samples Xj.

[0164] Here, Dist ΦΘ (Xi,Xj) is the distance function in the embedding space ΦΘ; that is:

[0165] Dist ΦΘ (Xi,Xj)=Dist(ΦΘ(Xi),ΦΘ(Xj))

[0166] Here, Dist is the usual distance function.

[0167] In the above formula, the function Dist can be any distance function; for example, the L2 norm:

[0168] Dist(ΦΘ(Xi),ΦΘ(Xj))=||ΦΘ(Xi)-ΦΘ(Xj)||.

[0169] The neighborhood NXi can be determined in different ways.

[0170] For example, the neighbors of sample Xi can be identified as samples Xj that maximize the "cosine similarity" between ΦΘ(Xi) and ΦΘ(Xj). In general, the cosine similarity between two vectors V and W is the dot product between these vectors, normalized to:

[0171] Cosine similarity(V,W)=VW / (||V||.||W||).

[0172] In addition to step S22, if there is prior knowledge about how the samples are clustered, an optional step S24 may be performed.

[0173] Step S24 can be performed in two different ways.

[0174] When it is known in advance that the new sample Yi constitutes one or more clusters, step S24 can be performed in a first manner by adding the sample Yi to the data set to be used in step S30. In other words, in each of these clusters, it is known in advance that all added samples Yi of the cluster are similar to each other.

[0175] In that case, the neighborhood NYi of each of these added similar samples Yi can be defined as including other samples Yj of the same cluster (other similar samples Yj).

[0176] Thereafter, step S30 may be performed based on the enlarged data set including samples Xi and Yi.

[0177] Step S24 can be performed in a second manner simply by considering the prior information that the sample sets of the data set to be used in step S30 constitute clusters. That is, it is known in advance that the different samples of the data set used for step S30 are similar samples and therefore must be classified as belonging to the same cluster. Multiple such sample sets can be identified as constituting clusters.

[0178] In this case, the neighborhood NXi of these samples Xi whose clusters are known in advance is defined as including other samples Xj of the same cluster. That is, the definition of the neighborhood NXi of one of these samples is thus clarified based on the prior knowledge of the clusters, and conversely, the neighborhood NXi determined based on the pre-model is not considered. Then, considering the neighborhood NXi optimized as defined in step S24, step S30 is performed based on the source data set.

[0179] S30) Training the classifier

[0180] The neighborhood set NXi identified in step S20 constitutes the precursor of the clusters that the classifier Φη will have to identify.

[0181] Once these neighborhoods NXi have been determined for all samples Xi of the dataset, the classifier Φη is trained.

[0182] In order to speed up the training, the weights of the core neural network of the previous model ΦΘ can be used as the initial values ​​of the core neural network of the classifier Φη.

[0183] The classifier Φη is trained to classify the samples of the dataset into a set of clusters C. The classifier Φη achieves this classification by performing a soft assignment of the input sample Xi among the clusters C = {1...C}, where Φη(Xi)∈[0,1] C Therefore, the purpose of training is to learn a clustering function parameterized by a neural network Φη with weights η to classify samples Xi and their neighbor sets NXi together.

[0184] The training is performed using a second training criterion, which is a loss function Λ that takes into account, for each sample Xi, the corresponding distances between the output (Φη(Xi)) of the sample (Xi) via the classifier (Φη) and the outputs of the neighbors of the sample (Xi) in the set of neighbors (NXi) of the sample, and tends to minimize these distances.

[0185] In the proposed embodiment, during training, the weights of Φη are obtained by minimizing the following second training criterion (or loss function) Λ:

[0186]

[0187] In the above formula, <,> represents the dot product operator.

[0188] The first term of this formula (the double summation) enforces Φη to make consistent predictions for the sample Xi and its neighbors in the set of neighbors (NXi). Note that the dot product will be maximum when two predictions are equal to 1 or very close to 1 (very certain) and are assigned to the same cluster (consistent).

[0189] In addition, in order to prevent the loss function from guiding Φη to assign all samples to one cluster, the formula also includes an entropy term (the second term in the above formula), which evenly distributes the predictions among clusters C, thereby tending to force samples Xi to be distributed throughout the entire cluster set C.

[0190] In this example, the weight of the entropy term is set to λ = 2. A higher weight avoids samples from being grouped too early during training.

[0191] If the probability distribution over the clusters C is known in advance (which is not the case here), the second term can be replaced by the KL divergence.

[0192] During step S30, the trained data set can be expanded by including the initial sample of the data set in the data set. In this case, the clustering loss function Λ is suitable for ensuring consistency between the sample and its expansion. This consistency is ensured by including the expansion of the sample in its neighbor value set NXi.

[0193] Here, the reference to “consistency” means that a sample and its amplification should be classified in the same cluster, which means that Dist ΦΘ (Xi,Xj) should be small.

[0194] S40) Fine-tune the classifier Φη

[0195] The clustering step S30 above provides an initial value for the weight η of the classifier Φη. In step S30, the second training objective Λ used for the initial training of Φη has strengthened the consistency between the sample Xi and the neighbor value set NXi. In step S40, this consistency is used as follows to define the classification or clustering of the data set (as described below, step S40 can be repeated):

[0196] Each fine-tuning step (S40) includes the following sub-steps:

[0197] S42) Identify high confidence samples (X hc i)

[0198] The following notation is used: For sample Xi,

[0199] The probability or confidence level that a sample Xi belongs to cluster Cj is expressed as Φη j (Xi), where j is an integer from 1 to C; and

[0200] The highest probability or maximum confidence level of sample Xi at its different coordinates j (j = 1...C) is expressed as Φη max (Xi). In other words:

[0201] Φη max (Xi)=Max j (Φη j (Xi)).

[0202] Experimentally, it is observed that with a high degree of certainty prediction (Φη max (Xi)≈1) or "high confidence sample X hc i" tends to be correctly classified as a cluster.

[0203] Based on this observation, based on the high confidence sample X hc i to determine the clusters Cj used to fine-tune the classifier Φη:

[0204] First, a confidence threshold Thr is selected.

[0205] Then, identify the highest probability (Φη max (Xi)) High confidence samples X that are higher than the confidence threshold Thr hc i.

[0206] Calculate high confidence samples X HC The number of i N HC .

[0207] On this basis, clusters Cj (j=1...C) are defined: each cluster Cj includes (only) the identified Φη max (Xi)=Φη j (Xi)>Thr high confidence sample X hci. (Some clusters can be empty).

[0208] Based on examples that are already well classified, this method can be called a self-labeling method. In each classification Cj, the examples with a particularly high confidence level Φη j The sample Xi of (Xi) is regarded as the prototype of this classification.

[0209] S44) The high confidence sample (X hc i) Associated with the corresponding cluster

[0210] Then, each high confidence sample X hc i and the maximum coordinate (Φη) of the prediction Φη(Xi) output by the sample through the classifier max (Xi)) is associated (or labeled) with the cluster indicated.

[0211] Preferably, a semantic label is assigned to each cluster. For example, the semantic label can be selected based on at least one high confidence sample of each cluster. Then, all other samples of the cluster are automatically associated with the semantic label.

[0212] Of course, it is also possible to assign semantic labels to clusters that do not contain high-confidence samples.

[0213] A further optional step S43 may be performed in parallel with steps S42 and S44 to group the samples into clusters.

[0214] In fact, for step S24, in some cases, it is known in advance that some samples are similar to each other and should therefore belong to the same cluster. In this case, for these samples, step S43 may be performed.

[0215] Step S43 includes associating all samples in a group of samples considered to be similar samples to form a cluster.

[0216] S46) Training classifier Φη

[0217] The classifier Φη is then trained to identify clusters Cj. This training is performed using the third training criterion.

[0218] The third training criterion is configured to hc i, so that the high confidence sample X hc i belongs to the high confidence sample X hc i The maximum coordinate (Φη) of the prediction (Φη(Xi)) output by the classifier max The probability (Φη) of the cluster (Cj) indicated by (Xi) j (Xi)) is maximized. That is, the third training criterion is configured to maximize the confidence level of each high confidence sample X hc i, so that the probability Φη j(Xi) is maximized, where j is the maximum coordinate Φη max The rank of (Xi).

[0219] For the high confidence sample X considered hc i, this can be achieved by including a term (which may be the sum of one or more sub-terms) in the third training criterion, which is the probability (Φη j (Xi)) is an increasing function, where Φη j (Xi) is the maximum coordinate Φη of the predicted Φη(Xi) max (Xi).

[0220] For example, the third training criterion may penalize the classifier's prediction when the probability that a high confidence sample belongs to the cluster associated therewith is less than 1.

[0221] In this embodiment, for each high confidence sample X hc i, the loss function includes a term where the probability Φη of the considered sample belonging to its associated cluster Cj j When (Xi) approaches 1, the term decreases.

[0222] The loss function can be cross entropy loss.

[0223] Specifically, in this embodiment, a weighted cross entropy loss is used to compensate for the imbalance between the determined samples between clusters. To this end, the weight assigned to the cluster is inversely proportional to the number of high confidence samples in the cluster.

[0224] In the case where step S43 is executed, all similar sample sets identified in step S43 are included in the training data set in step S46.

[0225] Furthermore, the third training criterion is thereby configured to maximize, during training, for all similar sample sets already included in the data set, the likelihood that similar samples in the considered set of similar samples belong to the same cluster.

[0226] S48) Check whether the fine-tuning step should continue

[0227] After retraining the classifier Φη in step S46, the prediction Φη(Xi) can be recalculated for each sample. The high confidence sample X is determined based on the updated prediction. HC i.

[0228] Also update the high confidence sample X HC The number of i N HC .

[0229] Determine the updated number N HC Is it higher than its previous value.

[0230] If the answer is yes (the number N HC is still increasing), then it is concluded that fine-tuning will continue to improve the prediction quality of the classifier Φη. In this case, at least one additional fine-tuning step S40 is performed.

[0231] On the contrary, if the answer is no (the number N HC has reached a steady state), then it is concluded that fine-tuning will no longer improve the prediction quality of the classifier Φη. Therefore, the fine-tuning step S40 is stopped.

[0232] Classification systems and methods

[0233] refer to Figures 2 to 4 , a classification method and system according to the present disclosure will now be presented.

[0234] exist Figure 2 The material architecture of the system 1000 is schematically shown in FIG. The system 1000 is both a classifier training system and a sample classification system.

[0235] The system 1000 mainly consists of a central computer 100 installed on the vehicle 1 and a group of sensors 110 .

[0236] Most of the hardware components of the system 1000 are shared components for performing multiple tasks. Therefore, the hardware components constituting the system 1000 perform the functions of the classification system 1000, but may also perform other functions.

[0237] Although the system 1000 (particularly the central computer 100) in this embodiment is physically located in the vehicle 1, it need not be in the vehicle. In fact, it can be located anywhere as long as a communication device is provided to transmit the necessary inputs to the central computer 100.

[0238] In addition, despite the Figure 1 The central computer 100 is represented by a single box in the figure, but it may include one or more processors, and thus may be a distributed computing system distributed on multiple processors and possibly physically arranged in different locations.

[0239] The set of sensors 110 comprises four external cameras 111 and a Lidar unit 112. Among the cameras 111, the front camera 111 is arranged to acquire an image of a scene in front of the vehicle 1.

[0240] The functions of the system 100 are implemented on a central computer 100 .

[0241] The central computer 100 includes a storage device 101 , one or more processors 102 , a memory 103 , an operating system 104 , a communication infrastructure 105 , and other application programs 106 .

[0242] The one or more processors 102 are intended to represent the presence of any one or more processors or processing devices in any form.

[0243] The communication infrastructure 105 is a data bus to which the above-mentioned sensors 110 are connected and through which the signals output by these sensor units are transmitted to the central computer 100 .

[0244] Storage device 101 , processor 102 , memory 103 , and operating system 104 are communicatively coupled via communication infrastructure 105 .

[0245] The classification computer program C1 is stored in the memory 103, and the classification computer program C1 includes instructions, and when the classification computer program C1 is run on one or more processors 102, the instructions are suitable for executing the steps of the classification method of the present disclosure. In addition, the training computer program Tr is also stored in the memory 103, and the training computer program Tr includes instructions, and when the training computer program Tr is run on one or more processors 102, the instructions are suitable for executing the steps of the training method of the present disclosure. Programs C1 and Tr and the memory 103 are examples of computer programs and computer-readable recording media according to the present disclosure, respectively. The memory 103 of the computer system 100 in fact constitutes a recording medium according to the present invention, which can be read by one or more processors 102, and the program is recorded on the recording medium.

[0246] The first function of the system 1000 when executing procedure Tr is to train the classifier Φη.

[0247] Another function of the system 1000 based on procedure C1 is to classify objects detected on the road ahead of the vehicle 1 during the journey.

[0248] To this end, the computer 100 is configured to execute program C1 to perform the following steps: Figure 3 ):

[0249] A. The computer 100 periodically acquires the point cloud output by the laser radar unit 112 and the image output by the camera 111 at a relatively high frequency.

[0250] C. The computer 100 detects whether the acquired data includes abnormal data. Such abnormal data may be points located above the road surface, indicating the presence of an object on the road.

[0251] The computer 100 is configured to trigger classification upon detecting the presence of such abnormal data:

[0252] D. The last image acquired by the front camera 111 is processed by the classifier Φη. Thus, the classifier Φη predicts the cluster corresponding to the object detected in front of the vehicle 1, i.e. the object type. This information is transmitted to the driver of the vehicle and / or the driving system of the vehicle.

[0253] In one embodiment ( Figure 4 ), the computer 100 is configured to perform an additional step B of generating a set of data sub-images based on the acquired image for each image acquired by the front camera 111. For example, in each time step, a matrix of 16 sub-images may be created by dividing the image acquired by the camera 111 into 16 regions (these regions preferably have some overlap).

[0254] Then, based on these data sub-images, step C is performed: in some or each data sub-image, detecting whether there is abnormal data.

[0255] If no anomalous data is detected, the algorithm stops.

[0256] Otherwise, the data sub-images in which abnormal data have been detected are classified by the classifier Φη. To this end, the classifier must have been trained on the sub-images (data sub-images) of the images (data) acquired by the front camera 111.

[0257] Based on the results of the predictions of the classifier Φη for all analyzed data sub-images, a list of different objects recognized in the scene in front of the car is determined.

[0258] Based on this information, the driver or driving system of the vehicle can take appropriate action in response to the object appearing on the road.

[0259] Step C may also be performed before step B.

[0260] In this case, step C for detecting whether abnormal data exists is performed based on the image acquired by the sensor (e.g., the front camera 111). Then, if abnormal data has been identified in the image, a sub-image of the acquired image is generated (step B). Then, a classification step D is performed. If the portion of the image where the abnormal data is located has been determined in step C, step D can be performed only based on the sub-image corresponding to the portion of the image; otherwise, step D can be performed on all sub-images.

[0261] Although the present disclosure has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure.

[0262] It is intended that the specification and examples be considered exemplary only, with the true scope of the disclosure being indicated by the following claims.

Claims

1. A computer-implemented training method for training a classifier Φη to classify objects detected in an image, wherein: Source samples are data of a source dataset SD, said data being based on one or more images acquired by one or more cameras and / or non-camera sensors; The transformed samples are samples obtained by performing a transformation T on the source samples; The training method is characterized by comprising: S10) using a first training criterion to train a pre-model ΦΘ based on the source data set SD to learn the pre-task, wherein the first training criterion minimizes the distance between the output of the source sample via the pre-model ΦΘ and the output of the corresponding transformed sample via the pre-model ΦΘ for each source sample of the source data set; S20) for at least one sample Xi in the samples of the source data set SD, determining a neighborhood NXi of the at least one sample Xi; Wherein, for the at least one sample Xi, the neighborhood NXi of the at least one sample Xi includes the K nearest neighbor values ​​of the sample Xi, K is an integer, K≥1, and the K nearest neighbor values ​​of the sample Xi are the K samples Xj whose minimum distance in the data set is between ΦΘ(Xi) and ΦΘ(Xj); S30) Use the second training criterion to train the classifier Φη to predict the corresponding estimated probability Φη of the sample Xi belonging to each cluster Cj j (Xi), j = 1...C, C is the number of clusters, the second training criterion: - maximizing the probability that the sample and the neighboring values ​​Xj of the sample Xi belonging to the neighborhood NXi of the sample belong to the same cluster; and - Force the samples to be distributed across multiple clusters.

2. The training method according to claim 1, wherein: For the sample under consideration, the first training criterion comprises a term that increases as the difference between the prediction ΦΘ(Xi) for the sample under consideration and the prediction ΦΘ(T(Xi)) for the corresponding transformed sample T(Xi) increases.

3. The training method according to claim 1 or 2, wherein: In step S20, for at least one set of two or more similar samples that have been determined to belong to the same cluster, a neighborhood is determined for each of the similar samples, the neighborhood at least including another one or some of the similar samples.

4. The training method according to claim 1 or 2, wherein: The second training criterion comprises the sum: in, f is an increasing continuous function; <,> is the dot product; D is the data set used to train the classifier in step S30; and |D| is the number of samples in the dataset.

5. The training method according to claim 4, wherein: The increasing continuous function is a logarithm.

6. The training method according to claim 1 or 2, wherein: The second training criterion comprises the sum: in in, c is the index of the coordinate of the predicted Φη(Xi) output by the classifier, and C is a set containing all c indices; g is a function that measures the distance between the distribution of the predictions on each cluster and the expected distribution of the predictions; D is the data set used to train the classifier in step S30; and |D| is the number of samples in the dataset.

7. The training method according to claim 6, characterized in that: The function g is defined as: g(Φη′ c )=Fi′ c logΦη′ c 。 8. The training method according to claim 1 or 2, wherein: In step S30, for at least one set of two or more similar samples of the data set that have been determined to belong to the same cluster, the second training criterion is configured to maximize the possibility that the similar samples in the considered set of similar samples belong to the same cluster.

9. The training method according to claim 1 or 2, further comprising performing at least one fine-tuning step S40: S40) training the classifier Φη based on the third training criterion Λ, for the highest probability Φη max (Xi) High confidence samples X that are higher than the predetermined threshold Thr HC For each considered sample Xi in i, the third training criterion is configured to make the considered sample belong to the prediction Φη for the considered sample Xi. max The maximum coordinate Φη of (Xi) max The probability Φη of cluster Cj represented by (Xi) j (Xi) Maximization.

10. The training method according to claim 9, wherein: When it is determined that the number of high-confidence samples no longer increases, the fine-tuning step S40 is stopped.

11. The training method according to claim 9, wherein: In step S40, for at least one set of two or more similar samples of the data set that have been determined to belong to the same cluster, the third training criterion is configured to maximize the possibility that the similar samples in the considered set of similar samples belong to the same cluster.

12. The training method according to claim 9, wherein: The high confidence sample X HC i is added to the data set used in step S40, the sample called "strongly augmented" is a transformed sample obtained by applying a transformation to the sample that modifies the sample content more deeply than the transformation of the conventional augmentation; and The third training criterion is configured to make the strong augmentation SA(Xi) and the high confidence sample X HC The probability that i belongs to the same cluster is maximized.

13. The training method according to claim 9, wherein: In one or more of steps S20, S30 and S40, the training method includes taking into account prior knowledge that a plurality of samples, referred to as similar samples, constitute clusters; thus, In step S20, when defining the neighborhood, the similar samples are regarded as neighboring values, and / or In step S30 or S40, the classifier must be trained to classify the similar samples into the same cluster.

14. The training method according to claim 1 or 2, wherein: In step S20 and / or S30, the training method includes taking into account prior knowledge that a plurality of samples, called similar samples, constitute clusters; thus, In step S20, when defining the neighborhood, the similar samples are regarded as neighboring values, and / or In step S30, the classifier must be trained to classify the similar samples into the same cluster.

15. A classification method, comprising step D: D) Classifying the sample using a classifier trained by the training method according to any one of claims 1 to 14.

16. The classification method according to claim 15, in the case where the sample Xi comprises one or more acquired images, Also includes step B: B) generating at least one sample sub-image based on the sample; and In step D, the sample is classified by classifying the at least one sample sub-image.

17. The classification method according to claim 15 or 16, comprising the following steps: A) acquiring data based on output of one or more cameras and / or non-camera sensors of the vehicle; and C) detecting whether the acquired data includes abnormal data; in Abnormal data is information showing abnormal objects on the road ahead of the vehicle, and Step D is performed only when it is detected in step C that the acquired data includes abnormal data.

18. The classification method according to claim 17, wherein the acquired data comprises one or more acquired images. Also includes step B: B) generating at least one data sub-image based on the acquired data; and In step C, the abnormal data is detected in the at least one data sub-image.

19. The classification method according to claim 18, wherein in step D, the sample is classified by classifying the at least one data sub-image.

20. A classifier training system, comprising one or more processors and a memory, wherein the memory stores instructions, and when the one or more processors execute the instructions, the instructions cause the one or more processors to train a classifier using the training method according to any one of claims 1 to 14.

21. A sample classification system, comprising one or more processors and a memory, wherein the memory stores instructions, and when the one or more processors execute the instructions, the one or more processors use the classification method according to any one of claims 15 to 19 to classify the sample.

22. A computer-readable recording medium comprising instructions of a computer program suitable for execution on one or more processors, wherein when the computer program is run on the one or more processors, the instructions are suitable for executing the steps of the training method according to any one of claims 1 to 14 or the classification method according to any one of claims 15 to 19.

Images