Apparatus and computer-implemented method for classifying digital content

Abstract

An apparatus and a computer-implemented method for classifying digital content, in particular digital images (102), wherein a first classifier (202) is used to determine, for the digital content (102), a class from a set of classes together with an accuracy (212) associated with the class, and depending on the accuracy (212), the class determined using the first classifier (202) is output for classifying the digital content (102); or a second classifier (204) is used to determine, for the digital content (102), a class from a true subset of the set of classes, the true subset including at least two classes from the set of classes, the true subset including the class determined using the first classifier (202), and the class (112) determined using the second classifier (204) is output (306) for classifying the digital content (102).

Description

[Technical Field]

[0001] Conventional technology The present invention relates to an apparatus and computer-implemented method for classifying digital content, particularly digital images. Summary of the Invention [Problem to be solved by the invention]

[0002] Disclosure of the Invention Improved classification is achieved by an apparatus and a computer-implemented method for classifying digital content, in particular digital images, according to the independent claims. [Means for solving the problem]

[0003] A computer-implemented method for classifying digital content, particularly digital images, envisions using a first classifier to determine a class from a set of classes together with an accuracy associated with the class, and depending on the accuracy, the class determined using the first classifier is output for classifying the digital content; or using a second classifier to determine a class from a true subset of the set of classes, the true subset including at least two classes from the set of classes, the true subset including the class determined using the first classifier, and the class determined using the second classifier is output for classifying the digital content. Depending on the accuracy, a single-layer or multi-layer classification is first determined via the set and then via the subset. The first classifier distinguishes classes from the set of classes. The second classifier distinguishes classes from the subset. This results in better classification results overall.

[0004] Preferably, digital contents are provided, and for each digital content, a prediction for a class is determined using a first classifier, and depending on the prediction for the digital content for a class from the set of classes, a variable characterizing the accuracy of the prediction for that class is determined, and that class for the true subset is selected or not selected depending on the variable determined for that class. Classes predicted with higher accuracy are separated from classes with lower accuracy depending on the variable. Classes with higher accuracy are carried forward, and classes with lower accuracy are reclassified.

[0005] For a class from the set, it is preferably checked whether the variables show an accuracy lower than a threshold, and if the accuracy is lower than the threshold, a second classifier with the class is provided, which means that if the accuracy of the first classifier is too low, the second classifier is used for the class, thereby improving the classification.

[0006] The true subset for the second classifier preferably includes the class determined using the first classifier and at least one additional class from the set of classes, and for the at least one additional class from the set of classes, the first classifier selects the class determined using the first classifier with a certain frequency, in particular more frequently than at least one other class. The class determined by the first classifier is a base class. The second classifier considers the base class and other classes, and for this other class, the base class is selected by prediction using the first classifier, but the classifier must correctly select this other class. In the simplest case, the second classifier is provided to distinguish between the base class and other classes. The second classifier is also provided to distinguish between the base class and multiple other classes, for example, for other classes different from the base class for which the first classifier incorrectly predicts the base class more frequently than other classes from the set of classes.

[0007] The first classifier is preferably trained independently of the second classifier. The second classifier is preferably trained independently of the first classifier.

[0008] In one embodiment, the first classifier and the second classifier are trained using the same digital content.

[0009] In one embodiment, a first classifier is trained with digital content from a set of classes and a second classifier is trained with digital content from a subset of the classes, meaning that the first classifier is trained to distinguish between classes from the set of classes, and that the second classifier is trained to distinguish between classes from the subset.

[0010] Preferably, it is envisaged that digital content, in particular digital images, digital text or objects represented as vectors, are mapped to embeddings using an encoder, and a first classifier is trained to map the embeddings to a respective one value for each class from a set of classes, and / or the embeddings are mapped to a respective one value for each class from a set of classes using the first classifier. The encoder provides the embeddings for classification.

[0011] Preferably, it is assumed that the second classifier is trained to map the embeddings of the digital image to a value for each class from the proper subset of classes, and / or that the embeddings are mapped to a value for each class from the proper subset of classes using the second classifier, which means that the second classifier uses the embeddings in the same way as the first classifier.

[0012] It is envisaged that the first and second classifiers are preferably trained using the same embedding and / or the same embedding is mapped using the first and second classifiers, and by temporarily storing the embeddings other computing resources are saved.

[0013] In one embodiment, a plurality of classifiers is provided, including a first classifier and a set of second classifiers for different true subsets of the set of classes, the different true subsets each corresponding to one class from the set of classes, the class being determined using the first classifier or by one of the second classifiers depending on the accuracy, and the second classifier for determining the class being selected from the set of classifiers depending on the class determined using the first classifier.

[0014] An apparatus for classifying digital content includes at least one processor and at least one memory, the at least one memory storing instructions executable by the at least one processor, the instructions, when executed by the at least one processor, performing the method.

[0015] The program for classifying digital content includes instructions executable by at least one processor, the instructions, when executed by the at least one processor, performing the method.

[0016] Further advantageous embodiments can be seen from the following description and drawings. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a schematic diagram of an apparatus for classifying digital content; [Figure 2] FIG. 1 is a schematic diagram of an architecture for classifying digital content. [Figure 3]1 is a flowchart with steps of a method for classifying digital content. [Figure 4] 1 is a flowchart with steps of a method for training a classifier. DETAILED DESCRIPTION OF THE INVENTION

[0018] FIG. 1 shows a schematic diagram of an apparatus 100 for classifying digital content 102 .

[0019] Digital content 102 is, for example, data that is created and provided in a digital format, such as a digital image, a photo or video file, a music file, a computer program, a digital game, digital text, an e-book, or an object represented as a vector.

[0020] The device 100 includes at least one processor 104 and at least one memory 106. In this example, the device 100 includes an input 108 configured to receive digital content 102. In this example, the device includes an output 110 configured to output a class 112 for classifying the digital content 102.

[0021] At least one memory 106 stores instructions executable by at least one processor 104 .

[0022] The instructions, when executed by at least one processor 104, perform a method for classifying digital content.

[0023] The program for classifying digital content includes instructions executable by at least one processor 104, which instructions, when executed by at least one processor 104, perform the method.

[0024] FIG. 2 shows a schematic diagram of an architecture 200 for classifying digital content 102.

[0025] The architecture 200 includes a first classifier 202 and a second classifier 204. The architecture 200 includes an encoder 206 that, in this example, is configured to map the digital content 102 into embeddings 208.

[0026] It is possible to envision that the architecture 200 includes a further classifier 210 or multiple further classifiers.

[0027] In this example, the class 112 output for classifying the digital content 102 is determined by the prediction of the first classifier 202 about the class 112 or by the prediction of the second classifier 204 about the class 112. Which of these predictions is output as the class 112 is determined in this example depending on the accuracy 212 of the prediction of the first classifier 202 about the class 112. This means that the class 112 is determined using either the first classifier 202 or the second classifier 204.

[0028] If at least one further classifier 210 is provided, it can be assumed that depending on the accuracy 212 of the prediction of the first classifier 202 for the class 112, the class 112 is determined using the first classifier 202 or the second classifier 204 or the further classifier 210 or one of multiple further classifiers.

[0029] The architecture 200 may be implemented, for example, as an artificial neural network with an input layer for inputs characterizing the digital content 102. The encoder 208 may include, for example, a fully connected layer. The classifiers may include, for example, one fully connected layer each. The neural network may include, for example, an output for the class 112.

[0030] In FIG. 3 a flow chart with steps of the method for classification is shown.

[0031] The method includes step 300 .

[0032] In step 300, digital content is provided, which is then mapped using the encoder 206 into respective embeddings of the digital content.

[0033] For example, the digital content 102 is mapped using an encoder 206 into an embedding 208 of the digital content 102 .

[0034] The method includes step 302 .

[0035] In step 302, the digital content is classified using the first classifier 202. In step 302, for example, for each digital content, a prediction for a class from a set of classes is determined using the first classifier 202.

[0036] For example, for the digital content 102, a first classifier 202 is used to determine a class from a set of classes.

[0037] In this example, the embeddings 208 are mapped to classes using a first classifier 202 .

[0038] For example, the embeddings 208 of the digital content 102 are mapped using the first classifier 202 to one value for each class from a set of classes.

[0039] The class is determined along with the accuracy 212 associated with that class.

[0040] For example, depending on the prediction for the digital content for each class, a variable characterizing the accuracy 212 of the prediction for that class is determined.

[0041] Depending on the precision 212, either step 304 or step 306 is performed.

[0042] In step 304, the second classifier 204 is used to determine a class from a proper subset of the set of classes.

[0043] For example, the embeddings 208 of the digital content 102 are mapped using the second classifier 204 to one value for each class from the proper subset of classes.

[0044] In one embodiment, the true subset includes a class determined using the first classifier 202 as a base class and at least one class from the set of classes, which the first classifier 202 incorrectly predicts more frequently or most frequently than other classes.

[0045] A proper subset includes the base class and one or more other classes. The selection of the other classes is described below. If more than one other class is selected, this is performed, for example, as described for one other class.

[0046] In order to select the other class, it is essential in this example that the majority of the false positive predictions of the first classifier 202, i.e., the majority of the predictions that the first classifier 202 predicts to be the base class, actually belong to the other class.

[0047] The base class of the second classifier 204 is determined depending on the accuracy of the first classifier 202. The other classes are determined depending, for example, on a comparison between the predictions of the first classifier 202 and the respective classes that should actually be predicted.

[0048] For example, if classes a, b, and c are considered, the evaluation of incorrect predictions is analyzed using a confusion matrix that shows the frequency of correct and incorrect predictions for a*, b*, and c* depending on the classes a, b, and c that are actually associated with the digital image.

[0049] An example of this is that for the true class a, there are 8 times as many occurrences of the correct prediction, 2 times as many occurrences of the incorrect prediction of b instead of a, and 3 times as many occurrences of the incorrect prediction of c instead of a.

[0050] An example of this is that for the true class b, there are 16 times as many occurrences of the correct prediction, 2 times as many occurrences of the incorrect prediction of a instead of b, and 5 times as many occurrences of the incorrect prediction of c instead of b.

[0051] An example of this is for a true class c, 7 times more occurrences of the correct prediction, no occurrences of the incorrect prediction a instead of c, and 2 times more occurrences of the incorrect prediction b instead of c.

[0052] This means that the accuracy for class a is 0.8, the accuracy for class b is 0.8, and the accuracy for class c is 0.47.

[0053] This means, for example, that if the threshold is 0.7, the results of the first classifier 202 for classes a and b are used, and the true subset {c, b} is used for the second classifier 204.

[0054] The proper subset {c, b} of the set of classes includes at least two classes from the set of classes a, b, c. The proper subset includes class c with an accuracy less than a threshold, as determined using the first classifier 202.

[0055] If a further classifier 210 or more further classifiers are provided, in one embodiment a classifier is selected to determine the class from the true subset.

[0056] It can be envisioned that a set of classifiers is provided, each containing a different classifier for a number of different true subsets.

[0057] It can be envisioned that a classifier configured to select a class from the proper subset is selected from the set.

[0058] The class for the proper subset is determined, for example, depending on the variables determined for the class.

[0059] The subset includes at least two classes from the set of classes.

[0060] The subset includes the classes determined using the first classifier 202 .

[0061] In step 306, the class determined using the first classifier 202 or the class determined using the second classifier 204 is output as the class 112 for classifying the digital content 102.

[0062] FIG. 4 shows a flow chart with steps of a method for training a classifier.

[0063] The method for training optionally includes step 400. Step 400 assumes that digital content is provided.

[0064] Step 400 assumes that the digital content is mapped using the encoder 206 into an embedding of the digital content.

[0065] For example, the digital content 102 is mapped using an encoder 206 into an embedding 208 of the digital content 102 .

[0066] The method for training includes step 402 .

[0067] In step 402, the first classifier 202 is trained without reliance on the second classifier 204. The first classifier 202 is trained, in this example, using digital content from a set of classes.

[0068] The first classifier 202 is, for example, trained to map each embedding to a value for each class from a set of classes.

[0069] For example, the first classifier 202 is trained to map the embeddings 208 of the digital content 102 using the first classifier 202 to one value for each class from a set of classes.

[0070] The method for training includes step 404 .

[0071] In step 404, the second classifier 204 is trained without reliance on the first classifier 202. The second classifier 204 is trained, in this example, using digital content from a subset of the classes.

[0072] The second classifier 204 is, for example, trained to map each embedding to a value for each class from a subset of classes.

[0073] For example, the second classifier 204 is trained to map the embeddings 208 of the digital content 102 using the second classifier 204 to one value for each class from the proper subset of classes.

[0074] It can be assumed that the first classifier 202 and the second classifier 204 are trained using the same digital content.

[0075] It can be assumed that the first classifier 202 and the second classifier 204 are trained using the same embedding 208 for the digital content 102. It can be assumed that the same embedding 208 is mapped using the first classifier 202 and the second classifier 204. This means that the embedding 208 is, for example, determined only once.

[0076] It is possible to envision training a further classifier 410 or multiple classifiers to classify different true subsets of the set of classes.

[0077] In one embodiment, a plurality of classifiers, in particular a cascade of classifiers, is provided, including a first classifier 202 and a set of second classifiers 204 for different true subsets of the set of classes, each of which corresponds to a class from the set of classes, i.e., a base class.

[0078] The class 112 is determined using the first classifier 202 or by one of the second classifiers 204 depending on the accuracy 212 .

[0079] If the first classifier 202 predicts the class 112 with too low an accuracy 212, a second classifier 204 for determining the class 112 is selected from the set of classifiers depending on the class determined using the first classifier 202, i.e., the base class.

[0080] It is possible to envisage that the encoder 206 is trained in a previous step or jointly during training. Preferably, the encoder includes pre-trained parameters that are kept constant during training of the classifier.

[0081] For inference purposes, the first classifier 202 for the digital content 102 is used to determine the class of the digital content 102, i.e., the base class of the digital content 102. If no other classifiers are provided for this base class, the first classifier 202 is used to determine the class 112.

[0082] If a second classifier 204 from the set of classifiers associated with the base class is provided for this base class, then this second classifier 204 is used to determine the class 112 in the inference.

[0083] This means that in inference, depending on the accuracy 212 determined during training, a class determined using a first classifier 202 for the digital content 102 is output to classify the digital content 102, or a class determined using a second classifier 202 for the digital content 102 is output to classify the digital content 102.

[0084] For inference, in one embodiment, a first classifier 202 is used first. If the first classifier 202 predicts a class that had too low an accuracy 212 during training, the instance is again reclassified using a second classifier 204 associated with this predicted class. This means that the first classifier 202 and the second classifier 204 form a cascade, and the class predicted using the second classifier 204 represents the final decision of the cascade, i.e., the output class 112.

Claims

1. A computer-implemented method for classifying digital content, in particular digital images (102), comprising: A class from a set of classes is determined (302) for the digital content (102) using a first classifier (202), along with an accuracy (212) associated with the class; Depending on the accuracy (212), The classes determined using the first classifier (202) are output (306) for classifying the digital content (102); or A second classifier (204) is used to determine (304) a class from a proper subset of the set of classes, the proper subset including at least two classes from the set of classes, the proper subset including the class determined using the first classifier (204); The classes (112) determined using the second classifier (204) are output (306) for classifying the digital content (102). A method characterized by:

2. Digital content (102) is provided (300); For each piece of digital content (102), a prediction for the class (112) is determined (302) using the first classifier (202); A variable characterizing the accuracy (212) of the prediction for the digital content (102) for a class from the set of classes is determined (302); The class for the proper subset is selected or not selected depending on the variables determined for that class (304). The method of claim 1.

3. For a class from said set, it is checked whether said variable exhibits an accuracy (212) less than a threshold; If the accuracy (212) is less than the threshold, the second classifier (204) with the class is provided (304). The method of claim 2.

4. the true subset for the second classifier (204) comprises a class determined using the first classifier (202) and at least one further class from the set of classes, and with respect to the at least one further class from the set of classes, the first classifier (202) selects the class determined using the first classifier with a certain frequency, in particular more frequently than at least one other class.

4. The method according to any one of claims 1 to 3.

5. The first classifier (202) is trained (402) independently of the second classifier (204), and / or The second classifier (204) is trained (404) independently of the first classifier (202); 5. The method according to any one of claims 1 to 4.

6. The first classifier (202) and the second classifier (204) are trained (402, 404) using the same digital content (102); 6. The method according to any one of claims 1 to 5.

7. The first classifier (202) is trained (402) using digital images (102) from the set of classes; The second classifier (204) is trained (404) using digital images (102) from a subset of the classes.

6. The method according to any one of claims 1 to 5.

8. The digital content (102) is mapped (300, 400) to an embedding (208) using an encoder (206); The first classifier (202) is trained (402) to map the embeddings (208) to a value for each class from the set of classes, and / or The embeddings (208) are mapped (302) to a value for each class from the set of classes using the first classifier (202).

8. The method according to any one of claims 1 to 7.

9. the second classifier (204) is trained (404) to map embeddings (208) of the digital content (102) to a respective one value for each class from the proper subset of classes; and / or The embeddings (208) are mapped (304) to a value for each class from the proper subset of classes using the second classifier (204).

9. The method according to any one of claims 1 to 8.

10. the first classifier (202) and the second classifier (204) are trained (402, 404) using the same embedding (208); and / or The same embedding (208) is mapped (302, 402) using the first classifier (202) and the second classifier (204).

10. The method according to any one of claims 1 to 9.

11. a plurality of classifiers are provided, including the first classifier (202) and a set of second classifiers (204) for different true subsets of the set of classes; the different proper subsets are each associated with a class from the set of classes; The class (112) is determined using the first classifier (202) or by one of the second classifiers (204), depending on the accuracy (212); the second classifier (204) for determining the class (112) is selected from the set of classifiers (204) depending on the class determined using the first classifier (202); 11. The method according to any one of claims 1 to 10.

12. An apparatus (100) for classifying digital content, in particular digital images (102), comprising: The device (100) comprises: at least one processor (104); at least one memory (106); Including, the at least one memory (106) storing instructions executable by the at least one processor (104); The instructions, when executed by the at least one processor (104), perform the method of any one of claims 1 to 11.

1. An apparatus (100) comprising:

13. A program for classifying digital content, in particular digital images (102), comprising: The program includes instructions executable by at least one processor (104); The instructions, when executed by the at least one processor (104), perform the method of any one of claims 1 to 11. A program characterized by:

Images