Device and method, in particular a computer-implemented method, for classifying digital images, especially for optical inspection

Abstract

Method, in particular a computer-implemented method, for classifying digital images, especially for optical inspection, wherein a first classification is determined for each image using a model for classifying images (204), wherein the model for classifying images is configured to map the images to features and the features to the first classification, wherein the images are mapped to the features and the features to the first classification, wherein a value for explaining the contribution of the feature to the first classification is determined for each feature using a method for explaining the first classification (206), wherein a second classification is determined for each image depending on the values ​​determined for the image (208), wherein a measure for the quality of the method for explaining the first classification is determined depending on the first classification and depending on the second classification (210).where it is recognized (212) that the method of explanation is unsuitable for the images if the measure of quality deviates from an expected quality.

Description

State of the art

[0001] The invention relates to a device and method, in particular a computer-implemented method, for the classification of digital images, especially for optical inspection.

[0002] The classification of digital images using classification models can be made understandable to a user through methods for explaining the classification. Disclosure of the invention

[0003] A method, in particular a computer-implemented method, for classifying digital images, especially for optical inspection, provides that a first classification is determined for each image using a model for classifying images, wherein the model for classifying images is configured to map the images to features and the features to the first classification, wherein the images are mapped to the features and the features to the first classification, wherein a value for explaining the contribution of the feature to the first classification is determined for each feature using a method for explaining the first classification, wherein a second classification is determined for each image depending on the values ​​determined for the image, and wherein a measure for the quality of the method for explaining the first classification is determined depending on the first classification and depending on the second classification.The system recognizes that the explanation method is unsuitable for the images if the measure of quality deviates from the expected level of quality. For images for which the explanation method is suitable, the values ​​provide an appropriate explanation. To prevent misinterpretation by a user due to an explanation that does not match the classification, the system recognizes, based on the images themselves, that the method is unsuitable for classifying the images.

[0004] Suitability can be verified through a statistical comparison.

[0005] For example, it may be provided that the first classification assigns the respective image to at least one of several classes, wherein the second classification assigns the respective image to at least one of the several classes, wherein for each image a distribution over the values ​​is determined depending on the values ​​determined for the image, wherein for each class of the several classes a reference for the distribution over the values ​​is specified, wherein the image is assigned to at least one class in the second classification depending on a similarity between the distribution determined for the image and the respective references specified for the classes for the distribution.

[0006] It may be provided that the similarity is determined by means of a statistical test based on the shape of the respective distribution and the shape of the reference for the distribution.

[0007] For example, it is intended that similarity will be determined if it is found that the values ​​intended for the image in a class come from the same distribution as the values ​​specified for the class.

[0008] Suitability can be verified through a graphical comparison.

[0009] For example, it can be provided that for each image, a representation is generated, particularly by means of color coding, depending on the values ​​specified for the image. For each of the several classes, a reference for the representation is specified, and the representation is mapped to a tensor. For each class, the reference is mapped to a corresponding reference for the tensor. Depending on the similarity between the tensor specified for the image and the respective references for the tensor specified for the classes, the image is assigned to at least one class in the second classification. The representation by means of color coding, i.e., a heat map, allows for a simplified understanding of the significance of the features from the image for the classification.

[0010] It may be provided that the similarity is determined depending on a distance between the tensor and the respective reference for the tensor.

[0011] It may be provided that the model for classifying images is designed to map an image to a tensor and the tensor to the features, wherein the tensor is determined by the model for classifying images depending on the representation and / or wherein the reference for the tensor is determined by the model for classifying images depending on the reference for the representation.

[0012] For example, the image of the class with the greater similarity is assigned.

[0013] For optical inspection, for example, it is intended that an image is captured, the image is classified using the image classification model, and, depending on the values ​​determined for the image, a representation, particularly using color coding, is generated to explain the classification. The classification and the representation explaining the classification are then displayed. This allows a user to assess the classification during the optical inspection or to verify it subsequently.

[0014] It may be provided that a message is displayed indicating that the method of explanation is unsuitable for an image when it is detected that the method of explanation is unsuitable for the image, or that a message is displayed indicating that the method of explanation is unsuitable for the images when it is detected that the method of explanation is unsuitable for the images.

[0015] A device for classifying digital images, in particular for optical inspection, provides that the device comprises at least one processor and at least one memory, wherein the at least one memory comprises instructions executable by the at least one processor, the execution of which by the at least one processor causes the device to execute the method.

[0016] A computer program may be provided, wherein the computer program comprises instructions executable by a computer, the execution of which by the computer carries out the procedure.

[0017] Further advantageous embodiments can be found in the following description and the drawing. The drawing shows: Fig. 1 a schematic representation of a device for classifying digital images, in particular for optical inspection, Fig. 2 a flowchart with steps of a procedure for classifying digital images, especially for optical inspection.

[0018] In Fig. Figure 1 shows a device 100.

[0019] The device 100 comprises at least one processor 102 and at least one memory 104.

[0020] The device 100 may include a camera 106 for capturing a digital image. The device 100 may also include a screen 108 for displaying the classification.

[0021] The device 100 is for classifying the digital images x1,...,x N A model M is formed for classifying images, where N represents the number of images. The model M is formed, and the images x1, ..., x N on one classification each y1, ...,y N to depict.

[0022] Model M is formed, each image xi on a tensor, the tensor on features, and the features on the classification y i of the image x i to represent. For example, model M is an artificial neural network trained to classify images.

[0023] Device 100 is used to explain the classification of digital images x1,...,x N The model M for classifying images was developed using a method E to explain the classification.

[0024] Method E is trained, per image x i Depending on the characteristics, one value a is assigned to each characteristic. ij to determine the contribution of the feature to the first classification. For images x1, ..., x N The values ​​a1, ..., a N certainly.

[0025] An example of method E is described in Visual Explanations from Deep Networks via Gradient-based Localization (GradCAM), arXiv:1610.02391.

[0026] The values ​​a1, ..., a N These are, for example, attention scores.

[0027] The device 100 is designed to carry out a method, in particular a computer-implemented method, for the classification of digital images, especially for optical inspection.

[0028] In Fig. Figure 2 shows a flowchart illustrating the steps of the procedure.

[0029] The procedure includes step 202.

[0030] In step 202, an image x is displayed. i captured. The image x i is used for optical inspection, e.g. of a component.

[0031] The procedure includes step 204.

[0032] In step 204, the image x i classified using the model for classifying images M.

[0033] This means that for image x i The model for classifying images M is used to perform a first classification y.i certainly.

[0034] The first classification y i arranges the image x i at least one class c i of several classes c1, ..., c C to.

[0035] This means the image x i The model for classifying images M is mapped to the tensor. The tensor is mapped to the features. The features are then mapped to the first classification y. i shown.

[0036] The procedure includes step 206.

[0037] In step 206, the method for explaining the first classification E assigns a value a to each characteristic. i,j intended to explain the contribution of the feature to the first classification.

[0038] For optical inspection, for example, depending on the values ​​determined for the image, a j,j A representation, particularly using color coding, is generated to explain the classification.

[0039] The procedure includes step 208.

[0040] In step 208, the image x i depending on the settings for image x i certain values ​​a i,j a second classification ya i certainly.

[0041] The second classification ya i The respective image is arranged x i at least one of the several classes c1, ..., c C to.

[0042] The second classification ya i is determined, for example, by a stochastic test or based on the representation.

[0043] Determination of the second classification ya i through the stochastic test: Each class of the several classes c1, ..., c C A reference for the distribution across the values ​​is specified. For image x i will depend on the settings for image x i A distribution across the values ​​is determined by specific values.

[0044] Depending on the similarity between the distribution determined for the image and the respective references specified for the classes, the image is assigned to the distribution of at least one class in the second classification.

[0045] For example, similarity is determined using a statistical test based on the shape of the respective distribution and the shape of the reference distribution. The distribution can have the shape of a standard normal distribution or another shape. The type of test is chosen, for example, depending on the shape of the distribution.

[0046] For example, similarity is established when it is determined that the values ​​intended for the image in a class come from the same distribution as the values ​​specified for the class.

[0047] Determination of the second classification ya i based on the representation: For image x iwill depend on the settings for image x i A representation is generated based on specific values, particularly using color coding. For example, a heat map is generated. Each class of the several classes c1, ..., c C A reference for the representation is specified.

[0048] The representation is mapped onto a tensor.

[0049] For each class, the reference to a respective reference for the tensor is mapped.

[0050] The tensor is determined, for example, using the model for classifying images M depending on the representation.

[0051] The reference for the tensor is determined, for example, using the model for classifying images M, depending on the reference for the representation.

[0052] The image x iDepending on the similarity between the tensor determined for the image and the respective references determined for the classes, the tensor is assigned to at least one class in the second classification.

[0053] The similarity is determined, for example, depending on the distance between the tensor and the respective reference for the tensor.

[0054] Similarity can also be determined using other methods based on the similarity of images. For example, a histogram is used instead of the representation, and a reference for the histogram is used instead of the reference for the representation. It may also be possible to directly compare the representation to a pattern.

[0055] For example, the image of the class with the greater similarity is assigned.

[0056] The procedure includes step 210.

[0057] In step 210, a measure of the quality of the method for explaining the first classification E is determined.

[0058] The measure of quality in the example depends on the first classification y. i and the second classification ya i certainly.

[0059] To assess the quality, the measure of quality is, for example, dependent on the values ​​for several digital images x1, ..., x N certain first classifications y1, ...,y N and the second classifications ya1, ..., ya N certainly.

[0060] For example, steps 202 to 208 are used to determine the quality for the multiple digital images x1, ...,x N executed.

[0061] The procedure includes step 212.

[0062] In step 212, it is recognized that the method for explaining E is unsuitable for the image x. i or the images x1, ...,x NThis occurs when the measure of quality deviates from an expected level of quality.

[0063] The procedure includes step 214.

[0064] Step 214 displays the classification and the explanation of the classification.

[0065] It may be intended to indicate that the method for explaining E is unsuitable for the image x. i or the images x1, ..., x N is when it is recognized that the method for explaining E is unsuitable for the image x i or the images x1, ..., x N is. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] Visual Explanations from Deep Networks via Gradient-based Localization (GradCAM), arXiv:1610.02391

[0025]

Claims

[1] Method, in particular a computer-implemented method, for classifying digital images, especially for optical inspection, characterized by, that for each image a first classification is determined using a model for classifying images (204), wherein the model for classifying images is designed to map the images to features and the features to the first classification, wherein the images are mapped to the features and the features to the first classification, wherein for each image a value for explaining the contribution of the feature to the first classification is determined using a method for explaining the first classification (206), wherein a second classification is determined for each image depending on the values ​​determined for the image (208), wherein a measure for the quality of the method for explaining the first classification is determined depending on the first classification and depending on the second classification (210), wherein it is recognized (212) that the method for explanation is unsuitable for the images,when the measure of quality deviates from an expected quality. [2] Method according to claim 1, characterized by , that the first classification assigns the respective image to at least one of several classes, wherein the second classification assigns the respective image to at least one of the several classes, wherein for each image a distribution over the values ​​is determined depending on the values ​​determined for the image, wherein for each class of the several classes a reference for the distribution over the values ​​is specified, wherein the image is assigned to at least one class in the second classification depending on a similarity between the distribution determined for the image and the respective references specified for the classes for the distribution. [3] Method according to claim 2, characterized by that the similarity is determined by means of a statistical test based on the shape of the respective distribution and the shape of the reference for the distribution. [4] Method according to claim 3, characterized by , that the similarity is established when it is found for a class that the values ​​determined for the image come from the same distribution as the values ​​specified for the class. [5] Method according to claim 1, characterized by , that for each image, depending on the values ​​determined for the image, a representation is generated, in particular by means of color coding, wherein for each class of the several classes a reference for the representation is specified, wherein the representation is mapped to a tensor, wherein for each class the reference is mapped to a respective reference for the tensor, and wherein the image is assigned to at least one class in the second classification depending on a similarity between the tensor determined for the image and the respective references for the tensor determined for the classes. [6] Method according to claim 5, characterized by, that the similarity is determined depending on a distance between the tensor and the respective reference for the tensor. [7] Method according to one of claims 5 or 6, characterized by , that the model for classifying images is designed to map an image to a tensor and the tensor to the features, wherein the tensor is determined by the model for classifying images depending on the representation and / or wherein the reference for the tensor is determined by the model for classifying images depending on the reference for the representation. [8] Method according to any one of claims 2 to 7, characterized by , that the image of the class with the greater similarity is assigned. [9] Method according to any one of the preceding claims, characterized by, that an image is captured (202), wherein the image is classified using the model for classifying images (204), wherein, depending on the values ​​determined for the image, a representation, in particular by means of color coding, is generated to explain the classification (206), and wherein the classification and the representation to explain the classification are displayed (214). [10] Method according to any of the preceding claims, characterized by , that it is indicated (214) that the method of explanation is unsuitable for an image when it is recognized that the method of explanation is unsuitable for the image, or that it is indicated (214) that the method of explanation is unsuitable for the images when it is recognized that the method of explanation is unsuitable for the images. [11] Device (100) for classifying digital images, in particular for optical inspection, characterized by, that the device (100) comprises at least one processor (102) and at least one memory (104), wherein the at least one memory (104) comprises instructions executable by the at least one processor (102), the execution of which by the at least one processor (102) of the device (100) executes the method according to one of claims 1 to 10. [12] Computer program, characterized by that the computer program comprises instructions executable by a computer, the execution of which by the computer performs the method according to one of claims 1 to 10.

Images