Method and device for determining an interpretation diagram

Abstract

A method for determining an interpretation map #imgabs0# of an image (x), in which all pixels of the image (x) that are decisive for the classification of the image (x) determined using a deep neural network (f) are highlighted, wherein the interpretation map #imgabs1# is selected such that it selects as small a subset of pixels of the image (x) as relevant, and then if the interpretation map #imgabs2# is fed to the deep neural network (f) for classification, it leads to the same classification result #imgabs3# as for the image (x), wherein the interpretation map #imgabs4# is selected such that in the feature map (h of the deep neural network (f) l ) caused by the explanatory image #imgabs5# does not substantially exceed the activation (h l (x)).

Description

Technical Field

[0001] The invention relates to a method for determining an interpretation diagram, a computer program, one or more machine-readable storage media and one or more computers, a machine classification system and a display system. Background Art

[0002] The non-prepublished DE 10 2018 205 561 discloses a device for classifying retina images, the device comprising an artificial neural network having at least one convolutional layer,

[0003] wherein the convolutional layer comprises a plurality of trainable convolutional kernels, and an activation map is provided for each convolutional kernel, the activation map assigning output values ​​to discrete positions in the input of the at least one convolutional layer, respectively, the output value being a measure of the local consistency of the input with the respective convolutional kernel,

[0004] wherein at least one classifier layer is connected after the at least one convolutional layer, which classifier layer maps its input by means of trainable weights into a set of probabilities that the input belongs to one of a plurality of predefined classes, in particular whether the tissue is healthy or pathologically altered,

[0005] An evaluation unit is provided which is designed to determine a correlation map of the output of the at least one convolutional layer by weighted summation of an activation map obtained by the at least one convolutional layer and weights stored in the aggregation layer, wherein the correlation map displays areas of the tissue that are decisive for the evaluation.

[0006] Advantages of the present invention

[0007] In comparison, the method having the features of independent claim 1 has the advantage that it allows an improved determination of an interpretation map which in particular allows a pixel-accurate representation, so that the information contained in the interpretation map is perceptible to humans.

[0008] Advantageous developments are the subject matter of the dependent claims. Summary of the invention

[0009] In particular in systems for assisting medical diagnosis, in which possible diagnoses are identified with the aid of deep neural networks based on certain features of the image, it is important to clearly show the attending physician which parts of the image led to the diagnosis.

[0010] In a first aspect, the invention therefore relates to a method for determining an interpretation map of an image, i.e. an image region that is decisive for a classification result, wherein the image is classified (i.e. assigned to one of a plurality of predefinable classes) by means of a deep neural network, in which pixels of the image, in particular all pixels, are changed depending on how much they contribute to the classification of the image, wherein the interpretation map selects pixels of the image as relevant and then leads to the same classification result as the image if the interpretation map is fed to the deep neural network for classification, wherein the interpretation map is selected such that in a feature map of the deep neural network the activations caused by the interpretation map then (i.e. when fed accordingly to the deep neural network) do not substantially exceed the activations caused by the image.

[0011] The pixels can be changed, for example, in such a way that all those pixels of the image are selected which are decisive for the classification of the image.

[0012] This means that provision is made for both the image and also the interpretation map to be fed to the deep neural network, wherein the classifications obtained in each case as output variables of the deep neural network should be identical.

[0013] Selecting a pixel of the image as relevant may mean that the activation of this pixel in the interpretation image has an absolute value greater than a predefinable threshold value, in particular zero.

[0014] Here, a deep neural network is generally a neural network including a plurality of layers.

[0015] A particularly accurate representation is obtained since the explanation map is created at the size of the input image and not at the size of the activation map of the (especially the last) convolutional layer or the size of the output variable, since in the described alternative the explanation map would have to be scaled up to the size of the input image. The relevant edge and color information is preserved.

[0016] In one embodiment, it can be provided that the explanation graph is selected such that a cost function L (English: loss function) is optimized. In other words, the proposed method is based on an optimization problem with auxiliary conditions, i.e., for example

[0017] in: .

[0018] Equation (1) can advantageously be solved with the aid of a mask b, which can be multiplicative, for example, i.e. , where optimization , advantageously using the starting condition b(0) = 1. Alternatively, the mask b can also be additive, i.e. Here, b is optimized with the initial condition b(0) = 0. If b* is used to represent the optimized mask, the solution to the optimization problem of equation (1) is or .

[0019] In the optimization problem, we start from the image x and find an explanation graph or representation that satisfies certain properties. These properties can be represented particularly simply and flexibly using the cost function.

[0020] So the image x and representation The relationship between x Initial state (or by the corresponding starting conditions of mask b) and by the auxiliary conditions of the optimization. One possible implementation is to represent contains only the features that are also present in image x. Therefore, the initial state can be , an auxiliary condition that should advantageously be satisfied for each ReLU layer of the deep neural network can be , where h l is the activation of the first ReLU layer of the deep neural network (other activation functions can of course be used instead of the ReLU function).

[0021] The cost function advantageously has multiple (additive) components, namely L = L h +L w +L c Of course, these additive components can be weighted differently by means of parameters, but these parameters can also be integrated into the definition of these components. When solving the optimization problem of equation (1), these weighted parameters can optionally be adapted together. The first component can be selected , so that it characterizes how many pixels of the image are selected as relevant. For example, this component can be expressed by the L1 norm Alternatively, this component can be multiplied by the L1 norm of the mask b to be used . Thus, this component causes all irrelevant pixels to be pulled to an average value. Alternatively, irrelevant pixels can be set to any other value.

[0022] Alternatively or additionally, the cost function may have a second component which characterizes how well the classification result obtained for the explanation image corresponds to the classification result obtained for the image. This can be achieved by the component , where f describes the deep neural network or a part of the deep neural network. The parameters of the deep neural network are expressed as The output of the deep neural network, i.e., the classification is represented by or express.

[0023] The second component can be given by the KL divergence, namely .

[0024] Alternatively, the second component can be completed region by region, in particular pixel by pixel, by a function that characterizes how much the respective pixel contributes to the classification result obtained for the image. For example, the logit function of category i can be used This function is given by That is, the second component is given by the negative probability of class i (in the case of minimizing the probability of class i). Alternatively, the probability can be given by a function that determines all pixels that activate feature i. If represents the i-th feature map of the l-th layer, then the associated term of the cost function can be obtained by Other paradigms are possible, such as the L2 paradigm.

[0025] The auxiliary condition that the activation by the interpretation image does not substantially exceed the activation by the image can be achieved particularly easily by means of the optimization approach via the third component L in the cost function. c The third component penalizes such an excess, i.e., for example . Other paradigms are possible, such as the L2 paradigm.

[0026] Alternatively, the auxiliary condition is satisfied by activating the characteristic map when determining the interpretation map. is set to a predeterminable value, in particular to a maximum permissible value according to the auxiliary condition, for example by If a gradient-based optimization method is used, this can be done by Give the corresponding gradient in back propagation The notation [...] here usually means if the condition is met, then value = 1, if the condition is not met, then value = 0.

[0027] Alternatively, the auxiliary condition can be satisfied in the following manner: when using a gradient-based optimization method, if the explanation graph is used in a layer with an activation function (particularly a ReLU layer) of the deep neural network Activation Over the activations of the image x , i.e. , then the feature map of this layer Back-propagating gradients (ie the gradient present at the output of the activation function) is set equal to a predefinable gradient value, in particular equal to zero.

[0028] The interpretation image thus determined can now advantageously be determined and displayed, in particular next to the image. Thus, by means of a position-true resolution and the selection of only relevant pixels, a representation can be generated that can be examined by a human expert, in particular a doctor. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The embodiments of the present invention are explained in more detail below with reference to the accompanying drawings. In the accompanying drawings:

[0030] Figure 1 The structure of an embodiment of the present invention is schematically shown;

[0031] Figure 2 The flow chart shows the process of a method according to one aspect of the present invention. DETAILED DESCRIPTION

[0032] In a preferred embodiment of the present invention, Figure 1 A machine classification system 40 is shown, which is arranged to carry out the invention. The machine classification system 40 comprises a computer 45 and a machine-readable storage medium 46. The machine classification system further comprises a machine learning system 60, which determines an output signal y from an image x fed to it. The image x may be determined, for example, from a sensor signal S determined by an imaging sensor 30 via an optional input interface 50.

[0033] The machine learning system determines the classification of the image x with the help of a deep neural network f . Use the parameter The deep neural network f is parameterized and the parameters are stored in the parameter memory Q. The machine learning system 60 also uses, for example, Figure 2 The method shown in Explanation diagram and classification The output signal y is formed together, from which the control signal A is determined together with the image x, for example using the optional output interface 80. The control signal A may include, for example, the interpretation image ,Classification Then, the control signal is transmitted to the display unit 10, which includes, for example, two monitors 11 and 12. The image x is transmitted to the first monitor 11 and displayed, and the explanation image The data are transmitted to the second monitor 12 and displayed. Optionally, the classification data can also be displayed. .

[0034] Figure 2 shows the interpretation diagram used to determine First (1000) a deep neural network f is trained, for example, to be able to distinguish between different clinical images in an image. Then (1100) for an image x fed to the machine learning system 60, a classification of the image x is determined by means of the deep neural network f. These steps are not necessarily part of the method according to the invention, but rather determine the context of said method and may, for example, be performed in advance by a separate unit.

[0035] Then (1200) the initial value of the explanation map is set ,For example = 0. Then (1300) solves the optimization problem defined by equation (1), preferably using a gradient descent method, to determine the explanation graph .

[0036] Then (1400), the diagram will be explained The image x is fed to the monitors 11, 12 and displayed on the monitors 11, 12 as described. The display can be performed side by side or superimposed on each other. The method is thus terminated.

[0037] In another embodiment, the imaging sensor 30 is implemented in the manufacturing facility.The method can then be used to detectably display reject portions to a human operator via the display unit, thereby enabling the operator to identify via the display on the display unit 10 whether reject portions are actually present.

[0038] In another embodiment, any image acquisition unit is provided instead of the imaging sensor 30, which may be the imaging sensor 30 or, for example, load and provide images from a database. The method can then be used to check the operability of the deep neural network f, so that a human developer can analyze via the display on the display unit 10 which image areas lead to possible misclassifications.

[0039] In yet another embodiment, the imaging sensor 30 is part of a surveillance system that enables a human expert to identify, through display on the display unit 10, whether an abnormal scene (eg, an abandoned suitcase) identified by the machine learning system 60 inferentially actually exists.

[0040] It is understandable to those skilled in the art that the computer 46 may be provided by multiple computers, and the machine-readable storage medium 45 may be provided by multiple machine-readable storage media.

Claims

1. An explanation graph for determining image x In the explanation map, all pixels of the image x that are decisive for the classification of the image x determined by the deep neural network f are highlighted, wherein the explanation map is chosen so that the optimization problem with auxiliary conditions for the cost function L is optimized: in: The optimization problem is solved with the aid of a mask b, which is either multiplicative, i.e. s x = x·b, where the optimization b∈[0,1] is either additive, i.e., s x =x+b, where b is optimized with the initial condition b(0)=0; The cost function L has a first component L h , the first component represents how many pixels of the image x are selected as relevant; The cost function L has a second component L w , the second component represents the explanation graph The classification results obtained How much does it correspond to the classification result f(x|θ) obtained for the image x, so that if the explanation graph is transmitted to the deep neural network f for classification, then the explanation graph leads to the same classification result as the image x The cost function L has a third component L c , the auxiliary condition is realized by the third component, that is, for all layers in the deep neural network, the explanation graph Activation No more than the activation h of the image x l (x), wherein the third component L c Penalize such an excess so that the explanation diagram is chosen so that the feature map h of the deep neural network f l The explanation diagram Activation caused by No more than the activation h induced by the image x l (x).

2. The method according to claim 1, wherein: The first component L h Given by the L1 paradigm: h (s x )=||s x ||1.

3. The method according to claim 2, wherein: The second component L w By KL divergence (D KL ) is given.

4. The method according to claim 2, wherein: The second component L w Region by region, by characterizing the respective pixel pairs for the interpretation map The classification results obtained How much contribution the function has to complete.

5. The method according to claim 4, wherein: The second component L w This is done pixel by pixel through the function.

6. The method according to any one of claims 1 to 4, wherein: By means of the explanation diagram Activation No more than the activation h of the image x l The auxiliary condition (x) is satisfied by: When the feature map h l Activation in Set to a predeterminable value.

7. The method according to claim 6, wherein: The predeterminable value is set to the maximum permissible value according to the auxiliary condition 8. The method according to any one of claims 1 to 4, wherein: By means of the explanation diagram Activation No more than the activation h of the image x l The auxiliary condition of (x) is satisfied by using a gradient-based optimization method and if in layer l of the deep neural network the explanation graph Activation Over the activation h of the image x l (x), then the feature map h of this layer l Back-propagated gradient δ l Set to a predeterminable gradient.

9. The method according to claim 8, wherein: The predefinable gradient is set to zero.

10. A method for displaying an image, wherein: Determining and displaying an explanation diagram using the method according to any one of claims 1 to 9 11. The method according to claim 10, wherein: Also in the explanation diagram The image x is displayed next to it.

12. A computer program product arranged to perform the method according to any one of claims 1 to 11.

13. A machine-readable storage medium (45) having stored thereon a computer program product according to claim 12.

14. A machine sorting system (40) arranged to perform the method according to any one of claims 1 to 9.

15. A display system (13) arranged to perform the method according to any one of claims 10 or 11.

Images