Method and apparatus for identifying disturbances in an image

Abstract

Computer-implemented method and device for detecting interference in an image, wherein a class for an object from the image is determined based on the image (202), wherein a property for the object is determined based on the image by means of a prediction function, in particular an artificial neural network (206), wherein it is determined (210) based on the property and the class whether the image contains interference that is hostile to the prediction function, in particular the artificial neural network.

Description

Technical Field

[0001] The invention is based on a method and a device for processing images, in particular using an artificial neural network. Background Art

[0002] Y. Dong, H. Su, J. Zhu and F. Bao's Towards interpretable deep neural networks by leveraging adversarial examples (preprint website arXiv: 1708.05493, 2017) discloses some aspects that should be used to understand the output of artificial neural networks. To this end, the internal state of the artificial neural network is considered, which appears when analyzing an image containing interference that is hostile to the artificial neural network. The interference that is hostile to the artificial neural network in the image to be analyzed is used to achieve a trustworthy output of the artificial neural network, which is intended by the creator of the image but is wrong relative to the content shown in the image. For example, in an image, changes (such as noise) that are difficult to recognize or unrecognizable to the human eye when observing the image are integrated into the image to be analyzed, and due to the changes, the image is incorrectly classified by the artificial neural network.

[0003] It is desirable to reliably detect such disturbances in the image. Summary of the invention

[0004] This is achieved by the subject matter of the independent claims.

[0005] In machine learning-based methods for image recognition, the input Assign output ,in, X represents the input space, and Y Represents the output space.

[0006] Models for image recognition include compatibility functions and the prediction function ,in, .

[0007] Prediction Function Including the model parameter vector w, the compatibility function F represents the given parameter vector In the case of A measure of compatibility.

[0008] The method is described below using the example of an artificial neural network. The method can be applied to any function that is parameterized and differentiable with respect to w.

[0009] From having and A training set of N input / output pairs To begin, in order to train the model, a prediction with the following objective is performed: Learning the StructuredJoint Embedding function , in such a way that the risk from experience Minimize, where When measuring costs, When characterizing applicable categories, it is important to predict , the cost occurs. For example, for ,use , and for ,use , that is, 0 / 1 loss.

[0010] For the input space , for example using input embedding , that is, the input space Mapping to the transformed input For the output space , for example using the output embedding , that is, the transformed output Mapping to output space Y For training the model, in this case, the transformed input and the transformed output .

[0011] Attribute Label Embedding is a method to embed a certain label. For the output embedding , from the number E attributes Starting from, the number E attributes describe the number C categories .value Representation attributes For Category y The measure of the belongingness of . Arranged row by row, all C categories Y Represents the dimension for attribute A The output embedding matrix .

[0012] If the parameter vector w Embedding based on input The dimension D and the output embedding The dimension E is expressed as D×E matrix W , then the compatibility function can be embedded according to the input and output embedding and is represented as .

[0013] For example, by using a binary value Setting properties Belongs to category Instead of using binary values, confidence values ​​from a continuous value range can be used, which continuously describe a measure of belonging. Starting from the following data set, which contains the characteristic image A collection of values, such as Assigns a binary value (ie, {0, 1}) or a value from the continuous range of values.

[0014] The aspects described below are not limited to monitored learning but can also be applied in conjunction with unmonitored or graded learning, in particular on a text-based basis.

[0015] These and additional aspects are described, for example, in Z. Akata, S. Reed, D. Walter, H. Lee, and B. Schiele, “Evaluation of output embeddings for fine-grained image classification,” in Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on IEEE, 2015, pp. 2927-2936.

[0016] The described structured joint embedding algorithm is particularly suitable for predicting attributes with the aid of zero-shot learning methods, with the aid of which a classifier is trained for classifying images into categories, in which the training images and the images to be classified are assigned to two mutually exclusive sets of categories.

[0017] The target learning function is defined as

[0018]

[0019] in,

[0020]

[0021] Among them, the stochastic gradient descent method is used to sample Start by determining the parameters in the matrix W and find the highest evaluated class If from sampling If the category is inconsistent with the applicable category, the weight is updated using the following equation, for example

[0022]

[0023] in,

[0024] Calculation steps

[0025] The learning rate at computation step t.

[0026] In the image When classifying, Represents attribute prediction, that is, for an image The predicted attributes , the image Assigned to category .

[0027] The result of the classification relates to the following objects from the image: the object is contained in one or more image segments (i.e., bounding boxes) of the image. The bounding box defines the following geometric contour in the image: the object is located in the geometric contour. In order to locate the bounding box in the image, a fast region-based convolutional neural network (Faster-Region-Based-Convolutional-Neural Network, Faster-RCNN) is used, for example, and the fast region-based convolutional neural network is trained with the Visual Genome data set. This approach is described, for example, in "Bottom-up and top-down attention for image captioning and vqa" by P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould and L. Zhang (preprint website arXiv:1707.07998, 2017).

[0028] Additionally, the bounding box assigned to the object is assigned an attribute In this example, the attribute vector A includes the attributes For this purpose, we use a margin-based ranking loss. A trained artificial neural network in which the pairwise assignment of attributes to bounding boxes that is not applicable in the analyzed image is Negative number of features , where the pairwise assignment of attributes to bounding boxes that applies in the analyzed image Number of positive features For a bounding box, for example, determine all positive attributes , and treat the remaining attributes as negative attributes Use. From the bounding boxes located in the image, the bounding box with the highest number of features is determined and output, for example, together with the image and the bounding boxes. For example, the order of the bounding boxes is set according to the number of features. This method for positioning is described, for example, in "Grounding visual explanations" by L. A. Hendricks, R. Hu, T. Darrell and Z. Akata (preprint site arXiv:1807.09685, 2018).

[0029] In the following aspects, a distinction is made between clean images and hostile images. Hostile images differ from clean images in that interference is present. The interference is preferably not recognizable or can only be recognized with difficulty for a human observer. The interference is a hostile interference for the artificial neural network, which leads to a reliable but incorrect classification of the hostile image. This means that the interference is designed so that the artificial neural network classifies the hostile image into an existing but incorrect category for the image.

[0030] For example, by giving the clean image the method described in "Adversarial examples in the physical world" by A. Kurakin, I. Goodfellow and S. Bengio (preprint website arXiv:1607.02533, 2016) x Add the disturbance. For this purpose, the following optimization equation is used:

[0031]

[0032] in,

[0033] The steps in this calculation are

[0034] Clean images,

[0035] The disturbed image in step n,

[0036] The gradient of the cost function for the perturbed image in step n,

[0037] function, which restricts it from x to The maximum distance of the independent variable,

[0038] The true category of image x,

[0039] Step parameters, such as ,

[0040] Optional hyperparameters.

[0041] The hostile image is determined by iteratively computing the optimization equation for a selectable number of iterations.

[0042] Further aspects of this type of interference are described, for example, in “Intriguing properties of neural networks” by C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow and R. Fergus (preprint website Arxiv: 1312.6199, 2013).

[0043] The following aspects may be applied to such hostile images as well as other perturbations applied in hostile images.

[0044] Not only for the clean image, but also for the hostile image, a bounding box or multiple bounding boxes and the attribute with the maximum number of features for each bounding box can be determined as previously described. The attribute or the category assigned to the attribute or both can be included in the output together with the bounding box. They can be visualized, that is, they can be displayed together with the analyzed image. If not only clean images but also hostile images are analyzed, these images can also be shown side by side or in sequence so that a person can evaluate the quality of the positioning of the bounding box in relation to the objects contained in the two images, or in relation to the association of the attributes shown with the bounding box, or in relation to the category for the bounding box shown with the bounding box. Alternatively, a trained artificial neural network can be used for the evaluation.

[0045] The computer-implemented method for identifying interference in an image provides for determining a class for an object from the image based on the image, wherein a property for the object is determined based on the image by a prediction function, in particular an artificial neural network, and wherein it is determined based on the property and the class whether the image contains interference that is hostile to the prediction function, in particular the artificial neural network. By predicting the property for the same object that is to be classified into the class, the output of the artificial neural network for the object can be analyzed to determine whether interference that is hostile to the artificial neural network is contained in the analyzed image.

[0046] In a driver assistance system, for example, the following feature is achieved by prediction of attributes (especially under the assumption that the attacker has no access to the attribute prediction itself): the attacker deceives the object detector that determines the category for the object from the image, so that the object detector assigns the object to the wrong category. The object is, for example, an illustration of a parking sign according to German street traffic regulations. Applicable categories are, for example, parking signs, and inapplicable categories are, for example, people. Attribute prediction assigns at least one attribute to an object identified as a person. Attributes represent concepts that are easy to understand for people, for example, as word vectors. Other representations of concepts that are easy to understand for people or attributes that are not easy to understand for people can also be set. For example, the attributes "octagon", "white text", "red background" are assigned, which are not suitable for people, but may be suitable for parking signs. Attributes that are suitable for two categories (that is, people and parking signs) can also be assigned. Potential deception attempts can be identified based on attributes that are not suitable for the predicted category.

[0047] Other application areas are, for example, (semi) autonomous robots, which also have to recognize objects and can be deceived. Here, for example, when a potential attack is recognized, a fallback mode is also switched, which for example provides for remote control of a person.

[0048] In one aspect of the method, it is provided that the property is determined by a first artificial neural network, wherein it is determined by a second artificial neural network whether interference that is hostile to the artificial neural network is contained in the image. This allows automated detection of attempts at deception. Advantageously, the attacker has no knowledge of the existence of the second neural network or has no access to it. By means of corresponding technical measures, which render the second neural network inaccessible, for example to a potential attacker, interference is avoided in such a way that both networks can be deceived at the same time.

[0049] Advantageously, it is provided that the properties for the object are outputted with the image and / or the region of the image for the object for image display. Thus, the positioning quality of the region related to the object or the relevance of the properties for the object can be evaluated by a person on the display screen.

[0050] Advantageously, provision is made for assigning a region in the image to the attribute, wherein a measure of the positioning of the region in relation to the object is determined, and wherein, based on the measure of the positioning, it is determined whether the image contains disturbances that are hostile to the artificial neural network. The measure of the positioning indicates, for example, how well the region is assigned to the object. For example, the artificial neural network is trained with a training data set, which is obtained as an input bounding box and, for example, manually annotated attributes, and the artificial neural network learns to predict whether the bounding box is well positioned. The manually annotated bounding boxes of the training data set are considered to be well positioned. Poorly positioned bounding boxes can be generated by random movement of these bounding boxes, which can also be used for the training. The artificial neural network trained in this way is applied to the predicted bounding boxes and checked whether the bounding boxes can be classified as well positioned. Poorly positioned bounding boxes indicate that the image has disturbances.

[0051] Preferably, provision is made to determine at least one first attribute for a first image, wherein a first distance is determined between at least one first attribute for the first image and at least one second attribute predicted by the artificial neural network for an undisturbed second image, wherein the first distance is compared with a threshold value in the comparison, and wherein it is determined based on the result of the comparison whether the first image contains disturbances that are hostile to the artificial neural network. An undisturbed second image is an image from which it is known that it does not contain hostile disturbances. The threshold value can be selected so that the disturbances can be identified. For a first image with hostile disturbances, the distance becomes, for example, greater than the threshold value. For a first image without hostile disturbances, the distance is less than or equal to the threshold value. For example, determining the first attribute and the second attribute . Subscript Relating to the category applicable to the first image . Subscript y At least one category that is not applicable to the first image y . and represents the attribute vector. For example, determine the first spacing .

[0052] In this aspect, the threshold is preferably defined based on a second distance between a plurality of attributes for a category applicable to the first image and a plurality of attributes for at least one category not applicable to the first image. , Second Attribute , for categories that are not applicable to the first image y Multiple attributes of and for the category applicable to the first image Multiple attributes of . The second spacing is determined by When the first spacing Smaller than the second spacing The effect of classification can be seen in this example.

[0053] Alternatively, the threshold value may be defined as a second distance between the at least one first attribute for the first image and at least one attribute predicted by the artificial neural network for an undisturbed second image from a class that is not applicable to the first image. Determine the second spacing . In this example, the corresponding attribute vector is represented. In this aspect, for example, when the first spacing Equal to the second spacing or approximately equal to the second spacing except for the tolerance , we can see the effect of classification.

[0054] Preferably, in a further aspect, it is provided that the at least one attribute for the image is determined, wherein a first distance between the at least one attribute for the image and a plurality of attributes for a category that is not applicable to the image is determined, wherein the first distance is compared with a threshold value in the comparison, and wherein it is determined based on the result of the comparison whether the image contains interference that is hostile to the artificial neural network. y Multiple attributes of and the first spacing The threshold may be selected such that when there is an image with hostile interference, the first spacing exceeds the threshold.

[0055] In this respect, it is preferably provided that the threshold is defined as a function of a second distance between the at least one attribute for the first image and a plurality of attributes for a category applicable to the first image. Multiple attributes of and the second spacing For example, change the first spacing With the second spacing For images containing adversarial perturbations, the attributes are closer to the attributes of the inapplicable class and farther away from the attributes of the applicable class.

[0056] Preferably, the driver assistance system is operated in a first operating mode or in a second operating mode different from the first operating mode, depending on the result of the recognition, wherein in the second operating mode human intervention is required or taken into account, or wherein the result of the object recognition not based on machine learning is used or is weighted more heavily than in the first operating mode. This forms a driver assistance system that is robust to such disturbances.

[0057] A corresponding device for detecting a fault comprises a computing device, in particular at least one microcontroller or processor, and a memory, wherein the computing device is designed to carry out the described method.

[0058] A driver assistance system for a vehicle comprises the described device, an interface for connecting to a camera for detecting an image and an interface for controlling the vehicle according to a result of recognition or for outputting the result of recognition and / or a human-machine interface for outputting the result of recognition.

[0059] Furthermore, a computer program having computer-readable instructions may be provided, which, when executed, implement the described method by a computer. Furthermore, a computer program product may be provided, the computer program product comprising a computer-readable memory, the computer program being stored in the computer-readable memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Further advantageous embodiments emerge from the following description and the accompanying drawings. In the drawings:

[0061] Figure 1 A schematic illustration of a device for detecting interference is shown,

[0062] Figure 2 The steps in a method for detecting interference are shown.

[0063] Figure 3 A schematic representation showing some parts of the method is shown. DETAILED DESCRIPTION

[0064] Figure 1The device 102 is shown schematically. The device 102 includes a computing device 104, which is configured to identify interference that is hostile to the artificial neural network in the image. In one aspect, a driver assistance system for a vehicle includes the device 102. The driver assistance system includes an interface 106 for connecting to a camera 108 for detecting an image and an interface 110 for controlling the vehicle according to the result of the recognition or for outputting the result of the recognition and / or a human-machine interface 112 for outputting the result of the recognition. They can be connected via a data line 114 for communication. The driver assistance system is configured to be regulated according to the result of the recognition so as to operate in a first operating mode or in a second operating mode. In the first operating mode, in this example, the image is used as an undisturbed image for the driver assistance function. In the second operating mode, the image is treated as a disturbed or potentially disturbed image.

[0065] Refer to the following Figure 2 A computer-implemented method for identifying interference in an image is described. The method begins, for example, when an image to be analyzed is prepared.

[0066] After the start, in step 202 a class for an object from the image to be analyzed is determined based on the image.

[0067] Next, in step 204, a property for the object is determined based on the image by a prediction function (in this example, an artificial neural network). In this example, the property is determined by a first artificial neural network.

[0068] Next, in step 206, regions in the image are assigned to the attributes.

[0069] Next, in step 208, it is determined based on the attributes and the category whether the image contains interference that is hostile to the artificial neural network. If no interference is detected in the image, step 210 is implemented. Otherwise, step 212 is implemented. It can be provided that a second artificial neural network is used to determine whether interference that is hostile to the artificial neural network is contained in the image. The attributes for the object can also be output with the image and / or a region of the image for the object for image display. A human observer can then determine whether it is an image with interference or an image without interference.

[0070] Provision may be made to determine a measure of the localization of the region associated with the object to which the attribute is assigned. In this respect, in addition or alternatively, it may be determined whether the image contains disturbances hostile to the artificial neural network based on the measure of the localization.

[0071] In one aspect, in step 204, additionally or alternatively, at least one first attribute for the first image is determined In this aspect, determining at least one first attribute and at least one second attribute The first spacing between . At least one second attribute For example, a prediction is made by the artificial neural network for the undisturbed second image.

[0072] In this aspect, in step 208, in addition or alternatively, the first distance A comparison is made with a threshold value and, depending on the result of the comparison, it is determined whether the first image contains interference that is hostile to the artificial neural network.

[0073] For example, according to the category applicable to the first image Multiple attributes of and for at least one category that is not applicable to the first image Multiple attributes of The second spacing between To define the threshold.

[0074] Alternatively, for example, according to at least one first attribute and at least one second attribute The second spacing between To define the threshold. For example, assigned to a category that is not applicable to the first image For example, the artificial neural network predicts at least one second attribute for the undisturbed second image. Alternatively or additionally, at least one property for the image is determined in step 206 In this aspect, determining at least one attribute and for categories that are not applicable to the first image Multiple attributes of The first spacing between .

[0075] In step 208, in this respect, alternatively or additionally, the first distance The image is compared with a threshold value, and it is determined whether the image contains interference that is hostile to the artificial neural network according to the result of the comparison.

[0076] In this aspect, for example, according to at least one attribute of the first image and for the category applicable to the first image Multiple attributes of The second spacing between To define the threshold.

[0077] In step 210 , in the present example, the driver assistance system is operated in a first operating mode 212 depending on the result of the identification.

[0078] In step 212 , in the present example, the driver assistance system is operated in a second operating mode that is different from the first operating mode.

[0079] In the second operating mode, for example, human intervention is required for a driver assistance function or is taken into account for this function. In addition or as an alternative thereto, it can be provided that the result of the object recognition not based on machine learning is used for this function or that the result is weighted more heavily for this function than in the first operating mode.

[0080] Next, in step 214, it is checked whether there are further images for the analysis. If this is the case, step 202 is carried out. Otherwise, the operating mode is maintained until further images arrive. For this purpose, step 214 is carried out again.

[0081] Figure 3 Some aspects of the method are schematically shown.

[0082] Device 102 is designed to process both undisturbed image 302 and also a disturbed image 306 which is based on undisturbed image 302 and is changed by disturbance 304 .

[0083] The undisturbed image 302 forms the input for the artificial neural network X In one aspect, the artificial neural network determines at least one applicable attribute for the undisturbed image 302 For example, in Figure 3 Possible values ​​308a, ..., 308e for this are shown in FIG. The undisturbed image 302 results at the output Y with at least mainly suitable properties The result 310 is identified as an image without interference.

[0084] In another aspect, the artificial neural network determines at least one inapplicable attribute for the disturbed image 304 For example, in Figure 3 Possible values ​​312a, ..., 312e for this are shown in FIG. The disturbed image 306 results in at least one or at least predominantly unsuitable property. The result 314 is identified as an image containing interference.

[0085] For training, both the undisturbed images 302 and the disturbed images 306 generated therefrom and their respective classification as disturbed or undisturbed are used. The training can provide that the disturbed images 306 are generated from the undisturbed images 302 as described above. In the training, the parameters of the parameter vector w are learned as described above. The artificial neural network trained in this way can then be used to recognize images containing disturbances that are hostile to the artificial neural network.

[0086] In addition to the use in driver assistance systems, the use may also be provided in any other system in which it is desirable to reliably detect whether an image to be evaluated contains such interference.

Claims

1. A computer-implemented method for identifying interference in an image, characterized in that The method comprises determining a class of an object from the image (202), wherein a property of the object is determined from the image by an artificial neural network (206), and wherein it is determined (210) based on the property and the class whether the image contains disturbances that are hostile to the artificial neural network, wherein the hostile disturbances are designed such that the artificial neural network classifies the image into a class that exists but is incorrect for the image, wherein at least one first attribute of the first image is determined (206), wherein determining and at least one attribute predicted by the artificial neural network for the undisturbed second image The first distance between z ), wherein in the comparison the first distance (dz) is compared with a threshold value, and wherein it is determined based on the result of the comparison whether the first image contains interference that is hostile to the artificial neural network (210), Wherein, according to the category (y n )'s multiple attributes (φ(y n )) and a second distance (d) between a plurality of attributes (φ(y)) of at least one category (y) that is not applicable to the first image zφ ) to define the threshold.

2. A computer-implemented method for identifying interference in an image, characterized in that The method comprises determining a class of an object from the image (202), wherein a property of the object is determined from the image by an artificial neural network (206), and wherein it is determined (210) based on the property and the class whether the image contains disturbances that are hostile to the artificial neural network, wherein the hostile disturbances are designed such that the artificial neural network classifies the image into a class that exists but is incorrect for the image, wherein at least one attribute of the image is determined (206), wherein determining and a first distance (d ) between a plurality of attributes (φ(y)) of a class (y) that is not applicable to the image y ), wherein the first distance (dy) is compared with a threshold value in the comparison, and wherein it is determined based on the result of the comparison whether the image contains interference that is hostile to the artificial neural network (210).

3. The method according to claim 1 or 2, characterized in that: The property is determined by a first artificial neural network, wherein it is determined by a second artificial neural network whether the disturbance hostile to the artificial neural network is contained in the image (210).

4. The method according to claim 1 or 2, characterized in that: The properties of the object are output with the image and / or a region of the image that is specific to the object for image display.

5. The method according to claim 1 or 2, characterized in that: A region in the image is assigned to the attribute (208), a measure of the location of the region relative to the object is determined, and a determination is made based on the measure of the location whether the image contains disturbances that are hostile to the artificial neural network (210).

6. The method according to claim 1, wherein: According to the at least one first attribute and at least one second attribute (Z) predicted by the artificial neural network for an undisturbed second image from a class that is not applicable to the first image m，y ) between the second spacing (d Zm ) to define the threshold.

7. The method according to claim 2, characterized in that According to the at least one attribute of the image and the category applicable to the image (y n )'s multiple attributes (φ(y n ))The second spacing between To define the threshold.

8. The method according to claim 1 or 2, characterized in that: Depending on the result of the recognition, a driver assistance system is operated in a first operating mode (212) or in a second operating mode (214) different from the first operating mode, wherein in the second operating mode human intervention is required or taken into account, or wherein a result of an object recognition not based on machine learning is used or is weighted more heavily than in the first operating mode (214).

9. A device (102) for identifying interference, characterized in that The device (102) comprises a computing device (104) which is designed to carry out the method according to any one of claims 1 to 8.

10. A driver assistance system (100) for a vehicle, characterized in that: The driver assistance system (100) comprises a device (102) according to claim 9, an interface (106) for connecting to a camera (108) for detecting the image, and an interface (110) for controlling the vehicle according to the result of the recognition or for outputting the result of the recognition and / or a human-machine interface (112) for outputting the result of the recognition.

11. A computer program product, characterized in that The computer program product comprises computer-readable instructions, and when the computer-readable instructions are executed, the method according to any one of claims 1 to 8 is implemented by a computer.

12. A computer readable memory, characterized in that: A computer program is stored in the computer-readable memory. The computer program includes computer-readable instructions. When the computer-readable instructions are executed, the method according to any one of claims 1 to 8 is implemented by a computer.

Images