Image recognition model training method, recognition method, electronic device and medium

Abstract

The present disclosure provides a training method, a recognition method, an electronic device and a medium for an image recognition model. The method can be applied to the field of artificial intelligence. The training method includes: adjusting the gradient value of at least one pixel according to the importance corresponding to each of at least one pixel in a first sample image to obtain a second sample image; inputting the second sample image into a deep learning model to obtain a sample recognition class; adjusting the model parameters of the deep learning model according to the sample recognition class to obtain an image recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a training method, a recognition method, an electronic device and a medium for an image recognition model. Background Art

[0002] At present, deep learning models are closely integrated with the field of image recognition and are widely used in the fields of segmented image classification, object detection, face recognition, and autonomous driving. The security issues of deep learning models are also receiving increasing attention, for example, the adversarial attack problem of deep learning models.

[0003] In the adversarial attack problem, the attacker adds a tiny special perturbation to the original image input by the deep learning model to obtain an attack image. This can deceive the deep learning model based on the attack image and cause the deep learning model to classify incorrectly.

[0004] The human eye may not be able to quickly identify small special perturbations of the attack image, and may not even be able to detect the difference between the original image and the attack image. Adversarial attack problems may have negative effects in special circumstances. For example, special processing of road signs may mislead autonomous driving to make wrong decisions. Therefore, in order to solve the adversarial attack problem faced by deep learning models, a defense method is needed to improve the defense performance of deep learning models against adversarial attack problems, thereby improving the security of deep learning models. Summary of the invention

[0005] In view of this, the present disclosure provides a training method, a recognition method, an electronic device and a medium for an image recognition model, in order to partially solve at least one of the above-mentioned technical problems.

[0006] In one aspect of the present disclosure, a method for training an image recognition model is provided, comprising:

[0007] According to the importance corresponding to each of the at least one pixel in the first sample image, the gradient value of the at least one pixel is adjusted to obtain a second sample image;

[0008] Input the second sample image into the deep learning model to obtain a sample recognition class; and

[0009] According to the above sample recognition class, the model parameters of the above deep learning model are adjusted to obtain a trained image recognition model.

[0010] According to an embodiment of the present disclosure, the model parameters of the deep learning model are adjusted according to the sample recognition class to obtain the image recognition model, including:

[0011] Determine a sample identification class vector according to the preset centroid and the sample identification class, wherein the sample identification class vector represents a directed relationship between the preset centroid and the sample identification class; and

[0012] The model parameters of the above-mentioned deep learning model are adjusted according to the above-mentioned sample recognition class vector to obtain the above-mentioned image recognition model.

[0013] According to an embodiment of the present disclosure, the step of adjusting the gradient value of at least one pixel in the first sample image according to the importance corresponding to each pixel to obtain the second sample image includes:

[0014] Determine a first pixel set and a second pixel set according to the importance corresponding to at least one pixel in the first sample image, wherein the first pixel set includes at least one first pixel, the second pixel set includes at least one second pixel, and the at least one first pixel corresponds to a first gradient value, and the at least one second pixel corresponds to a second gradient value;

[0015] Determine at least one target pixel from the first pixel set;

[0016] Adjusting the first gradient value corresponding to each of the at least one target pixel to obtain a third gradient value corresponding to each of the at least one target pixel; and

[0017] The second sample image is obtained according to the second gradient value corresponding to each of the at least one second pixel, the third gradient value corresponding to each of the at least one target pixel, and the first gradient value corresponding to each of the at least one other pixel, wherein the other pixel is any pixel in the first pixel set except the at least one target pixel.

[0018] According to an embodiment of the present disclosure, determining the first pixel set and the second pixel set according to the importance corresponding to each of at least one pixel in the first sample image includes:

[0019] For each pixel in the at least one pixel in the first sample image, if it is determined that the importance corresponding to the pixel satisfies a preset importance condition, determining the pixel as the first pixel; and

[0020] When it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition, the pixel is determined as the second pixel.

[0021] According to an embodiment of the present disclosure, the training method further includes:

[0022] Determine a mean value and a standard deviation corresponding to the first sample image according to the gradient values ​​corresponding to at least one pixel in the first sample image;

[0023] Determine the importance threshold based on the preset hyperparameters and the above standard deviation;

[0024] For each pixel in at least one pixel in the at least one sample image, determine the absolute value of the difference between the average value and the gradient value of the pixel to obtain a reference value corresponding to the pixel;

[0025] In the case where it is determined that the reference value corresponding to the pixel is less than or equal to the importance threshold, determining that the importance corresponding to the pixel satisfies the preset importance condition; and

[0026] When it is determined that the reference value corresponding to the pixel is greater than the importance threshold, it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition.

[0027] According to an embodiment of the present disclosure, the model parameters of the deep learning model are adjusted according to the sample recognition class vector to obtain the image recognition model, including:

[0028] Determine the modulus value of the above sample identification class vector;

[0029] Determine the angle between the sample recognition class vector and the normal vector of the preset decision boundary;

[0030] Based on the loss function, a loss function value is obtained according to the modulus value, the angle value and the sample identification class vector; and

[0031] The model parameters of the above-mentioned deep learning model are adjusted according to the above-mentioned loss function value to obtain the above-mentioned image recognition model.

[0032] According to an embodiment of the present disclosure, the above loss function is:

[0033]

[0034] in, is the loss function; is a loss function term associated with the angle value; is a loss function term associated with the sample identification class vector; is the cross entropy loss function between the label and the sample identification class; are the proportion coefficients of each item in the loss function respectively.

[0035] Another aspect of the present disclosure provides an image recognition method, comprising:

[0036] acquiring a target image; and

[0037] Input the above target image into the image recognition model to obtain the target recognition class;

[0038] The above image recognition model is trained using any of the above methods.

[0039] Another aspect of the present disclosure provides an electronic device, including:

[0040] one or more processors;

[0041] a memory for storing one or more instructions,

[0042] When the one or more instructions are executed by the one or more processors, the one or more processors implement any of the methods described above.

[0043] Another aspect of the present disclosure provides a computer-readable storage medium, characterized in that executable instructions are stored thereon, and when the executable instructions are executed by a processor, the processor implements any of the above methods.

[0044] Based on the above technical solutions, it can be seen that the embodiments of the present disclosure have the following beneficial effects compared with the prior art:

[0045] During the training process, importance sampling is performed on the first sample image based on the importance corresponding to each pixel, and the gradient value of at least one pixel is adjusted according to the importance to obtain a second sample image. Due to the adjustment of the gradient value of the pixel, the feature extraction process of the deep learning model is affected accordingly in the second sample image, that is, the feature vector of the second sample image with the pixel gradient value adjusted may produce misclassification when the deep learning model is classified. The second sample image is processed by the deep learning model to obtain a sample recognition class, and the model parameters of the deep learning model are adjusted according to the sample recognition class to obtain a trained image recognition model. The trained image recognition model can recognize and classify images with adjusted pixel gradient values, and can achieve the correct recognition class even when the pixel gradient value is adjusted. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure schematically shows a flow chart of the problem of deep learning model coping with adversarial attacks to be solved by the embodiments of the present disclosure.

[0047] Figure 2 The flowchart of the training method of the image recognition model according to the embodiment of the present disclosure is schematically shown.

[0048] Figure 3The figure schematically shows a flow chart of adjusting the model parameters of a deep learning model according to a sample recognition class to obtain an image recognition model according to an embodiment of the present disclosure.

[0049] Figure 4 The flowchart of obtaining the second sample image by adjusting the gradient value of at least one pixel according to the importance corresponding to each of at least one pixel in the first sample image according to an embodiment of the present disclosure is schematically shown.

[0050] Figure 5 The following is a schematic diagram schematically showing a method for determining the importance of image pixels according to an embodiment of the present disclosure.

[0051] Figure 6 A schematic diagram of a training loss function according to an embodiment of the present disclosure is schematically shown.

[0052] Figure 7 The figure schematically shows a flow chart of a method for training an image recognition model according to another embodiment of the present disclosure.

[0053] Figure 8 A schematic diagram of a flow chart of a method for identifying adversarial interference images according to an embodiment of the present disclosure is shown.

[0054] Fig. 9 A block diagram of an electronic device applied to a method for recognizing adversarial interference images and a method for training an image recognition model according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0056] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0057] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0058] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0059] In the process of implementing the concept of the present disclosure, it is found that there are at least the following problems:

[0060] In the adversarial attack problem, the attacker adds a tiny special perturbation to the original image input by the deep learning model to obtain an attack image. This can deceive the deep learning model based on the attack image and cause the deep learning model to classify incorrectly.

[0061] Figure 1 The figure schematically shows a flow chart of the problem of deep learning model coping with adversarial attacks to be solved by the embodiments of the present disclosure.

[0062] like Figure 1 As shown in the figure, taking a handwriting recognition image "7" (original image) in the handwriting recognition dataset as an example, after the attacker adds a small perturbation to the original image, the generated attack image can still recognize the number "7" when observed by the human eye, and does not affect human recognition behavior. When the original image is input into the defensive deep learning model and the defenseless deep learning model respectively, both can accurately recognize the information on the original image (the number "7"), and when the attack image is input into the defensive deep learning model and the defenseless deep learning model respectively, the defensive deep learning model can obtain the correct classification result, while the defenseless deep learning model will produce classification errors. Therefore, in practical applications, adversarial defense training is required for deep learning models in order to ensure and improve the security of deep learning models.

[0063] In the related art, adversarial attack methods may include white-box attacks and black-box attacks.

[0064] A white-box attack can be an attack method in which the attacker fully understands all the information of the attacked deep learning model, for example, it may include the neural network structure, weight information, etc.

[0065] A black-box attack can be an attack method in which the attacker does not understand the target deep learning model being attacked.

[0066] In order to at least partially solve the existing technical problems, the present disclosure provides a training method, a recognition method, an electronic device and a medium for an image recognition model, which can be applied to the field of artificial intelligence.

[0067] like Figure 2 As shown, the training method of the image recognition model may include operations S210 to S230.

[0068] In operation S210, a gradient value of at least one pixel is adjusted according to the importance corresponding to each pixel in the first sample image to obtain a second sample image.

[0069] In operation S220, the second sample image is input into the deep learning model to obtain a sample recognition class.

[0070] In operation S230, model parameters of the deep learning model are adjusted according to the sample recognition class to obtain a trained image recognition model.

[0071] According to the embodiment of the present disclosure, the importance corresponding to each pixel can correspond to the difference between at least one pixel in the first sample image and the average value of the pixel gradient in the first sample image. The importance corresponding to each pixel can be used to obtain the degree to which at least one pixel in the first sample image affects the recognition and classification of the first sample image. According to the importance, the gradient value of at least one pixel is adjusted to obtain the second sample image. According to the embodiment of the present disclosure, the deep learning model can obtain the characteristics of the training set by using the second sample image (i.e., the attack image), and obtain the contribution of a pixel block composed of a certain part of the pixels in the image in the sample training set to different prediction results, so as to learn the influence of the pixel gradient on the image recognition and classification according to the contribution. Through such an attack image, the deep learning model trained based on the same sample training set can complete the migration attack by using the commonality of the sample training set. For example, the attacker can use another alternative deep learning model to generate the attack image, and then use the attack image to attack the deep learning model. Therefore, the characteristics of the attack image corresponding to the black box attack can be learned by training the deep learning model, and the recognition and classification can still be completed in the presence of the black box attack.

[0072] According to the embodiment of the present disclosure, the first sample image may be an existing image recognition dataset. Those skilled in the art may select an image recognition dataset according to actual needs, which will not be elaborated herein.

[0073] According to the embodiment of the present disclosure, the first sample image can be processed to obtain a second sample image, which is used as a sample attack image. In practical applications, the attacker can use the same sample training set as the image recognition model to generate sample attack images through other deep learning models, which will not be repeated here.

[0074] According to the embodiment of the present disclosure, since the gradient corresponding to the pixel block can affect the prediction result, the points in the first sample image that are more important and less important to the prediction result can be screened out by judging the importance, and then the gradient value of at least one pixel is adjusted according to the importance to obtain the second sample image. Due to the adjustment of the gradient value of the pixel in the second sample image, the feature extraction process of the deep learning model is affected accordingly, that is, the classification of the deep learning model may be wrong.

[0075] According to an embodiment of the present disclosure, a deep learning model is trained using a second sample image. The training may include forward propagation, calculation of loss function values, calculation of weight gradients based on back propagation, updating of neural network weights according to loss function values, etc., and a trained image recognition model may be obtained. The trained image recognition model can recognize and classify images in which image pixel gradient values ​​are adjusted, and has the ability to recognize black box attack samples.

[0076] According to an embodiment of the present disclosure, the second sample image is input into the deep learning model as a sample attack image, so that the deep learning model learns the aggressive characteristics of the sample attack image, thereby enabling the trained image recognition model to learn and recognize the characteristics of the actual attack image obtained based on the commonality of the sample training set, thereby achieving the correct classification operation.

[0077] According to an embodiment of the present disclosure, during a training process, importance sampling is performed on a first sample image based on the importance corresponding to each pixel, and the gradient value of at least one pixel is adjusted according to the importance, to obtain a second sample image. Due to the adjustment of the gradient value of the pixel, the second sample image affects the feature extraction process of the deep learning model, that is, the feature vector of the second sample image with the adjusted pixel gradient value may produce an incorrect classification when the deep learning model is classified. The second sample image is processed by a deep learning model to obtain a sample recognition class, and the model parameters of the deep learning model are adjusted according to the sample recognition class to obtain a trained image recognition model. The trained image recognition model can perform recognition and classification on images with adjusted image pixel gradient values, and can achieve the correct recognition class even when the pixel gradient value is adjusted.

[0078] Reference below Figure 3~Figure 7 , combined with specific embodiments Figure 2The training method of the image recognition model shown in the figure is further explained according to the embodiment of the present disclosure.

[0079] Figure 3 The figure schematically shows a flow chart of adjusting the model parameters of a deep learning model according to a sample recognition class to obtain an image recognition model according to an embodiment of the present disclosure.

[0080] like Figure 3 As shown, in operation S230, operations S331 to S332 may be included.

[0081] In operation S331, a sample identification class vector is determined according to a preset centroid and a sample identification class, wherein the sample identification class vector represents a directed relationship between the preset centroid and the sample identification class.

[0082] In operation S332, model parameters of the deep learning model are adjusted according to the sample recognition class vector to obtain an image recognition model.

[0083] According to the disclosed embodiment, during the training method of the image recognition model, it is desired to control the range of the sample recognition class vector within a preset region, so that during the loss function calculation process, when the loss function is calculated for the sample recognition class, the change of the feature function caused by the attack image will affect the sample recognition class, resulting in recognition class errors. Thus, a preset centroid can be pre-determined for the location of the sample recognition class, so that the sample recognition class obtained by the trained image recognition model is as close as possible to the correct sample recognition class range corresponding to the preset centroid.

[0084] According to the embodiments of the present disclosure, based on the directed relationship between the preset centroid and the sample recognition class, the sample recognition class vector can be determined, and an improved loss function is constructed based on the range of the sample recognition class vector, so that the range of the sample recognition class vector is controlled within a preset area, and the image recognition model trained based on the improved loss function can maintain a certain training accuracy, and then in the case of image recognition of the attack image, the corresponding recognition class can still fall within a relatively accurate classification range. In the process of adjusting the model parameters of the deep learning model, the loss function is improved so that after the attacked deep learning model obtains the sample recognition class, the corresponding loss function changes less, and the influence of the attack image on the judgment of the deep learning model is eliminated as much as possible to improve the security and reliability of the trained image recognition model.

[0085] Figure 4 The flowchart of obtaining the second sample image by adjusting the gradient value of at least one pixel according to the importance corresponding to each of at least one pixel in the first sample image according to an embodiment of the present disclosure is schematically shown.

[0086] like Figure 4 As shown, in operation S210, the method of adjusting the gradient value of at least one pixel according to the importance corresponding to each pixel in the first sample image to obtain the second sample image may include operations S411 to S414.

[0087] In operation S411, a first pixel set and a second pixel set are determined based on importances corresponding to at least one pixel in the first sample image, wherein the first pixel set includes at least one first pixel, the second pixel set includes at least one second pixel, a first gradient value corresponds to each of the at least one first pixel, and a second gradient value corresponds to each of the at least one second pixel.

[0088] In operation S412, at least one target pixel is determined from the first pixel set.

[0089] In operation S413, the first gradient value corresponding to each of the at least one target pixels is adjusted to obtain a third gradient value corresponding to each of the at least one target pixels.

[0090] In operation S414, a second sample image is obtained according to the second gradient value corresponding to the at least one second pixel, the third gradient value corresponding to the at least one target pixel, and the first gradient value corresponding to the at least one other pixel, wherein the other pixel is any pixel in the first pixel set except the at least one target pixel.

[0091] According to an embodiment of the present disclosure, before performing operation S411, the following may also be included: determining the mean value and standard deviation corresponding to the first sample image based on the gradient values ​​corresponding to at least one pixel in the first sample image; determining the importance threshold based on preset hyperparameters and the standard deviation; determining, for each pixel in at least one pixel in the at least one sample image, the absolute value of the difference between the mean value and the gradient value of the pixel to obtain a reference value corresponding to the pixel; it may be set that, when it is determined that the reference value corresponding to the pixel is less than or equal to the importance threshold, it is determined that the importance corresponding to the pixel satisfies the preset importance condition; when it is determined that the reference value corresponding to the pixel is greater than the importance threshold, it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition.

[0092] According to the embodiment of the present disclosure, a specific gradient value corresponding to at least one pixel in the first sample image can be obtained through deep learning model calculation. According to the gradient value corresponding to at least one pixel in the first sample image, the importance corresponding to at least one pixel in the first sample image can be obtained according to the correspondence, and then the pixels in the first sample image can be classified to obtain a first pixel set and a second pixel set.

[0093] According to an embodiment of the present disclosure, operation S411 may also include: for each pixel of at least one pixel in the first sample image, when it is determined that the importance corresponding to the pixel satisfies a preset importance condition, determining the pixel as a first pixel; when it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition, determining the pixel as a second pixel.

[0094] Figure 5 The following is a schematic diagram schematically showing a method for determining the importance of image pixels according to an embodiment of the present disclosure.

[0095] like Figure 5 As shown, R, G, and B in the figure can represent the three color channels of the first sample image respectively when the first sample image is a color image. The mean value of the gradient of the pixels of the first sample image can be calculated according to the gradient value corresponding to the pixels of the first sample image. , and the standard deviation of the gradient of the first sample image pixel The importance of pixels can be determined based on how close the gradient value is to the mean gradient value, that is, the standard deviation of the gradient value can be used as the basis for determining the importance of pixels. of times as the importance threshold. satisfy In the case of , it is determined that the importance corresponding to the pixel does not meet the preset importance condition, that is, the pixel can be considered as an "important point" and the corresponding pixel is determined as the second pixel. satisfy In the case of, it is determined that the importance corresponding to the pixel meets the preset importance condition, the pixel is regarded as an "unimportant point", and the corresponding pixel is determined as the first pixel. These are preset training parameters, which can be adjusted based on the actual training process and are not limited here.

[0096] According to an embodiment of the present disclosure, after obtaining the first pixel and the second pixel according to the gradient of the pixels of the first sample image, the pixel gradient corresponding to the pixel value of the first pixel and the pixel gradient corresponding to the pixel value of the second pixel are converted to obtain the gradient direction corresponding to each pixel of the first sample image.

[0097] According to the embodiment of the present disclosure, the importance of image pixels is determined by using the gradient of image pixels generated by a preset deep learning model, and the positions of important points and unimportant points are obtained. The important point matrix is ​​obtained based on the positions of important points, unimportant points and the first sample image. and the matrix of unimportant points . You can set The value corresponding to the important point is 1, and the value corresponding to the other positions is 0; you can set The value corresponding to the position of the unimportant point is 1, and the value corresponding to the rest of the positions is 0. The gradient value of the image pixel is input into the sign function sign() to obtain the gradient direction corresponding to the first sample image pixel. , The values ​​include 1 and -1. You can also set a random matrix consisting of 1 and -1 , and The same size, The probability of each element being 1 or -1 is 0.5. , Important Point Matrix , unimportant point matrix and a random matrix , according to the formula , get the second sample image ,in, is the preset attack boundary. and the second sample image Input the deep learning model for subsequent training, which may include forward propagation, calculation of loss function value, back propagation calculation of weight gradient, and updating of neural network weights.

[0098] According to an embodiment of the present disclosure, in operation S320, it is possible to set the first pixel set to be associated with a random matrix. The pixel corresponding to the value "-1" is determined as the target pixel, and the random matrix The superposition operation of the value "-1" in the target pixel realizes the addition of reverse disturbance in the gradient direction, that is, the addition of interference items that interfere with the classification of the deep learning model.

[0099] According to the embodiments of the present disclosure, by using a deep learning model to process a second sample image with added interference items, and by using a sample recognition class obtained according to the second sample image to adjust the model parameters of the deep learning model, the deep learning model can learn the characteristics of black box attacks, thereby improving the defense performance of the deep learning model against black box attacks.

[0100] According to the disclosed embodiments, the cross entropy loss function is often used in classification problems and can show good accuracy, but the cross entropy loss function cannot significantly increase the distance between different classes in the classification space, which may cause the deep learning model using cross entropy as the loss function to be vulnerable to attacks by attackers, and the attack success rate is relatively high. In addition, the core of the gradient-based white box attack method is to add perturbations to the image along the direction of the gradient to increase the loss function value of the deep learning model, and ultimately make the attack image cross the decision boundary of the deep learning model, resulting in the deep learning model classification error, thereby achieving the effect of the attack. Therefore, while using the cross entropy loss function, an improved loss function can be constructed based on the range of the sample recognition class vector to achieve the training of the deep learning model.

[0101] According to an embodiment of the present disclosure, in the process of adjusting the model parameters of the deep learning model according to the sample recognition class vector to obtain the image recognition model, it can include: determining the modulus value of the sample recognition class vector; determining the angle value between the sample recognition class vector and the normal vector of the preset decision boundary; based on the loss function, obtaining the loss function value according to the modulus value, the angle value and the sample recognition class vector; adjusting the model parameters of the deep learning model according to the loss function value to obtain the image recognition model.

[0102] According to the disclosed embodiment, in the reasoning behavior of the deep learning model, the predicted output of the deep learning model is the class corresponding to the neuron with the highest confidence obtained after the confidence is calculated by the softmax layer as the predicted result of the deep learning model. The softmax function is a monotonically increasing function, that is, the output result corresponding to the maximum value when inputting the softmax layer must be the largest, that is, only the value of the output neuron of the previous layer of the softmax layer, that is, the logits layer, can be used to judge the final prediction result of the neural network, that is, the class corresponding to the neuron with the maximum value in the logits layer is the prediction result of the neural network, which is also the commonality of the deep learning model using the "unique hot code" output. Therefore, an improved loss function is constructed based on the range of the sample recognition class vector, so that the range of the sample recognition class vector is controlled within a preset area, and the "sector clustering" method can be constructed to ensure that the class corresponding to the second sample image input into the deep learning model has the largest output value in the neurons of the logits layer.

[0103] Figure 6 A schematic diagram of a training loss function according to an embodiment of the present disclosure is schematically shown.

[0104] like Figure 6As shown in the figure, the decision conditions for the deep learning model to perform binary classification are schematically shown. Since the final decision of the deep learning model is that the class corresponding to the neuron with the largest median value in the output layer is the sample identification class, the deep learning model only needs to ensure that the sample identification class after clustering is on the decision boundary. The purpose of the new loss function is to cluster the image into a fan-shaped area.

[0105] According to the embodiments of the present disclosure, the clustering range of different classes in the classification space can be limited by using cosine terms and modulus terms, thereby increasing the distance between the clustering space and the decision boundary, and ultimately increasing the difficulty of attack for attackers, which can better enhance the white box defense performance of the neural network.

[0106] According to the embodiment of the present disclosure, the constructed loss function can be expressed by the following formula (1).

[0107] (1)

[0108] in, is the loss function for binary classification in deep learning models, that is, ; is the angle value; Identify the class vector for the sample; is the modulus length value; is the normal vector of the preset decision boundary; crossentropy is the cross entropy loss function of the label and sample recognition class; They represent the coefficients of each item in the loss function.

[0109] According to the embodiment of the present disclosure, the "sector clustering" method in two-dimensional space can also be extended to M-dimensional space, that is, when the number of categories in the classification task is M, for the image recognition model of M recognition categories, the upper layer of the softmax layer, that is, the logits layer, can output an M-dimensional vector , set based on M-dimensional vector It can be determined that the corresponding classification result is , the corresponding decision boundary-based classification can be expressed by the following formula (2).

[0110] (2)

[0111] in, are different M-dimensional vector elements The corresponding preset M-dimensional decision boundary is In case of The corresponding target recognition class can be obtained.

[0112] According to the embodiment of the present disclosure, based on the M-dimensional vector and formula (2), M-1 decision boundaries can be obtained, and correspondingly M-1 unit normal vectors can be obtained. The unit normal vector can be expressed as . Among them, the unit normal vector The corresponding value can be expressed by the following formula (3).

[0113] (3)

[0114] in, .

[0115] According to the embodiment of the present disclosure, for The corresponding fan-shaped area (the fan-shaped area defined in the two-dimensional space corresponds to an area in the M-dimensional space that is no longer a fan-shaped area, but for the sake of convenience in understanding, the definition of "fan-shaped area" in the two-dimensional space is continued to be used, that is, the specified clustering space is still called a fan-shaped area), the M-dimensional preset centroid can be expressed as Among them, the value corresponding to the M-dimensional preset centroid can be expressed by the following formula (4).

[0116] (4)

[0117] in, is the preset centroid parameter; ; N is a positive integer.

[0118] According to the embodiment of the present disclosure, based on the above formulas (2) to (4), the sample identification class vector corresponding to the M-dimensional space can be obtained as follows: .

[0119] Then the angle value related to the M-dimensional space, that is, the corresponding cosine regularization term, is expressed by the following formula (5).

[0120]

[0121] = (5)

[0122] in, is the cosine regularization term, that is, the loss function term related to the angle value.

[0123] Then the modulus value related to the M-dimensional space, that is, the corresponding second-order norm regularization term, is expressed by the following formula (6).

[0124] (6)

[0125] in, is the second-order norm regularization term, that is, the loss function term related to the sample recognition class vector.

[0126] Therefore, the constructed loss function can also be expressed by the following formula (7).

[0127] (7)

[0128] in, Respectively represent the proportion coefficients of each item in the loss function, which can usually be Set to a positive number.

[0129] Figure 7 The figure schematically shows a flow chart of a method for training an image recognition model according to another embodiment of the present disclosure.

[0130] like Figure 7 As shown, the training method of the image recognition model may further include operations S710 to S780.

[0131] In operation S710, a deep learning model, a first sample image, and an attack boundary are acquired.

[0132] In operation S720, gradient values ​​of pixels of the first sample image are calculated.

[0133] In operation S730, a second sample image is obtained according to gradient values ​​of pixels of the first sample image.

[0134] In operation S740, it is determined whether the training is completed. If the training is completed, operation S780 is performed; if the training is not completed, operation S750 is performed.

[0135] In operation S750, the second sample image is input into the deep learning model to obtain a sample recognition class.

[0136] In operation S760, a loss function value is obtained using the constructed loss function according to the sample identification class.

[0137] In operation S770, model parameters of the deep learning model are adjusted according to the loss function value.

[0138] In operation S780, a trained image recognition model is obtained.

[0139] According to the embodiment of the present disclosure, the attack boundary can characterize the degree of change of the first sample image that the image recognition model can accept, that is, it can characterize the degree of attack. The attack boundary can be changed according to the actual problem to be solved to obtain a trained image recognition model suitable for different attack problems.

[0140] According to an embodiment of the present disclosure, a deep learning model is trained based on the range of sample recognition class vectors and an improved loss function constructed using a cross entropy loss function. Also, the gradient of the corresponding pixel of the first sample image is calculated using a deep learning model, importance sampling is performed based on the gradient of the corresponding pixel of the first sample image, the gradient value corresponding to the pixel of the important point of the first sample image is fixed, and interference items are added to the pixels of the unimportant points of the first sample image to obtain a second sample image that can be used for training, and the process of obtaining the second sample image is performed in each training iteration; the deep learning model is subsequently trained using the obtained second sample image to obtain an image recognition model. The image recognition model can simultaneously cope with and solve the problem of adversarial attacks. And the training method of the image recognition model is a universal training method that can be applied to deep learning models of different structures to improve the reliability of deep learning models of different structures.

[0141] According to the embodiment of the present disclosure, the specific contents of operations S710 to S780 may refer to the specific contents of operations S210 to S230, which will not be repeated here.

[0142] Figure 8 A schematic diagram of a flow chart of a method for identifying adversarial interference images according to an embodiment of the present disclosure is shown.

[0143] like Figure 8 As shown, the recognition method applied to the adversarial interference image may include operations S810 to S820.

[0144] In operation S810, a target image is acquired.

[0145] In operation S810, the target image is input into an image recognition model to obtain a target recognition class; wherein the image recognition model is trained using the above-mentioned image recognition model training method.

[0146] According to the embodiments of the present disclosure, an input target image is analyzed by an image recognition model and features of the image are extracted to obtain a feature vector corresponding to the target image. Classification and recognition are performed based on the feature vector to obtain a recognition result, thereby achieving classification of the target image or detection of the target included in the target image.

[0147] According to an embodiment of the present disclosure, the target image may be an acquired image to be identified and classified, and may be an object included in the image subjected to at least one of the following identification and classification: binary classification, multi-classification, multi-label classification, and fine-grained classification.

[0148] According to the disclosed embodiment, by extracting features from the target image, a corresponding target recognition class can be obtained. After the target image is input from the input layer of the image recognition model, feature extraction of the target image can be implemented through the hidden layer of the image recognition model, and one or more neurons can be set in the output layer based on actual needs to obtain one or more target recognition classes.

[0149] According to an embodiment of the present disclosure, an image recognition model trained by the above-mentioned image recognition model training method, wherein the network parameters are adjusted, can realize image recognition of attack images obtained by at least one attack method based on white box attack and black box attack.

[0150] Fig. 9 A block diagram of an electronic device applied to a method for recognizing adversarial interference images and a method for training an image recognition model according to an embodiment of the present disclosure is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0151] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present application includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0152] In RAM903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM902 and RAM903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present application by executing the program in ROM902 and / or RAM903. It should be noted that the program can also be stored in one or more memories other than ROM902 and RAM903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present application by executing the program stored in one or more memories.

[0153] According to an embodiment of the present application, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0154] The present application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0155] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM902 and / or RAM903 described above and / or one or more memories other than ROM902 and RAM903.

[0156] The embodiments of the present application also include a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided in the embodiments of the present application.

[0157] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the processor 901 executes the computer program. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0158] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0159] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0160] According to an embodiment of the present application, the program code for executing the computer program provided in the embodiment of the present application can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0161] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0162] It is understood that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present application. In particular, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in various ways without departing from the spirit and teachings of the present application. All of these combinations and / or combinations fall within the scope of the present application.

[0163] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0164] The embodiments of the present application are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present application. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present application is limited by the attached claims and their equivalents. Without departing from the scope of the present application, various substitutions and modifications can be made, and these substitutions and modifications should all fall within the scope of the present application.

Claims

1. A method for training an image recognition model, comprising: Determine a first pixel set and a second pixel set according to the importance corresponding to at least one pixel in the first sample image, wherein the first pixel set includes at least one first pixel, the second pixel set includes at least one second pixel, and the at least one first pixel corresponds to a first gradient value, and the at least one second pixel corresponds to a second gradient value; determining at least one target pixel from the first set of pixels; Adjusting the first gradient value corresponding to each of the at least one target pixels to obtain a third gradient value corresponding to each of the at least one target pixels; obtaining a second sample image according to the second gradient value corresponding to each of the at least one second pixel, the third gradient value corresponding to each of the at least one target pixel, and the first gradient value corresponding to each of the at least one other pixel, wherein the other pixel is any pixel in the first pixel set except the at least one target pixel; Inputting the second sample image into a deep learning model to obtain a sample recognition class; and According to the sample recognition class, adjusting the model parameters of the deep learning model to obtain a trained image recognition model; The method further comprises: Determine a mean value and a standard deviation corresponding to the first sample image according to the gradient value corresponding to at least one pixel in the first sample image; Determining an importance threshold according to a preset hyperparameter and the standard deviation; For each pixel in at least one pixel in the at least one sample image, determine an absolute value of a difference between the average value and a gradient value of the pixel to obtain a reference value corresponding to the pixel; In the case where it is determined that the reference value corresponding to the pixel is less than or equal to the importance threshold, determining that the importance corresponding to the pixel satisfies the preset importance condition; In the case where it is determined that the reference value corresponding to the pixel is greater than the importance threshold, it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition.

2. The method according to claim 1, wherein: The step of adjusting the model parameters of the deep learning model according to the sample recognition class to obtain the image recognition model includes: Determine a sample identification class vector according to a preset centroid and the sample identification class, wherein the sample identification class vector represents a directed relationship between the preset centroid and the sample identification class; and The model parameters of the deep learning model are adjusted according to the sample recognition class vector to obtain the image recognition model.

3. The method according to claim 1, wherein: The determining of the first pixel set and the second pixel set according to the importance corresponding to each of at least one pixel in the first sample image comprises: For each pixel of at least one pixel in the first sample image, if it is determined that the importance corresponding to the pixel satisfies a preset importance condition, determining the pixel as the first pixel; and When it is determined that the importance corresponding to the pixel does not satisfy the preset importance condition, the pixel is determined as the second pixel.

4. The method according to claim 2, wherein: The step of adjusting the model parameters of the deep learning model according to the sample recognition class vector to obtain the image recognition model includes: Determining the modulus value of the sample identification class vector; Determine the angle between the sample identification class vector and the normal vector of the preset decision boundary; Based on the loss function, a loss function value is obtained according to the modulus value, the angle value and the sample identification class vector; and The model parameters of the deep learning model are adjusted according to the loss function value to obtain the image recognition model.

5. The method according to claim 4, wherein: The loss function is: in, is the loss function; is a loss function term associated with the angle value; is a loss function term associated with the sample identification class vector; is the cross entropy loss function between the label and the sample identification class; are the proportion coefficients of each item in the loss function respectively.

6. An image recognition method, comprising: Get the target image; as well as Inputting the target image into an image recognition model to obtain a target recognition class; Wherein, the image recognition model is trained using the method according to any one of claims 1 to 5.

7. An electronic device comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the executable instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1 to 5.

Images