Image recognition model training method, image recognition method and device

Abstract

The application provides an image recognition model training method, an image recognition method and device. The image recognition model training method comprises: inputting original sample images and adversarial sample images in each set of training samples in a training set into a to-be-trained image recognition model to obtain a predicted classification result and an adversarial sample image feature vector; determining a first loss according to a first feature vector and a second feature vector in the adversarial sample image feature vector, and determining a second loss according to the predicted classification result and label information of the original sample images; when there is a gradient conflict between a first gradient corresponding to the first loss and a second gradient corresponding to the second loss, performing gradient conflict elimination processing on the first gradient and the second gradient, adjusting parameters of the image recognition model based on the processed first gradient and the processed second gradient, and obtaining a trained image recognition model. The application can improve the accuracy of image recognition by the image recognition model and the generalization ability of the image recognition model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image recognition model training method, an image recognition method, and an apparatus. Background Technology

[0002] With the rapid development of image recognition technology, image recognition models are increasingly needed in various fields for image recognition processing. Specifically, an image containing the content to be recognized is used as input to a trained image recognition model, which can then identify relevant information about the content.

[0003] Currently, when training image recognition models, perturbations are typically added to the original training sample images to obtain adversarial sample images. The initial image recognition model is then trained based on both the original training sample images and the adversarial sample images to obtain a trained image recognition model. The perturbation is usually achieved by adding uniformly distributed random noise to the original training sample images, but image recognition models obtained in this way have poor recognition accuracy. Summary of the Invention

[0004] This application provides an image recognition model training method, an image recognition method, and an apparatus to improve the recognition accuracy of the trained image recognition model. While accurately identifying the target object in the image to be recognized, it can also accurately determine whether the image to be recognized is an adversarial image, thereby greatly improving the generalization ability of the trained image recognition model.

[0005] In a first aspect, this application provides an image recognition model training method, including:

[0006] Obtain the original sample image, the annotation information of the original sample image, and the adversarial sample image from each group of training samples in the training set. The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is the image obtained by adding adversarial perturbation to the original sample image.

[0007] The original sample image and the adversarial sample image are input into the image recognition model to be trained to obtain the predicted classification result and the adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes a first feature vector and a second feature vector. The first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained.

[0008] The first loss is determined based on the first feature vector and the second feature vector, and the second loss is determined based on the predicted classification result and the annotation information. It is also determined whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss. The first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained.

[0009] In response to the gradient conflict between the first gradient and the second gradient, gradient conflict elimination processing is performed on the first gradient and the second gradient to obtain the processed first gradient and second gradient.

[0010] Based on the processed first and second gradients, the parameters of the image recognition model to be trained are adjusted to obtain the trained image recognition model.

[0011] Secondly, this application provides an image recognition method, including:

[0012] Acquire the image to be recognized;

[0013] The image to be recognized is input into the trained image recognition model to obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image. The trained image recognition model is a model trained using the image recognition model training method described in the first aspect of this application.

[0014] Thirdly, this application provides an image recognition model training device, comprising:

[0015] The first acquisition module is used to acquire the original sample image, the annotation information of the original sample image, and the adversarial sample image in each group of training samples in the training set. The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is the image obtained after adding adversarial perturbation to the original sample image.

[0016] The second acquisition module is used to input the original sample image and the adversarial sample image into the image recognition model to be trained, and obtain the prediction classification result and the adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes a first feature vector and a second feature vector, wherein the first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained.

[0017] The first determining module is used to determine a first loss based on a first feature vector and a second feature vector, and to determine a second loss based on the predicted classification result and the annotation information, and to determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss. The first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained.

[0018] The first processing module is used to perform gradient conflict elimination processing on the first gradient and the second gradient in response to the existence of gradient conflict between the first gradient and the second gradient, so as to obtain the processed first gradient and the second gradient.

[0019] The second processing module is used to adjust the parameters of the image recognition model to be trained based on the processed first and second gradients, so as to obtain the trained image recognition model.

[0020] Fourthly, this application provides an image recognition device, comprising:

[0021] The acquisition module is used to acquire the image to be recognized;

[0022] The processing module is used to input the image to be recognized into the trained image recognition model and obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image. The trained image recognition model is a model trained using the image recognition model training method described in the first aspect of this application.

[0023] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0024] The memory stores the instructions that the computer executes;

[0025] The processor executes computer execution instructions stored in memory to implement the image recognition model training method as described in the first aspect of this application or the image recognition method as described in the second aspect.

[0026] Sixthly, this application provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the image recognition model training method as described in the first aspect or the image recognition method as described in the second aspect of this application.

[0027] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the image recognition model training method as described in the first aspect or the image recognition method as described in the second aspect of this application.

[0028] The image recognition model training method, image recognition method, and apparatus provided in this application acquire the original sample images, the annotation information of the original sample images, and the adversarial sample images from each group of training samples in the training set; input the original sample images and the adversarial sample images into the image recognition model to be trained to obtain the predicted classification result and the adversarial sample image feature vector output by the image recognition model to be trained, wherein the adversarial sample image feature vector includes a first feature vector and a second feature vector; determine a first loss based on the first feature vector and the second feature vector, and determine a second loss based on the predicted classification result and the annotation information, and determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss; in response to the existence of a gradient conflict between the first gradient and the second gradient, perform gradient conflict elimination processing on the first gradient and the second gradient to obtain the processed first gradient and the second gradient; adjust the parameters of the image recognition model to be trained based on the processed first gradient and the second gradient to obtain the trained image recognition model. Because this application considers that the image recognition model to be trained may move in the direction of increasing second loss when fitting adversarial sample images, that is, it considers the conflict between the loss targets corresponding to the original sample image and the adversarial sample image, when it is determined that there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, gradient conflict elimination processing is performed on the first gradient and the second gradient, that is, to eliminate the gradient conflict between the gradient calculated for the training loss of the original sample image and the gradient calculated for the training loss of the adversarial sample image, so as to ensure that the fitting of the adversarial sample image does not affect the fitting of the original sample image. Therefore, it can greatly improve the recognition accuracy of the image recognition model. While accurately recognizing the image to be recognized, it can accurately determine whether the image to be recognized is an adversarial image, thereby greatly improving the generalization ability of the trained image recognition model. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram illustrating an application scenario provided in one embodiment of this application;

[0031] Figure 2 A flowchart illustrating an image recognition model training method provided in an embodiment of this application;

[0032] Figure 3 This is a schematic diagram of the structure of an image recognition model provided in an embodiment of this application;

[0033] Figure 4 A flowchart illustrating an image recognition model training method provided in another embodiment of this application;

[0034] Figure 5 A flowchart illustrating an image recognition model training method provided in yet another embodiment of this application;

[0035] Figure 6 A flowchart illustrating an image recognition method provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0037] Figure 8 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this application;

[0038] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0041] Image recognition technology is widely used, and deep neural network models are a common type of image recognition model. Generally, image data is used as training samples to train the original deep neural network, resulting in a trained deep neural network. However, the performance of deep neural network models drops sharply when processing adversarial example images with pixel perturbations, due to the inherent vulnerability of deep neural network models. To obtain deep neural network models with better generalization ability, adversarial training can be used. This involves simultaneously performing adversarial attacks on the original training sample images during the training process to obtain adversarial example images, which are then used as training samples to train the deep neural network model. However, generating adversarial example images through adversarial attacks in each training iteration is time-consuming, and the accuracy of the adversarially trained deep neural network model in recognizing normal natural data also decreases.

[0042] In one related technique, a lightweight Bayesian network method is used to transform the last few layers of the original deep neural network model into a Bayesian network to obtain an initial image recognition model. During training, perturbations are added to the original sample images to obtain adversarial sample images. The initial image recognition model is trained based on both the original and adversarial sample images to ensure that the feature vectors corresponding to the adversarial sample images output by the image recognition model have a large variance, resulting in a well-trained image recognition model. One method of adding perturbations is, for example, adding uniformly distributed random noise to the original sample images. However, the training process does not consider the conflict between the optimization objectives of the original and adversarial sample images, causing some adversarial sample images to increase the feature variance of the original sample images, thus leading to poor generalization ability of the obtained image recognition model.

[0043] To address the aforementioned issues, this application provides an image recognition model training method, image recognition method, and apparatus. During the training process, adversarial sample images of specific adversarial types, which are difficult to simulate with uniformly distributed random noise, are added to train the image recognition model. Simultaneously, considering that the image recognition model uses different training loss functions for the original sample images and adversarial sample images during training, i.e., corresponding to two different constraints, when training with adversarial sample images that are close to the original sample images, the calculated gradient may veer in a direction unfavorable to the constraints of the original sample images. That is, when the image recognition model fits the adversarial sample images, it may veer in a direction that increases the loss of the original sample images. Therefore, orthogonal projection is used to eliminate the gradient conflict between the gradients calculated for the training loss of the original sample images and the gradients calculated for the training loss of the adversarial sample images, removing the parts unfavorable to the constraints of the original sample images. This ensures that training with adversarial sample images does not affect the fitting of the original sample images, achieving the goal of maximizing the optimization of both constraints. This results in an image recognition model with higher recognition accuracy, significantly improving the generalization ability of the image recognition model.

[0044] The following section provides examples illustrating the application scenarios of the solution provided in this application.

[0045] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1As shown, in this application scenario, the attendance device 101 acquires the user's facial image and sends it to the server 102. The server 102 obtains the image recognition result corresponding to the facial image and sends the image recognition result back to the attendance device 101. The attendance device 101 outputs the corresponding attendance result to the user based on the image recognition result. The specific implementation process of the server 102 obtaining the image recognition result corresponding to the facial image can be found in the solutions of the following embodiments.

[0046] It should be noted that, Figure 1 This is merely a schematic diagram illustrating one application scenario provided by an embodiment of this application. This embodiment does not necessarily represent... Figure 1 The included equipment is not limited, nor is it restricted. Figure 1 The positional relationships between devices are defined. For example, in Figure 1 The application scenario shown may also include a data storage device, which may be an external storage device relative to the server 102 or an internal storage device integrated into the server 102.

[0047] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0048] Figure 2 This is a flowchart illustrating an image recognition model training method according to an embodiment of this application, used for training an image recognition model. The method of this embodiment can be applied to an electronic device, which may be a server or server cluster, etc. Figure 2 As shown, the method in this application embodiment includes:

[0049] S201. Obtain the original sample images, the annotation information of the original sample images, and the adversarial sample images from each group of training samples in the training set.

[0050] The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is the image obtained by adding adversarial perturbation to the original sample image.

[0051] In this embodiment, the original sample image is a normal image without added adversarial perturbation; the annotation information of the original sample image can be understood as the label of the original sample image; the adversarial sample image is an image obtained by adding adversarial perturbation to the original sample image, such as an image obtained by adding random noise to the original sample image and an image generated according to a preset adversarial type. For example, each training sample may include a batch of original sample images (e.g., represented by X), the annotation information of the batch of original sample images (e.g., represented by Y), and a batch of adversarial sample images (e.g., represented by X). ′ express).

[0052] S202. Input the original sample image and the adversarial sample image into the image recognition model to be trained to obtain the predicted classification result output by the image recognition model to be trained and the adversarial sample image feature vector. The adversarial sample image feature vector includes the first feature vector and the second feature vector.

[0053] The first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained; the prediction classification result is used to indicate the probability that a target object exists in the original sample image.

[0054] For example, Figure 3 This is a schematic diagram of the structure of an image recognition model provided in an embodiment of this application, as shown below. Figure 3 As shown, assuming a pre-trained face recognition model (let's call it F) is provided as the image recognition model, the face recognition model F consists of a backbone network connected to a classification network. The backbone is used to extract features from the input image. Figure 3 The text uses ResNet50 as an example to illustrate the backbone structure, which includes N residual network blocks. A classification network is used to classify the features of the input image and calculate the cross-entropy loss with the ground truth labels of the input image. A Bayesian network transformation is applied to the face recognition model F to obtain the image recognition model M. The specific Bayesian network transformation process involves replacing the last few layers of the backbone with n copies, for example... Figure 3 The diagram shows the replacement of the last residual network block N in the face recognition model F with n residual network blocks (i.e., residual network blocks N1 to Nn), which can be denoted as the Bayesian sub-network B. The modified backbone is denoted as M. B The resulting image recognition model M includes the unreplaced part of the face recognition model F (represented by F_) and a Bayesian subnetwork B. The weight parameters of the Bayesian subnetwork B are initialized as follows: The random uniform sampling values ​​are obtained. After obtaining the image recognition model M, a batch of original sample images X are input into the image recognition model M. Each image in the batch of original sample images X is indexed in the order of all processed original sample images. One of the n residual network blocks in the Bayesian sub-network B is selected (for example, the index of each image in the batch of original sample images X is modulo n) for forward inference. The other network parts of the image recognition model M, except for the Bayesian sub-network B, are forward inferred in the normal way. The predicted classification result corresponding to the batch of original sample images X can be obtained through the image recognition model M. At the same time as inputting the batch of original sample images X into the image recognition model M, the batch of adversarial sample images X′ are also input into the image recognition model M. The batch of adversarial sample images X′ are copied into two identical copies, for example, represented by X′1 (i.e., the batch of adversarial sample images) and X′2 (i.e., the copied images of the batch of adversarial sample images). The image recognition model M can output the first feature vector corresponding to each adversarial sample image in X′1 and the second feature vector corresponding to each adversarial sample image in X′2 through one of the residual network blocks in the Bayesian sub-network B contained in the backbone network. It can be understood that the first feature vector and the second feature vector are different.

[0055] S203. Determine the first loss based on the first feature vector and the second feature vector, and determine the second loss based on the prediction classification result and the annotation information, and determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss.

[0056] Wherein, the first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained.

[0057] The first loss represents the Euclidean distance between the first and second feature vectors, and the second loss represents the cross-entropy loss of the predicted classification result relative to the labeled information.

[0058] In this step, after obtaining the predicted classification result output by the image recognition model, the first feature vector corresponding to the adversarial sample image, and the second feature vector corresponding to the copy image of the adversarial sample image, the first loss can be determined based on the first feature vector and the second feature vector, and the second loss can be determined based on the predicted classification result and the annotation information.

[0059] Further, optionally, the adversarial sample image includes a first adversarial sample image and a second adversarial sample image. The first adversarial sample image is an image obtained by adding random noise to the original sample image, and the second adversarial sample image is an image generated according to a preset adversarial type. Determining the first loss based on the first feature vector and the second feature vector may include: if the number of training iterations of the image recognition model to be trained is not an integer multiple of the second preset value, then the first loss is determined based on the first feature vector corresponding to the first adversarial sample image and the second feature vector corresponding to the copy image of the first adversarial sample image; if the number of training iterations of the image recognition model to be trained is an integer multiple of the second preset value, then the first loss is determined based on the first feature vector corresponding to the second adversarial sample image and the second feature vector corresponding to the copy image of the second adversarial sample image.

[0060] For example, for the first adversarial sample image, a portion of the original sample images (e.g., half of the original sample images) can be obtained from the batch of original sample images and uniformly distributed random noise can be added. Specifically, a random noise image U is obtained, which has the same size as the original sample images, and each pixel value is uniformly randomly selected from the interval [-∈, +∈]. By adding uniformly distributed random noise to a portion of the original sample images using the random noise image U, a batch of first adversarial sample images (e.g., denoted by Xs) is obtained. For the second adversarial sample image, assuming there are k preset adversarial types, one of which is denoted by Ai, then A = {A1, A2, ..., Ai, ..., Ak} represents a set containing adversarial sample images generated by applying each preset adversarial type. It can be understood that A contains adversarial sample images that are still difficult to detect after training with random noise. X″ can be used to represent a batch of adversarial sample images corresponding to a certain preset adversarial type Ai, i.e., a batch of second adversarial sample images. The number of second adversarial sample images in X″ is the same as the number of first adversarial sample images in Xs. The second preset value is, for example, 25. If the number of training iterations for the image recognition model to be trained is not a multiple of 25, then the first loss is determined based on the first feature vector corresponding to the first adversarial sample image and the second feature vector corresponding to the copy image of the first adversarial sample image. If the number of training iterations for the image recognition model to be trained is a multiple of 25, then the first loss is determined based on the first feature vector corresponding to the second adversarial sample image and the second feature vector corresponding to the copy image of the second adversarial sample image. It can be understood that X″ is taken once every 25 training iterations.

[0061] Optionally, determining the first loss based on the first feature vector and the second feature vector may include: determining the Euclidean distance between the first feature vector and the second feature vector; determining the target Euclidean distance as the smaller value between the Euclidean distance and a preset Euclidean distance threshold; and determining the first loss based on the target Euclidean distance.

[0062] For example, refer to Figure 3 The image recognition model includes a backbone network and a classification network. The backbone network includes a Bayesian sub-network. The first loss is determined based on the first feature vector and the second feature vector, and can be obtained according to the following formula:

[0063]

[0064] Where X′ represents the batch of adversarial sample images; |X′| represents the number of adversarial sample images in X′. X′ is copied into two identical copies, X′1 and X′2. Each adversarial sample image in X′1 is represented by X′. 1i This means that each adversarial example image in X′2 is represented by X′. 2i Indicates; M B M represents the backbone network. B (X′ 1i ) indicates that through M B The output X′ of the included Bayesian subnetwork 1i The corresponding first feature vector, M B (X′ 2i ) indicates that through M B The output X′ of the included Bayesian subnetwork 2i The corresponding second feature vector; γ represents the preset Euclidean distance threshold.

[0065] For example, refer to Figure 3 The batch of adversarial sample images X′ are input into the image recognition model M, and X′ is copied into two identical copies, X′1 and X′2. The first feature vector M corresponding to each adversarial sample image in X′1 is output through one of the residual network blocks in the Bayesian sub-network B included in the image recognition model M. B (X′ 1i ) and the second feature vector M corresponding to each adversarial sample image in X′2 B (X′ 2i Therefore, based on the above formula, the first loss can be obtained, which can be understood as the regularized loss.

[0066] For example, for the second loss (e.g., denoted by L), assuming the annotation information of the batch of original sample images is represented by Y, the batch of original sample images X are input into the image recognition model M, and the predicted classification result M(X) corresponding to the batch of original sample images X output by the image recognition model M can be obtained. Then L(X,Y)=cross-entropy(M(X),Y), which represents the cross-entropy loss of the predicted classification result M(X) relative to the annotation information Y.

[0067] In this step, after determining the first loss and the second loss, it can be determined whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss. For example, this can be determined by calculating the projection of the first gradient onto the second gradient. For details on how to determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, please refer to subsequent embodiments; these will not be elaborated upon here.

[0068] S204. In response to the gradient conflict between the first gradient and the second gradient, gradient conflict elimination processing is performed on the first gradient and the second gradient to obtain the processed first gradient and second gradient.

[0069] In this step, after determining that there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, gradient conflict elimination processing can be performed on the first and second gradients to obtain the processed first and second gradients. For details on how to perform gradient conflict elimination processing on the first and second gradients to obtain the processed first and second gradients, please refer to subsequent embodiments; these will not be elaborated here.

[0070] S205. Based on the processed first and second gradients, adjust the parameters of the image recognition model to be trained to obtain the trained image recognition model.

[0071] In this step, after obtaining the first and second gradients after gradient conflict elimination processing, the parameters of the image recognition model to be trained can be adjusted based on the processed first and second gradients to obtain the trained image recognition model.

[0072] Further, optionally, adjusting the parameters of the image recognition model to be trained based on the processed first and second gradients to obtain the trained image recognition model may include: obtaining the adjusted parameters of the image recognition model to be trained based on the processed first and second gradients and the learning rate parameter; iteratively training the image recognition model to be trained based on the adjusted parameters of the image recognition model to be trained until a preset number of iterations is reached or both the first loss and the second loss converge to obtain the trained image recognition model.

[0073] For example, based on the processed first and second gradients, the parameters of the image recognition model are adjusted. These parameters can be understood as the weight parameters of the image recognition model (e.g., represented by W), i.e. Where α and τ are both preset learning rate parameters. Indicates the first gradient. The second gradient is represented. Based on the adjusted parameters W of the image recognition model to be trained, the model is iteratively trained. When the preset number of iterations is reached or both the first and second losses converge (i.e., tend to stabilize), the trained image recognition model is obtained.

[0074] The image recognition model training method provided in this application involves acquiring the original sample images, annotation information of the original sample images, and adversarial sample images from each group of training samples in the training set; inputting the original sample images and adversarial sample images into the image recognition model to be trained to obtain the predicted classification result and adversarial sample image feature vectors output by the image recognition model to be trained, wherein the adversarial sample image feature vectors include a first feature vector and a second feature vector; determining a first loss based on the first feature vector and the second feature vector, and determining a second loss based on the predicted classification result and annotation information, and determining whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss; in response to the existence of a gradient conflict between the first gradient and the second gradient, performing gradient conflict elimination processing on the first gradient and the second gradient to obtain the processed first gradient and the second gradient; and adjusting the parameters of the image recognition model to be trained based on the processed first gradient and the second gradient to obtain the trained image recognition model. Since the embodiments of this application consider that the image recognition model to be trained may move in the direction of increasing second loss when fitting adversarial sample images, that is, considering the conflict between the loss targets of the original sample image and the adversarial sample image, when it is determined that there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, gradient conflict elimination processing is performed on the first gradient and the second gradient, that is, eliminating the gradient conflict between the gradient calculated for the training loss of the original sample image and the gradient calculated for the training loss of the adversarial sample image, so as to ensure that the fitting of the adversarial sample image does not affect the fitting of the original sample image. Therefore, the recognition accuracy of the image recognition model can be greatly improved. While accurately recognizing the image to be recognized, it can accurately determine whether the image to be recognized is an adversarial image, thereby greatly improving the generalization ability of the trained image recognition model.

[0075] Figure 4 This is a flowchart illustrating an image recognition model training method according to another embodiment of this application. Based on the above embodiments, this application further describes the image recognition model training method. Figure 4 As shown, the method in this application embodiment may include:

[0076] S401. Obtain the original sample images, the annotation information of the original sample images, and the adversarial sample images from each group of training samples in the training set.

[0077] For a detailed description of this step, please refer to [link / reference]. Figure 2The relevant description of S201 in the illustrated embodiment will not be repeated here.

[0078] S402. Input the original sample image and the adversarial sample image into the image recognition model to be trained, and obtain the predicted classification result and the adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes the first feature vector and the second feature vector.

[0079] For a detailed description of this step, please refer to [link / reference]. Figure 2 The relevant description of S202 in the illustrated embodiment will not be repeated here.

[0080] S403. Determine the first loss based on the first feature vector and the second feature vector, and determine the second loss based on the predicted classification result and the annotation information.

[0081] For a detailed description of this step, please refer to [link / reference]. Figure 2 The relevant description of S203 in the illustrated embodiment will not be repeated here.

[0082] In the embodiments of this application, Figure 2 Step S203 can further include the following two steps, S404 and S405:

[0083] S404. Determine whether the first projection from the first gradient corresponding to the first loss to the second gradient corresponding to the second loss is less than a first preset value.

[0084] In this step, the first preset value is, for example, 0. Referring to the examples in steps S202 and S203, the first gradient corresponding to the first loss is calculated. And calculate the second gradient corresponding to the second loss. Among them, the first gradient The first gradient represents the gradient of the first loss relative to the weight parameters of the image recognition model; the second gradient represents the gradient of the second loss. Let represent the gradient of the second loss with respect to the weight parameters of the image recognition model; W represents the weight parameters of the image recognition model. Based on the first and second gradients, calculate the first projection from the first gradient to the second gradient. Specifically, for example, the first projection can be calculated using the vector dot product method. After obtaining the first projection, it can be determined whether the first projection is less than 0.

[0085] S405. In response to the first projection being less than the first preset value, determine that there is a gradient conflict between the first gradient and the second gradient; in response to the first projection being not less than the first preset value, determine that there is no gradient conflict between the first gradient and the second gradient.

[0086] For example, after obtaining the first projection, if the first projection is less than 0, it is determined that there is a gradient conflict between the first gradient and the second gradient; if the first projection is not less than 0, it is determined that there is no gradient conflict between the first gradient and the second gradient.

[0087] Based on steps S404 and S405, optionally, it is determined whether the second projection of the second gradient to the first gradient is less than a first preset value. If the second projection is less than the first preset value, it is determined that there is a gradient conflict between the first gradient and the second gradient. If the second projection is not less than the first preset value, it is determined that there is no gradient conflict between the first gradient and the second gradient.

[0088] In the embodiments of this application, Figure 2 Step S204 can further include the following two steps, S406 and S407:

[0089] S406. In response to the gradient conflict between the first gradient and the second gradient, the first projection and the first gradient are summed to obtain the processed first gradient.

[0090] For example, after determining that there is a gradient conflict between the first gradient and the second gradient, i.e., the first projection... If the value is less than 0, the first projection and the first gradient are summed to obtain the processed first gradient. It is understandable that if it is determined that there is no gradient conflict between the first gradient and the second gradient, then the current gradient can be maintained.

[0091] S407. The second projection and the second gradient are summed to obtain the processed second gradient, wherein the second projection is the projection of the second gradient onto the first gradient.

[0092] For example, such as using If the second projection is represented by a second gradient, then the second projection and the second gradient are summed to obtain the processed second gradient.

[0093] It can be understood that the projection of the processed first gradient to the processed second gradient is greater than or equal to the first preset value, and the projection of the processed second gradient to the processed first gradient is greater than or equal to the first preset value.

[0094] S408. Based on the processed first and second gradients, adjust the parameters of the image recognition model to be trained to obtain the trained image recognition model.

[0095] For a detailed description of this step, please refer to [link / reference]. Figure 2 The relevant description of S205 in the illustrated embodiment will not be repeated here.

[0096] The image recognition model training method provided in this application involves acquiring the original sample images, annotation information of the original sample images, and adversarial sample images from each group of training samples in the training set; inputting the original sample images and adversarial sample images into the image recognition model to be trained to obtain the predicted classification result and adversarial sample image feature vectors output by the image recognition model to be trained, wherein the adversarial sample image feature vectors include a first feature vector and a second feature vector; determining a first loss based on the first feature vector and the second feature vector, and determining a second loss based on the predicted classification result and annotation information; determining whether the first projection from the first gradient corresponding to the first loss to the second gradient corresponding to the second loss is less than a first preset value; in response to the first projection being less than the first preset value, determining that there is a gradient conflict between the first gradient and the second gradient; in response to the first projection being not less than the first preset value, determining that there is no gradient conflict between the first gradient and the second gradient; in response to the gradient conflict between the first gradient and the second gradient, summing the first projection and the first gradient to obtain the processed first gradient; summing the second projection and the second gradient to obtain the processed second gradient, wherein the second projection is the projection of the second gradient to the first gradient; and adjusting the parameters of the image recognition model to be trained based on the processed first gradient and the second gradient to obtain the trained image recognition model. Because the embodiments of this application consider that the image recognition model to be trained may move in the direction of increasing second loss when fitting adversarial sample images, that is, considering the conflict between the loss targets corresponding to the original sample image and the adversarial sample image, when it is determined that there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, the first projection and the first gradient are summed to obtain the processed first gradient; the second projection and the second gradient are summed to obtain the processed second gradient, that is, the gradient conflict between the gradient calculated for the training loss of the original sample image and the gradient calculated for the training loss of the adversarial sample image is eliminated, so as to ensure that the fitting of the adversarial sample image does not affect the fitting of the original sample image. Therefore, the recognition accuracy of the image recognition model can be greatly improved. While accurately recognizing the image to be recognized, it can accurately determine whether the image to be recognized is an adversarial image, thereby greatly improving the generalization ability of the trained image recognition model.

[0097] Based on the above embodiments, Figure 5 A flowchart illustrating an image recognition model training method provided in yet another embodiment of this application. For example... Figure 5 As shown, the method in this application embodiment may include:

[0098] S501. Initialize the training iteration count e = 0.

[0099] S502. Obtain batch of original sample images X and batch of original sample image annotation information Y, as well as batch of adversarial sample images X′.

[0100] S503. Determine whether (e+1)%25 is 0.

[0101] If (e+1)%25 is 0, then execute step S504; if (e+1)%25 is not 0, then execute step S505.

[0102] S504. Obtain the batch of second adversarial sample images X″ as the batch adversarial sample images X′.

[0103] S505. Obtain the first batch of adversarial sample images Xs as the batch of adversarial sample images X′.

[0104] S506. Input X and X′ into the image recognition model and calculate the first loss R(X′) and the second loss L(X,Y).

[0105] S507. Calculate the first gradient corresponding to the first loss. The second gradient corresponding to the second loss

[0106] S508, Determine the first projection from the first gradient to the second gradient. Is it less than 0?

[0107] If the first projection from the first gradient to the second gradient is less than 0, then execute step S509; if the first projection from the first gradient to the second gradient is not less than 0, then execute step S511.

[0108] S509. Determine that there is a gradient conflict between the first gradient and the second gradient.

[0109] S510. The first projection and the first gradient are summed to obtain the processed first gradient. Calculate the second projection of the second gradient onto the first gradient. The second projection and the second gradient are summed to obtain the processed second gradient.

[0110] S511. Determine that there is no gradient conflict between the first gradient and the second gradient.

[0111] S512, Update the weight parameters of the image recognition model.

[0112] S513. Increment the training iteration count e by 1, and determine whether e is less than the preset number of iterations.

[0113] If e is less than the preset number of iterations, then execute step S502; if e is greater than or equal to the preset number of iterations, then execute step S514.

[0114] S514. The iterative training ends, and the trained image recognition model is obtained.

[0115] The image recognition model training method provided in this application, when the number of training iterations of the image recognition model is an integer multiple of a second preset value, obtains a first loss based on a second adversarial sample image. The second adversarial sample image is an image generated according to a preset adversarial type, rather than simply using a first adversarial sample image obtained by adding random noise to the original sample image to obtain the first loss. On this basis, orthogonal projection is used to eliminate the gradient conflict between the gradient calculated for the training loss of the original sample image and the gradient calculated for the training loss of the adversarial sample image, so as to ensure that the training of the adversarial sample image does not affect the fitting of the original sample image. Therefore, it can greatly improve the recognition accuracy of the image recognition model. While accurately recognizing the image to be recognized, it can accurately determine whether the image to be recognized is an adversarial image, thereby greatly improving the generalization ability of the trained image recognition model.

[0116] Figure 6 This is a flowchart illustrating an image recognition method provided in an embodiment of this application. The method described in this embodiment can be applied to an electronic device, which may be a server or a server cluster, etc. Figure 6 As shown, the method in this application embodiment includes:

[0117] S601. Obtain the image to be recognized.

[0118] In this embodiment of the application, the image to be identified may be input by a user to an electronic device performing this method embodiment, or it may be sent by another device to the electronic device performing this method embodiment. For example, the image to be identified may be a face image.

[0119] S602. Input the image to be recognized into the trained image recognition model to obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image.

[0120] The trained image recognition model is the model trained using the image recognition model training method in any of the above method embodiments.

[0121] For example, refer to Figure 3 Assuming the image to be identified is a face image, inputting the face image into an image recognition model can yield the predicted classification result of the face image output by the image recognition model. This allows us to determine the user's identity information and whether the face image is an adversarial image.

[0122] Further, optionally, the image recognition model includes a backbone network and a classification network. The backbone network includes a Bayesian sub-network. The image to be recognized is input into the trained image recognition model to obtain the image recognition result output by the trained model. This can include: extracting features from the image to be recognized through the Bayesian sub-network to obtain a preset number of target feature vectors output by the Bayesian sub-network; obtaining the mean variance among the preset number of target feature vectors through the classification network, and determining the image to be recognized as an adversarial image when the mean variance is greater than or equal to a variance threshold; determining the image to be recognized as not an adversarial image when the mean variance is less than the variance threshold; obtaining the average value of the preset number of target feature vectors; and determining the predicted classification result corresponding to the image to be recognized based on the similarity between the average value and the feature vectors corresponding to the preset image.

[0123] For example, refer to Figure 3 It can be achieved through the backbone network M B The Bayesian sub-network B extracts features from the image to be recognized, outputting n target feature vectors. Then, a classification network obtains the mean variance among these n target feature vectors. This mean variance is used to determine whether the image to be recognized is adversarial. The average of the n target feature vectors is obtained, and the similarity between this average and the feature vector corresponding to each preset image is determined. Based on this similarity, the predicted classification result for the image to be recognized is determined. For example, the preset image corresponding to the feature vector with the highest similarity (i.e., the predicted classification result) is the recognized image.

[0124] The image recognition method provided in this application acquires an image to be recognized, inputs the image to be recognized into a trained image recognition model, and obtains the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image. Since the image recognition model is trained using the image recognition model training method in any of the method embodiments, it can perform image recognition more accurately, improve the image recognition accuracy of the image recognition model, and maximize the detection capability of adversarial images.

[0125] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0126] Figure 7 This is a schematic diagram of the structure of an image recognition model training device provided in one embodiment of this application, used for training an image recognition model. Figure 7 As shown, the model training device 700 of this application embodiment includes: a first acquisition module 701, a second acquisition module 702, a first determination module 703, a first processing module 704, and a second processing module 705. Wherein:

[0127] The first acquisition module 701 is used to acquire the original sample image, the annotation information of the original sample image, and the adversarial sample image in each group of training samples in the training set. The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is an image obtained by adding adversarial perturbation to the original sample image.

[0128] The second acquisition module 702 is used to input the original sample image and the adversarial sample image into the image recognition model to be trained, and obtain the predicted classification result and the adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes a first feature vector and a second feature vector, wherein the first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained.

[0129] The first determining module 703 is used to determine a first loss based on a first feature vector and a second feature vector, and to determine a second loss based on the predicted classification result and annotation information, and to determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss. The first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained.

[0130] The first processing module 704 is used to perform gradient conflict elimination processing on the first gradient and the second gradient in response to the existence of gradient conflict between the first gradient and the second gradient, so as to obtain the processed first gradient and the second gradient.

[0131] The second processing module 705 is used to adjust the parameters of the image recognition model to be trained based on the processed first and second gradients, so as to obtain the trained image recognition model.

[0132] In some embodiments, the first determining module 703 may be specifically used to: determine whether the first projection from the first gradient to the second gradient is less than a first preset value; in response to the first projection being less than the first preset value, determine that there is a gradient conflict between the first gradient and the second gradient; in response to the first projection being not less than the first preset value, determine that there is no gradient conflict between the first gradient and the second gradient.

[0133] Optionally, the first processing module 704 may be specifically used to: sum the first projection and the first gradient to obtain the processed first gradient; sum the second projection and the second gradient to obtain the processed second gradient, wherein the second projection is the projection of the second gradient onto the first gradient; wherein the projection of the processed first gradient onto the processed second gradient is greater than or equal to a first preset value, and the projection of the processed second gradient onto the processed first gradient is greater than or equal to the first preset value.

[0134] Optionally, the adversarial sample images include a first adversarial sample image and a second adversarial sample image. The first adversarial sample image is an image obtained by adding random noise to the original sample image, and the second adversarial sample image is an image generated according to a preset adversarial type. The first determining module 703 can be specifically used to: if the number of training iterations of the image recognition model to be trained is not an integer multiple of the second preset value, then determine a first loss based on the first feature vector corresponding to the first adversarial sample image and the second feature vector corresponding to the copy image of the first adversarial sample image; if the number of training iterations of the image recognition model to be trained is an integer multiple of the second preset value, then determine a first loss based on the first feature vector corresponding to the second adversarial sample image and the second feature vector corresponding to the copy image of the second adversarial sample image.

[0135] Optionally, the first determining module 703 may be specifically used to: determine the Euclidean distance between the first feature vector and the second feature vector; determine the target Euclidean distance as the smaller value between the Euclidean distance and the preset Euclidean distance threshold; and determine the first loss based on the target Euclidean distance.

[0136] In some embodiments, the second processing module 705 may be specifically used to: obtain the parameters of the adjusted image recognition model to be trained based on the processed first gradient and second gradient and the learning rate parameter; and iteratively train the image recognition model to be trained based on the parameters of the adjusted image recognition model to be trained until a preset number of iterations or both the first loss and the second loss converge, thereby obtaining the trained image recognition model.

[0137] The apparatus of this embodiment can be used to execute the image recognition model training method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0138] Figure 8 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this application. Figure 8 As shown, the image recognition device 800 of this application embodiment includes: an acquisition module 801 and a processing module 802. Wherein:

[0139] The acquisition module 801 is used to acquire the image to be recognized.

[0140] The processing module 802 is used to input the image to be recognized into the trained image recognition model and obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image. The image recognition model is a model trained using the image recognition model training method of the first aspect of this application.

[0141] Optionally, the image recognition model includes a backbone network and a classification network. The backbone network includes a Bayesian sub-network. The processing module 802 can be specifically used to: extract features from the image to be recognized through the Bayesian sub-network to obtain a preset number of target feature vectors output by the Bayesian sub-network; obtain the mean variance among the preset number of target feature vectors through the classification network, and determine that the image to be recognized is an adversarial image when the mean variance is greater than or equal to a variance threshold; determine that the image to be recognized is not an adversarial image when the mean variance is less than the variance threshold; obtain the average value of the preset number of target feature vectors; and determine the predicted classification result corresponding to the image to be recognized based on the similarity between the average value and the feature vectors corresponding to the preset image.

[0142] The apparatus of this embodiment can be used to execute the image recognition method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0143] Figure 9 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Exemplarily, the electronic device may be provided as a server or a computer. (Refer to...) Figure 9 The electronic device 900 includes a processing component 901, which further includes one or more processors, and memory resources represented by memory 902 for storing instructions, such as application programs, that can be executed by the processing component 901. The application programs stored in memory 902 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 901 is configured to execute instructions to perform any of the method embodiments described above.

[0144] Electronic device 900 may also include a power supply component 903 configured to perform power management of electronic device 900, a wired or wireless network interface 904 configured to connect electronic device 900 to a network, and an input / output (I / O) interface 905. Electronic device 900 may operate on an operating system stored in memory 902, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0145] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-mentioned image recognition model training method and image recognition method.

[0146] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image recognition model training method and image recognition method.

[0147] The aforementioned computer-readable storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0148] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an image recognition model training device or an image recognition device.

[0149] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training an image recognition model, characterized in that, include: Obtain the original sample image, the annotation information of the original sample image, and the adversarial sample image from each group of training samples in the training set. The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is an image obtained by adding adversarial perturbation to the original sample image. The original sample image and the adversarial sample image are input into the image recognition model to be trained to obtain the predicted classification result and the adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes a first feature vector and a second feature vector, wherein the first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained. A first loss is determined based on the first feature vector and the second feature vector, and a second loss is determined based on the predicted classification result and the annotation information. It is also determined whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss. The first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained. In response to the gradient conflict between the first gradient and the second gradient, gradient conflict elimination processing is performed on the first gradient and the second gradient to obtain the processed first gradient and second gradient. Based on the processed first and second gradients, the parameters of the image recognition model to be trained are adjusted to obtain the trained image recognition model.

2. The image recognition model training method according to claim 1, characterized in that, Determining whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss includes: Determine whether the first projection from the first gradient to the second gradient is less than a first preset value; In response to the first projection being less than the first preset value, it is determined that there is a gradient conflict between the first gradient and the second gradient; In response to the first projection being not less than the first preset value, it is determined that there is no gradient conflict between the first gradient and the second gradient.

3. The image recognition model training method according to claim 2, characterized in that, The step of performing gradient conflict elimination processing on the first gradient and the second gradient to obtain the processed first gradient and second gradient includes: The first projection and the first gradient are summed to obtain the processed first gradient. The second projection and the second gradient are summed to obtain the processed second gradient, wherein the second projection is the projection of the second gradient onto the first gradient; Wherein, the projection of the processed first gradient to the processed second gradient is greater than or equal to the first preset value, and the projection of the processed second gradient to the processed first gradient is greater than or equal to the first preset value.

4. The image recognition model training method according to any one of claims 1 to 3, characterized in that, The adversarial sample image includes a first adversarial sample image and a second adversarial sample image. The first adversarial sample image is obtained by adding random noise to the original sample image, and the second adversarial sample image is generated according to a preset adversarial type. Determining the first loss based on the first feature vector and the second feature vector includes: If the number of training iterations for the image recognition model to be trained is not an integer multiple of the second preset value, then the first loss is determined based on the first feature vector corresponding to the first adversarial sample image and the second feature vector corresponding to the copy image of the first adversarial sample image. If the number of training iterations for the image recognition model to be trained is an integer multiple of the second preset value, then the first loss is determined based on the first feature vector corresponding to the second adversarial sample image and the second feature vector corresponding to the copy image of the second adversarial sample image.

5. The image recognition model training method according to claim 4, characterized in that, Determining the first loss based on the first feature vector and the second feature vector includes: Determine the Euclidean distance between the first feature vector and the second feature vector; The target Euclidean distance is determined to be the smaller value between the Euclidean distance and the preset Euclidean distance threshold; The first loss is determined based on the target Euclidean distance.

6. The image recognition model training method according to any one of claims 1 to 3, characterized in that, The step of adjusting the parameters of the image recognition model to be trained based on the processed first and second gradients to obtain the trained image recognition model includes: Based on the processed first and second gradients and the learning rate parameter, the parameters of the adjusted image recognition model to be trained are obtained. Based on the adjusted parameters of the image recognition model to be trained, the image recognition model to be trained is iteratively trained until a preset number of iterations is reached or both the first loss and the second loss converge, thus obtaining a trained image recognition model.

7. An image recognition method, characterized in that, include: Acquire the image to be recognized; The image to be identified is input into the trained image recognition model to obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be identified and whether the image to be identified is an adversarial image. The trained image recognition model is a model trained using the image recognition model training method as described in any one of claims 1 to 6.

8. The image recognition method according to claim 7, characterized in that, The image recognition model includes a backbone network and a classification network. The backbone network includes a Bayesian sub-network. The process of inputting the image to be recognized into the trained image recognition model to obtain the image recognition result output by the trained image recognition model includes: The Bayesian subnetwork is used to extract features from the image to be identified, and a preset number of target feature vectors are obtained from the output of the Bayesian subnetwork. The classification network obtains the mean variance among the preset number of target feature vectors, and when the mean variance is greater than or equal to a variance threshold, the image to be identified is determined to be an adversarial image; when the mean variance is less than the variance threshold, the image to be identified is determined not to be an adversarial image. Obtain the average value of the preset number of target feature vectors; The predicted classification result of the image to be identified is determined based on the similarity between the average value and the feature vector corresponding to the preset image.

9. An image recognition model training device, characterized in that, include: The first acquisition module is used to acquire the original sample image, the annotation information of the original sample image, and the adversarial sample image in each group of training samples in the training set. The annotation information is used to indicate the target object information present in the original sample image, and the adversarial sample image is an image obtained by adding adversarial perturbation to the original sample image. The second acquisition module is used to input the original sample image and the adversarial sample image into the image recognition model to be trained, and obtain the prediction classification result and adversarial sample image feature vector output by the image recognition model to be trained. The adversarial sample image feature vector includes a first feature vector and a second feature vector, wherein the first feature vector is the feature vector corresponding to the adversarial sample image output by the backbone network of the image recognition model to be trained, and the second feature vector is the feature vector corresponding to the copy image of the adversarial sample image output by the backbone network of the image recognition model to be trained. The first determining module is configured to determine a first loss based on the first feature vector and the second feature vector, and to determine a second loss based on the predicted classification result and the annotation information, and to determine whether there is a gradient conflict between the first gradient corresponding to the first loss and the second gradient corresponding to the second loss, wherein the first gradient is the gradient of the first loss relative to the parameters of the image recognition model to be trained, and the second gradient is the gradient of the second loss relative to the parameters of the image recognition model to be trained. The first processing module is configured to perform gradient conflict elimination processing on the first gradient and the second gradient in response to the existence of gradient conflict between the first gradient and the second gradient, and obtain the processed first gradient and second gradient. The second processing module is used to adjust the parameters of the image recognition model to be trained based on the processed first and second gradients, so as to obtain the trained image recognition model.

10. An image recognition device, characterized in that, include: The acquisition module is used to acquire the image to be recognized; The processing module is used to input the image to be recognized into the trained image recognition model to obtain the image recognition result output by the trained image recognition model. The image recognition result includes the predicted classification result corresponding to the image to be recognized and whether the image to be recognized is an adversarial image. The trained image recognition model is a model trained using the image recognition model training method as described in any one of claims 1 to 6.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the image recognition model training method as described in any one of claims 1 to 6, or the image recognition method as described in claim 7 or 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the image recognition model training method as described in any one of claims 1 to 6, or the image recognition method as described in claim 7 or 8.

Images