Image recognition method and device, electronic equipment and storage medium

Abstract

The invention provides an image recognition method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining an initial image; inputting the initial image into a pre-trained confrontation model, performing first calculation processing on the initial image through the confrontation model to obtain a first image, and performing second calculation processing on the initial image to obtain a second image; performing feature extraction on the first image to obtain a first feature, and performing feature extraction on the second image to obtain a second feature; and based on the first feature and the second feature, outputting an image category corresponding to the initial image and a data source address to which the initial image belongs through the adversarial model, thereby solving the technical problem of inaccurate image recognition in the prior art, and achieving the purpose of accurately recognizing the image.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image recognition method, device, electronic device and storage medium. Background Art

[0002] Generative Adversarial Network (GAN) has shown significant application potential in many fields such as image generation, text-to-image conversion, style transfer, etc. In recent years, the concept of GAN has been further extended to the generation and defense of adversarial samples, becoming one of the key technologies to improve the security of machine learning models. Especially in the fields of image recognition, natural language processing, and speech recognition, using GAN to generate adversarial samples has become an effective method to test and enhance the robustness of models.

[0003] In the prior art, generative adversarial networks have the following disadvantages: (1) Dependence on a specific model: Many existing adversarial sample generation techniques rely on the knowledge of the internal structure and parameters of the model, i.e., white-box attacks. This dependence significantly reduces the effectiveness of these techniques when faced with models of unknown structures (black-box models). (2) Single function: They can only generate adversarial samples, but cannot defend against adversarial samples. The above disadvantages of generative adversarial networks result in inaccurate image processing using models of generative adversarial networks. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose an image recognition method, device, electronic device and storage medium to overcome all or part of the deficiencies in the prior art.

[0005] Based on the above-mentioned purpose, the present application provides an image recognition method, including: obtaining an initial image; inputting the initial image into a pre-trained adversarial model, performing a first computational processing on the initial image through the adversarial model to obtain a first image, and performing a second computational processing on the initial image to obtain a second image; performing feature extraction on the first image to obtain a first feature, and performing feature extraction on the second image to obtain a second feature; based on the first feature and the second feature, outputting the image category and the data source address corresponding to the initial image through the adversarial model.

[0006] Optionally, performing a first calculation process on the initial image to obtain the first image includes: performing a first calculation process on the initial image using the following formula: z=x+ε·sign(▽ x J(x,y true ), where z is the first image, x is the initial image, ε is a hyperparameter for the noise addition intensity, sign(·) represents a sign function, is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

[0007] Optionally, performing a second computational process on the initial image to obtain the second image includes: adding noise to the initial image to obtain the second image.

[0008] Optionally, based on the first feature and the second feature, the image category and the data source address corresponding to the initial image are output through the adversarial model, including: splicing the first feature and the second feature to obtain a spliced ​​feature; performing feature extraction on the spliced ​​feature to obtain a target feature; and outputting the image category and the data source address corresponding to the target feature through the adversarial model.

[0009] Optionally, the loss function of the discriminator in the adversarial model is determined by the following formula, including: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents the expectation; the loss function of the generator in the adversarial model is determined by the following formula, including: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

[0010] Optionally, instance noise is added to the discriminator in the adversarial model.

[0011] Optionally, before inputting the initial image into a pre-trained adversarial model, the method includes: preprocessing the initial image.

[0012] Based on the same inventive concept, the present application also provides an image recognition device, including: an acquisition module, configured to acquire an initial image; a computational processing module, configured to input the initial image into a pre-trained adversarial model, perform a first computational processing on the initial image through the adversarial model to obtain a first image, and perform a second computational processing on the initial image to obtain a second image; a feature extraction module, configured to perform feature extraction on the first image to obtain a first feature, and perform feature extraction on the second image to obtain a second feature; an output module, configured to output the image category and the data source address corresponding to the initial image through the adversarial model based on the first feature and the second feature.

[0013] Optionally, the calculation processing module is further configured to: perform a first calculation processing on the initial image by using the following formula: Wherein, z is the first image, x is the initial image, ε is a hyperparameter of the noise adding intensity, sign(·) represents the sign function, is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

[0014] Optionally, the calculation and processing module is further configured to: add noise to the initial image to obtain the second image.

[0015] Optionally, the output module is further configured to: concatenate the first feature and the second feature to obtain a concatenated feature; extract features from the concatenated feature to obtain a target feature; and output the image category and the data source address corresponding to the target feature through the adversarial model.

[0016] Optionally, the device further includes a first determination module, and the first determination module is configured to: determine the loss function of the discriminator in the adversarial model by the following formula: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents expectation; the device also includes a second determination module, which is configured as follows: the loss function of the generator in the adversarial model is determined by the following formula: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

[0017] Optionally, the device further comprises an adding module, wherein the adding module is configured to add instance noise to the discriminator in the adversarial model.

[0018] Optionally, before inputting the initial image into the pre-trained adversarial model, the device further includes a preprocessing module, and the preprocessing module is configured to: preprocess the initial image.

[0019] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0020] Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described above.

[0021] As can be seen from the above, the image recognition method, device, electronic device and storage medium provided by the present application include obtaining an initial image, inputting the initial image into a pre-trained adversarial model, performing a first calculation process on the initial image through the adversarial model to obtain a first image, and performing a second calculation process on the initial image to obtain a second image, so that the relationship between pixels that are easily ignored in the initial image is made more accurate. Feature extraction is performed on the first image to obtain a first feature, and feature extraction is performed on the second image to obtain a second feature, which not only improves the processing efficiency and accuracy of the image, but also reduces the redundancy of image data. Based on the first feature and the second feature, the image category and the data source address corresponding to the initial image are output through the adversarial model, so as to achieve the purpose of accurately identifying the initial image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A schematic diagram of a flow chart of an image recognition method according to an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a framework of an adversarial model according to an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the network structure of the attack module and the defense module of an embodiment of the present application;

[0026] Figure 4 A schematic diagram of a network structure of a discrimination module according to an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the structure of an image recognition device according to an embodiment of the present application;

[0028] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0030] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0031] As described in the background technology section, generative adversarial networks have shown significant application potential in many fields such as image generation, text-to-image conversion, style transfer, etc. In recent years, the concept of generative adversarial networks has been further extended to the generation and defense of adversarial samples, becoming one of the key technologies to improve the security of machine learning models. Especially in the fields of image recognition, natural language processing, and speech recognition, using generative adversarial networks to generate adversarial samples has become an effective method to test and enhance the robustness of models. In the field of deep learning security, a variety of technical solutions for defending against adversarial samples have been developed, including common neural network architectures such as residual networks (Residual Networks, ResNet), adversarial training methods such as Adversarial Training, Trades method, and strategies based on generative adversarial networks such as Rob-GAN method. Among them, ResNet allows gradients to flow directly through multiple layers by introducing residual connections, which helps the model maintain the integrity of information during learning, thereby improving the stability of the model to input perturbations. Adversarial Training is the most basic form of adversarial training, which enhances the robustness of the model by adding adversarial samples to the training data and performing a standard training process on these samples. Trades is an improved adversarial training technique that considers not only accuracy but also the impact of adversarial samples when training models. Specifically, Trades balances the overall performance and adversarial robustness of the model by optimizing a combined objective function that includes clean sample loss and adversarial sample loss. Rob-GAN is a method that uses generative adversarial networks to enhance the adversarial robustness of models. In this method, the generator is trained to generate adversarial samples, while the discriminator tries to distinguish between real samples and adversarial samples. In this way, the discriminator gradually learns to identify and resist adversarial attacks, thereby improving the overall security of the model. Generative adversarial networks also include bidirectional generative adversarial networks (BiGAN), which include an encoder that maps data to a latent space, while the generator maps the vectors of the latent space back to the data space. This structure allows BiGAN to learn the representation and reverse mapping of generated data at the same time, making it particularly useful in feature extraction and anomaly detection, which is crucial for defending against adversarial sample attacks.

[0032] Common adversarial sample generation methods mainly include: Basic Iterative Method (BIM) and DeepFool method. BIM is an iterative version of FGSM. It fine-tunes adversarial samples through multiple steps to make the attack more effective. The DeepFool method aims to deceive the classifier by minimizing the necessary input perturbations and approximating the decision boundary of the classifier through an iterative process to generate adversarial samples with as little change as possible, thereby causing the model to misclassify. These methods of generating adversarial samples not only help researchers identify and strengthen the weaknesses of machine learning models, but also promote the development of defense mechanisms, such as enhancing the robustness of models by training models to recognize and resist these adversarial samples.

[0033] However, in the prior art, generative adversarial networks have the following disadvantages: (1) Dependence on a specific model: Many existing adversarial sample generation techniques rely on the knowledge of the internal structure and parameters of the model, i.e., white-box attacks. This dependence significantly reduces the effectiveness of these techniques when faced with models of unknown structures (black-box models). (2) Single function: It can only generate adversarial samples, but cannot defend against adversarial samples. The above disadvantages of generative adversarial networks result in inaccurate image processing using models of generative adversarial networks.

[0034] In view of this, the present application embodiment proposes an image recognition method, referring to Figure 1 , including the following steps:

[0035] Step 101, obtaining an initial image.

[0036] In this step, with the development of imaging technology, the number of images has increased dramatically. If users want to obtain relevant content of an image, they rely on the original record of the image. However, not all images have original records, and there may be cases where the original record of the image cannot be found. Therefore, if users want to understand an image, they first need to obtain the image. The above image is the initial image, so that the initial image can be processed accordingly later.

[0037] Step 102: input the initial image into a pre-trained adversarial model, perform a first computational processing on the initial image through the adversarial model to obtain a first image, and perform a second computational processing on the initial image to obtain a second image.

[0038] In this step, the initial image is processed using the trained adversarial model and input into the adversarial model, where the adversarial model includes a generative adversarial network, which consists of two parts: a generator and a discriminator. The goal of the generator is to create data samples that are realistic enough to deceive the discriminator, while the discriminator attempts to distinguish between generated samples and real samples. This setting allows the two to compete with each other during the training process and continuously improve their performance. In addition, the advantage of the generative adversarial network in simulating complex data distribution enables it to better understand and identify abnormal patterns when defending against adversarial attacks.

[0039] The adversarial model is used to perform a first computational processing on the initial image to obtain a first image, wherein the first computational processing makes relatively large changes to the initial image, wherein the first image is used to attack the adversarial model and the first image can be regarded as an attack image. The initial image is subjected to a second computational processing to obtain a second image, wherein the second computational processing makes relatively small changes to the initial image, wherein the second image is used to counter the attack on the first image and the second image can be regarded as an adversarial image. The initial image is processed respectively by two computational processing techniques, and the relatively simple initial image is relatively complicated, so that the relationship between pixels in the initial image that is easily overlooked is made, and the display of the initial image is made more accurate.

[0040] It should be noted that the framework of the adversarial model is Figure 2 As shown, Figure 2 E represents the attack module, through which the initial image is subjected to the second computational processing, and G represents the defense module, through which the initial image is subjected to the first computational processing. The network structure of the attack module and the defense module is shown in Figure 3 As shown in Figure 2. M represents the adversarial sample generation module, x represents the initial input image, and x~P x Represents the distribution P from the input image x Sample input image x, z~P z Represents the adversarial sample distribution P z Sample adversarial examples z, Indicates the output of clean data from the defense module Indicates the generation of adversarial samples from the attack module output distribution E(x) L c represents the classification loss of the discriminative module, L s represents the adversarial loss of the discriminative module, ∑ represents the summation symbol, L D represents the total loss of the discriminant module. H, F, and Q are all feature extractors.

[0041] Step 103: extract features from the first image to obtain first features, and extract features from the second image to obtain second features.

[0042] In this step, feature extraction technology can more accurately identify and process specific information in the image, whether through color, texture, shape or other types of features, which can significantly improve the efficiency and accuracy of image processing. The first image and the second image are images obtained through different computational processing. Therefore, it is necessary to perform feature extraction on the first image and the second image respectively through the adversarial model to obtain the first feature corresponding to the first image and the second feature corresponding to the second image. By extracting features from the first image and the second image, the processing efficiency and accuracy of the image are improved, and the redundancy of image data is reduced.

[0043] Step 104: Based on the first feature and the second feature, output the image category and the data source address corresponding to the initial image through the adversarial model.

[0044] In this step, based on the first feature and the second feature, the adversarial model performs classification processing, and can accurately output the image category and the data source address corresponding to the initial image, wherein the image category can reflect what category the initial image contains, for example, the image category is vehicle or person; the data source address reflects the source of the image, for example, which computer the image comes from. By outputting the image category and the data source address of the adversarial model, the user can obtain accurate information about the initial image without relying on the original record of the initial image, thereby achieving the purpose of accurately identifying the initial image.

[0045] Through the above scheme, an initial image is obtained, and the initial image is input into a pre-trained adversarial model, and the initial image is subjected to a first calculation process by the adversarial model to obtain a first image, and the initial image is subjected to a second calculation process to obtain a second image, so that the relationship between pixels that are easily ignored in the initial image is made more accurate. Feature extraction is performed on the first image to obtain a first feature, and feature extraction is performed on the second image to obtain a second feature, which not only improves the processing efficiency and accuracy of the image, but also reduces the redundancy of image data. Based on the first feature and the second feature, the image category and the data source address corresponding to the initial image are output through the adversarial model, thereby achieving the purpose of accurately identifying the initial image.

[0046] In some embodiments, performing a first computation on the initial image to obtain the first image includes: performing a first computation on the initial image using the following formula: z=x+ε·sign(▽ x J(x,y true )), where z is the first image, x is the initial image, ε is a hyperparameter for the noise addition intensity, sign(·) represents a sign function, ▽x J(·) is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

[0047] In this embodiment, the above formula is used to make the initial image disturbed accordingly, so that the change of the initial image is relatively large, aiming to make the adversarial model make wrong predictions or classifications. The above formula is derived from the Fast Gradient Sign Method (FGSM), which uses the gradient information of the input data to generate new samples that mislead the model. The first image is an image used to influence the decision of the adversarial model, aiming to reveal the potential security vulnerabilities of the adversarial model and promote the development of defense mechanisms. It can not only generate a first image with adversarial effects, but also effectively defend against adversarial models, generate adversarial network defenses, build a safer application environment, eliminate external interference with adversarial models, and thus make the subsequent recognition of adversarial models accurate.

[0048] In some embodiments, performing a second computational process on the initial image to obtain the second image includes: adding noise to the initial image to obtain the second image.

[0049] In this embodiment, noise is added to the initial image. The noise refers to unnecessary or redundant interference information existing in the image data. It can be various, for example, salt and pepper noise and Gaussian noise. These noises can simulate various interference factors in the real environment, such as noise of electronic equipment, interference in the transmission process, etc. This embodiment simulates the interference factors existing in the real environment, and can more realistically simulate the image quality in the actual environment, so as to better evaluate the performance and effect of the image processing algorithm. It can generate an adversarial second image without knowing the detailed internal structure of the adversarial model, and the process of adding noise to the initial image does not depend on gradient information, and the execution speed is faster.

[0050] In some embodiments, based on the first feature and the second feature, the image category and the data source address corresponding to the initial image are output through the adversarial model, including: splicing the first feature and the second feature to obtain a spliced ​​feature; performing feature extraction on the spliced ​​feature to obtain a target feature; and outputting the image category and the data source address corresponding to the target feature through the adversarial model.

[0051] In this embodiment, the first feature and the second feature are spliced ​​to obtain a spliced ​​feature, which contains both features and can reflect relatively comprehensive features of the initial image. Feature extraction is performed on the spliced ​​feature so that the features in the spliced ​​feature can be fully extracted to obtain the target feature. Given the target feature, the adversarial model outputs the image category and the data source address, thereby achieving the purpose of accurately identifying the initial image.

[0052] It should be noted that adversarial models also include Figure 4 The network structure of the discriminant module is shown, and feature extraction is performed in the discriminant module.

[0053] In some embodiments, the loss function of the discriminator in the adversarial model is determined by the following formula, including: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents the expectation; the loss function of the generator in the adversarial model is determined by the following formula, including: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

[0054] In this embodiment, the loss function, also called the cost function, is a function that maps the value of a random event or random variable to a non-negative real number, and is used to represent the "risk" or "loss" of the random event. The input image in the loss function is an image pair, and the image pair includes an initial image and a first image. In the discriminator, the loss function is mainly used to evaluate and optimize the performance of the model, measure the degree of inconsistency between the true value and the predicted value, that is, measure the accuracy of the model prediction and guide the model to optimize, thereby improving the accuracy of classification and the performance of the model.

[0055] The reconstruction loss of the generator is calculated by the following formula:

[0056] Among them, L p For perceptual loss, VGG-16 network is used as F(·) to extract image features, L e is the pixel loss and λ is the weight factor.

[0057] The role of the loss function in the generator is to measure the degree of error between the model's predicted results and the actual results, thereby optimizing the model performance.

[0058] The loss function in the generator has the following functions:

[0059] (1) Measuring model performance: The loss function provides a way to measure model performance by calculating the error between the model's predictions and the actual results. A smaller loss value means that the model's predictions are closer to the actual results, while a smaller loss value means that the error is larger, which helps to evaluate the accuracy and reliability of the model.

[0060] (2) Parameter optimization: When training machine learning and deep learning models, the loss function is used as the objective function of the optimization algorithm. By minimizing the loss function, the parameters of the model can be adjusted so that the model can better approximate the real results. This process is crucial to improving the model's predictive and generalization capabilities.

[0061] (3) Back propagation: In deep learning, the gradient of the loss function with respect to the model parameters is calculated through the back propagation algorithm. These gradients are used to update the parameters in order to optimize the model. The loss function plays an important role in back propagation, guiding the adjustment direction of the parameters and ensuring that the model can continue to improve.

[0062] In some embodiments, instance noise is added to the discriminator in the adversarial model.

[0063] In this embodiment, increasing the training difficulty of the discriminator is beneficial to improving the robustness of the adversarial model. Adding instance noise ψ to the discriminator D is done by the following formula:

[0064] D(x,z)→D(x+ψ,z),ψ∈N(μ,σ 2 ),

[0065] Among them, μ represents the mean of normal distribution N, σ represents the standard deviation of normal distribution N, T cur Indicates the current iteration number, T max represents the maximum number of iterations, σ t represents the current standard deviation, σ min Represents the current minimum standard deviation, σ max Indicates the current maximum standard deviation.

[0066] In some embodiments, before inputting the initial image into a pre-trained adversarial model, the method includes: pre-processing the initial image.

[0067] In this embodiment, image preprocessing is an important step in the fields of image analysis, machine learning, and computer vision. It involves a series of operations and transformations on the input image, aiming to make the image easier to process and improve the performance of the model. These operations include, but are not limited to, resizing, normalization, cropping, and enhancement. The purpose of preprocessing is to ensure the quality and consistency of image data, because in practical applications, the quality of image samples is often not uniform, which may affect the training and prediction effects of the model. The main function of image preprocessing is to eliminate irrelevant information in the image, enhance the detectability of relevant information, and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching, and recognition.

[0068] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0069] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0070] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an image recognition device.

[0071] refer to Figure 5 , the image recognition device comprises:

[0072] The acquisition module 10 is configured to acquire an initial image.

[0073] The computing and processing module 20 is configured to input the initial image into a pre-trained adversarial model, perform a first computing and processing on the initial image through the adversarial model to obtain a first image, and perform a second computing and processing on the initial image to obtain a second image.

[0074] The feature extraction module 30 is configured to perform feature extraction on the first image to obtain a first feature, and to perform feature extraction on the second image to obtain a second feature.

[0075] The output module 40 is configured to output the image category and the data source address corresponding to the initial image through the adversarial model based on the first feature and the second feature.

[0076] Through the above-mentioned device, an initial image is obtained, and the initial image is input into a pre-trained adversarial model. The initial image is subjected to a first calculation process by the adversarial model to obtain a first image, and the initial image is subjected to a second calculation process to obtain a second image, so that the relationship between pixels that are easily ignored in the initial image is made more accurate. Feature extraction is performed on the first image to obtain a first feature, and feature extraction is performed on the second image to obtain a second feature, which not only improves the processing efficiency and accuracy of the image, but also reduces the redundancy of image data. Based on the first feature and the second feature, the image category and the data source address corresponding to the initial image are output through the adversarial model, so as to achieve the purpose of accurately identifying the initial image.

[0077] In some embodiments, the calculation processing module 20 is further configured to: perform a first calculation processing on the initial image by the following formula: z=x+ε·sign(▽ x J(x,y true )), where z is the first image, x is the initial image, ε is a hyperparameter for the noise addition intensity, sign(·) represents a sign function, ▽ x J(·) is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

[0078] In some embodiments, the computing and processing module 20 is further configured to: add noise to the initial image to obtain the second image.

[0079] In some embodiments, the output module 40 is further configured to: concatenate the first feature and the second feature to obtain a concatenated feature; extract features from the concatenated feature to obtain a target feature; and output the image category and the data source address corresponding to the target feature through the adversarial model.

[0080] In some embodiments, the apparatus further includes a first determination module, wherein the first determination module is configured to: determine the loss function of the discriminator in the adversarial model by the following formula: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents expectation; the device also includes a second determination module, which is configured as follows: the loss function of the generator in the adversarial model is determined by the following formula: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

[0081] In some embodiments, the apparatus further comprises an adding module configured to: add instance noise to the discriminator in the adversarial model.

[0082] In some embodiments, before inputting the initial image into a pre-trained adversarial model, the device further includes a preprocessing module, wherein the preprocessing module is configured to: preprocess the initial image.

[0083] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0084] The device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0085] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image recognition method as described in any of the above embodiments is implemented.

[0086] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0087] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0088] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0089] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0090] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0091] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0092] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0093] The electronic device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0094] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image recognition method described in any of the above embodiments.

[0095] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0096] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image recognition method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0097] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the image recognition method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0098] It should be noted that the embodiments of the present application can be further described in the following manner:

[0099] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0100] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0101] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0102] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0103] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0104] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0105] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0106] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for recognizing an image, characterized in that: include: Get the initial image; Inputting the initial image into a pre-trained adversarial model, performing a first computational process on the initial image through the adversarial model to obtain a first image, and performing a second computational process on the initial image to obtain a second image; Performing feature extraction on the first image to obtain a first feature, and performing feature extraction on the second image to obtain a second feature; Based on the first feature and the second feature, the adversarial model outputs the image category and the data source address corresponding to the initial image.

2. The method according to claim 1, characterized in that: The performing a first calculation process on the initial image to obtain a first image includes: The initial image is first calculated using the following formula: z=x+ε·sign(▽ x J(x,y true )), Wherein, z is the first image, x is the initial image, ε is the hyperparameter of the noise adding intensity, sign(·) represents the sign function, ▽ x J(·) is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

3. The method according to claim 1, characterized in that The performing a second calculation process on the initial image to obtain a second image includes: Noise is added to the initial image to obtain the second image.

4. The method according to claim 1, characterized in that The step of outputting the image category and the data source address corresponding to the initial image through the adversarial model based on the first feature and the second feature includes: Splicing the first feature and the second feature to obtain a spliced ​​feature; Performing feature extraction on the splicing features to obtain target features; The adversarial model outputs the image category and the data source address corresponding to the target feature.

5. The method according to claim 1, characterized in that: The loss function of the discriminator in the adversarial model is determined by the following formula, including: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents expectation; The loss function of the generator in the adversarial model is determined by the following formula, including: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

6. The method according to claim 1, characterized in that Add instance noise to the discriminator in the adversarial model.

7. The method according to claim 1, characterized in that Before inputting the initial image to the pre-trained adversarial model, the method includes: The initial image is preprocessed.

8. An image recognition device, characterized in that: include: An acquisition module, configured to acquire an initial image; a computing and processing module configured to input the initial image into a pre-trained adversarial model, perform a first computing and processing on the initial image through the adversarial model to obtain a first image, and perform a second computing and processing on the initial image to obtain a second image; a feature extraction module, configured to perform feature extraction on the first image to obtain a first feature, and to perform feature extraction on the second image to obtain a second feature; The output module is configured to output the image category and the data source address corresponding to the initial image through the adversarial model based on the first feature and the second feature.

9. The device according to claim 8, characterized in that The computing and processing module is further configured to: The initial image is first calculated using the following formula: z=x+ε·sign(▽ x J(x,y true )), Wherein, z is the first image, x is the initial image, ε is the hyperparameter of the noise adding intensity, sign(·) represents the sign function, ▽ x J(·) is the gradient of the generator’s loss function relative to the initial image, y true is the category label corresponding to the initial image.

10. The device according to claim 8, characterized in that The computing and processing module is further configured to: Noise is added to the initial image to obtain the second image.

11. The device according to claim 8, characterized in that The output module is further configured as: Splicing the first feature and the second feature to obtain a spliced ​​feature; Performing feature extraction on the splicing features to obtain target features; The adversarial model outputs the image category and the data source address corresponding to the target feature.

12. The device according to claim 8, characterized in that The device further includes a first determining module, which is configured to: determine the loss function of the discriminator in the adversarial model by the following formula: Among them, L D is the loss function of the discriminator, is the adversarial loss of the discriminator, is the classification loss of the discriminator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S1 represents the output of the discriminator identifying the source of the input image, and the corresponding loss is C1 represents the output of the discriminator to identify the category of the input image, and the corresponding loss is c1 represents the input image category recognized by the discriminator, and E represents expectation; The device also includes a second determination module, which is configured to: determine the loss function of the generator in the adversarial model by the following formula: Among them, L GE is the loss function of the generator, is the adversarial loss of the generator, is the classification loss of the generator, L r is the reconstruction loss of the generator, P(·) is the predicted probability, z is the first image, x is the initial image, real represents the label of real data, fake represents the label of fake data, is the initial image reconstructed by the generator in the adversarial model, is the first image generated by the generator in the adversarial model, S2 represents the output of the generator recognizing the input image, and the corresponding loss is C2 represents the output of the generator to identify the category of the input image, and the corresponding loss is c2 represents the input image category recognized by the generator, and E represents expectation.

13. The device according to claim 8, characterized in that The apparatus further includes an adding module, which is configured to add instance noise to the discriminator in the adversarial model.

14. The device according to claim 8, characterized in that Before inputting the initial image into the pre-trained adversarial model, the apparatus further includes a pre-processing module, wherein the pre-processing module is configured to: The initial image is preprocessed.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images