Method for obtaining image recognition model and image recognition method

Abstract

The application provides an image recognition model obtaining method, an image recognition method, an image recognition device, an electronic device and a computer readable storage medium. The method comprises: obtaining an image adversarial sample, the image adversarial sample carrying a sample label; performing feature extraction processing on the image adversarial sample through an image recognition model to obtain a corresponding sample image feature; determining a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label based on the sample image feature, the interference label being different from the sample label; performing image recognition on the image adversarial sample based on the first mapping feature and the second mapping feature to obtain a corresponding first recognition result; and updating model parameters of the image recognition model based on the sample label and the first recognition result.

Description

TECHNICAL FIELD

[0001] The present application relates to computer technology, and particularly relates to an image recognition model obtaining method and an image recognition method. BACKGROUND

[0002] In the current adversarial training, a model usually directly learns features corresponding to adversarial samples, so that the model can predict accurate results after learning the feature range corresponding to the adversarial samples. The model in this adversarial training mode only focuses on the feature region corresponding to the label during the training process, and the feature focus region is small. The accuracy of the training set can reach a high level, but the accuracy in the test set is not high, the model has weak generalization ability and low robustness. SUMMARY

[0003] The image recognition model obtaining method, the image recognition method, the device, the electronic equipment and the computer readable storage medium provided in the embodiments of the present application can expand the feature learning range of the model, and improve the generalization ability and robustness of the model.

[0004] The technical solutions of the embodiments of the present application are as follows:

[0005] The image recognition model obtaining method provided in the embodiments of the present application comprises the following steps.

[0006] An image adversarial sample is obtained, and the image adversarial sample carries a sample label.

[0007] A feature extraction process is performed on the image adversarial sample by using an image recognition model, and a corresponding sample image feature is obtained.

[0008] Based on the sample image feature, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label are determined, and the interference label is different from the sample label.

[0009] Based on the first mapping feature and the second mapping feature, image recognition is performed on the image adversarial sample, and a corresponding first recognition result is obtained.

[0010] Based on the sample label and the first recognition result, model parameters of the image recognition model are updated.

[0011] In the above solution, the image recognition based on the first mapping feature and the second mapping feature and the first recognition result obtained by the image recognition comprises the following steps.

[0012] The first mapping feature and the second mapping feature are spliced to obtain a corresponding spliced feature.

[0013] perform image recognition on the image adversarial sample based on the splicing feature to obtain the first recognition result.

[0014] In the scheme, the splicing of the first mapping feature and the second mapping feature comprises:

[0015] determining a first ratio corresponding to the first mapping feature and a second ratio corresponding to the second mapping feature;

[0016] performing weighted summation on the first mapping feature and the second mapping feature based on the first ratio and the second ratio.

[0017] In the scheme, the image recognition model comprises a first prediction layer and a second prediction layer, the first mapping feature, the second mapping feature and the first recognition result are obtained through the first prediction layer, and the method further comprises:

[0018] determining, through the second prediction layer, a third mapping feature corresponding to the sample label based on the sample image feature;

[0019] performing image recognition on the image adversarial sample through the second prediction layer based on the third mapping feature to obtain a corresponding second recognition result;

[0020] updating the model parameters of the image recognition model based on the sample label and the first recognition result comprises:

[0021] updating the model parameters of the image recognition model based on the sample label, the first recognition result and the second recognition result.

[0022] In the scheme, updating the model parameters of the image recognition model based on the sample label, the first recognition result and the second recognition result comprises:

[0023] determining a first error between the sample label and the first recognition result and a second error between the sample label and the second recognition result;

[0024] updating the model parameters of the image recognition model based on the sum of the first error and the second error.

[0025] In the scheme, the feature extraction processing on the image adversarial sample to obtain the corresponding sample image feature comprises:

[0026] performing feature extraction on the image adversarial sample to obtain a corresponding first feature;

[0027] The first feature is subjected to image style transfer processing to obtain the corresponding second feature;

[0028] The first feature and the second feature are fused to obtain the sample image features.

[0029] In the above scheme, obtaining image adversarial examples includes:

[0030] Obtain a clean image sample, wherein the clean image sample carries the sample label;

[0031] Noise is added to the clean sample of the image to obtain the adversarial sample of the image.

[0032] This application provides an image recognition method, including:

[0033] Obtain the image to be recognized;

[0034] The image to be identified is processed by feature extraction using an image recognition model to obtain the corresponding image features;

[0035] Based on the image features, a fourth mapping feature corresponding to the recognition result and a fifth mapping feature corresponding to the interference result are determined;

[0036] Based on the fourth mapping feature and the fifth mapping feature, image recognition is performed on the image to be recognized to obtain the recognition result.

[0037] This application provides an apparatus for obtaining an image recognition model, comprising:

[0038] The acquisition module is used to acquire image adversarial samples, wherein the image adversarial samples carry sample labels;

[0039] The feature extraction module is used to perform feature extraction processing on the adversarial examples in the image through an image recognition model to obtain the corresponding sample image features;

[0040] The determining module is used to determine, based on the features of the sample image, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label, wherein the interference label is different from the sample label;

[0041] The recognition module is used to perform image recognition on the adversarial sample of the image based on the first mapping feature and the second mapping feature, and obtain the corresponding first recognition result;

[0042] The parameter update module is used to update the model parameters of the image recognition model based on the sample label and the first recognition result.

[0043] This application provides an image recognition device, including:

[0044] The acquisition module is used to acquire the image to be recognized;

[0045] The feature extraction module is used to perform feature extraction processing on the image to be recognized through an image recognition model to obtain the corresponding image features;

[0046] The determining module is used to determine, based on the image features, a fourth mapping feature corresponding to the recognition result and a fifth mapping feature corresponding to the interference result;

[0047] The recognition module is used to perform image recognition on the image to be recognized based on the fourth mapping feature and the fifth mapping feature, and obtain the recognition result.

[0048] This application provides an electronic device, including:

[0049] Memory, used to store executable instructions;

[0050] A processor, when executing executable instructions stored in the memory, implements the method provided in the embodiments of this application.

[0051] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the method provided in this application.

[0052] This application embodiment obtains adversarial images carrying sample labels, performs feature extraction processing on the adversarial images using an image recognition model to obtain corresponding sample image features, and then determines a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label based on the sample image features. The interference label is different from the sample label. Based on the first mapping feature and the second mapping feature, the image adversarial images are recognized to obtain a corresponding first recognition result. Finally, based on the sample label and the first recognition result, the model parameters of the image recognition model are updated so that the model can pay attention to the second mapping feature corresponding to the interference label other than the sample label. While learning the mapping feature of the interference label, it can also predict the prepared sample label, thus expanding the feature learning range of the model and improving the model's generalization ability and robustness. Attached Figure Description

[0053] Figure 1 This is an optional schematic diagram illustrating the feature range that the model focuses on during the training process of the image samples provided in this application embodiment;

[0054] Figure 2 This is an optional structural schematic diagram of the electronic device 200 provided in the embodiments of this application;

[0055] Figure 3 This is an optional flowchart illustrating the method for obtaining an image recognition model provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of an optional model structure of the image recognition model provided in this application embodiment;

[0057] Figure 5 This is an optional detailed flowchart of step 304 provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of an optional model structure of the image recognition model provided in this application embodiment;

[0059] Figure 7 This is an optional flowchart illustrating the method for obtaining an image recognition model provided in an embodiment of this application;

[0060] Figure 8 This is an optional detailed flowchart of step 302 provided in an embodiment of this application;

[0061] Figure 9 This is an optional structural diagram of the image recognition model provided in the embodiments of this application;

[0062] Figure 10A This is a schematic diagram of an optional accuracy result of the standard generalization method provided in the embodiments of this application;

[0063] Figure 10B This is a schematic diagram of an optional accuracy result of the robust generalization method provided in the embodiments of this application;

[0064] Figure 11A This is an optional schematic diagram of the image features extracted from the penultimate layer of the image recognition model obtained by the method provided in the embodiments of this application;

[0065] Figure 11B This is an optional schematic diagram of the image features extracted from the penultimate layer of an image recognition model obtained through related technical methods;

[0066] Figure 12 This is an optional flowchart of the image recognition method provided in the embodiments of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0069] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0071] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0072] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0073] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0074] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0075] The solutions provided in this application involve technologies such as machine learning in artificial intelligence, and are specifically illustrated through the following embodiments.

[0076] Among related technologies, deep neural networks (DNNs, Dense Nearest Neighbor Search) have achieved remarkable success in solving complex prediction tasks. However, they are particularly vulnerable to adversarial attacks, which can cause DNNs to make prediction errors by making small perturbations to the input. How to defend against adversarial attack samples has been extensively studied. Among the various existing defense strategies, adversarial training (AT) is one of the most effective defense methods.

[0077] Adversarial training (AT) refers to training a network on a dataset of adversarial examples. Therefore, adversarially trained models are more robust to adversarial examples than standard-trained models. DNNs trained via AT typically suffer from low robustness and generalization; that is, DNNs have high training robustness but low testing robustness. Furthermore, while AT improves the network's robustness to adversarial attacks, it severely impacts the standard accuracy on clean data.

[0078] To address the above issues, we experimentally explored the differences in the training phase between the standard model (trained with clean samples without AT) and the robust model (trained with AT). See [link to relevant documentation]. Figure 1 , Figure 1 This is an optional schematic diagram illustrating the feature range that the model focuses on during the training process of the image samples provided in this application embodiment. Figure 1Five columns of graphs are shown: (a) Column of graphs shows clean image samples, with a cat, a boat, and a dog in the top, middle, and bottom images, respectively. (b) Column of graphs shows the feature range of the standard model during the prediction phase. (c) Column of graphs shows the feature range of the robust model during the prediction phase. (d) Column of graphs shows the difference in feature ranges between the standard and robust models during the prediction phase. (e) Column of graphs shows the result after partially erasing features from the image sample using column of graphs (d) as a mask. It can be seen that for the same image sample, the standard model and the robust model have different feature ranges of interest, with the standard model generally having a larger feature range than the robust model. Through further experiments, we found that these different ranges of interest (see...) Figure 1 (d) and Table 1) affected the model's decisions on clean samples, but had little effect on its decisions on adversarial samples. This indicates that AT enables the model to learn robust features but neglects generalization features. In other words, adversarially trained DNNs tend to focus on specific visual concepts on training images, thus limiting their test robustness. This prompts us to design a method to broaden the range of features to focus on, ensuring that the model pays attention to robust features while enhancing attention to other features to improve the generalization ability of the robust model.

[0079] Table 1 shows the model's prediction accuracy for both raw and corrupted data.

[0080]

[0081] Based on this, embodiments of this application provide a method for obtaining an image recognition model, an image recognition method, an apparatus, an electronic device, and a computer-readable storage medium, which can expand the feature learning range of the model and improve the robustness of the model.

[0082] First, the method for obtaining the above-described image recognition model and the electronic device for implementing the image recognition method, as provided in the embodiments of this application, will be described. See [link to relevant documentation]. Figure 2 , Figure 2 This is an optional structural diagram of the electronic device 200 provided in this application embodiment. In practical applications, the electronic device 200 can be implemented as a terminal or a server. The terminal can be a laptop, tablet, desktop computer, smartphone, dedicated messaging device, portable gaming device, smart speaker, smartwatch, etc., but is not limited to these. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) services, and big data and artificial intelligence platforms.Figure 2 The illustrated electronic device 200 includes at least one processor 201, a memory 205, at least one network interface 202, and a user interface 203. The various components in the electronic device 200 are coupled together via a bus system 204. It is understood that the bus system 204 is used to implement communication between these components. In addition to a data bus, the bus system 204 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 204.

[0083] Processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0084] User interface 203 includes one or more output devices 2031 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 203 also includes one or more input devices 2032, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0085] The memory 205 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 205 may optionally include one or more storage devices physically located away from the processor 201.

[0086] The memory 205 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 205 described in this application embodiment is intended to include any suitable type of memory.

[0087] In some embodiments, the memory 205 can store data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof. In this embodiment, the memory 205 stores an operating system 2051, a network communication module 2052, a presentation module 2053, an input processing module 2054, and an image recognition model acquisition device 2055. Specifically,

[0088] Operating system 2051 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0089] The network communication module 2052 is used to reach other computing devices via one or more (wired or wireless) network interfaces 202, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0090] The presentation module 2053 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 2031 (e.g., a display screen, a speaker, etc.) associated with the user interface 203;

[0091] The input processing module 2054 is used to detect and translate one or more user inputs or interactions from one or more input devices 2032.

[0092] In some embodiments, the image recognition model acquisition device provided in this application can be implemented in software. Figure 2 An image recognition model acquisition device 2055 stored in memory 205 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: acquisition module 20551, feature extraction module 20552, determination module 20553, recognition module 20554, and parameter update module 20555. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0093] In other embodiments, the image recognition model acquisition device provided in this application can be implemented in hardware. As an example, the image recognition model acquisition device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image recognition model acquisition method and image recognition method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0094] The following will describe the method for obtaining the image recognition model and the image recognition method provided in the embodiments of this application, with reference to the exemplary application and implementation of the terminal provided in the embodiments of this application.

[0095] See Figure 3 , Figure 3 This is an optional flowchart illustrating a method for obtaining an image recognition model provided in an embodiment of this application, which will be combined with... Figure 3 The steps shown are explained.

[0096] Step 301: Obtain image adversarial samples, wherein the image adversarial samples carry sample labels;

[0097] Step 302: Using an image recognition model, feature extraction processing is performed on the adversarial examples in the images to obtain the corresponding sample image features;

[0098] Step 303: Based on the features of the sample image, determine a first mapping feature corresponding to the sample label and a second mapping feature corresponding to the interference label, wherein the interference label is different from the sample label;

[0099] Step 304: Based on the first mapping feature and the second mapping feature, perform image recognition on the adversarial sample in the image to obtain the corresponding first recognition result;

[0100] Step 305: Update the model parameters of the image recognition model based on the sample labels and the first recognition result.

[0101] Here, the image adversarial example is a sample obtained by adding noise to a clean image sample. In some embodiments, obtaining the image adversarial example can be achieved by: obtaining a clean image sample carrying the sample label; adding noise to the clean image sample to obtain the image adversarial example. In practice, the clean image sample carries a sample label, and the clean image sample is the original image sample, with a direct correspondence between it and the sample label, that is, there is a clear correspondence between the image features of the clean image sample and the sample label. However, for the image adversarial example obtained by adding noise, the region corresponding to the image features of the clean image sample may also be affected by noise interference. This application embodiment uses image adversarial examples for model training, enabling the model to learn the features after adding noise, thereby improving the model's generalization ability.

[0102] In practical implementation, the terminal inputs adversarial examples into an image recognition model. The image recognition model performs feature extraction on the adversarial examples to obtain corresponding sample image features. Here, sample image features include features of all regions of the adversarial example. Specifically, the terminal inputs the adversarial examples into the feature extraction layer of the image recognition model. Through the feature extraction layer, the adversarial examples are feature-encoded, and the image features of the adversarial examples are extracted to obtain sample image features. Then, based on the sample image features, the image recognition model determines a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label, where the interference label is different from the sample label. Specifically, the output layer of the image recognition model includes a first prediction layer. After the feature extraction layer extracts the sample image features, it inputs the sample image features into the first prediction layer. The first prediction layer identifies the first mapping feature corresponding to the sample label and the second mapping feature corresponding to the interference label in the sample image features. The interference label is different from the sample label. It should be noted that if the image recognition model can recognize N types, where N is a positive integer greater than or equal to 2, then both the sample label and the interference label belong to at least one of the N types, and the number of interference labels can be one or more. The output layer of the image recognition model can obtain mapping features corresponding to each of the N classes. After obtaining the first and second mapping features, the terminal uses the image recognition model to perform image recognition on adversarial examples based on the first and second mapping features, obtaining a first recognition result. Then, based on the sample labels and the first recognition result, the model parameters of the image recognition model are updated. In practical implementation, the model parameters of the image recognition model can be updated based on the error between the sample labels and the first recognition result. Specifically, the error between the sample labels and the first recognition result can be calculated using the cross-entropy loss function. For example, see... Figure 4 , Figure 4This is an optional model structure diagram of the image recognition model provided in the embodiments of this application.

[0103] In some embodiments, the terminal can train the image recognition model in the following ways:

[0104] During training, when the value of the objective function reaches the first threshold, the corresponding error signal is determined based on the objective function. The error signal is then backpropagated in the image recognition model, and the model parameters of each layer of the image recognition model are updated during the propagation process.

[0105] Here's an explanation of backpropagation: Training samples are input into the input layer of a neural network model, pass through the hidden layers, and finally reach the output layer to output the result. This is the forward propagation process of the neural network model. Since there is an error between the output result of the neural network model and the actual result, the error between the output result and the actual value is calculated and propagated backward from the output layer to the hidden layers until it reaches the input layer. During the backpropagation process, the values ​​of the model parameters are adjusted according to the error. The above process is iterated continuously until convergence.

[0106] Taking the objective function as an example, the terminal determines the error signal based on the objective function. The error signal propagates backward from the output layer of the image recognition model, and the first error signal propagates backward layer by layer. When the first error signal reaches each layer, the gradient (that is, the partial derivative of the Loss function with respect to the parameters of the layer) is solved in combination with the propagated first error signal, and the corresponding gradient value of the parameters of the layer is updated.

[0107] By obtaining adversarial examples of images carrying sample labels, an image recognition model is used to extract features from the adversarial examples to obtain corresponding sample image features. Then, based on the sample image features, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to the interference label are determined. The interference label is different from the sample label. Based on the first mapping feature and the second mapping feature, image recognition is performed on the adversarial examples to obtain a corresponding first recognition result. Finally, based on the sample label and the first recognition result, the model parameters of the image recognition model are updated so that the model can pay attention to the second mapping feature corresponding to the interference label other than the sample label. While learning the mapping feature of the interference label, it can also predict the prepared sample label, thus expanding the feature learning range of the model and improving the robustness of the model.

[0108] In some embodiments, see Figure 5 , Figure 5 This is an optional detailed flowchart of step 304 provided in the embodiments of this application. Step 304 includes:

[0109] Step 3041: Perform feature concatenation on the first mapping feature and the second mapping feature to obtain the corresponding concatenated feature;

[0110] Step 3042: Based on the splicing features, perform image recognition on the adversarial sample of the image to obtain the first recognition result.

[0111] In practice, the terminal performs feature concatenation on the first and second mapping features. Specifically, the terminal concatenates the first and second mapping features along a feature dimension, combining features from two different feature ranges into a larger feature range. For example, if the feature range corresponding to the first mapping feature belongs to region A in the adversarial example, and the feature range corresponding to the second mapping feature belongs to region B in the adversarial example, then the feature range corresponding to the concatenated feature belongs to region A+B in the adversarial example. It should be understood that the size of region A+B is not necessarily equal to the sum of the sizes of region A and region B. If regions A and B have overlapping areas, the size of the feature range of the concatenated feature will further exclude the overlapping areas. Then, the terminal performs image recognition on the adversarial example based on the concatenated features, enabling the image recognition model to learn the concatenated features and predict the sample label corresponding to the first mapping feature while also focusing on a larger range of concatenated features, thus providing robustness to the model.

[0112] In some embodiments, step 3041 includes: determining a first ratio corresponding to the first mapping feature and a second ratio corresponding to the second mapping feature; and performing a weighted summation on the first mapping feature and the second mapping feature based on the first ratio and the second ratio.

[0113] In practice, the first ratio and the second ratio can have preset or random initial values. The sum of the first ratio and the second ratio is 1. The terminal performs a weighted summation of the first mapping feature and the second mapping feature based on the first ratio and the second ratio, and uses the result of the weighted summation as the concatenated feature. The first ratio and the second ratio can be used as model parameters and updated during model training. For example, the concatenated feature can be obtained using the following formula:

[0114]

[0115] in, For splicing features, A l The sample image features are defined by α, where α is the first ratio and 1-α is the second ratio. The feature weights are the feature dimensions corresponding to the first mapped feature. The feature weights are the feature dimensions corresponding to the second mapping feature. It is a Cartesian product.

[0116] In practical implementation, in the first prediction layer of the image recognition model, average pooling is used to reduce the dimensionality of the sample image features, and then Cartesian product is used to increase the dimensionality of the reduced features. The feature concatenation method provided in this application embodiment allows for quick and convenient acquisition of the first mapping features based on the sample image features. and the second mapping feature It also achieves feature concatenation of the first mapping feature and the second mapping feature.

[0117] In some embodiments, see Figure 6 , Figure 6 This is a schematic diagram of an optional model structure of the image recognition model provided in this application embodiment. The output layer of the image recognition model includes a first prediction layer and a second prediction layer, wherein the first mapping feature, the second mapping feature, and the first recognition result are obtained through the first prediction layer. See also Figure 7 , Figure 7 This is an optional flowchart illustrating a method for obtaining an image recognition model provided in an embodiment of this application. Before step 305, the method further includes:

[0118] Step 701: Based on the sample image features, determine the third mapping feature corresponding to the sample label through the second prediction layer;

[0119] Step 702: Through the second prediction layer, based on the third mapping feature, perform image recognition on the adversarial sample of the image to obtain the corresponding second recognition result;

[0120] Step 305 includes: Step 3051, updating the model parameters of the image recognition model based on the sample label, the first recognition result, and the second recognition result.

[0121] In practical implementation, after obtaining the sample image features, the terminal inputs these features into the first prediction layer and the second prediction layer respectively. The second prediction layer, based on the sample image features, determines the third mapping feature corresponding to the sample label, and directly performs image recognition of adversarial examples based on the third mapping feature, obtaining the second recognition result. Then, based on the sample label, the first recognition result, and the second recognition result, the model parameters of the image recognition model are updated. It should be understood that both the first and second recognition results represent the probability that the model recognizes the adversarial example as the sample label. Furthermore, the second prediction layer provided in this embodiment only focuses on the mapping features corresponding to the sample label, and not on the mapping features under other labels. It performs corresponding feature processing based on the mapping features corresponding to the sample label, enabling the model to learn the features corresponding to the sample label, thereby learning more robust features and further improving the model's robustness.

[0122] In some embodiments, step 3051 is implemented by: determining a first error between the sample label and the first recognition result and a second error between the sample label and the second recognition result; and updating the model parameters of the image recognition model based on the sum of the first error and the second error.

[0123] In practical implementation, the terminal calculates a first error between the sample label and the first recognition result, and a second error between the sample label and the second recognition result. Then, the first and second errors are summed to obtain a total error. Based on this total error, the model parameters of the image recognition model are updated. For example, the total error can be obtained using the following loss function:

[0124]

[0125] Where L is the sum of errors, L CE () is the cross-entropy function, F ori (x adv The first identification result is ) y represents the first identification result, and y represents the sample label.

[0126] In some embodiments, see Figure 8 , Figure 8 This is an optional detailed flowchart of step 302 provided in the embodiments of this application. Step 302 includes:

[0127] Step 3021: Extract features from the adversarial examples in the image to obtain the corresponding first features;

[0128] Step 3022: Perform image style transfer processing on the first feature to obtain the corresponding second feature;

[0129] Step 3023: Perform feature fusion between the first feature and the second feature to obtain the sample image features.

[0130] In actual implementation, see Figure 9 , Figure 9 This is an optional structural diagram of the image recognition model provided in this application embodiment. Here, the feature extraction layer of the image recognition model is also used to perform feature fusion on the extracted features. Specifically, after extracting the first feature of the adversarial example of the image, the feature extraction layer performs feature shuffling on the first feature, then performs image style transfer processing based on the feature shuffling to obtain the second feature, and then fuses the first feature and the second feature to obtain the sample image feature. In actual implementation, feature shuffling and feature fusion can be repeated multiple times. For example, after obtaining the first feature A... l Then, the first feature A l The first feature shuffling is performed to obtain the shuffling feature A′. l Then based on the first feature A l and the characteristics of mixed washing A′ l Perform image style transfer processing. Specifically, image style transfer processing can be performed using the following formula:

[0131]

[0132]

[0133] in, and These are all second features obtained after image style transfer processing, and AdaIN() is the image style transfer processing function.

[0134] Next, the terminal uses the following formula based on the first feature A. l Mixed washing characteristics A′ l First feature Second feature Perform image fusion:

[0135]

[0136]

[0137]

[0138] Among them, FeatureFusion(A l () represents the sample image features obtained after feature fusion.

[0139] This application embodiment increases the diversity of sample image features through image style transfer processing and feature fusion, enabling the model to learn more types of features, thereby further improving the robustness of the model.

[0140] In this application's embodiments, we conducted experimental evaluations of the models under different training methods. See [link / reference]. Figure 10A and Figure 10B , Figure 10A This is a schematic diagram of an optional accuracy result for the standard generalization method provided in the embodiments of this application. Figure 10B This is a schematic diagram illustrating an optional accuracy result of the robust generalization method provided in this application embodiment. MART w / AGAIN and AWP w / AGAIN are the training methods provided in this application embodiment, while the others are training methods from related technologies. It can be seen that MART w / AGAIN and AWP w / AGAIN, compared to PGD-AT and MART, effectively reduce the standard generalization error and robust generalization error, improving test accuracy. Compared to other training methods, such as AWP and LAS, they have a smaller difference in generalization ability, but these are achieved at the cost of reducing the accuracy of the training dataset. The method in this application embodiment can maintain or even improve the accuracy of the training dataset while still achieving a smaller generalization error. Furthermore, when the method in this application embodiment is combined with AWP, the generalization error can be further reduced, improving accuracy on clean data and adversarial examples on the test set, while ensuring the model's accuracy on the training set.

[0141] See Figure 11A and Figure 11B , Figure 11A This is an optional schematic diagram of the image features extracted from the penultimate layer of the image recognition model obtained by the method provided in the embodiments of this application. Figure 11B This is an optional schematic diagram of image features extracted from the penultimate layer of an image recognition model obtained through related techniques. It should be understood that the image features extracted from the penultimate layer of the model are the features input to the model's output layer, corresponding to the sample image features in the above embodiments of this application during training. Figure 11A and 11B As can be seen, the features of a certain category learned by the method provided in this application are more compactly distributed among features of other different categories, making the model more difficult to be attacked based on marginal region features, thereby reducing the model's generalization error and improving its robustness. In contrast, the features of each category learned by related technical methods are independently distributed in different regions. If attacked by marginal region features, it is easy to cause errors in the recognition results, resulting in poor model robustness.

[0142] The image recognition method provided in the embodiments of this application will be further described below. See also... Figure 12 , Figure 12 This is an optional flowchart illustrating an image recognition method provided in an embodiment of this application, which will be combined with... Figure 12 The steps shown are explained.

[0143] Step 1201: Obtain the image to be recognized;

[0144] Step 1202: Using an image recognition model, perform feature extraction processing on the image to be recognized to obtain the corresponding image features;

[0145] Step 1203: Based on the image features, determine the fourth mapping feature corresponding to the recognition result and the fifth mapping feature corresponding to the interference result;

[0146] Step 1204: Based on the fourth mapping feature and the fifth mapping feature, perform image recognition on the image to be recognized to obtain the recognition result.

[0147] In practical implementation, after obtaining the image to be recognized, the terminal extracts features through an image recognition model to obtain image features. Based on the image features, the output layer of the image recognition model determines the fourth mapping feature corresponding to the recognition result and the fifth mapping feature corresponding to the interference result. It should be understood that if the image recognition model can recognize N types, then both the recognition result and the interference result are at least one of the N types. The number of interference results can be one or more. The output layer of the image recognition model can obtain the mapping feature corresponding to each of the N types. In the embodiment of this application, during image recognition, not only the fourth mapping feature corresponding to the recognition result is considered, but also the fifth mapping feature of the interference result is considered. While simultaneously considering the fourth and fifth mapping features, the model can still output accurate recognition results, providing the model's generalization ability and robustness.

[0148] The following description continues to illustrate the exemplary structure of the image recognition model acquisition device 2055 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module in the image recognition model acquisition device 2055 stored in the memory 205 may include:

[0149] The module 20551 is used to obtain image adversarial samples, wherein the image adversarial samples carry sample labels;

[0150] The feature extraction module 20552 is used to perform feature extraction processing on the adversarial examples of the image through an image recognition model to obtain the corresponding sample image features;

[0151] The determining module 20553 is used to determine, based on the features of the sample image, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label, wherein the interference label is different from the sample label;

[0152] The recognition module 20554 is used to perform image recognition on the image adversarial sample based on the first mapping feature and the second mapping feature, and obtain the corresponding first recognition result;

[0153] The parameter update module 20555 is used to update the model parameters of the image recognition model based on the sample label and the first recognition result.

[0154] In some embodiments, the recognition module 20554 is further configured to perform feature concatenation on the first mapping feature and the second mapping feature to obtain corresponding concatenated features; and to perform image recognition on the adversarial sample of the image based on the concatenated features to obtain the first recognition result.

[0155] In some embodiments, the identification module 20554 is further configured to determine a first ratio corresponding to the first mapping feature and a second ratio corresponding to the second mapping feature; and to perform a weighted summation on the first mapping feature and the second mapping feature based on the first ratio and the second ratio.

[0156] In some embodiments, the image recognition model includes a first prediction layer and a second prediction layer. The first mapping feature, the second mapping feature, and the first recognition result are obtained through the first prediction layer. The recognition module 20554 is further configured to determine a third mapping feature corresponding to the sample label based on the sample image features through the second prediction layer; and to perform image recognition on the adversarial sample of the image based on the third mapping feature through the second prediction layer to obtain a corresponding second recognition result. The parameter update module 20555 is further configured to update the model parameters of the image recognition model based on the sample label, the first recognition result, and the second recognition result.

[0157] In some embodiments, the parameter update module 20555 is further configured to determine a first error between the sample label and the first recognition result and a second error between the sample label and the second recognition result; and update the model parameters of the image recognition model based on the sum of the first error and the second error.

[0158] In some embodiments, the feature extraction module 20552 is further configured to extract features from the adversarial sample of the image to obtain a corresponding first feature; perform image style transfer processing on the first feature to obtain a corresponding second feature; and fuse the first feature and the second feature to obtain the sample image features.

[0159] In some embodiments, the obtaining module 20551 is further configured to obtain a clean image sample, the clean image sample carrying the sample label; and to add noise to the clean image sample to obtain the adversarial image sample.

[0160] The image recognition device provided in the embodiments of this application will be further described below. The image recognition device includes:

[0161] The acquisition module is used to acquire the image to be recognized;

[0162] The feature extraction module is used to perform feature extraction processing on the image to be recognized through an image recognition model to obtain the corresponding image features;

[0163] The determining module is used to determine, based on the image features, a fourth mapping feature corresponding to the recognition result and a fifth mapping feature corresponding to the interference result;

[0164] The recognition module is used to perform image recognition on the image to be recognized based on the fourth mapping feature and the fifth mapping feature, and obtain the recognition result.

[0165] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be repeated.

[0166] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in this application.

[0167] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and, when executed by a processor, will cause the processor to execute the method provided in this application.

[0168] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0169] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0170] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0171] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0172] In summary, the embodiments of this application can expand the feature learning range of the model and improve the model's generalization ability and robustness.

[0173] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for obtaining an image recognition model, comprising: Obtain image adversarial examples, wherein the image adversarial examples carry sample labels; The image adversarial examples are processed by an image recognition model to extract features and obtain the corresponding sample image features. Based on the features of the sample image, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to the interference label are determined, wherein the interference label is different from the sample label; Based on the first mapping feature and the second mapping feature, image recognition is performed on the adversarial sample in the image to obtain the corresponding first recognition result; Based on the sample labels and the first recognition result, the model parameters of the image recognition model are updated.

2. The method according to claim 1, wherein performing image recognition on the adversarial sample based on the first mapping feature and the second mapping feature to obtain a corresponding first recognition result includes: The first mapping feature and the second mapping feature are concatenated to obtain the corresponding concatenated feature; Based on the splicing features, image recognition is performed on the adversarial sample in the image to obtain the first recognition result.

3. The method according to claim 2, wherein the feature concatenation of the first mapping feature and the second mapping feature includes: Determine a first ratio corresponding to the first mapping feature and a second ratio corresponding to the second mapping feature; Based on the first ratio and the second ratio, the first mapping feature and the second mapping feature are weighted and summed.

4. The method according to claim 1, wherein the image recognition model includes a first prediction layer and a second prediction layer, the first mapping feature, the second mapping feature, and the first recognition result are obtained through the first prediction layer, and the method further includes: The second prediction layer determines a third mapping feature corresponding to the sample label based on the sample image features. Through the second prediction layer, based on the third mapping feature, image recognition is performed on the adversarial sample of the image to obtain the corresponding second recognition result; The step of updating the model parameters of the image recognition model based on the sample labels and the first recognition result includes: Based on the sample labels, the first recognition result, and the second recognition result, the model parameters of the image recognition model are updated.

5. The method according to claim 4, wherein updating the model parameters of the image recognition model based on the sample label, the first recognition result, and the second recognition result includes: Determine a first error between the sample label and the first recognition result, and a second error between the sample label and the second recognition result; The model parameters of the image recognition model are updated based on the sum of the first error and the second error.

6. The method according to claim 1, wherein the step of performing feature extraction processing on the adversarial examples of the image to obtain corresponding sample image features includes: Feature extraction is performed on the adversarial sample in the image to obtain the corresponding first feature; The first feature is subjected to image style transfer processing to obtain the corresponding second feature; The first feature and the second feature are fused to obtain the sample image features.

7. The method according to claim 1, wherein obtaining image adversarial examples comprises: Obtain a clean image sample, wherein the clean image sample carries the sample label; Noise is added to the clean sample of the image to obtain the adversarial sample of the image.

8. An image recognition method, comprising: Obtain the image to be recognized; The image to be identified is processed by feature extraction using an image recognition model to obtain the corresponding image features; Based on the image features, a fourth mapping feature corresponding to the recognition result and a fifth mapping feature corresponding to the interference result are determined; Based on the fourth mapping feature and the fifth mapping feature, image recognition is performed on the image to be recognized to obtain the recognition result; The image recognition model is obtained based on the method for obtaining the image recognition model according to any one of claims 1-7.

9. An apparatus for obtaining an image recognition model, comprising: The acquisition module is used to acquire image adversarial samples, wherein the image adversarial samples carry sample labels; The feature extraction module is used to perform feature extraction processing on the adversarial examples in the image through an image recognition model to obtain the corresponding sample image features; The determining module is used to determine, based on the features of the sample image, a first mapping feature corresponding to the sample label and a second mapping feature corresponding to an interference label, wherein the interference label is different from the sample label; The recognition module is used to perform image recognition on the adversarial sample of the image based on the first mapping feature and the second mapping feature, and obtain the corresponding first recognition result; The parameter update module is used to update the model parameters of the image recognition model based on the sample label and the first recognition result.

10. An image recognition device, comprising: The acquisition module is used to acquire the image to be recognized; The feature extraction module is used to perform feature extraction processing on the image to be recognized through an image recognition model to obtain the corresponding image features; The determining module is used to determine, based on the image features, a fourth mapping feature corresponding to the recognition result and a fifth mapping feature corresponding to the interference result; The recognition module is used to perform image recognition on the image to be recognized based on the fourth mapping feature and the fifth mapping feature, and obtain the recognition result; The image recognition model is obtained based on the image recognition model obtaining device according to claim 9.

Images